Like any other search engine, Brave search has a crawler to discover new pages and index their content.
The Brave Search crawler does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them. However, if a domain or page is not crawlable by Googlebot, then Brave Search’s bot will not crawl it either.
Note that robots.txt is not used to prevent a page from being indexed. A site owner can delist a page by using the robots noindex directive. As soon as your web page has been updated with the directive, Brave needs to re-fetch it in order to apply the changes - that is, delisting the page. You can submit your page for re-fetching here.
If your inquiry is about delisting web pages containing personal data about you, you should follow the guidance and process of the Right To Be Forgotten.
In case Brave Search is returning a page which does not exist anymore and you would like to get it delisted, you can submit your URL here. Additionally, you may contact us at not-found@brave.com.
Brave Search’s crawler is partially powered by information provided by users enrolled in the Web Discovery Project (WDP) option in Brave browser’s search settings, which is an off-by-default (aka opt-in), privacy-preserving system. WDP has multiple mechanisms to prevent Brave from knowing who is contributing what (WDP is also open-source for auditing and inspection by anyone).