YouBot

YouBot crawls pages to build the search index an AI platform answers from. Most operators use a separate user agent for each job, so you can usually allow or block one without affecting the others.

Crawlers that build search indexes used by AI platforms. Access supports discovery but does not guarantee a listing or citation.

Requests from YouBot carry this token in their user agent string. Match the token rather than a full string, which changes as You.com updates versions.

To block YouBot, add this to the robots.txt at the root of your site:

Blocking it keeps your pages out of that index, so the platform has less of your content to find and cite. Allowing it makes your pages eligible, not guaranteed, to appear in answers.

To allow it explicitly, for example after a broader rule blocks other crawlers:

Check what your robots.txt allows for YouBot and the other AI crawlers in this directory.

YouBot signs its requests with Web Bot Auth. A signature can be checked against the public keys You.com publishes, so a copied user agent cannot pass as YouBot.

• you.com/.well-known/http-message-signatures-directory

SurfacedBy's AI traffic tracking recognizes YouBot in the user agent and checks each request against You.com's published signing keys. Only requests that pass count as crawls; the rest are shown separately.

Details on this page come from You.com's own documentation: you.com/docs/youbot.