Skip to content

See When AI Crawlers Visit and Which Pages They Read

Ali Khallad2 min readUpdated
July 9, 2026 , 2 min read
Product update card reading When AI crawlers visit, and which pages, beside a SurfacedBy card titled When AI is active: a seven-weekday by 24-hour grid of blue tiles with the Live crawls toggle selected and the line Busiest: Mondays around 13:00. Quiet: 22:00-08:00 every day.
Share

The Tracking page gives you two ways to look beyond crawler totals: a weekly activity grid and a detail view for each page. You can see when AI requests cluster, then open a page to check which bots fetched it and whether your server let them through.

See the pattern across the week

The “When AI is active” grid shows the last four weeks by weekday and hour, in your local time. Switch between recorded visits from AI assistants and live retrieval requests. The latter are fetches made while an assistant answers a question; training and search-indexing traffic stay out of this view.

SurfacedBy card titled When AI is active, subtitle Hour-by-hour pattern over the last 4 weeks, shown in your local time (America/New_York), with AI visitors and Live crawls toggles, a grid of weekday rows Mon to Sun against hour columns labelled every three hours from 00 to 21 and shaded from light to dark blue, a Less to More legend, and the line Busiest: Mondays around 13:00. Quiet: 22:00-08:00 every day.
The activity grid with live crawls selected in the demo workspace. It groups four weeks of requests by local weekday and hour.

Look at a particular page

Click a page path in Crawler activity to open its details. The drawer shows requests over your selected period, the bots behind them, and any access errors. A chart puts crawler activity alongside recorded AI referral visits, so you can inspect both for the same page.

The purpose labels separate training, search indexing and live retrieval. Verification shows which requests could be matched to a vendor’s published addresses; not every bot can be verified. A request using an AI bot’s name is not, by itself, proof of its identity.

SurfacedBy drawer titled Crawled page detail for the path /sizing: In the last 30 days: first seen Aug 17, last seen Sep 15; tiles Crawls 928, Verified 928 of 928, Status All allowed; a Crawls by day chart of stacked bars for OpenAI, Anthropic, and Perplexity with an AI visits line; and a list titled Crawlers that fetched this page with ClaudeBot 211, GPTBot 203, OAI-SearchBot 138, PerplexityBot 129, ChatGPT-User 84, Perplexity-User 83, and Claude-User 80, each with a purpose chip reading Training, Search indexing, or Live citation.
The page drawer separates bots by purpose and shows access status. In this captured demo view, no attributable AI referral visit was recorded for the page.

These views show recorded requests and attributable visits. They cannot prove that a fetched page was used in an answer, and a flat referral line does not prove that no one arrived from AI. For pages with repeated retrieval activity, prompt suggestions based on crawler activity offer a way to investigate gaps in what you track.

Live tracking is included on paid plans and during a trial. Crawler data needs the WordPress plugin, Cloudflare edge tracker or NGINX server tracker; the browser snippet records visitors only. See the tracking setup guide to connect your site.

See SurfacedBy in actionWatch overview52 seconds