Skip to content
OpenAITraining

GPTBot

GPTBot is a crawler run by OpenAI. It trains OpenAI foundation models.

What it does

GPTBot collects pages to train or improve AI models. Most operators use a separate user agent for each job, so you can usually allow or block one without affecting the others.

Model training and content-use controls. Some also govern answer grounding; review the operator's policy before changing access.

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

This is a full user agent string GPTBot sends. Version numbers change over time; match the GPTBot token rather than the whole string.

robots.txt

To block GPTBot, add this to the robots.txt at the root of your site:

User-agent: GPTBot Disallow: /

Blocking it asks the operator not to use your pages for model training from then on. It does not remove content already collected, and it does not stop the operator's search crawler or live fetchers, which use their own user agents.

To allow it explicitly, for example after a broader rule blocks other crawlers:

User-agent: GPTBot Allow: /

Check what your robots.txt allows for GPTBot and the other AI crawlers in this directory.

Verifying requests

OpenAI publishes the IP addresses GPTBot uses. A request that claims to be GPTBot from any other address is not genuine.

In SurfacedBy

SurfacedBy's AI traffic tracking recognizes GPTBot in the user agent and checks each request against OpenAI's published addresses. Only requests that pass count as crawls; the rest are shown separately.

Operator documentation

Details on this page come from OpenAI's own documentation: developers.openai.com/api/docs/bots.

More from OpenAI

Common Questions

What is GPTBot?
GPTBot is a crawler run by OpenAI. It trains OpenAI foundation models.
How do I block GPTBot in robots.txt?
Add a group for GPTBot with "Disallow: /" to the robots.txt at the root of your site. Blocking it asks the operator not to use your pages for model training from then on. It does not remove content already collected, and it does not stop the operator's search crawler or live fetchers, which use their own user agents.
How can I tell if a request is really GPTBot?
OpenAI publishes the IP addresses GPTBot uses. A request that claims to be GPTBot from any other address is not genuine.