GPTBot: user agent, IP verification and robots.txt
GPTBot is operated by OpenAI. It is classified here as Training (AI training crawler). Verification method: Published CIDR list.
Exact user-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
robots.txt token: GPTBot. The user-agent string is self-asserted and can be
forged by anyone. It is a claim, not proof.
How to verify GPTBot is genuine
Match the source IP against OpenAI's published CIDR list: https://openai.com/gptbot.json
21 ranges published, as of 2025-10-30. Pulled live at build time, never hardcoded.
How to block or allow GPTBot
User-agent: GPTBot
Disallow: /Replace Disallow with Allow to permit it explicitly.
What blocking GPTBot costs you
Disallowing GPTBot signals that your content should not be used to train OpenAI's foundation models. Per OpenAI's own documentation it does not remove you from ChatGPT search results, which are governed separately by OAI-SearchBot. Blocking GPTBot alone is the cleanest available split between opting out of training and staying visible in ChatGPT.
What is specific to GPTBot
GPTBot and OAI-SearchBot are separate robots.txt tokens but not always separate crawls. OpenAI states that when a site allows both, it may use the results of a single crawl to serve both purposes rather than fetching twice. That means a GPTBot hit in your logs is not proof the content was used for training, and the absence of OAI-SearchBot hits is not proof you are missing from search. A second detail worth knowing when reading logs: when OpenAI fetches robots.txt itself, it may append a `robots.txt` marker to the user-agent string, so the same crawler produces two distinguishable strings depending on what it is requesting.
Verify a request claiming to be GPTBot
Check a GPTBot request against OpenAI's published ranges
Source
Primary documentation: https://platform.openai.com/docs/bots. Last verified 2026-08-02.
Other OpenAI crawlers
- OAI-SearchBot - Search index
- ChatGPT-User - User-triggered fetch
- OAI-AdsBot - Ad landing page validation