ClaudeBot: user agent, IP verification and robots.txt

ClaudeBot is operated by Anthropic. It is classified here as Training (AI training crawler). Verification method: Published CIDR list.

Exact user-agent string

Anthropic's crawler documentation names the robots.txt token but does not publish a full example user-agent string. Nothing is asserted here that Anthropic has not published.

robots.txt token: ClaudeBot. The user-agent string is self-asserted and can be forged by anyone. It is a claim, not proof.

How to verify ClaudeBot is genuine

Match the source IP against Anthropic's published CIDR list: https://claude.com/crawling/bots.json

20 ranges published, as of 2026-05-01. Pulled live at build time, never hardcoded.

How to block or allow ClaudeBot

User-agent: ClaudeBot
Disallow: /

Replace Disallow with Allow to permit it explicitly.

What blocking ClaudeBot costs you

Restricting ClaudeBot signals that your site's future material should be excluded from Anthropic's model training datasets. It does not affect whether Claude can reach your page when a user asks about it, which is Claude-User, or whether you are indexed for Claude's search, which is Claude-SearchBot.

What is specific to ClaudeBot

Anthropic publishes one IP list for all three of its robots, not one per bot. The same bots.json covers ClaudeBot, Claude-User and Claude-SearchBot, so a source IP match proves the request came from Anthropic but does not by itself prove which bot sent it. To distinguish them you still have to trust the user-agent, which means the IP check and the user-agent check answer different questions here than they do for OpenAI or Perplexity. Anthropic also explicitly warns against opting out by blocking their IPs: doing so stops them reading your robots.txt, which is the only mechanism that actually registers the opt-out.

Verify a request claiming to be ClaudeBot

Check a ClaudeBot request against Anthropic's published ranges

Source

Primary documentation: https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler. Last verified 2026-08-02.

Other Anthropic crawlers

All crawlers and the full comparison table