Claude-User: user agent, IP verification and robots.txt
Claude-User is operated by Anthropic. It is classified here as Not a crawler (User-triggered fetch). Verification method: Published CIDR list.
Exact user-agent string
Anthropic's crawler documentation names the robots.txt token but does not publish a full example user-agent string.
robots.txt token: Claude-User. The user-agent string is self-asserted and can be
forged by anyone. It is a claim, not proof.
How to verify Claude-User is genuine
Match the source IP against Anthropic's published CIDR list: https://claude.com/crawling/bots.json
20 ranges published, as of 2026-05-01. Pulled live at build time, never hardcoded.
How to block or allow Claude-User
User-agent: Claude-User
Disallow: /Replace Disallow with Allow to permit it explicitly.
What blocking Claude-User costs you
Anthropic states plainly that disabling Claude-User prevents their system retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search. This is the clearest published statement by any operator that blocking a user-triggered fetcher costs you visibility.
What is specific to Claude-User
Claude-User fires when a person asks Claude something that requires your page, so its traffic pattern is bursty and query-shaped rather than a systematic sweep. Anthropic's opt-out guidance applies to it the same way as the crawlers, through robots.txt rather than IP blocking, and it draws from the same shared bots.json range list. Because all three Claude robots resolve to the same IP set, a verified Claude-User request and a verified ClaudeBot request are indistinguishable at the network layer.
Verify a request claiming to be Claude-User
Check a Claude-User request against Anthropic's published ranges
Source
Primary documentation: https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler. Last verified 2026-08-02.
Other Anthropic crawlers
- ClaudeBot - AI training crawler
- Claude-SearchBot - Search index