Googlebot News: user agent, IP verification and robots.txt
Googlebot News is operated by Google. It is classified here as Search (Google News index). Verification method: Published CIDR list and reverse DNS.
Exact user-agent string
Google states Googlebot-News has no separate HTTP user-agent string. Crawling is done with the various Googlebot user agents; the token exists only as a robots.txt control.
robots.txt token: Googlebot-News. The user-agent string is self-asserted and can be
forged by anyone. It is a claim, not proof.
How to verify Googlebot News is genuine
Match the source IP against Google's published CIDR list: https://developers.google.com/static/crawling/ipranges/common-crawlers.json
169 ranges published, as of 2026-07-31. Pulled live at build time, never hardcoded.
A reverse DNS path also exists. See https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests.
How to block or allow Googlebot News
User-agent: Googlebot-News
Disallow: /Replace Disallow with Allow to permit it explicitly.
What blocking Googlebot News costs you
Disallowing Googlebot-News removes you from Google News, including news.google.com and the Google News app, while leaving ordinary Google Search results in place. For a publisher this is a large and very specific loss.
What is specific to Googlebot News
Googlebot-News is a control token wearing a crawler's name. It has no user-agent of its own, so you cannot tell from a log line whether a Googlebot request was made for News or for Search, and there is no separate crawl to rate-limit. It also carries two tokens, Googlebot-News and Googlebot, and Google applies a rule if either matches. The consequence catches publishers out: a blanket Googlebot disallow removes you from News as well, but a Googlebot-News disallow does not touch Search. The narrower token is the safe one to reach for.
Verify a request claiming to be Googlebot News
Check a Googlebot News request against Google's published ranges
Source
Primary documentation: https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers. Last verified 2026-08-02.
Other Google crawlers
- Googlebot - Search index
- GoogleOther - Internal fetch and analysis
- GoogleOther-Image - Internal fetch, images
- GoogleOther-Video - Internal fetch, video
- Google-CloudVertexBot - Site-owner requested crawl
- Google-Extended - Control token, not a crawler