Google-CloudVertexBot: user agent, IP verification and robots.txt
Google-CloudVertexBot is operated by Google. It is classified here as Retrieval (Site-owner requested crawl). Verification method: Published CIDR list and reverse DNS.
Exact user-agent string
Google-CloudVertexBot
robots.txt token: Google-CloudVertexBot. The user-agent string is self-asserted and can be
forged by anyone. It is a claim, not proof.
How to verify Google-CloudVertexBot is genuine
Match the source IP against Google's published CIDR list: https://developers.google.com/static/crawling/ipranges/common-crawlers.json
169 ranges published, as of 2026-07-31. Pulled live at build time, never hardcoded.
A reverse DNS path also exists. See https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests.
How to block or allow Google-CloudVertexBot
User-agent: Google-CloudVertexBot
Disallow: /Replace Disallow with Allow to permit it explicitly.
What blocking Google-CloudVertexBot costs you
Google states this token has no effect on Google Search or other products. It only affects crawls that site owners themselves requested for building Vertex AI Agents, so blocking it breaks your own Vertex integration and nothing else.
What is specific to Google-CloudVertexBot
Google-CloudVertexBot is the odd one out in Google's crawler list because it does not crawl on Google's initiative at all. It fetches sites at the request of the site owner, when someone is building a Vertex AI Agent over their own content. That inverts the usual reasoning: if you see it in your logs and did not set up a Vertex agent, the interesting question is who in your organisation did, not whether Google is overreaching. Google documents it as a user-agent substring rather than a full string, so match on the substring and do not expect a versioned bot URL. It also carries Googlebot as a secondary token, which means a blanket Googlebot disallow will stop your own Vertex crawl.
Verify a request claiming to be Google-CloudVertexBot
Check a Google-CloudVertexBot request against Google's published ranges
Source
Primary documentation: https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers. Last verified 2026-08-02.
Other Google crawlers
- Googlebot - Search index
- GoogleOther - Internal fetch and analysis
- Googlebot News - Google News index
- GoogleOther-Image - Internal fetch, images
- GoogleOther-Video - Internal fetch, video
- Google-Extended - Control token, not a crawler