FacebookExternalHit: user agent, IP verification and robots.txt
FacebookExternalHit is operated by Meta. It is classified here as Retrieval (Link preview fetch). Verification method: None available.
Exact user-agent string
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
robots.txt token: facebookexternalhit. The user-agent string is self-asserted and can be
forged by anyone. It is a claim, not proof.
How to verify FacebookExternalHit is genuine
Meta publishes no IP range file and no reverse DNS method for FacebookExternalHit. There is no way for anyone, including us, to prove a request claiming this identity is genuine.
How to block or allow FacebookExternalHit
User-agent: facebookexternalhit
Disallow: /Replace Disallow with Allow to permit it explicitly.
What blocking FacebookExternalHit costs you
Blocking FacebookExternalHit means links to your site shared on Facebook, Instagram or Messenger render without a title, description or thumbnail. The link still works, it just looks broken, which measurably suppresses clicks. This is the highest-cost block on this list for most consumer sites.
What is specific to FacebookExternalHit
FacebookExternalHit is the oldest crawler in this reference and the only one with documented performance requirements you can fail. Meta states your server must support gzip and deflate, that Open Graph properties must appear within the first 1 MB or they are cut off, and that the page must be crawlable within a few seconds or the preview cannot be displayed. It also either honours the Range header it sends or requires you to ignore it entirely. Meta additionally states the crawler may bypass robots.txt when performing security or integrity checks such as malware scanning, so a disallow is not a guarantee. When a preview looks wrong the fix is usually the 1 MB rule, not the block.
Verify a request claiming to be FacebookExternalHit
Check a FacebookExternalHit request against Meta's published ranges
Source
Primary documentation: https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/. Last verified 2026-08-02.
Other Meta crawlers
- Meta-ExternalAgent - AI training and indexing crawler
- Meta-ExternalFetcher - User-triggered fetch
- Meta-WebIndexer - Meta AI search index
- Meta-ExternalAds - Advertising and business products