How do you verify a visitor claiming to be Googlebot, GPTBot or ClaudeBot really is?

A User-Agent string is a claim, not proof

AdminKit’s User Agent tool parses the raw User-Agent header your own browser sends and shows exactly what it reveals — browser, engine, OS, coarse device type. The same parsing logic applies to any request a server receives, including one claiming to be a search-engine or AI crawler. But parsing only tells you what the string says; it cannot tell you whether the client sending it is actually the crawler it claims to be, because nothing in HTTP stops a client from setting that header to arbitrary text. See what a User-Agent string reveals for how loosely-structured and client-controlled the header really is.

For most traffic this doesn’t matter — a browser lying about its own identity has little to gain. It matters a great deal for a request claiming to be “Googlebot” or “GPTBot,” because many sites give those specific claims special treatment: lighter rate limits, access past a bot-challenge page, or inclusion in analytics that get less scrutiny than everything else. Verifying the claim means looking at something the requester doesn’t fully control — the IP address the request actually came from — against something the real crawler operator publishes.

Google documents a reverse-DNS method — and it requires both directions

Google’s own crawler-verification documentation states that Googlebot’s User-Agent header “is often spoofed by other crawlers,” and recommends forward-confirmed reverse DNS (FCrDNS) as the check:

1. Reverse DNS lookup. Take the source IP from your log and look up its PTR record — AdminKit’s reverse DNS tool does this directly. Confirm the returned hostname ends in googlebot.com, google.com or googleusercontent.com.
2. Forward DNS confirmation. Take that hostname and look up its own A/AAAA record — AdminKit’s DNS lookup tool works for this. Confirm it resolves back to the exact same IP you started with.

Both steps matter, and skipping the second is the most common mistake. A PTR record is set by whoever controls the reverse-DNS zone for that IP block — which, for an IP that genuinely belongs to Google, is Google itself, but for a spoofed setup could be anyone who controls their own IP range and chooses to publish a misleading hostname there. Only the forward lookup, landing back on the original IP under a domain (googlebot.com) that only Google can publish records for, confirms the hostname and the address genuinely belong together. See how to read a reverse DNS (PTR) lookup result and using reverse DNS to investigate an unfamiliar IP for more on why a PTR record alone is never sufficient on its own.

Google also publishes machine-readable IP range files (including a general list at gstatic.com/ipranges/goog.json and several crawler-specific JSON files) as an alternative to running FCrDNS on every request — useful for checking in bulk, though it requires re-downloading the list periodically rather than a one-off lookup.

GPTBot and ClaudeBot: published IP ranges, not reverse DNS

It’s tempting to assume every major crawler supports the same FCrDNS check Google documents. As of this writing, neither OpenAI nor Anthropic documents one:

• OpenAI publishes separate IP-range JSON files for each of its crawlers — openai.com/gptbot.json for GPTBot, plus separate files for OAI-SearchBot and ChatGPT-User — and its published crawler documentation describes checking a source IP against those ranges. It does not document a reverse-DNS verification path.
• Anthropic takes the same approach for ClaudeBot, Claude-SearchBot and Claude-User: its support documentation states that a source IP appearing on the list at claude.com/crawling/bots.json indicates the request came from Anthropic.

Practically, this means checking a “GPTBot” or “ClaudeBot” claim correctly is a different process than checking a “Googlebot” one: fetch the vendor’s current JSON file and test whether the source IP falls inside one of its listed CIDR ranges, rather than running a PTR lookup and expecting a googlebot.com-shaped answer. AdminKit’s reverse DNS tool will still return whatever PTR record (if any) that IP happens to have, but neither OpenAI nor Anthropic has documented that record as part of how they expect their crawlers to be verified — so treat a PTR-based check alone as inconclusive for those two, not as a pass or fail.

Putting it together

Crawler	Documented verification method	AdminKit tool(s) that help
Googlebot (and other Google crawlers)	Forward-confirmed reverse DNS, or Google’s published IP-range JSON files	Reverse DNS + DNS lookup
GPTBot / OAI-SearchBot / ChatGPT-User	Match source IP against OpenAI’s published JSON ranges	None yet — a manual or scripted CIDR check against OpenAI’s file
ClaudeBot / Claude-SearchBot / Claude-User	Match source IP against Anthropic’s published JSON range	None yet — a manual or scripted CIDR check against Anthropic’s file

Whichever crawler a request claims to be, start from the source IP in your own server log — AdminKit’s User Agent tool is useful for understanding what a header reveals about an ordinary visitor, but a crawler-identity question always comes down to the IP address the request actually arrived from, checked against whatever method that crawler’s own operator documents, not the header text it chose to send.

This guide is general information, not a security audit of your specific network — see our disclaimer for what AdminKit's guidance does and does not commit to.