AI crawler log analyzer
Drop your server access log to see which AI crawlers visited your site, which of them came from the operators' published addresses, and which pages failed for them. The log is read in your browser and never uploaded.
Drop log files here
or
.log, .txt, .json and .gz files. Several files are read into one report.
Choose a log file to see the report.
The analyzer reads nginx and Apache logs in the combined format, and Cloudflare Logpush HTTP request logs (NDJSON). Plain files and .gz files both work, and you can drop several rotated files at once. Very large logs are read as a stream, so the page stays responsive.
A user agent is only a name, and anyone can send GPTBot. So each hit is also checked against the IP addresses the operator publishes. A hit from those addresses is verified. A hit from outside them is flagged, but it is only “not from the published addresses”: operators add ranges now and then, and our copy of the list can lag. Some operators publish no list at all; their hits stay unverifiable.
Your log stays on your device. The only request the page makes is one for the operators' public address lists, and it carries nothing from your log.
Questions
Is my log uploaded to SagBot?
No. The file is read and counted in your browser. Only the operators' public IP address lists are fetched, and that request carries no log data. Close the tab and the report is gone.
Which log formats work?
The combined format used by nginx by default and by Apache, and Cloudflare Logpush HTTP request logs as NDJSON. If your server writes the Common Log Format, it has no user agent, so switch it to combined first. Our guide on AI crawler visits shows how.
What does impostor mean?
The user agent says it is an AI crawler, but the IP address is outside the ranges its operator publishes. That is the usual sign of a fake, a scraper borrowing the name. It can also be a genuine crawler on a range the operator added recently, so treat it as a lead, not a verdict.
Why does a crawler show as unverifiable?
Its operator publishes no machine-readable list of addresses, or we could not load it. Hits are then counted, but nobody can tell whether they are genuine from the log alone. Reverse DNS lookups can, and the guide explains how.
Why are some AI visits missing from my log?
Pages served from a CDN cache never reach your origin, so they are not in its log. If your domain on Cloudflare is not proxied, or you log somewhere else, the crawler's requests do not appear either. Use the log from the edge that actually receives the requests.