[{"data":1,"prerenderedAt":100},["ShallowReactive",2],{"tool-ai-crawler-log-analyzer-en":3},{"common":4,"copy":13},{"hubLink":5,"resultTitle":6,"fix":7,"fullTitle":8,"fullText":9,"fullCta":10,"faqTitle":11,"waiting":12},"All free tools","Results for {host}","How to fix it","See how AI reads this site","Run a free full scan of {host} to get both AI readiness scores and every fix.","Scan this site","Questions","Testing {host}. This can take several minutes, so keep this page open.",{"name":14,"title":15,"description":16,"h1":14,"lede":17,"empty":18,"labels":19,"about":20,"faq":24,"crawlerLogs":40},"AI crawler log analyzer","AI crawler log analyzer: which AI bots visit your site?","Drop your server access log to see which AI crawlers visit, which are genuine and which pages fail for them. Runs in your browser; nothing is uploaded.","Drop your server access log to see which AI crawlers visited your site, which of them came from the operators' published addresses, and which pages failed for them. The log is read in your browser and never uploaded.","",{},[21,22,23],"The analyzer reads nginx and Apache logs in the combined format, and Cloudflare Logpush HTTP request logs (NDJSON). Plain files and .gz files both work, and you can drop several rotated files at once. Very large logs are read as a stream, so the page stays responsive.","A user agent is only a name, and anyone can send GPTBot. So each hit is also checked against the IP addresses the operator publishes. A hit from those addresses is verified. A hit from outside them is flagged, but it is only “not from the published addresses”: operators add ranges now and then, and our copy of the list can lag. Some operators publish no list at all; their hits stay unverifiable.","Your log stays on your device. The only request the page makes is one for the operators' public address lists, and it carries nothing from your log.",[25,28,31,34,37],{"q":26,"a":27},"Is my log uploaded to SagBot?","No. The file is read and counted in your browser. Only the operators' public IP address lists are fetched, and that request carries no log data. Close the tab and the report is gone.",{"q":29,"a":30},"Which log formats work?","The combined format used by nginx by default and by Apache, and Cloudflare Logpush HTTP request logs as NDJSON. If your server writes the Common Log Format, it has no user agent, so switch it to combined first. Our guide on AI crawler visits shows how.",{"q":32,"a":33},"What does impostor mean?","The user agent says it is an AI crawler, but the IP address is outside the ranges its operator publishes. That is the usual sign of a fake, a scraper borrowing the name. It can also be a genuine crawler on a range the operator added recently, so treat it as a lead, not a verdict.",{"q":35,"a":36},"Why does a crawler show as unverifiable?","Its operator publishes no machine-readable list of addresses, or we could not load it. Hits are then counted, but nobody can tell whether they are genuine from the log alone. Reverse DNS lookups can, and the guide explains how.",{"q":38,"a":39},"Why are some AI visits missing from my log?","Pages served from a CDN cache never reach your origin, so they are not in its log. If your domain on Cloudflare is not proxied, or you log somewhere else, the crawler's requests do not appear either. Use the log from the edge that actually receives the requests.",{"drop":41,"dropOr":42,"choose":43,"dropHint":44,"fileCount":45,"pasteLabel":46,"pastePlaceholder":47,"pasteHint":48,"analyse":49,"privacyTitle":50,"privacy":51,"reading":52,"readingFile":53,"resultTitle":54,"summary":55,"formats":63,"table":66,"verdictsNote":74,"noRanges":75,"detailTitle":76,"topPages":77,"failing":78,"noFailing":79,"daily":80,"dailyBar":81,"hitsCount":82,"emptyTitle":83,"emptyIntro":84,"limits":85,"unsupportedTitle":90,"unsupported":91,"unsupportedLink":92,"gzipUnsupported":93,"tooLarge":94,"readFailed":95,"start":96,"otherPages":97,"moreFailing":98,"statusDone":99,"proxyNote":89},"Drop log files here","or","Choose files",".log, .txt, .json and .gz files. Several files are read into one report.","{n} files selected","Or paste log lines","203.0.113.7 - - [09\u002FNov\u002F2026:10:12:01 +0000] \"GET \u002Fpricing HTTP\u002F1.1\" 200 5120 \"-\" \"GPTBot\u002F1.2 (+https:\u002F\u002Fopenai.com\u002Fgptbot)\"","Up to 5 MB of text.","Analyse pasted text","Stays on your device","Your log is processed in this browser. Nothing from it is sent to SagBot. The page only downloads the operators' public IP address lists.","Reading the log…","Reading {read} of {total}","AI crawlers in this log",{"range":56,"lines":57,"unparsed":58,"other":59,"ai":60,"impostor":61,"format":62},"Period","Lines read","Unparsed lines","Other traffic","AI crawler hits","Not from published addresses","Format",{"combined":64,"cloudflare":65},"nginx \u002F Apache combined","Cloudflare Logpush",{"crawler":67,"operator":68,"kind":69,"hits":70,"verified":71,"impostor":72,"unverifiable":73},"Crawler","Operator","Kind","Hits","Verified","Impostor","Unverifiable","Verified: from the operator's published addresses. Impostor: not from those addresses. New ranges can take a while to be published, so check before blocking. Unverifiable: the operator publishes no list, or it could not be loaded.","The operators' address lists could not be loaded, so every hit is shown as unverifiable.","Detail per crawler","Top pages","Pages that failed","No 4xx or 5xx responses for this crawler.","Hits per day","{day}: {hits} hits","{n} hits","No AI crawler visits in this log","{lines} lines were read and none had an AI crawler user agent. That does not always mean none came:",[86,87,88,89],"Pages served from a CDN cache never reach your origin, so they are missing from its log.","If your domain is on Cloudflare but not proxied, the requests do not pass through it, so a Cloudflare log shows nothing.","A user agent can be faked, in both directions, and some crawlers do not announce themselves.","Behind a proxy or CDN, the log may record the proxy's address instead of the crawler's, so genuine crawlers can look like impostors.","This log format is not supported","No line matched the combined format or Cloudflare Logpush NDJSON.","See how to check and set the log format","This browser cannot unzip .gz files. Unzip the file first and drop the result.","This input is too large to read. Split the log or paste less text.","The file could not be read. Try again, or unzip it first if it is compressed.","Choose a log file to see the report.","Other pages","and {n} more","Done: {n} AI crawler hits found.",1794386918817]