server logs(3)

/ Date/ Name
2026.02.13 Bot traffic and AI crawlers: what server logs show

Summary:

Your analytics hides most bots. Your server logs show them all. How to read a log line, spot AI crawlers, tell real Googlebot from fakes, and decide what to do.

Topics:

How the internet works 5 min read

Read
2026.02.05 Reading server logs for questions a crawler tool cannot answer

Summary:

Server logs can show which paths were requested and what the server returned, but they cannot tell you whether a page was indexed. Start with one question and record the limits of the evidence.

Topics:

Technical SEO 4 min read

Read
2024.10.16 AI crawlers in your logs: who’s visiting and why

Summary:

Identify automated requests from their user agent and verified network details, then decide which crawlers to allow based on their stated purpose and your site's policy.

Topics:

AI & search 4 min read

Read