🕷 Crawl Budget Analyzer
Paste Access Log (Apache/Nginx format)
66.249.64.1 - - [01/Jan/2025:10:00:00] "GET / HTTP/1.1" 200 5432 "-" "Googlebot/2.1" 66.249.64.2 - - [01/Jan/2025:10:01:00] "GET /about HTTP/1.1" 200 3211 "-" "Googlebot/2.1" 157.55.39.1 - - [01/Jan/2025:10:02:00] "GET /products HTTP/1.1" 200 8712 "-" "bingbot/2.0" 157.55.39.1 - - [01/Jan/2025:10:03:00] "GET /wp-admin HTTP/1.1" 403 412 "-" "bingbot/2.0" 66.249.64.3 - - [01/Jan/2025:10:04:00] "GET /page?utm=xyz HTTP/1.1" 200 5432 "-" "Googlebot/2.1" 40.77.167.1 - - [01/Jan/2025:10:05:00] "GET /search?q=test HTTP/1.1" 200 2100 "-" "bingbot/2.0" 66.249.64.1 - - [01/Jan/2025:10:06:00] "GET /robots.txt HTTP/1.1" 200 120 "-" "Googlebot/2.1"
Analyze
Bot Breakdown
Recommendations