Common Crawl
1 supported bot or service.
Supported bots & services
CCBot
Common Crawl web dataset crawler.
Verify Common Crawl IP addresses
Confirm whether a connecting IP from your server logs belongs to Common Crawl's published ranges.
1. Find the connecting IP
Look up the connecting client IP ($remote_addr) in your server or proxy logs rather than relying on the User-Agent header.
2. Test against published ranges
Check the address in the tool against published network ranges for Common Crawl.
3. Read the result
A match confirms the address is in Common Crawl's published ranges. No match means the IP is unlisted or someone is spoofing the name.
Notes on published ranges
- Publisher additionally recommends forward-confirmed reverse DNS for IPv4.
Coverage reflects published network ranges. Some services publish no address lists, and shared ranges can return multiple bots for a single IP.
Sources reviewed .