robots.txt Tester
Fetches a domain's robots.txt, shows the content and counts the Disallow rules.
💡 What's this for?
This tool loads a website's robots.txt file and shows you which areas are blocked for Google and other search engines. That way you can quickly check whether important pages are being blocked by mistake.
🐤 Want Canary to keep an eye on this for you?
This tool checks once. Canary checks your whole website every day and alerts you instantly when something breaks technically and costs you Google traffic. Start a free health check →