robots.txt Generator
Build your robots.txt with User-agent, Disallow, Allow, Sitemap and Crawl-delay live in the browser.
💡 What's this for?
With this tool you put together your robots.txt file by choosing which areas of your site search engines may crawl and which they should leave alone. You add your rules and sitemap and get the finished file to upload, so crawlers like Google know exactly where they're allowed to go.
Trage deine Werte ein - die robots.txt wird sofort erzeugt. Disallow- und Allow-Pfade jeweils einer pro Zeile.
Ergebnis: robots.txt
🐤 Want Canary to keep an eye on this for you?
This tool checks once. Canary checks your whole website every day and alerts you instantly when something breaks technically and costs you Google traffic. Start a free health check →