Robots.txt Generator
Build a valid robots.txt for your site. Free.
🔒 Private by design: this tool runs 100% in your browser — nothing is uploaded anywhere.
Free online robots.txt generator
This free robots.txt generator builds the small text file that tells search engine bots which parts of your website they may crawl. Block admin pages, private folders or duplicate content, allow the pages you want indexed, and point bots to your sitemap.
Upload the finished file to your website root as robots.txt. Well-behaved bots like Googlebot follow it, though it is a request — not a security lock.
Frequently asked questions
What does Disallow do? It asks bots not to crawl the listed paths, for example /admin/ keeps admin pages out of search results.
Should I block my whole site? Only if you truly want it hidden from search engines. For a public site, leave Disallow empty.
What is crawl-delay? A hint asking bots to wait some seconds between requests, to reduce server load. Not all bots honor it.