Robots.txt Maker

Set a basic crawl policy, add private paths, and connect your sitemap without wrestling with the syntax.

Crawl rules

Allow public crawlingRecommended for public websitesBlock all crawlingUse only for non-public environments

Paths to block one per line

Sitemap URL optional

Good to know

Robots.txt guides well-behaved crawlers, but it does not protect private data. Use authentication for anything sensitive.

YOUR ROBOTS.TXT

User-agent: *
Allow: /

Upload this file to the root of your domain, for example example.com/robots.txt.