Robots.txt generator

Generate robots.txt file for your website. You can copy the generated robots.txt file and paste it in the root directory of your website.

Rules (one directive per line)

Example: Allow: / or Disallow: /private

Sitemap URL

robots.txt

User-agent: *
Allow: /
Disallow: /admin

Sitemap: https://example.com/sitemap.xml

FAQ

What is robots.txt?: It is a plain text file at the site root that tells crawlers which paths they may or may not fetch.