SEO Auditor Keyword Generator HTML Editor Word Counter YouTube Tools View All Tools →

Robots.txt Tester

Test if your robots.txt allows or blocks specific URLs for crawlers

Robots.txt Content

User-agent: *
Disallow: /admin/
Disallow: /private/
Disallow: /tmp/
Allow: /public/

User-agent: Googlebot
Disallow: /no-google/

Sitemap: https://example.com/sitemap.xml

URL to Test

User Agent

Test History

No tests yet. Test a URL to see results here.