robots.txt is published

What this check means

Checks whether a robots.txt is served at your domain root.

Why it matters

Without one, crawlers are permitted by default — so this is not a blocker. But robots.txt is also where you declare your sitemap and where you can explicitly welcome AI crawlers, and its absence means you have no way to express either.

How we detect it

We request https://yourdomain/robots.txt and require a 2xx response containing at least one directive.

How to fix it

Publish a robots.txt at your domain root declaring your sitemap and any crawler rules.

Example

User-agent: *
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml