All checks / AI Access

Error claudebot-blocked

ClaudeBot can reach your site

What this check means

Checks whether your robots.txt denies ClaudeBot, the crawler operated by Anthropic, access to your site.

Why it matters

Claude cannot read your pages. It then has nothing from your site to use when it answers questions about your products.

How we detect it

We parse robots.txt per RFC 9309 and ask whether ClaudeBot is permitted to fetch "/". The most specific matching user-agent group wins, so a named ClaudeBot group replaces any wildcard rules rather than combining with them. A bare "Disallow:" permits everything and is not a block.

How to fix it

Remove the blanket "Disallow: /" from the group covering ClaudeBot, or add an explicit group permitting it: User-agent: ClaudeBot Allow: /

Example

User-agent: ClaudeBot
Allow: /

Check whether this applies to your site.

Run an AI audit

Other AI Access checks