All checks / AI Access

Error claudebot-blocked

ClaudeBot can reach your site

What this check means

Checks whether your robots.txt denies ClaudeBot, the crawler operated by Anthropic, access to your site.

Why it matters

Claude cannot access your content when answering questions about your product or sector.

How we detect it

We parse robots.txt per RFC 9309 and ask whether ClaudeBot is permitted to fetch "/". The most specific matching user-agent group wins, so a named ClaudeBot group replaces any wildcard rules rather than combining with them. A bare "Disallow:" permits everything and is not a block.

How to fix it

Remove the blanket "Disallow: /" from the group covering ClaudeBot, or add an explicit group permitting it: User-agent: ClaudeBot Allow: /

Example

User-agent: ClaudeBot
Allow: /

Check whether this applies to your site.

Run an AI audit

Other AI Access checks