All checks / AI Access

Error oai-searchbot-blocked

OAI-SearchBot can reach your site

What this check means

Checks whether your robots.txt denies OAI-SearchBot, the crawler operated by OpenAI (ChatGPT search), access to your site.

Why it matters

ChatGPT search cannot retrieve or cite your pages, which is the crawler that produces live links in answers.

How we detect it

We parse robots.txt per RFC 9309 and ask whether OAI-SearchBot is permitted to fetch "/". The most specific matching user-agent group wins, so a named OAI-SearchBot group replaces any wildcard rules rather than combining with them. A bare "Disallow:" permits everything and is not a block.

How to fix it

Remove the blanket "Disallow: /" from the group covering OAI-SearchBot, or add an explicit group permitting it: User-agent: OAI-SearchBot Allow: /

Example

User-agent: OAI-SearchBot
Allow: /

Check whether this applies to your site.

Run an AI audit

Other AI Access checks