OAI-SearchBot

What is OAI-SearchBot?

Powers ChatGPT Search citations. Blocking = invisible in ChatGPT search results. Search crawlers build the index behind AI answer engines. When a user asks a question, the engine searches this index and cites matching pages with a link. No crawl means no citation, no link, no referral traffic — allowing search crawlers is the single highest-impact step for AI-era visibility.

Allowing it puts your pages in the citation pool for AI answers, which is where referral traffic increasingly comes from.

Does OAI-SearchBot respect robots.txt?

Yes. OAI-SearchBot honors robots.txt directives, including wildcard rules and Allow lines. A Disallow rule for this crawler is respected — but remember robots.txt is a voluntary protocol across the whole industry, not a wall.

How to block OAI-SearchBot

Add this group to your robots.txt to deny OAI-SearchBot across your whole site:

User-agent: OAI-SearchBot
Disallow: /

Blocking it removes you from that engine's answers entirely. There is no middle ground: uncrawled pages cannot be cited.

How to allow OAI-SearchBot

OAI-SearchBot is allowed by default on every site. It only gets blocked if a matching rule exists — most often a global User-agent: * group with Disallow: /. To exempt it while blocking other bots, put an empty Disallow group for it before the global block:

User-agent: OAI-SearchBot
Disallow:

User-agent: *
Disallow: /