ChatGPT Agent
OpenAI uses this crawler to open pages on a user's behalf inside AI tools. Browses the web on a user's behalf during ChatGPT agent tasks.
Crawler data last reviewed September 18, 2026 · all 61 crawlers
ChatGPT Agent at a glance
| Operator | OpenAI |
|---|---|
| Type | AI agent browser |
| Respects robots.txt | Yes — a Disallow rule is honored |
| If you allow it | Allowing it keeps your site reachable to AI agents acting for users — an emerging traffic channel worth keeping open. |
| If you block it | Blocking it stops agent-driven visits but provides no real content protection, since agents only fetch pages the user asked for. |
What is ChatGPT Agent?
Browses the web on a user's behalf during ChatGPT agent tasks. Agent crawlers browse the web on a user's behalf inside AI tools: opening pages, reading documentation, and completing tasks. Traffic arrives from engaged users who asked for exactly what you offer. Blocking agents does not protect content from training, but it does cut off this kind of high-intent visit.
Allowing it keeps your site reachable to AI agents acting for users — an emerging traffic channel worth keeping open.
Primary source: OpenAI's official crawler documentation.
Does ChatGPT Agent respect robots.txt?
Yes. ChatGPT Agent honors robots.txt directives, including wildcard rules and Allow lines. A Disallow rule for this crawler is respected — but remember robots.txt is a voluntary protocol across the whole industry, not a wall.
How to block ChatGPT Agent
Add this group to your robots.txt to deny ChatGPT Agent across your whole site:
User-agent: ChatGPT Agent Disallow: /
Blocking it stops agent-driven visits but provides no real content protection, since agents only fetch pages the user asked for.
How to allow ChatGPT Agent
ChatGPT Agent is allowed by default on every site. It only gets blocked if a matching rule exists — most often a global User-agent: * group with Disallow: /. To exempt it while blocking other bots, put an empty Disallow group for it before the global block:
User-agent: ChatGPT Agent Disallow: User-agent: * Disallow: /
Does blocking ChatGPT Agent hurt your rankings?
No — not for classic search. ChatGPT Agent is not a Google user agent, so a robots.txt rule for it has no effect on Google rankings or indexing. The only thing you give up is what this specific AI agent browser does with your pages, described above.
How to verify ChatGPT Agent is really hitting your site
robots.txt states what a compliant crawler may fetch; your access log shows what actually happened. Search the log for the token:
grep -i "ChatGPT Agent" /var/log/nginx/access.log | tail -50
On Cloudflare the free plan does not expose raw request logs, so use Security Events to see blocked requests, or add a console.log line in a Worker to record the user agent. On Vercel or Netlify, filter the request log by user agent string.
User agents are trivially spoofable, which is why "I saw ChatGPT Agent in my logs" is not proof. Because ChatGPT Agent is documented to honor robots.txt, a request from it that ignores your rules is worth investigating — but a single stray hit is usually a misconfigured tool, not the real crawler.
Should you block ChatGPT Agent?
- Allow it — agent traffic is a user asking an AI tool to do something on the open web, which is closer to a real visitor than a crawler is.
- Block it if the agent could trigger writes, purchases or state changes on your site, and you cannot tell agent traffic apart downstream.
- Expect growth here — agent browsers are the newest category and the least covered by classic bot management.
Is ChatGPT Agent currently allowed on your site?
Paste your robots.txt into the free auditor and see verdicts for all 61 AI crawlers — including the specific paths that matter.
Other AI crawlers from OpenAI
OpenAI runs 4 distinct user agents. They are configured independently in robots.txt, so blocking one does not block the others — a detail that catches people out when they assume a single rule covers a whole vendor.
| Crawler | Type | robots.txt | What it does |
|---|---|---|---|
| GPTBot | AI training | yes | Feeds GPT model training. Blocking removes your content from future ChatGPT knowledge. |
| OAI-SearchBot | AI search | yes | Powers ChatGPT Search citations. Blocking = invisible in ChatGPT search results. |
| ChatGPT-User | AI assistant fetcher | yes | Fetches pages a ChatGPT user explicitly requested. Blocking breaks user-initiated browsing. |
| ChatGPT Agent this page | AI agent browser | yes | Browses the web on a user's behalf during ChatGPT agent tasks. |
Related AI crawlers
Further reading
- How AI Crawlers Work (And What They Want From Your Site) — AI crawlers train models, build search indexes, or fetch pages on request. Meet the major bots, learn how they fetch pages, and why access is step zero.
- robots.txt for AI Crawlers: Every Rule That Matters — A 30-second robots.txt refresher, what each AI bot costs you when blocked, three copy-paste policies and the syntax mistakes that silently break your rules.