Windsurf

What is Windsurf?

Fetches docs during Windsurf coding-agent sessions. Agent crawlers browse the web on a user's behalf inside AI tools: opening pages, reading documentation, and completing tasks. Traffic arrives from engaged users who asked for exactly what you offer. Blocking agents does not protect content from training, but it does cut off this kind of high-intent visit.

Allowing it keeps your site reachable to AI agents acting for users — an emerging traffic channel worth keeping open.

Does Windsurf respect robots.txt?

Yes. Windsurf honors robots.txt directives, including wildcard rules and Allow lines. A Disallow rule for this crawler is respected — but remember robots.txt is a voluntary protocol across the whole industry, not a wall.

How to block Windsurf

Add this group to your robots.txt to deny Windsurf across your whole site:

User-agent: Windsurf
Disallow: /

Blocking it stops agent-driven visits but provides no real content protection, since agents only fetch pages the user asked for.

How to allow Windsurf

Windsurf is allowed by default on every site. It only gets blocked if a matching rule exists — most often a global User-agent: * group with Disallow: /. To exempt it while blocking other bots, put an empty Disallow group for it before the global block:

User-agent: Windsurf
Disallow:

User-agent: *
Disallow: /