assistant Mistral AI respects robots.txt

MistralAI-User

Mistral AI uses this crawler to fetch pages in real time when a user asks about them. Fetches pages for user-initiated Mistral/Le Chat queries.

What is MistralAI-User?

Fetches pages for user-initiated Mistral/Le Chat queries. Assistant crawlers fetch a page in real time when a user explicitly asks an AI assistant about it. They do not build a standing index of the web. Blocking them makes the assistant say it cannot access your page instead of summarizing it — a poor experience for users who were actively looking for you.

Allowing it means users who ask about you get an accurate, current answer with your page as the source.

Does MistralAI-User respect robots.txt?

Yes. MistralAI-User honors robots.txt directives, including wildcard rules and Allow lines. A Disallow rule for this crawler is respected — but remember robots.txt is a voluntary protocol across the whole industry, not a wall.

How to block MistralAI-User

Add this group to your robots.txt to deny MistralAI-User across your whole site:

User-agent: MistralAI-User
Disallow: /

Blocking it only affects user-requested fetches, so the traffic cost is small — but the users you do lose are the highest-intent ones.

How to allow MistralAI-User

MistralAI-User is allowed by default on every site. It only gets blocked if a matching rule exists — most often a global User-agent: * group with Disallow: /. To exempt it while blocking other bots, put an empty Disallow group for it before the global block:

User-agent: MistralAI-User
Disallow:

User-agent: *
Disallow: /

Is MistralAI-User currently allowed on your site?

Paste your robots.txt into the free auditor and see verdicts for all 61 AI crawlers — including the specific paths that matter.

Related AI crawlers