CrawlCheck.dev vs crawlcheck.io
Two unrelated products, one confusingly similar name. This page exists so you can tell in ten seconds which CrawlCheck you are looking at.
They are not the same product
crawlcheck.dev is this site: a free Generative Engine Optimization toolkit that audits your robots.txt and llms.txt against 61 AI crawlers. crawlcheck.io is a separate project run by different people. We do not operate it, we have no access to its data, and we cannot speak for its features, pricing or privacy practices — check its own site for that.
Because "CrawlCheck" reads as plain English ("crawl check"), independent teams arrived at the same name on different domains. Search engines occasionally mix the two up, which is the only reason this page exists.
| crawlcheck.dev this site | crawlcheck.io | |
|---|---|---|
| Focus | AI crawler visibility: which of 61 bots can read and cite your site | Unrelated product — see their site |
| Core tool | robots.txt auditor, policy generator, JSON-LD builder, live scan | Not operated by us |
| Account needed | None — scans run in your browser | Not operated by us |
| Price | Free | Not operated by us |
| Operated by | Independent project, contact hi@crawlcheck.dev | A different team |
How to tell which one you are on
- Look at the address bar. Everything in this project lives on crawlcheck.dev or api.crawlcheck.dev. If the domain is crawlcheck.io, you are on the other product.
- Look for an account prompt. CrawlCheck here has no signup, no login and no email gate. Any page asking you to create an account is not ours.
- Look for the crawler table. Our reports always list individual user agents — GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the rest — with an allow/block verdict per crawler. That per-agent breakdown is the product.
- Look at the navigation. This site links to an AI crawler directory, a set of GEO guides and a live scanner.
What crawlcheck.dev actually does
Auditor — paste your robots.txt and get a verdict for all 61 AI crawlers, grouped by search indexer, assistant fetcher, agent and training collector.
Generator — choose a strategy per crawler category and get a valid policy you can paste straight into your site.
Schema builder — produce FAQ and HowTo JSON-LD that answer engines parse reliably.
Live scanner — fetch your site as 8 AI crawlers and score how visible it really is.
Directory — every bot profiled with operator, purpose, robots.txt behavior and blocking instructions.
Which page should you be on?
If you wanted to know whether AI engines can read your site, run the free robots.txt audit. If you wanted to test a live URL end to end, use the scanner. If you were looking for a specific bot, find it in the directory. If you were looking for crawlcheck.io, go to crawlcheck.io — this page cannot help you with that product.
See what AI engines can see on your site
Audit your robots.txt against 61 AI crawlers in seconds — free, no signup, runs entirely in your browser.