AI Crawler Checker: is GPTBot, ClaudeBot or PerplexityBot blocked on your site?
Enter your domain to see which AI crawlers your robots.txt allows, the rule that decided each one, whether you have a sitemap and llms.txt, and how much of your homepage a crawler that doesn't run JavaScript can read. Free, no account, results in a few seconds.
What does this checker test?
- robots.txt rules for 14 AI crawlers and tokens, plus Googlebot and Bingbot, read per the robots.txt standard (a group naming one crawler replaces the * group).
- Whether a sitemap and an llms.txt file are served.
- The homepage as a crawler without JavaScript receives it: how much text, and which structured data types.
- Whether your Organization schema links your official profiles (sameAs), which helps AI tell your business apart from others with a similar name.
Want the rules explained? Read robots.txt for AI crawlers.
Common questions
Enter your domain above. The checker reads your robots.txt the way crawlers do, including groups named for one crawler, and says for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and the others whether your homepage is allowed, and which rule decided it.
Not from ChatGPT search. OpenAI uses OAI-SearchBot for its search index and GPTBot for training, so blocking only GPTBot opts you out of training.
Your site's bot protection refused our request for robots.txt. That says nothing about whether the file exists, so nothing is reported as blocked or missing. AI crawlers may be refused the same way, which is worth checking in your CDN or firewall settings.
It is the first step of one. This checks access: whether AI crawlers may read your homepage. The free AI Visibility check (no sign-up) goes on across up to 12 pages, adding structured data, how clearly your business is described and whether your content answers questions, with a score. With a free account it also asks ChatGPT your buyers' questions with live web search and shows whether it names and cites you, and who it names instead.