Skip to content
All free tools

Free AI Visibility Checker

Paste a URL and we'll read it the way an AI answer engine does: crawler access for GPTBot, ClaudeBot and PerplexityBot, llms.txt, structured data, and whether your content is extractable enough to be cited.

What this checker measures

AI engines can only cite what they can reach, read and attribute. This check covers the layers that decide that: whether the AI crawlers are allowed in (robots.txt and any CDN or firewall rules in front of it), whether your key content is present in the HTML the server sends rather than injected later by JavaScript, whether structured data gives engines typed facts about who you are, and whether the page is written so an answer can be lifted from it cleanly.

You get an AI discoverability score and an agent readiness score, then the specific findings behind them, each with the fix written out. It is a readiness audit: it tells you whether anything stops engines from using your site. It cannot tell you whether a given engine currently mentions you. That is a separate measurement, covered at the bottom of this page.

The AI crawlers that matter, and what blocking each one does

Most AI companies run more than one crawler, and they do different jobs. Blocking the wrong one is the most common self-inflicted AI visibility problem we see, usually from a copy-pasted robots.txt.

  • OpenAI — OAI-SearchBot and ChatGPT-User fetch pages for ChatGPT search results and for links users ask about. GPTBot is the training crawler. Blocking GPTBot alone does not remove you from ChatGPT answers; blocking OAI-SearchBot does.
  • Perplexity — PerplexityBot builds Perplexity's index and Perplexity-User fetches pages when someone asks about them.
  • Anthropic — Claude-SearchBot is the search crawler and ClaudeBot is the training crawler.
  • Google — AI Overviews are served from the normal Google index, so they depend on Googlebot. Google-Extended is a separate robots.txt token that controls use of your content for Gemini training and grounding; blocking it does not affect Search or AI Overviews, but blocking Googlebot does.
  • Training-only opt-outs — Applebot-Extended, CCBot (Common Crawl) and Meta-ExternalAgent are mostly about model training rather than live answers.

The practical rule: allow the search and user-fetch agents if you want to be cited, and decide separately whether you are comfortable with the training crawlers. The checker reports each family as open, partially blocked or blocked, and separates search from training so you can see which you have chosen.

A robots.txt that allows AI search but opts out of training

This is one reasonable configuration, not the only one. Delete the training blocks at the bottom if you are happy for your content to be used for model training.

robots.txt
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

User-agent: Claude-SearchBot
Allow: /

# Optional: opt out of model training while staying in AI search
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

A robots.txt that allows these bots is necessary but not sufficient. Many CDNs and security plugins block unfamiliar bots by default, or serve them a JavaScript challenge they cannot pass, so the crawler sees a blank page even though robots.txt says yes. The live check requests your page the way these crawlers do, which is how it catches that case.

Is your content in the HTML, or only in the browser?

Googlebot renders JavaScript. Most AI crawlers do not. If your headline, product details or answers only appear after a script runs, those crawlers receive an empty shell. You can test this yourself in a few seconds: fetch the page without a browser and search the raw HTML for a sentence that should be on it.

Terminal
curl -s https://yourdomain.com/your-page | grep -i "a sentence from your page"

If grep prints nothing, the text is being added client-side. The fix is server-side rendering or static generation for the pages you want cited. Client-rendered single-page apps are the usual culprit.

A sensible order to fix things in

Work from the cheapest, highest-impact change down. Re-run the check after each step so you can see the score move.

  1. Unblock the search and user-fetch crawlers in robots.txt, and check your CDN or firewall is not challenging them.
  2. Make sure the pages you want cited are server-rendered: the key text must be in the initial HTML.
  3. Add Organization structured data with a consistent name, logo and sameAs profile links so engines can resolve who you are.
  4. Write answer-first content: a direct answer near the top of the page, question-style headings, and specific facts rather than slogans.
  5. Optionally add llms.txt. It is cheap, but the evidence that AI engines use it is thin, so treat it as the last step rather than the first.

Readiness is not the same as being cited

Passing every check here means nothing is stopping AI engines from using your site. It does not mean they will mention you: that also depends on whether your content is the best answer and whether the engine trusts your domain, and it can take weeks to show after a fix. To measure actual mentions you need to run real prompts against the engines on a schedule and record which sources they cite. Website Verdict's paid plans include an AI visibility tracker that does this; see the pricing page.

Frequently asked questions

How do I check if ChatGPT can see my website?

Run this free check. It confirms that OpenAI's search and user-fetch crawlers (OAI-SearchBot and ChatGPT-User) are allowed in robots.txt and not blocked by your CDN, and that your key content is in the initial HTML. Those are the conditions for your pages to be fetched for ChatGPT answers.

Does blocking GPTBot remove me from ChatGPT?

No. GPTBot is OpenAI's training crawler. ChatGPT's search results use OAI-SearchBot, and pages users ask about are fetched with ChatGPT-User. Blocking GPTBot opts you out of training; blocking the other two removes you from answers.

Which AI crawlers should I allow?

If you want to be cited, allow the search and user-fetch agents: OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User and Claude-SearchBot, plus Googlebot for AI Overviews. Whether to allow the training crawlers (GPTBot, ClaudeBot, Google-Extended and similar) is a separate decision about how you are comfortable your content being used.

What is a good AI visibility score?

Above 80 means engines can crawl, parse and attribute your site with confidence. Between 50 and 80 you are partly readable and likely losing citations to clearer competitors. Below 50, fix crawler access and server-side rendering first, since those are usually the blockers.

Do I need an llms.txt file to be visible to AI?

No. llms.txt is a proposal rather than a standard, Google has said it does not use it, and independent server-log studies suggest most AI crawlers rarely request it. Fix crawler access, server-rendered content and structured data first. See our llms.txt generator page for the details.

Is this AI visibility check really free?

Yes, no account required. The full Website Verdict scan (SEO, security, performance and a prioritized Fix Pack alongside AI visibility) is also free from the homepage, with 3 scans per day on the free plan.

Go deeper

More free tools

Primary technical references & standards