RankEngineQuick answer: paste a domain and this checks what the AI answer engines are allowed to do with it — crawler rules for GPTBot, ClaudeBot, PerplexityBot and Google-Extended, whether an llms.txt exists, and whether the page is structured to be quoted.
Free tool
AI visibility checker — can AI engines read your site?
Access is necessary and nowhere near sufficient
There are two separate questions and they get conflated constantly. The first is whether an answer engine is allowed to read your site. The second is whether it chooses to cite you. This tool answers the first completely and only gestures at the second, because nobody can answer it for you.
Allowing GPTBot does not buy a citation. It buys eligibility. What earns the citation is being the clearest available answer to the question somebody asked, in a form that can be lifted without rewriting.
The four controls, and the one people confuse
GPTBot and OAI-SearchBot are OpenAI. ClaudeBot is Anthropic. PerplexityBot is Perplexity. CCBot feeds Common Crawl, which many models train from indirectly.
Google-Extended is the one that trips people. It is not Googlebot. Googlebot crawls for Search; Google-Extended governs whether your content trains and grounds Gemini. Blocking Google-Extended does not remove you from Search, and people block it believing otherwise.
Most sites name none of them, which means allowed. That is a defensible position — it just should not be an accidental one.
Why the check looks at headings and structure
An answer engine lifts a passage, not a page. Three things make a passage liftable: a heading shaped like the question, an answer in the first sentence or two beneath it, and structured data that says what kind of thing the page is.
A page can be perfectly readable and still never be quoted, because the useful sentence is buried in the ninth paragraph. That is a content problem rather than a crawler problem, and it is the one most sites actually have.
Where to go next
Answer engine optimization covers the content side in full, and llms.txt explains what that file is and is not. If the page failed the readable-without-JavaScript row, that is the first thing to fix — an engine that cannot read the raw HTML has nothing to lift.
Frequently asked questions
Does allowing AI crawlers mean I get cited?
No. Access is necessary and nowhere near sufficient. Allowing GPTBot means your pages can be read; whether an engine cites you on a given question depends on whether your page answers it better and more extractably than the alternatives. Nobody can promise a citation, and any tool that does is selling something.
What happens if I take no position?
Every crawler that respects robots.txt is allowed by default. Most sites are in this state, which means they are readable by accident rather than by decision. That is fine if being read is what you want — it is worth making it deliberate either way.
Is llms.txt a standard?
It is a proposal, not a standard, and no engine has committed to reading it. It costs nothing, it is easy to keep honest, and it gives an engine a curated list of the pages you would rather be cited from. Treat it as cheap insurance rather than a lever, and do not expect it to move anything on its own.
Is Google-Extended the same as Googlebot?
No, and this is the one people get wrong. Googlebot crawls for Search. Google-Extended is a separate control for whether your content trains and grounds Gemini. Blocking Google-Extended does not remove you from Search, and blocking Googlebot does not stop the other.
Why does the check look at headings and structured data?
Because an answer engine lifts passages, not pages. Clear question-shaped headings, short direct answers and valid structured data make a page easy to quote. A wall of undifferentiated prose can be perfectly readable and still never be the passage that gets used.
Should a Shopify store block AI crawlers?
Usually not. A store that sells things people ask assistants about — which is most stores — wants to be the source that gets named. The calculation is different for publishers whose text is itself the product.
All free tools
SEO Title Generator & Checker
Generate SEO titles from your keyword and check any title's character count, pixel width, and keyword position.
Meta Description Generator & SERP Preview
Generate high-converting meta descriptions and preview how your page looks in Google.
Keyword Density Checker
Paste any text and see keyword density, top 1–2–3-word phrases, and stuffing warnings.
Robots.txt Generator (AI-Crawler Aware)
Generate a correct robots.txt in seconds.
LSI & Related Keyword Generator
Generate related keywords, question keywords, and commercial long-tails from any seed term.
Image Alt Text Generator & Checker
Build descriptive, SEO-friendly alt text for product images and check existing alt text for length, redundancy, and stuffing.
Googlebot Simulator — See What Google Sees
Fetch any URL with Googlebot's user agent: status code, redirects, title, meta robots, canonical, X-Robots-Tag, H1s, and an indexability verdict.
Schema markup generator
Generate valid JSON-LD schema free: Product, FAQPage, BreadcrumbList, LocalBusiness and Article.
Hreflang tag generator
Build a valid hreflang cluster for every language version of a page, with x-default, and a built-in check that every URL points back at every other one..
UTM builder
Build consistent UTM-tagged campaign URLs free.
URL slug generator
Turn any title into a clean, lowercase, hyphenated URL slug.
Readability checker
Paste text and get Flesch Reading Ease, Flesch-Kincaid grade level, sentence length distribution and the specific sentences that are dragging the score down..
SERP preview tool
Preview how a title, URL and meta description will render in Google on desktop and mobile, measured in pixels rather than characters so truncation is accurate..
SEO ROI calculator
Estimate what a ranking improvement is worth: search volume, position-based CTR, conversion rate and order value, plus a payback period on your SEO spend..
Open Graph generator
Generate Open Graph and Twitter card meta tags with a live preview of how the link will look when shared on Facebook, LinkedIn, X, WhatsApp and Slack..
SEO report generator
Generate an on-page SEO report for any URL: title and meta lengths, headings, canonical, schema types, image alt coverage, links and indexability, scored..
Redirect checker
Follow any URL hop by hop: every status code, every intermediate URL, and the page that finally answers.
Robots.txt tester
Fetch any site robots.txt and test a path against Googlebot, Bingbot, GPTBot, ClaudeBot and PerplexityBot.
Sitemap checker
Fetch any XML sitemap or index and check it: URL count against the 50,000 limit, lastmod coverage and honesty, size and reachability..
HTTP header checker
Fetch any URL as Googlebot and read the full response headers, with the SEO-relevant ones called out: X-Robots-Tag, Cache-Control, Content-Type, Vary and HSTS..
hreflang checker
Read the hreflang cluster on any page: every declared alternate, whether the page includes itself, whether x-default is present, and whether the codes are valid..
Canonical checker
Check the canonical on any URL: present or missing, self-referencing or pointing elsewhere, and whether a redirect or noindex contradicts it..
Heading structure checker
Check the heading structure of any page: how many H1s, whether levels are skipped, and how many sections a reader or an answer engine can actually navigate..
Indexability checker
One verdict from every signal that decides it: status code, robots meta, X-Robots-Tag, canonical target, redirect chain and whether the HTML has content at all..
Related guides
ai.txt vs llms.txt: Which File Do AI Crawlers Read?Shopify Answer Engine Optimization Best PracticesHow to Get Indexed by LLMs Through an llms.txt Filellms.txt generator for Shopify (free): create one
Run this across your whole catalog
RankEngine applies and verifies these fixes on every product in your Shopify store — automatically.
Install RankEngine free