The result tells you which AI robots may read your home page, what ChatGPT’s search robot really receives, whether text is left without JavaScript, and what helps an AI understand you.
Would you rather I looked myself, for free, within 24 hours?
Website looking dated? Showcase website redesign from €1,500. A clear, fast website, well ranked on Google, built on up-to-date tools and easy to edit yourself. Free fixed-price quote, reply within 24 hours.
More and more people ask ChatGPT, Perplexity, Copilot or Google’s AI mode before choosing a provider. An AI can only cite your website if its robots are allowed to read it, and if they find text there. This test does what a robot does: it reads your robots.txt file, opens your home page once normally and once with the signature of ChatGPT’s search robot, and looks at what is left without JavaScript. For each point: what it saw, what it means, what to do.
Tool built by Léo Guillaume, freelance developer near Rennes, France. I build and redesign websites for small businesses, remotely, and I reply within 24 hours.
Updated on
In short
An AI like ChatGPT can only cite your website if its robots are allowed to read it and find text there. Three things can stop them: a line of the robots.txt file, a firewall that refuses robots, or a website whose text only appears with JavaScript. Being readable is a condition, not a guarantee of being cited.
A robot’s path
How an AI reads your website, in four steps
Before citing a page, an AI’s robot has to be able to open it and read it. Each step can block.
01
Ask
It reads robots.txt
First, the robot opens your website’s robots.txt file: a list of instructions that says which robots may read which pages. If it is forbidden there, it stops.
02
Enter
It asks for the page
It asks for your page with its signature, a name that says “I am the robot of this AI”. The server, the host or the firewall may answer, or refuse.
03
Read
It reads the HTML received
It reads the text as it arrives. Do not count on it to run JavaScript: a text that only appears with JavaScript may stay invisible to it.
04
Understand
It looks for who you are
The title, the description and the structured data tell it in black and white your trade, your town and your services.
The robots to know
The robots of the AIs, and what they are for
Each publisher has several robots. Blocking a training robot does not remove your website from the answers; blocking a search robot does. List taken from each publisher’s documentation, re-read on 16 November 2026.
Robot
Whose it is, what it is for
Official source
OAI-SearchBot
OpenAI · Search, answers with links. ChatGPT’s search robot. According to OpenAI, a website that blocks it does not appear in ChatGPT’s search answers, except as a plain navigational link.
OpenAI · Visit asked by a user. It opens a page when a ChatGPT user asks for it. According to OpenAI, robots.txt may not apply to these visits, and this robot does not decide your presence in search.
OpenAI · Model training. It collects pages that may be used to train OpenAI’s models. Blocking it means: no training. According to OpenAI, this setting is independent of OAI-SearchBot’s.
Anthropic · Search, answers with links. It browses the web to improve Claude’s search results. According to Anthropic, blocking it prevents your content from being indexed for search.
Anthropic · Visit asked by a user. It retrieves a page when a Claude user asks a question. According to Anthropic, blocking it prevents your content from being read in answer to that question.
Perplexity · Search, answers with links. It makes websites appear, with a link, in Perplexity’s results. According to Perplexity, it is not used to train models.
Perplexity · Visit asked by a user. It opens a page when a Perplexity user asks a question. According to Perplexity, it generally ignores robots.txt, since the visit is asked by a person.
Google · Model training. Not a separate robot: a name to use in robots.txt to say whether your pages may train the Gemini models. According to Google, it affects neither presence in Search nor ranking.
OpenAI · Search, answers with links. ChatGPT’s search robot. According to OpenAI, a website that blocks it does not appear in ChatGPT’s search answers, except as a plain navigational link.
OpenAI · Visit asked by a user. It opens a page when a ChatGPT user asks for it. According to OpenAI, robots.txt may not apply to these visits, and this robot does not decide your presence in search.
OpenAI · Model training. It collects pages that may be used to train OpenAI’s models. Blocking it means: no training. According to OpenAI, this setting is independent of OAI-SearchBot’s.
Anthropic · Search, answers with links. It browses the web to improve Claude’s search results. According to Anthropic, blocking it prevents your content from being indexed for search.
Anthropic · Visit asked by a user. It retrieves a page when a Claude user asks a question. According to Anthropic, blocking it prevents your content from being read in answer to that question.
Perplexity · Search, answers with links. It makes websites appear, with a link, in Perplexity’s results. According to Perplexity, it is not used to train models.
Perplexity · Visit asked by a user. It opens a page when a Perplexity user asks a question. According to Perplexity, it generally ignores robots.txt, since the visit is asked by a person.
Google · Model training. Not a separate robot: a name to use in robots.txt to say whether your pages may train the Gemini models. According to Google, it affects neither presence in Search nor ranking.
Good to know: According to Microsoft, Copilot fetches web information from the Bing search engine. For Copilot, being read well by Bing therefore matters too.
What the publishers say
Seven facts, with sources
Read on 16 November 2026 on the official pages of OpenAI, Anthropic, Perplexity, Google, Microsoft, Cloudflare and llmstxt.org. Each one links to its source.
OpenAI
Search and training, two settings
Each setting is independent: a website can allow OAI-SearchBot to appear in ChatGPT search and forbid GPTBot so that its pages are not used for training. OpenAI says a robots.txt change can take about 24 hours.
ClaudeBot collects for training, Claude-SearchBot for search, Claude-User when a user asks a question. Anthropic says it honours robots.txt and advises against blocking its robots by IP address.
PerplexityBot makes websites appear in the results and is not used to train models. Perplexity-User, which opens a page at a user’s request, generally ignores robots.txt.
Google-Extended says whether your pages may train the Gemini models. Google states that it affects neither a website’s inclusion in Google Search nor its ranking.
Since 1 July 2025, Cloudflare blocks by default the AI robots that come without permission, and asks every new domain whether it wants to let them in. A website behind Cloudflare can therefore close the door to AIs without its owner knowing.
Proposed by Jeremy Howard, the /llms.txt file sums up a website in Markdown, with links, for AIs. The official website does not say which AIs read it and promises no effect: the file stays optional.
These are the four points the test checks. One is enough to close the door.
1. A line in robots.txt
A “Disallow: /” line under a robot’s name, or under “User-agent: *” for all, and the robot may no longer read. It was sometimes added by a theme, a plugin or a former provider.
2. A firewall that refuses robots
Cloudflare, the host or a security plugin may answer with a refusal or a verification page. The robot leaves without having read anything.
3. A text that only exists with JavaScript
Some websites send an almost empty page that the browser fills afterwards. A robot that does not run JavaScript only sees the empty page.
4. A ban in the page
A “noindex” or “nosnippet” tag, or the same order in the X-Robots-Tag header, asks robots to ignore the page or not to quote it.
What to do
What to do, depending on what the test found
One case per line of the result.
1
A robot blocked in robots.txt
Open your website followed by /robots.txt. Remove the “Disallow” line aimed at the search robot, or add a group with its name and “Allow: /”. On WordPress, the file may be written by the SEO plugin: edit it from its settings.
The trap Blocking “User-agent: *” to keep out an annoying robot: the rule also closes the door to search robots.
2
A firewall or Cloudflare that refuses
In Cloudflare, look at the settings for AI robots and for robots in general, and allow at least the search robots. With a WordPress security plugin, look for the blocking of robots or “user agents”.
The trap Unblocking everything at once: allow the search robots, and decide separately for the training ones.
3
A website empty without JavaScript
The text has to arrive in the HTML. With Next.js, Nuxt or a tool of the same kind, server-side rendering or pages generated in advance must be switched on. With a website builder, check with the test that the text shows.
The trap Believing that “Google reads it, so AIs do too”: a robot that does not run JavaScript does not see what the browser adds.
4
A refusal that comes from the host
Some hosts filter robots before they even reach your website. Send them the test result and ask whether AI search robots are filtered, and how to allow them.
The trap Concluding too fast: a refusal of the test is a signal, not a proof. The real robot comes from its own addresses and may get through.
5
A ban in the page
On WordPress, untick “Discourage search engines from indexing this site” in Settings, Reading, then check the SEO plugin settings for the home page.
The trap Leaving the settings of a website under construction online: the “noindex” set during the work sometimes stays afterwards.
Experience
What I see on my own website
On dibodev.fr: in the week of 8 to 14 November 2026, 27 visits came from an AI, 22 of them from ChatGPT. At the end of October 2026, a first quote request came through a link cited in a ChatGPT answer. ChatGPT sends no referring website: its visits can be recognised by the utm_source=chatgpt.com parameter in the landing link.
Method and limits
What the test does, and what it does not see
The test reads your website from outside, like a robot, without installing or changing anything.
Four readings, nothing more
robots.txt, the home page with the test’s signature, the same page with the signature of OAI-SearchBot as OpenAI publishes it, and /llms.txt.
The rules of robots.txt
For each robot: its own group if there is one, otherwise the “*” group. The longest rule that matches the home page wins, “Allow” on a tie.
The text without JavaScript
The test counts the visible words of the HTML received, without scripts or styles. Fewer than 50 words with an application container (#app, #root, __next…): the page is empty for an AI.
A signature is not an address
The test borrows a robot’s signature, not its addresses. A refusal is a signal to check in the firewall, not a proof that the real robot is blocked.
What it does not see
The other pages of the website, your firewall settings, and what ChatGPT really answers. Being readable does not guarantee being cited.
What is kept
The link of the website tested, the result and a few indicators, for twelve months, for statistics. Never your IP address, never sent to PostHog.
Frequent questions
ChatGPT and your website: your questions
How do I appear in ChatGPT?
Nobody can promise it. There are conditions: that ChatGPT’s search robot, OAI-SearchBot, may read your website, that your text is there without JavaScript, and that your pages say clearly who you are, what you do and where. After that, being cited depends on the question asked and on the other websites. According to OpenAI, a website that blocks OAI-SearchBot does not appear in ChatGPT’s search answers.
How do I know if ChatGPT can read my website?
Paste your website link in the test at the top of the page. It reads your robots.txt file for OAI-SearchBot, ChatGPT-User and GPTBot, opens your home page with the signature of OAI-SearchBot and compares it with a normal visit, then measures the text readable without JavaScript.
Should I block GPTBot?
It is up to you. GPTBot collects pages that may be used to train OpenAI’s models. Blocking it means: no training on your pages. According to OpenAI, this setting is independent of OAI-SearchBot’s: you can forbid GPTBot and allow OAI-SearchBot to stay in ChatGPT search.
If I block GPTBot, do I disappear from ChatGPT?
No, not from search. According to OpenAI, OAI-SearchBot decides your presence in ChatGPT’s search answers. Blocking GPTBot only says that your pages must not be used to train the models.
What is the robots.txt file?
A small text file at the root of your website (yourwebsite.com/robots.txt) that tells robots which pages they may read. A “User-agent” line names the robot, a “Disallow” line forbids it a path, an “Allow” line allows one. Without a file, nothing is forbidden.
What is the llms.txt file?
A proposal published on 3 September 2024 by Jeremy Howard: a /llms.txt file in Markdown that sums up the website and links to its useful pages, for AIs. Its official website does not say which AIs read it and promises no effect. It is optional, and replaces neither robots.txt nor clear text on your pages.
How do I know if ChatGPT sends me visitors?
Look at your visit statistics. ChatGPT sends no referring website: its visits can be recognised by the utm_source=chatgpt.com parameter in the landing link. On dibodev.fr, in the week of 8 to 14 November 2026, 27 visits came from an AI, 22 of them from ChatGPT.
Does Cloudflare block AI robots?
It can. Since 1 July 2025, Cloudflare blocks by default the AI robots that come without permission, and asks every new domain whether it wants to let them in. If your website goes through Cloudflare, check the AI robot settings in your dashboard.
Does Copilot use Bing?
Yes, according to Microsoft: when web search is on, Copilot sends a query to the Bing search service and relies on its results. For Copilot, being read well by Bing therefore matters too.
My website is built with React, Vue or Angular: can an AI read it?
It depends on how it is served. If the server sends an almost empty page that the browser fills, a robot that does not run JavaScript sees nothing. If the pages are rendered on the server or generated in advance, the text is in the HTML. The test tells you in a few seconds.
Does the test record my website link?
Yes, for statistics only: the link of the website tested, the result and a few indicators (robots blocked, readable text, bans) are kept for twelve months. They are never used for prospecting and never published website by website. Your IP address is not kept, and the link is not sent to PostHog, this website’s analytics tool.
Blog
To go further
Being found by your customers, and what a website built with a JavaScript tool changes.
Is your website up to date, secure and compliant? In ten seconds, the tool reads your home page and checks eight points: HTTPS, PHP, WordPress, e-mails, phone display, title and description, legal notice, signs of an ageing site.
Has your Google traffic moved since 10 November? Google has been rolling out a broad update of its ranking since 10 November 2026. Drop your Search Console export, or answer 5 questions: the tool tells you whether your site is affected, and what to do.
Send me your website link. I reply within 24 hours with what I see: the robots to unblock, the firewall setting, the text to make readable without JavaScript, and a firm price if you want me to handle it.