Short answer: AI answer optimization asks three questions of the same page, and they have different answers. Can the page be found and fetched by the engine's own agents? That is the SEO and crawler-access question. Can a complete, correct answer be lifted out of it? That is the answer-engine (AEO) question. Does the brand behind the page get named when an assistant answers a buyer? That is the GEO question. Each one is a precondition for the next, so work through them in that order, and test each one directly instead of assuming it from the one before.
The distinction is practical, not academic. The three questions fail for different reasons and are fixed by different people: a developer, a writer, and whoever owns the brand's presence outside its own site. A page can rank well in search and still be impossible to quote, and a page can be quoted as a source while the answer recommends someone else. Treating "AI SEO" as one project hides which of the three you are actually failing.
Why AI answer optimization is three questions, not one
People often ask whether AI answer optimization is just SEO with a new name. Part of it is. The first question, whether the page can be found and fetched, is the same discipline SEO teams already practise, extended to a new set of crawlers. The other two have no real equivalent in a ranking report. A ranking tells you that a page was judged relevant as a whole. It says nothing about whether any single passage on it can be quoted, and nothing about whether the brand is the one the assistant puts forward.
The table below puts the three side by side. The most useful column is the last one: each question has its own failure signal, and they do not look alike.
| 1. Fetch | 2. Lift | 3. Name | |
|---|---|---|---|
| Discipline | SEO and crawler access | Answer-engine optimization (AEO) | Generative engine optimization (GEO) |
| Unit that is judged | The URL and its HTML | A passage cut out of the page | The brand, across many sources |
| Usual owner | Developer or platform admin | Writer or editor | Brand, PR or partnerships |
| Direct test | Request the page as the agent | Read two sentences out of context | Ask the question without your brand name |
| What failure looks like | A 403, a challenge, or a 200 with no text | No passage that stands on its own | Cited as a source, competitor recommended |
Read across a row and the ordering explains itself. A writer cannot fix a page the search agent never receives, and a brand team cannot be named on the strength of a page nobody can quote. Work done out of order is not wasted, but it cannot show up until the earlier question passes.
Question 1: can the page be found and fetched?
This is the part that carries over from SEO, with one important addition. Answer engines fetch pages with their own agents, and each major vendor runs more than one, on purpose. OpenAI documents GPTBot for content that may be used to train its models, OAI-SearchBot for the index behind ChatGPT's search features, and ChatGPT-User for pages a person asked ChatGPT to read. Perplexity separates PerplexityBot, which indexes pages so they can be surfaced and linked in its answers, from Perplexity-User. Anthropic splits ClaudeBot, Claude-SearchBot and Claude-User the same way. The primary sources are OpenAI's crawler documentation, Perplexity's bot guide and Anthropic's crawler help page.
That split is why "we rank fine on Google" says little about AI answers. A robots.txt rule or a CDN rule that blocks a search agent removes you from that engine's answers whatever your rankings look like, and a permissive robots.txt in front of a firewall that returns 403 to those agents is a block in practice. The split also means the training decision and the visibility decision are separate. Blocking GPTBot is a position on training use and does not remove you from ChatGPT search results; blocking OAI-SearchBot does. Sites that meant the first often wrote a rule that does the second.
The user-triggered agents behave differently again. They fetch a page because a person asked for it, and the vendors' own documentation notes that some of them may not follow robots.txt in that case. Do not rely on robots.txt to keep those requests out, and do not read their absence from your logs as a sign the search agent is reaching you.
The second half of "found" is "readable". Most AI crawlers do not execute JavaScript, so content that appears only after client-side rendering is an empty shell to them. A page can return 200, pass every check that runs in a browser, and still deliver no text to the agent that matters. Test with the real user agent and treat a 200 response with an empty body as a failure, not a pass. Run a plain browser request alongside, so a site that is simply down is not mistaken for one that blocks AI agents. Our guide to AI crawler access covers these checks in more depth.
Question 2: can an answer be lifted out of it?
This is where pages that pass the first question most often fall down, and the cause is structural rather than editorial. An answer engine does not read a page from top to bottom the way a person does. It works with passages and writes its reply from the ones it selects. So the operative test is: if you cut a short passage out of this page and pasted it into a reply, would it still be true, and would it still make sense on its own?
Pages fail that test for reasons that have nothing to do with how well they are written:
- The answer depends on the heading. "It is fully waterproof" means nothing once lifted; "The Ridge rain jacket is fully waterproof" survives the cut.
- The answer arrives late. If it comes after three paragraphs of context, the passage that gets selected may not contain it at all.
- The answer is split. Half sits in a table and half in a footnote, so two distant parts of the page have to be reassembled.
- The answer is hidden. It lives in an accordion, tab or modal that a script fills in, and therefore is not in the HTML a non-JavaScript crawler receives.
- The page hedges. "It depends on your use case" cannot be quoted as an answer, so it loses to a page that states the condition and the outcome.
Assistants tend to lift sentences that answer a constraint: a dimension, a material, a return window, a compatibility. Copy written for brand tone gives them nothing to lift, however well it reads. Buyers also ask in full sentences. On our own site, 181 of the 475 queries that produced impressions in a two-month Search Console window were full questions or phrases of seven words or more, carrying about 22% of impressions. Headings phrased the way those questions are phrased tell a retriever what the passage underneath answers.
The rewrite rules for question 2
| Do | Instead of |
|---|---|
| State the answer in the first sentence under the heading, then explain | Building up to it over several paragraphs |
| Repeat the subject in the sentence that carries the fact | Relying on "it", "this plan" or the heading above |
| One question per heading, phrased as a person would ask it | Headings such as "Our approach" or "Why us" |
| Facts in sentences or a table with real units and dates | Facts scattered through prose or trapped in images |
| Explicit dates on anything time-bound | "Recently", "currently", "this year" |
| Key answers in the server-rendered HTML | Answers inside script-filled tabs and accordions |
The quickest check is to read the first two sentences under each heading out of context. If they are a complete, correct answer that names its subject, the passage is liftable. Writing for extraction walks through before-and-after rewrites.
Question 3: does the brand behind it get named?
Passing the first two questions makes your page a candidate source. It does not make your brand the recommendation. Those are separate outcomes and should be tracked separately, because your page can be cited as a source while the answer recommends a competitor. Merging the two flatters a report and hides the real gap.
Being named depends less on any single page than on two things outside it. The first is entity clarity: whether a model is confident about who you are, what you do and which of the names in circulation refer to you. Spelling variants, a product line described as a separate company, or a legal name that differs from the trading name all blur that. The second is corroboration. A model needs the same fact from independent places before it will state it confidently, and a claim that exists only on your own domain is the weakest evidence you can offer, however well it is written or marked up. Third-party corroboration covers where to find those independent sources and how to prioritise them.
The common failure here is not silence but a plausible wrong detail: an outdated price, a market you left, a feature that belongs to a competitor. When you read an answer for question 3, look for those as carefully as for your name. They usually trace back to one stale external source rather than to any weakness on your own site, and fixing that source is a different job from rewriting your pages.
Question 3 also has to be measured honestly, or you will optimise the wrong thing. Exclude questions that contain your brand name: ask an assistant whether a named company is any good and it will of course name it. In one of our own runs, 13 of 15 generated questions contained the brand name, and the score read 13% when the honest value was zero. Count competitors with the same ruler as your own brand, too. An early version of our measurement showed one brand at 31% share of voice and rank two; measured symmetrically it was 0% and last.
Where AI answer optimization conflicts with habit
Answer-first writing can shorten time on page, which some teams still treat as a quality signal. Stating a specific condition or figure invites being wrong later, which is why so many pages hedge. Both resolve the same way. A passage that can be quoted is worth more than one that keeps a reader scrolling, and a fact with a date on it is safer than a vague claim, because the date tells the reader and the engine exactly what is being claimed and when.
A third habit is reporting. Teams used to one ranking number want one AI number. Report each engine on its own before any average: in our own benchmark one model named 3.5 providers per answer and another 2.1, so a blended figure describes neither. And keep the three questions as three lines in the report. A single "AI visibility" score that mixes access, extraction and naming cannot tell you which of the three people to call.
A five-minute AI answer optimization test
- Fetch. Request the page as a search agent and look for the sentence you care about:
No match means question 1 fails, whatever the browser shows.curl -s -A "OAI-SearchBot" https://example.com/returns/ | grep -i "return window" - Lift. Find the heading that matches a real buyer question and read the two sentences under it out of context. If they do not stand alone, question 2 fails.
- Name. Ask the same question, without your brand name, in the assistants you care about. Write down which brands are named and which sources are cited, as two separate lists. Repeat on a fixed schedule rather than trusting one sample, because answers vary between runs even when nothing has changed.
None of this guarantees a mention. What it does is tell you which of the three questions you are failing, so the work goes to the right person. If you want the first check run against several agents at once, the free checkers at InsightWonder Tools include one for crawler access.
Sources
- OpenAI, overview of OpenAI crawlers.
- Perplexity, Perplexity crawlers.
- Anthropic, how Anthropic crawls the web and how site owners can block it.
- Our own Search Console export for insightwonder.com, 2026-07-02 to 2026-09-07, and our measurement incident records, July and August 2026.