Short answer: when your brand is not in AI answers, the cause is usually mechanical, not mysterious. Check six things in this order: whether the retrieval agents are allowed, whether the content exists without JavaScript, whether a CDN or firewall blocks the agents anyway, whether the answer on the page can be quoted, whether the brand is an unambiguous entity, and whether you are targeting the questions buyers actually ask. The first three are yes-or-no checks you can run today. The last three are slower and more expensive, so fix them only after the first three pass.
Before any of that, confirm that the absence is real. Measurement errors can manufacture an absence, or hide one, and we have made each of the errors below in our own product before fixing it.
Before you start: is your brand really not in AI answers?
Three measurement errors produce a false absence, or hide a real one:
- Unmeasured shown as zero. A model that timed out produced no evidence either way. Counted as "not mentioned", it drags the percentage down and can look like a decline that never happened. See unmeasured vs zero.
- One average across engines. In our own benchmark one model named 3.5 providers per answer and another 2.1. You can be absent from one engine and present in another, and a blended number hides which.
- Brand-named questions in the set. These make you look present when you are not. In one of our own runs, 13 of 15 generated questions contained the brand name, and the score read 13% across five engines when the honest value was zero.
If the absence survives a fixed set of questions that do not name you, reported per engine, it is real. Then work through the causes below.
Three yes-or-no reasons your brand is not in AI answers
These three are binary and cheap to test, and any one of them makes the rest of this list irrelevant: if the agent cannot fetch or read the page, nothing written on it matters. Run them on the pages that should answer buyer questions, such as pricing, product and comparison pages, not only on the homepage.
1. The retrieval agent is not allowed to fetch the page
Check this first because it invalidates everything after it. Each major assistant uses separate agents for training and for search. OpenAI documents GPTBot for training, OAI-SearchBot for the index behind ChatGPT's search features, and ChatGPT-User for fetches a person asked for. Perplexity separates PerplexityBot, which indexes pages for its answers, from Perplexity-User. Blocking GPTBot is a defensible decision about training. Blocking OAI-SearchBot removes you from ChatGPT search results, and it is the agent most often blocked by accident by sites that meant to opt out of training.
Read your robots.txt for a blanket rule aimed at a whole vendor, or at every agent. The vendor pages are the primary source: OpenAI's crawler documentation and Perplexity's. Our AI crawler access guide lists every agent with what blocking it costs.
2. The content only exists after JavaScript runs
Most AI crawlers do not execute JavaScript. A page whose product names, prices or answers appear only after client-side rendering is an empty shell to them, and your analytics will never show it, because they run in a browser that executes scripts. Fetch a key page without a browser and search for the sentence you care about:
curl -s https://example.com/pricing/ | grep -i "per month"
A 200 response with the answer missing is a failure, not a pass. The fix is server-side or static rendering for those pages. JavaScript rendering and AI crawlers covers the common setups.
3. A CDN or firewall blocks the agents anyway
Your robots.txt can be permissive while an edge rule returns 403 to anything that looks like a bot. Reading the file will never catch this. Request the page as each agent, next to a normal browser request as a control:
for ua in "Mozilla" "GPTBot" "OAI-SearchBot" "PerplexityBot" "ClaudeBot"; do
printf "%-16s " "$ua"
curl -s -o /dev/null -w "%{http_code}\n" -A "$ua" https://example.com/
done
The browser row matters. Without it, a site that is simply down looks identical to one that blocks AI agents. Two limits apply to this test: it sends the agent's name but from your own IP address, and some edges verify agents by address. OpenAI publishes an IP list for each of its agents, and allow rules at the edge should be built from those lists rather than from the user-agent string alone.
Three slower causes: quotability, entity and questions
Once an engine can fetch and read your pages, the remaining causes are matters of degree. None of them has a single test that returns yes or no, and each takes weeks of work rather than a configuration change, which is why they come second.
4. The answer on the page cannot be quoted
Answer engines retrieve passages, not pages, and write a reply from what they selected. If the sentence that answers the question only makes sense next to the heading above it, or arrives after three paragraphs of preamble, it may never be in the selected passage. A page that says "pricing varies by use case" cannot be quoted. A page that states the price can. Put the answer in the first one or two sentences under each heading, name the product in full instead of "it", and state facts with units and dates. Writing for extraction has before-and-after examples.
5. The brand is ambiguous, or wrong somewhere else
An assistant has to know which entity you are before it can name you. Ambiguity comes from a name spelled several ways, a name shared with something unrelated, or a description that changes from site to site. The other failure is not silence but a confident wrong detail, such as an outdated price or a market you left, which usually traces back to one stale external source. A model needs the same fact from independent places before it states it confidently, so a claim that exists only on your own domain is the weakest evidence you can offer. Write one plain description and use it everywhere you control, add Organization markup with sameAs links, and correct the outside sources that disagree with you. Entity and sameAs covers the markup.
6. You are targeting questions nobody asks that way
This is the least comfortable cause. Buyers ask assistants in full sentences with a situation in them: a budget, a constraint, a use case. Those questions produce a different set of named brands from the bare category term. Even in classic search the shift shows: in our own Search Console data for July to September 2026, 181 of 475 queries that produced impressions were full questions or phrases of seven words or more, carrying about 22% of impressions. If your content is organised around short keywords, you may rank for terms that matter less and be absent from the questions that matter more.
Also check what you are up against. In our Q3 2026 benchmark, the leading brand appeared in 38% of answers about project management software, while every local-service category we measured had a leader below 13%. In a consolidated category, the realistic target is a specific sub-question the incumbent answers badly. Building a prompt set shows how to write the questions.
A useful corollary is that questions where the assistants currently name nobody are open ground, and cheaper to win than displacing a brand that already appears in a quarter or more of answers.
The order, in one table
| Cause | Type | How you confirm it |
|---|---|---|
| 1. Agent disallowed | Yes or no | Read robots.txt for each named agent |
| 2. JavaScript-only content | Yes or no | Fetch without a browser, search for the answer |
| 3. Edge block | Yes or no | Status codes per agent against a browser control |
| 4. Unquotable answer | Gradual | Read the first two sentences under each heading out of context |
| 5. Ambiguous or wrong entity | Gradual | Ask each assistant what you do; look for wrong details |
| 6. Wrong questions | Gradual | Compare your target list with full buyer questions |
Doing them in the wrong order means paying for content that still cannot be read. If you want the first three checked automatically, InsightWonder's crawlability check runs them against a browser control; the commands above do the same job by hand.
After a fix: how to tell it worked
The re-test has to use the same ruler as the original diagnosis, or it proves nothing. Keep the question set fixed, keep brand-named questions out of it, and report each engine separately. Change the question set between the two runs and any movement could be the questions rather than the brand, because adding easier questions raises the average while nothing improves.
Expect noise. AI answers vary between runs even when nothing changed, so a single improved run is not evidence. Treat a change as real when it appears in two consecutive runs. Treat a timeout as missing data rather than as a zero, otherwise an engine that failed to respond will look like a new absence.
Finally, match the re-test to the cause. A fix to robots.txt or an edge rule should show up in the probes immediately, long before it shows up in answers, so confirm the probe first. A fix to an entity description or an outside listing has no probe at all; only repeated answer runs over the following weeks can show it, and they can never prove which change caused the movement.