Short answer: when your brand is not in AI answers, the cause is usually mechanical, not mysterious. Check six things in this order: whether the retrieval agents are allowed, whether the content exists without JavaScript, whether a CDN or firewall blocks the agents anyway, whether the answer on the page can be quoted, whether the brand is an unambiguous entity, and whether you are targeting the questions buyers actually ask. The first three are yes-or-no checks you can run today. The last three are slower and more expensive, so fix them only after the first three pass.

Before any of that, confirm that the absence is real. Measurement errors can manufacture an absence, or hide one, and we have made each of the errors below in our own product before fixing it.

Step 0: confirm the zero is real (fixed question set, no brand names, per engine)Yes-or-no checkscheap to run, each one invalidates the rest01Agent disallowedrobots.txt blocks the searchor user agent by name02JavaScript-onlythe answer is missing fromthe raw HTML03Edge blockCDN or firewall returns 403whatever the file saysSlower causesfix only after the first three pass04Unquotable answerthe key sentence needs itsheading to make sense05Ambiguous entityseveral names, or a wrongdetail somewhere else06Wrong questionsshort keywords instead offull buyer situations
Figure 1. The six causes in the order to check them. The first row are yes-or-no checks that can each make everything after them irrelevant; the second row are slower causes worth paying for only once the first row passes. Source: the procedure in this article.

Before you start: is your brand really not in AI answers?

Three measurement errors produce a false absence, or hide a real one:

  • Unmeasured shown as zero. A model that timed out produced no evidence either way. Counted as "not mentioned", it drags the percentage down and can look like a decline that never happened. See unmeasured vs zero.
  • One average across engines. In our own benchmark one model named 3.5 providers per answer and another 2.1. You can be absent from one engine and present in another, and a blended number hides which.
  • Brand-named questions in the set. These make you look present when you are not. In one of our own runs, 13 of 15 generated questions contained the brand name, and the score read 13% across five engines when the honest value was zero.

If the absence survives a fixed set of questions that do not name you, reported per engine, it is real. Then work through the causes below.

Three yes-or-no reasons your brand is not in AI answers

These three are binary and cheap to test, and any one of them makes the rest of this list irrelevant: if the agent cannot fetch or read the page, nothing written on it matters. Run them on the pages that should answer buyer questions, such as pricing, product and comparison pages, not only on the homepage.

1. The retrieval agent is not allowed to fetch the page

Check this first because it invalidates everything after it. Each major assistant uses separate agents for training and for search. OpenAI documents GPTBot for training, OAI-SearchBot for the index behind ChatGPT's search features, and ChatGPT-User for fetches a person asked for. Perplexity separates PerplexityBot, which indexes pages for its answers, from Perplexity-User. Blocking GPTBot is a defensible decision about training. Blocking OAI-SearchBot removes you from ChatGPT search results, and it is the agent most often blocked by accident by sites that meant to opt out of training.

Read your robots.txt for a blanket rule aimed at a whole vendor, or at every agent. The vendor pages are the primary source: OpenAI's crawler documentation and Perplexity's. Our AI crawler access guide lists every agent with what blocking it costs.

2. The content only exists after JavaScript runs

Most AI crawlers do not execute JavaScript. A page whose product names, prices or answers appear only after client-side rendering is an empty shell to them, and your analytics will never show it, because they run in a browser that executes scripts. Fetch a key page without a browser and search for the sentence you care about:

curl -s https://example.com/pricing/ | grep -i "per month"

A 200 response with the answer missing is a failure, not a pass. The fix is server-side or static rendering for those pages. JavaScript rendering and AI crawlers covers the common setups.

3. A CDN or firewall blocks the agents anyway

Your robots.txt can be permissive while an edge rule returns 403 to anything that looks like a bot. Reading the file will never catch this. Request the page as each agent, next to a normal browser request as a control:

for ua in "Mozilla" "GPTBot" "OAI-SearchBot" "PerplexityBot" "ClaudeBot"; do
  printf "%-16s " "$ua"
  curl -s -o /dev/null -w "%{http_code}\n" -A "$ua" https://example.com/
done

The browser row matters. Without it, a site that is simply down looks identical to one that blocks AI agents. Two limits apply to this test: it sends the agent's name but from your own IP address, and some edges verify agents by address. OpenAI publishes an IP list for each of its agents, and allow rules at the edge should be built from those lists rather than from the user-agent string alone.

Three slower causes: quotability, entity and questions

Once an engine can fetch and read your pages, the remaining causes are matters of degree. None of them has a single test that returns yes or no, and each takes weeks of work rather than a configuration change, which is why they come second.

4. The answer on the page cannot be quoted

Answer engines retrieve passages, not pages, and write a reply from what they selected. If the sentence that answers the question only makes sense next to the heading above it, or arrives after three paragraphs of preamble, it may never be in the selected passage. A page that says "pricing varies by use case" cannot be quoted. A page that states the price can. Put the answer in the first one or two sentences under each heading, name the product in full instead of "it", and state facts with units and dates. Writing for extraction has before-and-after examples.

5. The brand is ambiguous, or wrong somewhere else

An assistant has to know which entity you are before it can name you. Ambiguity comes from a name spelled several ways, a name shared with something unrelated, or a description that changes from site to site. The other failure is not silence but a confident wrong detail, such as an outdated price or a market you left, which usually traces back to one stale external source. A model needs the same fact from independent places before it states it confidently, so a claim that exists only on your own domain is the weakest evidence you can offer. Write one plain description and use it everywhere you control, add Organization markup with sameAs links, and correct the outside sources that disagree with you. Entity and sameAs covers the markup.

6. You are targeting questions nobody asks that way

This is the least comfortable cause. Buyers ask assistants in full sentences with a situation in them: a budget, a constraint, a use case. Those questions produce a different set of named brands from the bare category term. Even in classic search the shift shows: in our own Search Console data for July to September 2026, 181 of 475 queries that produced impressions were full questions or phrases of seven words or more, carrying about 22% of impressions. If your content is organised around short keywords, you may rank for terms that matter less and be absent from the questions that matter more.

Testing cause 6: do buyers ask the question the way your pages are built?1List the shortkeywords your pagesare built around2Rewrite each as afull question with abudget, constraintor use case3Ask the assistantsboth versions, twice(answers varybetween runs)4Compare who isnamed in eachversionKeyword“payroll software”Question a buyer asks“payroll software for a café with part-time staff”Named for the keyword, absent from the questions: it is cause 6.Reorganise the content around the questions buyers ask.Absent from both:check causes 1–5 first
Figure 2. A test for cause 6: the same need asked as a short keyword and as a buyer’s full question can produce different named brands. The example is illustrative. Source: the diagnostic order in this guide.

Also check what you are up against. In our Q3 2026 benchmark, the leading brand appeared in 38% of answers about project management software, while every local-service category we measured had a leader below 13%. In a consolidated category, the realistic target is a specific sub-question the incumbent answers badly. Building a prompt set shows how to write the questions.

Read the current answers before choosing a question to winOPEN GROUNDNobody is namedCheapest to win: be the firstpage that answers it well.Look here firstCONTESTEDSeveral brands,none dominantWinnable with corroboratedfacts and a clear answer.Not yet consolidated: everylocal-service category we measuredCONSOLIDATEDOne brand in a quarteror more of answersTarget a sub-question theincumbent answers badly.Our benchmark: project mgmtsoftware, skincare, e-bikesCheaper to winCostlier to displace
Figure 3. Three kinds of ground a question can be, read from what the current answers name; the cost of winning rises from left to right. Source: the method in this guide; category placements from the InsightWonder Q3 2026 benchmark, 12 categories across five models.

A useful corollary is that questions where the assistants currently name nobody are open ground, and cheaper to win than displacing a brand that already appears in a quarter or more of answers.

The order, in one table

CauseTypeHow you confirm it
1. Agent disallowedYes or noRead robots.txt for each named agent
2. JavaScript-only contentYes or noFetch without a browser, search for the answer
3. Edge blockYes or noStatus codes per agent against a browser control
4. Unquotable answerGradualRead the first two sentences under each heading out of context
5. Ambiguous or wrong entityGradualAsk each assistant what you do; look for wrong details
6. Wrong questionsGradualCompare your target list with full buyer questions

Doing them in the wrong order means paying for content that still cannot be read. If you want the first three checked automatically, InsightWonder's crawlability check runs them against a browser control; the commands above do the same job by hand.

After a fix: how to tell it worked

The re-test has to use the same ruler as the original diagnosis, or it proves nothing. Keep the question set fixed, keep brand-named questions out of it, and report each engine separately. Change the question set between the two runs and any movement could be the questions rather than the brand, because adding easier questions raises the average while nothing improves.

Expect noise. AI answers vary between runs even when nothing changed, so a single improved run is not evidence. Treat a change as real when it appears in two consecutive runs. Treat a timeout as missing data rather than as a zero, otherwise an engine that failed to respond will look like a new absence.

Finally, match the re-test to the cause. A fix to robots.txt or an edge rule should show up in the probes immediately, long before it shows up in answers, so confirm the probe first. A fix to an entity description or an outside listing has no probe at all; only repeated answer runs over the following weeks can show it, and they can never prove which change caused the movement.

Sources