An AI visibility score is good at one thing, noticing that something changed, and poor at the thing people usually buy it for, deciding what to do next. The number only means something when three things are visible behind it: the question set it was computed over, the engines and method that produced it, and the components it is made of. If any of those is hidden, the score cannot be checked, and a number that cannot be checked will eventually mislead you, usually in the flattering direction.
This is an opinion piece from a company that sells measurement, so the bias is worth stating up front: every rule below is one we apply to our own product, and most of them exist because we got it wrong first. The failures are ours, they are dated, and each one has been fixed.
What an AI visibility score is good for
It tells you to go and look. Nobody reads every answer across several engines every week, so a summary that moves when something real moves earns its place on a dashboard. A drop is a prompt to open the evidence, not a finding in itself, and a rise is a prompt to check whether the rise is yours or the ruler's.
Even that modest use needs one guard. AI answers vary between runs when nothing has changed, so a single-run movement is weak evidence. Treat a change as real when it shows up in two consecutive runs. An alert that fires on one run trains people to ignore alerts, which is worse than having none, because the one alert that matters arrives already discounted.
The second guard is cadence. Answer runs cost quota, and in our product they come out of a shared account-level pool, which is why the default schedule is monthly rather than weekly. A score refreshed more often than the underlying answers can be afforded is mostly measuring noise.
What an AI visibility score cannot tell you
The moment someone asks "so what do we fix?", a single number stops helping. Two brands can share a score while having opposite problems, and the score cannot tell them apart (Figure 2). The composite is a sum of things that need different work, and the work is decided by the parts, not by the total.
| What the composite merges | Why it has to be split |
|---|---|
| Being cited and being recommended | Your page can be used as a source while the answer recommends a competitor. Merged, that looks like success. |
| Different engines | Engines name different numbers of brands per answer, so the same presence means different things on each. |
| Different buyer intents | Being named in research questions and absent from purchase questions is a different problem, with a different fix, from the reverse. |
| Citation volume and breadth | One answer citing twelve of your pages is not twelve times the reach. The number of distinct questions that cite you is much harder to inflate. |
Engines are the clearest case. In our own benchmark one model named 3.5 providers per answer and another 2.1. Being one of three or four names is a different competitive position from being one of two, and an average of the two engines describes neither. That is why every result should be reported per engine before any average is shown.
A score earns its keep only when you can open it on screen and see these components next to it, because the components are what you act on. The GEO measurement framework sets out which components to keep separate and why.
Six ways a score misleads
Each of these has happened in our own product or is a rule we added after a failure. All of them make the number move without anything about your brand changing, and most of them move it upward.
- Questions that contain your brand name. Ask an assistant whether a named company is any good and it will name it. One generated question set contained the brand name in 13 of 15 questions, and the resulting score read 13% across five engines when the honest value was zero. These are self-referential questions, and they should never be scored.
- Counting the retrieval pool as citations. An assistant retrieves many pages and cites a few. Counting the candidate pool instead of the sources marked in the answer body overstated our citation figures by two to four times.
- Detecting your brand differently from competitors. Loose matching for your own name and a fixed list for competitors inflates your share by construction. Asymmetric detection once showed a brand at 31% share of voice and rank two; measured symmetrically it was 0% and last.
- Treating unmeasured as zero, or dropping it. A model that timed out produced no evidence either way. Rendering it as 0% invents a decline; silently dropping it changes the denominator. We once found that measurement runs taking longer than 10 minutes were never snapshotted, so the trend line quietly left them out. The honest display is a visible gap, which is the whole point of unmeasured versus zero.
- Changing the method mid-series. Enabling web search once moved a site from 3% to 13% while the site did nothing. Every stored run needs a methodology version, and runs on different versions should not be compared.
- Changing the question set. Every percentage has the question set as its denominator. Add easier questions and the average rises while nothing has improved.
Notice the pattern. Four of the six errors make the brand look better than it is, and the other two make normal months look like losses. Neither direction is harmless: a flattering number defers the work, and a false decline sends the team to rewrite pages that were never the problem.
A percentage needs its denominator
Even a correctly computed share can mislead without context. Being 1% of 117 brands is a completely different situation from being 1% of five, and the percentage alone cannot tell them apart. A useful score therefore travels with the number of questions, the number of engines, the date range and the total number of brands named. When the sample is too thin, the honest option is to show no share at all, because a percentage built on a handful of answers is noise dressed as a signal and will swing on the next run.
The same applies to reports built from the score. A report should state its question set, engines and date range before any finding, and say plainly when a section could not be computed rather than filling it with plausible content. Its numbers should be recomputed from stored runs, not copied from a dashboard snapshot, so that a report produced weeks later still adds up. What to put in an AI visibility report goes through the structure.
What to ask before you trust a score
Show me the question set. Show me which runs failed. Show me one answer where I was counted as cited, with the source marker.
These three requests are cheap to satisfy if the measurement is sound. Add two more if you are comparing vendors or tracking a trend: which methodology version produced each point, and whether competitors are detected by the same code as your own brand. If any of these requests meets friction, the score is doing work it cannot support.
The difference between a score you can lean on and one you cannot is rarely the formula. It is almost always whether the inputs are exposed.
| Question | A score you can check | A score you cannot |
|---|---|---|
| Which questions? | A fixed, versioned set, brand-named questions excluded | "Hundreds of prompts", not shown |
| Which engines? | Each engine reported on its own first | One blended number |
| What counts as cited? | Sources marked in the answer body, each resolving to a real URL | Anything the engine retrieved |
| What about failures? | Shown as gaps, excluded from the denominator | Counted as zero, or silently dropped |
| Can the trend be trusted? | Every point carries a methodology version | The line runs straight through method changes |
When the score drops, check the boring causes first
A falling score invites a content response. Before writing anything, rule out the causes that have nothing to do with content, in this order:
- Access. Confirm that the search and user-triggered agents can still fetch your pages, by requesting them with the real user agent rather than reading robots.txt. A new firewall rule or CDN setting can block them while robots.txt is unchanged. Perplexity cites sources on nearly every answer, so a block there shows up in citation data almost at once; its agents are listed in Perplexity's crawler guide, and OpenAI's are in its bot documentation.
- Method. Check whether the engines, the retrieval setting or the question set changed between the two runs.
- Failed runs. Check whether the lower number includes engines or questions that simply were not measured.
- Repetition. Check whether the drop appears in two consecutive runs.
Only when all four are clean is the drop worth a content investigation, and then the components tell you where to look: which engine, which intent, cited or recommended. If you want to see the access check on your own site before anything else, the AI crawlability check runs the real-agent probes with a browser control group alongside.
Sources
- Perplexity, Perplexity crawlers (agent names and roles).
- OpenAI, Overview of OpenAI crawlers (agent names and roles).
- InsightWonder first-party incident records, July to September 2026: brand-named question set, citation counting, share-of-voice detection and snapshot anchoring, each described above with the fix applied.