Short answer: run a GEO audit in gate order. First, can each AI agent reach the page? Second, is the content there without JavaScript? Third, can a self-contained answer be lifted out of it? Fourth, is the claim confirmed anywhere you do not own? Each gate is cheap to test and close to pass-or-fail, and there is no point polishing gate three while gate one is closed. Save a baseline measurement before you change anything, so you can tell afterwards whether the fixes did anything.
An audit does not guarantee a mention. Assistants decide what to name answer by answer. What an audit establishes is whether your pages are eligible to be used at all, and that part is entirely in your hands. The first three gates are about your own site and can be fixed this week. The fourth depends on pages other people publish, which is why it comes last and takes longest.
The GEO audit checklist at a glance
| Gate | Test | Passes when |
|---|---|---|
| 1. Reachable | Request key pages as each AI agent and as a browser | Same status code and similar response size for agents and browser |
| 2. Readable | Fetch the raw HTML with no JavaScript | The facts you care about are in the response text |
| 3. Extractable | Read the first two sentences under each heading out of context | They answer the heading's question and name their own subject |
| 4. Corroborated | For each key claim, find a source that is not your domain | At least one independent page states the same fact |
Pick a handful of pages before you begin: the homepage, the main product or service pages, pricing, and whichever pages answer the questions buyers actually ask. Run every gate on the same set, and write the results into one sheet with a row per page and a column per gate. A sheet makes the pattern visible: a site whose every page fails gate two has a rendering problem, not a writing problem, and one fix in the theme clears the whole column.
Record three states, not two. "Pass" and "fail" are not enough, because some tests will not complete, and an audit that turns an unfinished test into a verdict will send someone to fix a thing that is already fine.
| What you observed | Record it as | Not as |
|---|---|---|
| The request timed out or the connection dropped | Unknown, retry later | Blocked, or "page does not exist" |
| robots.txt could not be fetched | Unknown | "No robots.txt" |
| The agent and the browser both failed | Site unavailable | "Blocks AI agents" |
| 200 response, key sentence missing | Fail (gate 2) | Pass because the status was 200 |
| Agent got 403, browser got 200 | Fail (gate 1), edge rule | A robots.txt problem |
Gate 1: can each named agent reach the page?
Do not stop at "robots.txt allows it". Request the page as each agent and compare status codes and bytes served with a normal browser request. This is the one gate where a robots.txt reading and a real request can disagree, so the request wins. A CDN or firewall rule that returns 403 to these agents is a block in practice while robots.txt reads as permissive. The browser request is your control group: without it, a site that is briefly down looks exactly like one that blocks AI agents.
for ua in OAI-SearchBot ChatGPT-User PerplexityBot Claude-SearchBot Applebot; do
printf "%s " "$ua"
curl -s -o /dev/null -w "%{http_code} %{size_download}\n" -A "$ua" https://example.com/pricing/
done
Check the search and user-triggered agents first, because blocking those removes you from answers. OAI-SearchBot builds the index behind ChatGPT's search features and is the agent sites most often block by accident while meaning to opt out of training; the agent list is in OpenAI's bot documentation. Training agents such as GPTBot and ClaudeBot are a policy choice, not an audit failure. Our AI crawler access guide covers every agent and what blocking it costs. One more rule for this gate: a timeout is "could not fetch", not "does not exist". Record it as unknown and retry.
Robots.txt is read per host, so run the probe on every hostname that serves content you care about. A shop on one subdomain and a help centre on another each have their own file and their own edge rules, and a clean result on the main domain says nothing about them.
Platform traps worth checking by name
| Platform | Trap | Check |
|---|---|---|
| Shopify | The generated robots.txt is not permissive by default, and only the published theme's robots.txt.liquid is served. | Read the live /robots.txt, and edit it in the published theme template, not a draft theme. |
| WordPress | "Discourage search engines from indexing this site" also disables core sitemaps, so /wp-sitemap.xml returns 404. | Confirm the box is unticked and that /wp-sitemap.xml and /robots.txt both respond. |
| Webflow | The webflow.io staging subdomain can serve a different, blocking robots.txt. | Audit the custom domain only; the staging file is not what visitors' domain serves. |
| HubSpot | Domains containing hs-sites are permanently no-index; a blanket disallow also blocks HubSpot's own site-search indexer. | Check each connected domain in the robots.txt settings. |
While you are at the URL level, check that each page has exactly one address. On our own site a trailing-slash inconsistency left 46 pages indexed under two URLs each, splitting 1,980 impressions between the twins. Answer engines face the same ambiguity. Canonical and duplicate URLs explains the fix.
Gate 2: is the content there without JavaScript?
Most AI crawlers do not execute JavaScript. Fetch the raw HTML and search it for the facts that matter: product names, prices, the answer to the page's main question.
curl -s -A "OAI-SearchBot" https://example.com/pricing/ | grep -i "per month"
A 200 response with an empty body is a failure, not a pass, and nothing in your analytics will reveal it, because analytics run in a browser that executes scripts. Where the content tends to go missing depends on the platform. On Shopify, normal pages arrive as complete HTML, but anything loaded through the Ajax Section Rendering API, such as faceted filtering, and app-injected blocks such as reviews and Q&A are not in it. On WordPress the risk is plugins: lazy-load scripts, load-more lists and builder tabs. On HubSpot, text inside forms and CTAs populates client-side. JavaScript rendering and AI crawlers walks through the fixes.
Gate 3: can an answer be lifted out?
Find the heading that matches a real buyer question. Read the first two sentences under it as if nothing else on the page existed. If they form a complete, correct answer that names its own subject, the passage passes. If you had to scroll up to understand them, it fails, and no amount of link building will fix that.
- Facts as sentences, not only as specs. Dimensions, compatibility, prices and return windows should be stated in prose as well as in a table. Assistants lift sentences that answer a constraint.
- Nothing important behind interaction. Content inside tabs, accordions or a load-more button often fails gate two as well.
- Named subjects. Replace "it", "this plan" and "the above" with the product name in the opening sentence of each section.
Structured data belongs in this gate too, because what matters is whether it is correct, not whether it exists. On WordPress, view the source of one post and count the Organization nodes: there should be exactly one, and duplicates from competing plugins are the usual failure. On Shopify, the structured_data filter emits ProductGroup rather than Product for products with variants, so a checker that insists on Product will report a false failure across your catalogue. Rewriting techniques are in writing for extraction.
Gate 4: is the claim corroborated somewhere you do not own?
List the claims you want an assistant to repeat about you: who you serve, what you cost, where you operate, what you integrate with. For each one, find a page that is not on your own domain and states the same thing. A claim that exists only on your own site is the weakest evidence you can offer, however well it is marked up.
Also look for the opposite problem. A common failure is not silence but a confident wrong detail, such as an outdated price or a market you left, and it usually traces back to one stale external source. Correcting that source is often faster than writing anything new. Third-party corroboration covers how to find and prioritise those pages.
Then measure the GEO audit's effect, and keep the baseline
Before you fix anything, run a fixed set of buyer questions on each engine and save the answers. After the fixes, run the same set again. Three rules keep the comparison honest:
- No brand names in the questions. In one of our own runs, 13 of 15 generated questions contained the brand name, and the score read 13% when the honest value was zero.
- Same method before and after. Enabling web search once moved a site from 3% to 13% while the site did nothing. A method change is not an improvement.
- Two consecutive runs before you believe a change. Answers vary between runs even when nothing changed.
Save the raw answers, not only the score. When a number moves, the first question is always whether the brand changed or the measurement did, and only the stored answer text can settle it. Report each engine separately, too: in our own benchmark one model named 3.5 providers per answer and another 2.1, so an average across them hides where the work is. Measuring change over time explains how to set up the comparison.
What a GEO audit cannot tell you
It cannot tell you whether the questions you are optimising for are the ones buyers ask. That comes from your own query data. In our own Search Console export, 181 of 475 queries that produced impressions were full questions or phrases of seven words or more, carrying about 22% of impressions.
Those long questions are the closest thing you have to the prompts people type into assistants, so start the question set there. A technically perfect site that answers the wrong questions will pass every gate and still stay absent.
It also cannot tell you why an assistant chose a competitor. You can observe that a rival was named and you were not; the reason is not something the answers reveal, and inventing one sends the next month's work in the wrong direction. Treat the audit as the part of GEO that is fully under your control, then let the measurement show what changed. If you would rather not script gate one yourself, the free checkers at InsightWonder Tools include an AI bot access check.
Sources
- OpenAI, overview of OpenAI crawlers.
- Shopify, robots.txt.liquid and the structured_data filter.
- WordPress, XML sitemaps in WordPress 5.5.
- Webflow, disable search engine indexing.
- HubSpot, customise the robots.txt file.
- Our own Search Console exports for insightwonder.com (2026-07-02 to 2026-09-07) and our measurement incident records, 2026.