Short answer: to get into AI answers, a brand needs six things: engines that can fetch its pages, content they can read without running JavaScript, self-contained passages that answer the question, an identity they cannot confuse with anyone else, the same facts confirmed on other sites, and an honest way to measure whether any of it worked. That is the whole job, in roughly that order, because each piece is a precondition for the next.
None of this guarantees a mention. Assistants decide what to name answer by answer, and nobody outside the engine controls that. What you control is whether your brand is eligible to be named, and whether you can see the result honestly. The order matters more than it looks: a brand that rewrites every page before checking whether the search agents can reach them has spent its budget on the fourth step while failing the first.
1. Let the right crawlers in, and test it rather than assume it
Each major assistant uses more than one agent, and they do different jobs. OpenAI documents three that matter here: GPTBot crawls content that may be used to train its models, OAI-SearchBot builds the index behind ChatGPT's search features, and ChatGPT-User fetches a page because a person asked about it. Anthropic splits ClaudeBot, Claude-SearchBot and Claude-User the same way, and Perplexity separates PerplexityBot from Perplexity-User. Google-Extended is not a crawler at all; it is a usage control over content Googlebot has already fetched.
| Agent | Role | What blocking it costs |
|---|---|---|
| GPTBot, ClaudeBot | Training corpus | Nothing in today's answers; a position on training use |
| OAI-SearchBot, Claude-SearchBot, PerplexityBot | Search or answer index | Removal from that engine's search answers |
| ChatGPT-User, Claude-User, Perplexity-User | Fetch on a user's request | A user who asks the assistant to read your page gets nothing back |
| Google-Extended | Usage control, does not crawl | Removal from grounded Gemini answers; Google Search unaffected |
That split lets you make a deliberate choice. If you want to opt out of model training but stay eligible for search answers, block the training agents and allow the search and user agents:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Claude-SearchBot
Allow: /
The most common way sites remove themselves from answers is one rule that blocks every agent at once. The second most common is invisible in robots.txt: a CDN or firewall rule that returns 403 to these agents while the file says they are allowed. That is why access has to be tested with the real user agent, not inferred from the file, and why the test needs a normal browser request alongside it: without that control, a site that is simply down looks exactly like a site that blocks AI agents. The agent list and each vendor's documentation are collected in our AI crawler access guide, and the primary sources are OpenAI's bot documentation and Anthropic's crawler article.
One caveat applies to the user-triggered agents. OpenAI, Anthropic and Perplexity each note that agents fetching a page on a user's behalf may not follow robots.txt the way crawlers do. Treat robots.txt as a statement of intent for crawlers, not as access control.
2. Make the content readable without JavaScript
Most AI crawlers do not execute JavaScript. A page whose product names, prices or answers appear only after client-side rendering is an empty shell to them, and nothing in your own analytics will tell you, because your analytics run in a browser that does execute scripts.
The check takes a minute. Fetch a key page with scripts disabled and look for the text you care about:
curl -s -A "OAI-SearchBot" https://example.com/pricing/ | grep -i "per month"
If the answer is missing, the fix is server-side or static rendering for those pages, not more content. A 200 response with an empty body should be treated as a failure, not a pass. The usual culprits depend on the platform. Normal Shopify storefront pages arrive as complete HTML, but content fetched through its Ajax Section Rendering API and blocks injected by apps such as reviews do not. On WordPress, both classic and block themes render on the server; the risks are plugins that lazy-load content, load-more listings and builder-injected tabs. Our guide to JavaScript rendering and AI crawlers walks through the common framework setups.
3. Write passages that AI answers can lift
Answer engines quote passages, not pages. A passage that only makes sense after the paragraph before it is hard to quote, however good the page is as a whole. Three habits make a page easier to lift from:
- Answer first. Put the direct answer in the first one or two sentences under each heading, then the evidence and the nuance.
- Name things in full. Write the product, the brand and the condition in the sentence itself instead of relying on "it", "this plan" or "the above".
- Use headings that state the question or the fact. A heading such as "How long does delivery to Canada take?" tells a retriever what the passage under it answers. A heading such as "Our approach" tells it nothing.
Write facts rather than adjectives. Assistants lift sentences that answer a constraint: a dimension, a material, a return window, a compatibility. Copy written for brand tone gives them nothing to lift, however well it reads, and putting those facts only in a specification table is not enough either; write them into sentences as well. A quick test is to read the first two sentences under each heading out of context. If they stand on their own as an answer, the passage is liftable. Writing for extraction covers this with before-and-after rewrites.
4. Make the brand unambiguous
An assistant has to know which entity you are before it can recommend you. Ambiguity comes from small inconsistencies: one category on the homepage, another on LinkedIn, a third in a directory listing, and a brand name shared with something else. Write one plain description of what you are and who it is for, and use it everywhere you control. The common failure here is not silence but a plausible wrong detail, such as an outdated price, a market you left or a feature that belongs to a competitor, and it traces back to one stale source more often than to any weakness on your own site.
Structured data helps as disambiguation: an Organization block with your official name, URL, logo and sameAs links to the profiles that are really yours. How you add it depends on the platform, and the defaults are thinner than most people assume. Shopify does not inject JSON-LD for you; its structured_data Liquid filter has to be called explicitly in templates, and it emits ProductGroup rather than Product for products with multiple variants. WordPress core emits no schema.org markup for content, so it all comes from the theme or plugins, and the common failure there is duplicated or conflicting Organization graphs rather than missing ones.
Be realistic about what markup does. None of the crawler documentation linked in this article says that schema markup raises the chance of being cited. Treat it as a way to remove ambiguity, not as a ranking lever. Entity and sameAs explains which links to include and which to leave out.
5. Be corroborated somewhere other than your own site
Your own site is the source of truth about your product, but it is not independent evidence. A model needs the same fact from independent places before it will state it confidently, and a claim that exists only on your own domain is the weakest form of evidence you can offer, however well marked up it is. When an assistant names brands for a category question, it is easier to name one whose claims appear consistently in places the brand does not control: reviews, trade press, comparison pages, community discussions and directories.
How much this matters depends on your category. In our own Q3 2026 benchmark, some product categories had already consolidated around a default answer: the top brand appeared in 38% of project management software answers, 35% of skincare answers and 28% of electric bike answers. No local-service category had a leader above 13% of answers. That difference decides which goal is realistic (Figure 3).
In a consolidated category you are displacing an incumbent, not filling a void, and the realistic goal is to become the answer to a specific sub-question the leader answers badly: a constraint, a budget, a use case. In the product categories we measured, a large share of citations also went to marketplaces, review sites and publisher round-ups rather than to brand sites, which is exactly where corroboration lives.
The useful work is unglamorous. Check that the facts other sites state about you, such as your category, prices, locations and what you integrate with, match what your own site says, and correct the ones you can. Contribute genuinely useful answers where your buyers already ask questions, under your real name. Avoid anything that looks like seeded praise, because it is easy to spot and hard to undo. Third-party corroboration covers how to find and prioritise those sources.
6. Measure whether you got into AI answers, honestly
Measurement is where most programmes quietly go wrong, because the easiest numbers to produce are the most flattering ones. Three rules keep it honest:
- Exclude questions that contain your brand name. Ask an assistant whether a named company is any good and it will name it. In one of our own runs, 13 of 15 generated questions contained the brand name, and the score read 13% when the honest value was zero.
- Report each engine separately before any average. In our own benchmark one model named 3.5 providers per answer and another 2.1. A single cross-engine number describes neither and hides where the work is.
- Count only sources marked in the answer text as citations. Counting everything the engine retrieved inflated our own citation figures by two to four times before we fixed it.
Keep the question set fixed between runs, written as full questions a buyer would ask without knowing you exist, and keep the answer text beside every number so you can tell a detection bug from a real change. Building a prompt set explains how to write one.
Sources
- OpenAI: overview of OpenAI crawlers
- Anthropic: does Anthropic crawl data from the web
- Perplexity: Perplexity crawlers
- Google: common crawlers (Google-Extended)
- Shopify: the structured_data Liquid filter and the Section Rendering API
- WordPress: the wp_head hook
A note on this revision: an earlier version of this article showed percentage charts labelled as estimates, a statistic about small brands and several unnamed case studies, none of which we could trace to a source. We removed them. Every number that remains comes either from a vendor's own documentation or from our own measurement, and says which.