Short answer: a sitemap does not get you into AI search answers, and we have no vendor documentation saying how, or whether, AI crawlers such as OAI-SearchBot or PerplexityBot use sitemaps. What a sitemap does is tell crawlers which URLs you consider canonical and when they last changed. When it is wrong, it quietly advertises redirects, duplicates and fake freshness, and nothing in your build will fail. So treat it as hygiene with four checks: every listed URL returns 200, each page has one URL form, lastmod is real, and you know exactly what your platform generates.
That is a smaller role than many GEO checklists give it, and a more useful one. The sections below cover where the sitemap sits among the files crawlers read, the duplicate-URL failure we found on our own site, the four checks with a one-line test, how to keep lastmod honest, and what Shopify, WordPress, Webflow and HubSpot generate for you.
What a sitemap does, and does not do, for AI search
AI engines reach your pages in different ways. OpenAI's OAI-SearchBot builds the index behind ChatGPT's search features, PerplexityBot indexes pages for Perplexity's answers, and Claude-SearchBot crawls for Claude's search results. Gemini is different again: Google-Extended is not a crawler but a usage control over content Googlebot has already fetched, so what Gemini can ground answers on depends on Google's own crawl. The vendor pages for each agent are collected in OpenAI's bot documentation and Google's list of common crawlers.
None of that tells you whether a given AI crawler reads your sitemap, so do not count on one to get a page into an answer. Count on it for something narrower: a clean, accurate list of the URLs you want crawled, which every crawler that does read it can trust. The three files people tend to lump together do quite different jobs:
| File | What it tells a crawler | What AI vendors document about it | Typical failure |
|---|---|---|---|
robots.txt | Which agents may fetch which paths | Each major operator documents its agents' names and how they treat it | A rule meant for training agents also blocks a search agent |
| Sitemap | Which URLs are canonical and when each changed | Nothing we could find from the AI operators themselves | Lists redirects, duplicates and build-time dates |
llms.txt | A plain-text summary of the site, written for language models | Supported by some platforms, such as Shopify and Webflow; not documented as read by the engines in our sources | Served as HTML, or behind a redirect |
The things that decide whether a page can be used in an answer at all, such as crawler access and server-rendered content, sit upstream of the sitemap. A perfect sitemap cannot rescue a page that returns 403 to OAI-SearchBot or renders its text only in JavaScript. If you have not checked those yet, they are the better use of the hour. If you want the plain-text summary file as well, the llms.txt guide covers what to put in it.
The failure that hides
Sitemaps break silently. A routing change, a trailing-slash rule, a CMS migration, and the file starts listing URLs that redirect or return 404. Nothing errors, the build passes and the file is still valid XML. It has simply stopped being an accurate list of canonical pages.
We hit a version of this on our own site. A trailing-slash inconsistency left 46 pages indexed under two URLs each, splitting 1,980 impressions between the twins in our Search Console data. Each pair competed with itself, and neither URL got the full credit.
The fix was to choose one URL form, enforce it everywhere, and make the sitemap, the canonical tags and the internal links agree. Any one of the three pointing at the other form is enough to keep the twin alive. Canonical and duplicate URLs covers that alignment in detail.
Four sitemap checks for AI search
Each check catches a different way the file stops telling the truth:
- Every listed URL returns 200. Not a redirect, not a 404. A sitemap is a list of canonical destinations, not of addresses that eventually work.
- One URL form per page. Pick trailing slash or none, and http or https once, enforce it in code, and make sure the sitemap matches the canonical tag and the internal links.
lastmodis real. A file in which every page claims to have changed today carries no information.- Only one sitemap is advertised. Two generators running side by side, each listed in robots.txt, is a common platform side effect.
The first check takes one command:
# List every URL in a sitemap and print the ones that do not return 200.
# If the file is a sitemap index, run it again on each child sitemap it lists.
curl -s https://example.com/sitemap.xml \
| grep -o '<loc>[^<]*</loc>' | sed 's/<[^>]*>//g' \
| while read -r u; do
printf "%s %s\n" "$(curl -s -o /dev/null -w '%{http_code}' "$u")" "$u"
done \
| grep -v '^200 '
Any line it prints is a URL to fix or remove. Run it after every deploy that touches routing, and run it against the production domain rather than a staging host, because staging hosts often serve different files. For the second check, compare the URL form in the sitemap with the canonical tag on a handful of pages; any mismatch is a twin waiting to happen.
lastmod and dates: do not let the build lie
Questions that imply a current answer favour dated, recently updated pages. Undated pages, or pages whose modified date is bumped by every deploy, lose to competitors that state a real date, and a bumped date is a claim a crawler can check against the content. The same discipline applies to the sitemap's lastmod: it should change when the content changes, and only then.
In practice that means taking lastmod from the content's own updated date, not from the build time, and keeping the visible "last updated" date on the page, the Article markup and the sitemap in agreement. Decide what counts as a change worth a new date, too. A corrected typo is not; a new price, a new section or a changed recommendation is. Content freshness and dates walks through how to set that up.
What your platform actually generates
Most sites do not write their sitemap by hand, which is why the platform's behaviour matters more than the format:
| Platform | Default behaviour | What to watch |
|---|---|---|
| Shopify | Auto-generated at /sitemap.xml as an index over products, collections, blogs and pages, and it updates itself. | No documented way to customise it or exclude single URLs. A password-protected store's sitemap is not reachable by crawlers at all. |
| WordPress | Since WordPress 5.5, a sitemap index at /wp-sitemap.xml, adjustable through filters. | Most SEO plugins replace it at a different URL, so two sitemaps often coexist. The "Discourage search engines" setting also disables core sitemaps. |
| Webflow | Generated at /sitemap.xml once enabled, rebuilt on every publish, with a lastmod per URL and hreflang alternates for Localize sites. | You can paste a hand-written sitemap instead, but not both, or robots.txt ends up advertising two. |
| HubSpot | Website pages and blog posts are added automatically. | Landing pages are not; each one is added by hand. Edits can take up to thirty minutes to appear, and removing a URL does not deindex it. |
The WordPress trap deserves a sentence of its own: a staging site promoted to production with "Discourage search engines from indexing this site" still ticked loses /wp-sitemap.xml silently, and an SEO plugin that has replaced the sitemap URL will not warn you. The details are in the WordPress 5.5 sitemap announcement, Shopify's sitemap help page and HubSpot's sitemap article, and summarised on our WordPress page.
Where hreflang fits
If you publish in more than one language, the sitemap is a tidy place to declare which URLs are language versions of each other. Webflow does this for you on Localize sites. Elsewhere, generate the alternates from the same source as the pages, list every language version consistently, and use exactly the URL form your canonical tags use, or you recreate the duplicate-URL problem in a second language. Multilingual GEO covers the rest of the setup.
A sitemap will not make an engine name you. A broken one can make it harder for any crawler to see which of your pages are real, current and canonical, and that is the part worth an hour of checking.
Sources
- OpenAI, Overview of OpenAI crawlers; Google, Google's common crawlers. Both checked 2026-09-09.
- WordPress, New XML sitemaps functionality in WordPress 5.5.
- Shopify Help Center, Finding your sitemap.
- Webflow Help Center, Create a sitemap in Webflow.
- HubSpot Knowledge Base, View and edit a HubSpot-hosted domain sitemap.