Short answer: llms.txt is a proposed Markdown file served at the root of your domain that gives language models a short, curated map of your site: what you are, who it is for, and which pages matter. It is a convention, not a standard that any AI vendor has committed to read. Write one because it is cheap and forces useful decisions about your own content, not because it will put your brand into AI answers.

This guide covers what the file is for, what to put in it, how each major site platform serves it (they differ more than you would expect), how to check it, and where it belongs in your priorities.

What llms.txt is, and what it is not

The proposal is simple. A plain Markdown document at /llms.txt starts with the name of the site or product, follows with a short summary, and then lists the important pages in sections, each link with a one-line description. The idea is that a model, or an agent acting for a user, can read one small file instead of crawling the whole site and guessing which pages are authoritative.

It is not an access control and it is not a ranking signal. Whether an AI agent can fetch your pages is decided by robots.txt and, in practice, by what your CDN or firewall returns to that agent. The vendors document their agents in detail: OpenAI separates GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic separates ClaudeBot, Claude-SearchBot and Claude-User, and Perplexity separates PerplexityBot from Perplexity-User. None of the vendor pages we checked for our AI crawler list says that llms.txt is read when an answer is composed. Treat any promise that the file "gets you into ChatGPT" accordingly.

The three root files are often confused, so it helps to set them side by side:

FileIts jobControls access?Documented by AI vendors as read for answers?
robots.txtTells each named agent which paths it may fetchYes, for agents that honour itYes: every vendor page we checked describes how its agents treat it
SitemapLists canonical URLs and when each changedNoNot in the vendor pages we checked
llms.txtA curated, described map of the pages that matterNoNot in the vendor pages we checked

There are still good reasons to write one. It costs little, it makes you decide which of your pages are canonical, and one major commerce platform now serves the file by default, so many sites already have one whether they wrote it or not.

What to put in llms.txt

A structure that holds up on a real site looks like this:

# Example Co

> Example Co sells modular standing desks to home and small-office buyers
> in the UK and EU. This file lists the pages that describe the products,
> prices, delivery and returns.

## Products
- [Standing desks](https://example.com/desks/): frame options, sizes, load limits
- [Accessories](https://example.com/accessories/): monitor arms, cable trays

## Buying
- [Pricing](https://example.com/pricing/): current prices and what each bundle includes
- [Delivery and returns](https://example.com/returns/): regions served, return window

## Company
- [About](https://example.com/about/): who runs the company and where it is based
# Example Co> Example Co sells modular standing desks> to home and small-office buyers in the> UK and EU. This file lists the pages ...## Products- [Standing desks](…/desks/): frame options, sizes, load limits## Buying- [Pricing](…/pricing/): current prices- [Delivery and returns](…/returns/): regions## Company- [About](…/about/): who runs the companyTitle lineThe name a stranger would search forSummary: the most important partWhat you do, for whom, where scope endsSections by the reader’s decisionProducts, then buying, then companyLink text plus a descriptionSays what each page answersOnly crawlable, server-rendered pagesGenerated from routes, not typed by hand
Figure 1. The parts of an llms.txt file and what each one is for, using the example above. Source: the llms.txt convention as described in this guide.

Six habits make the difference between a useful file and a decorative one:

  • Treat the summary as the most important part. Say what the business is, who it serves and where its scope ends, in the words a stranger would use. Marketing copy that would fit any competitor tells a model nothing.
  • Group links by the decision a reader is making, not by your navigation menu. Products, then buying information such as prices and policies, then documentation or guides, then company pages.
  • Write link text and a description for every link. A bare URL carries no meaning; the description is the part that explains what the page answers.
  • Define your own terms if your product uses names or metrics a stranger would not know. That is exactly the ambiguity a model would otherwise resolve by guessing.
  • List only pages that are crawlable and readable without JavaScript. A file that points at blocked or client-rendered pages helps nobody. Our guide to JavaScript rendering and AI crawlers explains how to check.
  • Generate it from your routes or CMS rather than writing it by hand. A hand-written file drifts as pages are renamed or deleted, and a file that points at pages that no longer exist is worse than none.

For more worked examples of the format, see llms.txt example and format.

How each platform serves llms.txt

Where the file lives and how you change it depends almost entirely on your platform. These are the documented behaviours:

PlatformHow /llms.txt is servedWhat to watch for
ShopifyServed by default with no theme file present: Shopify serves /agents.md and mirrors it to /llms.txt and /llms-full.txt. The lookup order for /llms.txt is llms.txt.liquid, then agents.md.liquid, then Shopify's managed default.Adding agents.md.liquid changes all three files; adding llms.txt.liquid changes only /llms.txt.
WordPress (self-hosted)Upload llms.txt to the document root. The recommended server configuration serves existing files before routing to index.php, so the static file wins.Does not apply to WordPress.com hosted plans, where filesystem access is restricted. Confirm the file is not swallowed by the rewrite.
WebflowThe help centre documents a dedicated llms.txt upload that serves the file from the domain root on a custom domain.Other arbitrary root-level files are still unsupported. Check on the custom domain, never on the webflow.io staging subdomain.
HubSpotNo documented way to serve an arbitrary static file at the root; File Manager assigns URLs under /hubfs/.A page whose slug is llms.txt resolves at the root but is served as text/html. A reverse proxy in front of HubSpot is the reliable option.
Request for /llms.txt on Shopify1 llms.txt.liquid in the published theme?YesServed; changes /llms.txt only2 agents.md.liquid in the published theme?YesServed; also drives /agents.md and /llms-full.txtNeitherShopify’s managed default is servedA store with neither template lands here,publishing a file nobody on the team wrote.
Figure 2. How Shopify decides what to serve at /llms.txt. Source: Shopify llms.txt.liquid and agents.md.liquid template documentation, checked 2026-09-09.

The Shopify row deserves a second look if you run a store. Because Shopify serves a managed default, your store is probably already publishing an llms.txt you have never read. Open it before deciding whether to override it, and if you do override it, prefer agents.md.liquid so that all three files say the same thing. The sources are Shopify's llms.txt template documentation, Webflow's llms.txt upload article, WordPress's nginx guide and HubSpot's supported file types. Platform-specific detail is collected on our Shopify platform page.

How to check your llms.txt

Two commands cover most of the check:

curl -sI https://example.com/llms.txt
curl -s https://example.com/llms.txt | head -n 5
  1. It answers at the root itself, not after a redirect. Programs that fetch control files commonly do not follow a redirect, so a file that only exists behind one is effectively missing.
  2. The content type is plain text, not text/html. This is the failure the HubSpot slug workaround produces, and it is invisible in a browser.
  3. The body starts with your title line, not with an HTML error page, a cookie banner or a login wall.
  4. Every linked page is reachable by the agents you care about. Test with the real user agent rather than reading robots.txt: a permissive robots.txt behind a CDN rule that returns 403 to those agents is a block in practice. Our AI crawler access guide walks through that test.
  5. You checked the production domain. On Webflow in particular, the staging subdomain serves different control files from the custom domain.

Re-run the check after any redesign, migration or platform change, because those are the moments when root files quietly disappear.

Where it belongs, and common mistakes

If you have limited time for machine readability, spend it in this order: confirm the search and user agents can actually fetch your content pages; make sure the key answers exist in the raw HTML; write passages that stand on their own; make your brand unambiguous with consistent descriptions and Organization markup; and then write llms.txt. The order is about dependencies, not about a measured effect: each earlier step is a precondition for the later ones. An llms.txt that points at pages an agent cannot fetch, or that render empty without JavaScript, changes nothing.

5 llms.txtA curated map of pages that already work4 An unambiguous brand entityConsistent descriptions and Organization markup3 Passages that stand on their ownAnswer first, subject named in the sentence2 Key answers in the raw HTMLReadable without running JavaScript1 Agents can fetch your content pagesrobots.txt allows them and the edge returns 200Each layer is a precondition for the ones above it. The order shows dependencies, not a measured effect size.
Figure 3. Where llms.txt sits among the work that makes a site machine-readable. Read from the bottom: each layer depends on the ones below it. Source: the priority order described in this guide.

Read the stack from the bottom. If a lower layer is missing, fix it before spending time on anything above it; if all the lower layers hold, llms.txt is a sensible, cheap addition rather than a gamble.

Common mistakes

  • Using it as robots.txt. The convention has no access-control meaning. Rules about which agents may fetch what belong in robots.txt and in your CDN configuration.
  • Listing every URL. That is a sitemap. The value of llms.txt is the curation and the descriptions.
  • Letting it go stale. If it is not generated, put a review date next to it and an owner on it.
  • Measuring success by whether the file exists. The outcome you care about is in the answers. Measure that directly, with a fixed set of buyer questions, rather than inferring it from the presence of a file.

Sources

An earlier version of this guide said writing the file takes about an hour and that its priority order reflected how much each step changes an answer. Neither was measured, so both are gone. The Webflow and HubSpot rows are taken from their help centres rather than tested on a live site.