What a generator has to produce

An llms.txt file is a Markdown file served at the root of your domain. A generator is only useful if its output follows the shape that the proposal describes, because anything that reads the file expects that shape and nothing else. In order, the file contains:

  1. One H1 with the name of the site or project. This is the only required line.
  2. A blockquote summary: a sentence or two saying what the site is and who it is for, written so it still makes sense when quoted on its own.
  3. Optional plain paragraphs with context a reader needs before the links, such as what the product does not do.
  4. H2 sections that group the links, for example Docs, Guides, Pricing.
  5. One line per entry inside each section: a Markdown link followed by a colon and a short description of what that page answers.
# Example Co

> Example Co makes inventory software for independent bookshops.

Example Co does not offer payments or point-of-sale hardware.

## Docs
- [Getting started](https://example.com/docs/start/): connect a shop and import stock
- [Pricing](https://example.com/pricing/): plans, limits and what each tier includes

## Optional
- [Changelog](https://example.com/changelog/): release notes, safe to skip

The proposal reserves a section called Optional for links a reader can drop when space is short. For the full format and a longer worked file, see our llms.txt guide and the example-and-format walkthrough.

Which pages a generator should pick

The hard part of a generator is not the Markdown, it is the selection. A file that lists every URL in your sitemap is just a worse sitemap. Pick pages that answer a question on their own:

  • Product or service overviews, pricing, and the pages that define what you do and do not offer.
  • Documentation and how-to pages that a buyer or user would actually ask about.
  • Canonical URLs only: no tag archives, paginated lists, search results, or parameter variants.
  • Pages that are indexable and return a normal response. Never list a page you block in robots.txt or mark noindex; the file would contradict your own access rules.

The description after each link matters more than the title. Write what the page answers, not a slogan, because that line is the only context a reader gets before deciding whether to fetch the page.

How to stop the file going stale

A hand-written llms.txt is usually correct on the day it is written and wrong a few releases later: renamed URLs, retired pages, a pricing page that moved. The fix is to generate the file from the same route data that builds your site, not from a crawl and not from memory.

  • Build it at deploy time from your route list or CMS, the same source your sitemap uses, so a deleted page disappears from both at once.
  • Keep the descriptions next to the content, for example as a field on each page record, so editing a page is the moment its description gets updated.
  • Fail the build if any listed URL does not resolve, redirects, or is noindex.
  • Serve it as plain text from the root path, and check the live response rather than the file in your repository.

What llms.txt will not do

Be clear about the limits before you spend time on a generator. No major AI engine has publicly committed to reading llms.txt when it builds answers, so publishing one is not a visibility lever you can measure on its own. It does not grant or block crawler access; that is still robots.txt. Treat it as a cheap, well-kept index for any tool or agent that chooses to read it, and keep your effort on pages that answer real questions. The glossary entry has the short definition. Once your file is live, you can check its structure against the format with the validator on our free tools site.