What is llms.txt, and should your docs have it?
A map of your docs for AI readers, adopted fast and read rarely. What that means for yours.
More and more of the readers hitting your documentation are AI tools, not people. Assistants answer customer questions straight from your help center, and llms.txt is the web's first file written specifically for those readers: a map of your best pages, left at your site's root. Whether you need one depends less on the hype than on what it costs you to have.
Our short answer: have one if your documentation platform generates it for free, and skip it as a hand-maintained project.
What is llms.txt?
llms.txt is a plain text file at the root of your website that gives AI tools a curated map of your documentation: your most important pages, grouped under headings, each described in one line, so an assistant answering questions about your product starts from your best material. Your site already has two files written for machines, and this one does a different job. robots.txt tells crawlers where they may not go, and sitemap.xml lists every page for search engines. llms.txt recommends: you pick the pages an AI should read first and leave out what would waste its attention, like archived pages and duplicates.
A minimal file looks like this:
Most sites that publish llms.txt also publish two related files. llms-full.txt packs the entire documentation into one text file, for systems that want everything in a single fetch. And many sites serve a Markdown copy of each page (append .md to the URL), because plain text is far easier for a model to read accurately than a webpage wrapped in navigation and scripts.
Pro tip: while you are making your docs readable to machines, give every image alt text. A model reading the Markdown version of an article cannot see your screenshots; the alt text is all it gets, and it is one of the cheapest AI-readability improvements you can make.
Who publishes it, and who reads it?
Publishing has taken off. When Ahrefs studied 137,000 domains, 28% had an llms.txt file, a striking share for a proposal that is barely two years old. And the publishers are not only small sites chasing an edge: we checked the big names ourselves while writing this, and Stripe, GitHub, Cloudflare, Vercel, Zapier, Supabase, and Perplexity all serve one today. By the normal measures of a web convention, llms.txt has already won its adoption phase.
And because so many sites now publish it, LLMs have become aware of llms.txt as a convention. That awareness is itself a reason to have one.
Whether AI tools actually read the file is the other half of the picture, and it looks very different. The same Ahrefs study found that 97% of published llms.txt files received zero requests from AI crawlers in the month they measured. OpenAI's site does not have one and its crawler documentation does not mention the format. Google's John Mueller has publicly questioned the point of it. Even Anthropic, which maintains a well-groomed llms.txt of its own, makes no promise that its crawler reads anyone else's.
Today's agents can already fetch what they need from your documentation without it. They follow sitemaps, crawl pages, and read Markdown copies of articles directly. llms.txt is a convenience, not a prerequisite.
So the honest state of llms.txt in 2026: widely published, rarely read. That gap is not proof the file is pointless, since sitemaps also took years before every crawler expected them. It does mean the payoff is a bet rather than a guarantee, which is why the cost side matters so much.
Should your docs have one?
Yes, as long as it costs you close to nothing. The documentation platforms have all reached that conclusion: GitBook and Mintlify generate llms.txt automatically for sites they host, and Pageloop does the same for its hosted knowledge bases, in beta, along with llms-full.txt and a Markdown copy of every article. If your platform makes the file, turn it on and spend your attention elsewhere.
Writing one by hand is also fine, and small:
List your most important pages under a few headings, each with a one-line description of what the page answers.
Save it as llms.txt at your site root, and revisit it when pages move or get deleted, because a map of pages you removed last quarter is worse than no map.
The frequency of agents accessing documentation keeps going up, and every small improvement to AI readability compounds as that traffic grows. Not because llms.txt is essential today, but because the bet improves as more agents read your docs.
llms.txt points at your content and says nothing about whether the content is correct. If the article behind a link describes last year's pricing, an AI assistant repeats last year's pricing. Keeping the pages accurate is the harder half of being AI-ready, and it is the half Pageloop is built for: it watches the tools where product changes surface, flags the articles and screenshots a change affects, and drafts the fix for a person to approve. AI will describe your product either way; llms.txt only decides which pages it reads first, and the accuracy of those pages decides what it says.
Frequently asked questions
Is llms.txt an official standard?
No, llms.txt is not an official standard. It is a proposal published by Jeremy Howard in September 2024, and no standards body or major AI company has formally adopted it. There is no fixed schema: a typical llms.txt file contains a site title, a short summary, and links to important pages grouped under headings, each with a one-line description.
Does llms.txt help SEO?
llms.txt does not affect search engine rankings. Search engines use sitemap.xml and normal crawling to index pages, and Google has confirmed it does not use llms.txt. The file is aimed at AI assistants and agents, which can read it to find a site's most important pages and their Markdown versions.
Do you need llms.txt for a help center?
A customer help center benefits from llms.txt in the same way a docs site does: the file points AI tools to the most important articles and their plain text versions. Help desk platforms such as Zendesk and Intercom do not generate llms.txt today. Pageloop generates llms.txt and llms-full.txt automatically for the knowledge bases it hosts (in beta), along with a Markdown copy of every article.
Author
Fatema works across marketing and content at Pageloop. She has an academic background in Ecology, a side-life in fashion, and an irrational loyalty to milk coffee.
Other related content you might be interested in
