llms.txt: what it does, what it doesn’t, and whether you need one

By Matija Konjić

Every few months a new file promises to make AI models notice your site. Right now that file is llms.txt, a plain markdown document you place at your root to tell language models what your site covers and where the useful pages live.

The pitch is reasonable. The question worth answering is whether anything on the other end is actually reading it. In 2026 the honest answer is that some tools are, the ones you most want are not, and it may still be worth an afternoon.

What llms.txt actually is

llms.txt was proposed in September 2024 by Jeremy Howard at Answer.AI. The reasoning behind it is sound. Language models work inside a limited context window, and a normal web page buries its useful content under navigation, scripts, cookie banners and markup. A curated markdown file hands the model a clean map instead.

The specification is deliberately small. A valid file has a defined shape, and only the first part is mandatory:

• An H1 with your site or project name. This is the only required element in the entire file.
• A blockquote summary. One short paragraph explaining what the site is and how to read the rest of the file.
• H2 sections holding link lists. Each entry is a markdown link followed by an optional note describing what that page covers.
• An optional section. Anything flagged here can be skipped by a model that is running short on context.

There is a second file in the proposal, llms-full.txt, which takes the opposite approach. Where llms.txt is an index, llms-full.txt compiles your site’s text into a single markdown document so a whole knowledge base can be loaded from one URL.

It is easy to confuse this with robots.txt, but the two do different jobs. One controls access and is enforced. The other suggests importance and is entirely advisory.

That distinction explains why the two files have had such different fates. A crawler that ignores robots.txt is misbehaving. A model that ignores llms.txt is simply doing what nobody promised it would not do.

Who actually uses it in 2026

Google is the clearest voice on this. Gary Illyes confirmed at Search Central Live that Google does not support llms.txt and has no plans to. John Mueller went further, comparing it to the old keywords meta tag and pointing out that server logs show AI services do not even request the file.

The objection is about incentives rather than effort. A signal that the site owner writes about their own site is trivially easy to game, which is precisely why the keywords tag was abandoned. Not everyone accepts the comparison. Search Engine Land argued that llms.txt is different in kind, because it points at real pages rather than asserting claims about them, and Search Engine Journal noted that Google’s own guidance shifts depending on which product you ask.

OpenAI has not committed either way. Its crawler documentation points site owners to robots.txt and never mentions llms.txt, though site owners do occasionally see AI bots requesting the file in their logs.

Anthropic and Perplexity come closest to genuine support. Both publish llms.txt files for their own documentation, and Perplexity has said it retrieves the file to help decide which pages to read first. Even so, no provider has confirmed that its assistant consults llms.txt at the moment it answers a user question. Publishing one and reading one are different things.

The place it demonstrably works is quieter than the AI search debate. Coding agents and IDEs such as Cursor, Continue and Cline, along with various MCP integrations, read llms.txt to pull a library’s documentation straight into context. If your audience builds software, that use case alone can justify the file.

What the adoption numbers show

Research through mid 2026 puts valid llms.txt files on roughly 5.86 to 10 percent of top sites, and that adoption is heavily concentrated in documentation and developer tooling rather than across the web generally.

The split by traffic tier is the interesting part, because it runs the opposite way to most technical SEO adoption. Mid-traffic sites are slightly more likely to ship the file than the largest domains.

One analysis went a step further and tested whether the file predicts anything at all. When llms.txt was removed as a variable from a model predicting how often a site gets cited by AI, the model became more accurate. In other words, the file behaved as noise rather than signal. That is the strongest evidence available today that it is not yet doing the job people hope it does.

How to write one

You can write a good llms.txt by hand in under an hour. The syntax is trivial. The work is deciding what deserves to be in it.

The parts that matter

Start with the H1, since it is the only line the spec actually requires. Follow it with a blockquote that explains in one or two sentences what your site is and who it serves. Then group your best pages under H2 headings, giving each link a short note that says what a reader gains from it. Those notes are what let a model choose between two pages that look similar.

What to leave out

The most common mistake is treating the file as a second sitemap and dumping every URL into it. That defeats the purpose. The value of llms.txt is curation, and a model working with limited context is better served by twenty pages that genuinely explain your business than by two thousand that merely exist.

When llms-full.txt is worth it

The full-text version earns its place on documentation sites, where someone genuinely wants to load an entire reference into a coding assistant. For a marketing site it is usually the wrong tool, because you end up publishing one enormous file that nobody reads and that you will forget to update.

Is it worth shipping

On the evidence available today, llms.txt will not increase how often AI systems cite you. Nothing credible shows that it does, and the one attempt to measure its predictive value found it made things worse. Anyone selling it as an AI visibility tactic is ahead of the facts.

That is not the same as saying do not bother. The calculation is about cost, and the cost is small:

• Ship it if you publish documentation. Developer tools, API references and knowledge bases are exactly where coding agents already read the file today.
• Ship it if the half day is genuinely spare. There is no downside beyond the time, and you are already compliant if a major assistant starts honouring it.
• Skip it if that half day has a better use. For most service and local businesses, the same afternoon spent improving the content itself will do more.

The trap is not the file. The trap is letting a cheap, tidy, technical task stand in for the harder work that actually moves AI visibility.

What actually earns AI citations

The anxiety behind llms.txt is well founded. AI answers are absorbing clicks that used to reach websites, and a 2026 field experiment found that AI Overviews cut organic clicks by 38 percent on the queries where they appear. Wanting a lever to pull is a rational response.

The lever is just not a file you write about yourself. Models assemble answers from sources they already trust, and that trust is built the slow way: pages other credible sites link to, a brand mentioned repeatedly in the same context, and writing clear enough to quote without ambiguity. A model reaches for the source it has seen corroborated elsewhere, not the one that described itself most helpfully.

That is the work behind our AI SEO service, earning the links and brand mentions on the publications these models actually read, so you appear inside the answer instead of underneath it. It is relationship-driven, it runs over months rather than afternoons, and it is genuinely better handled by people doing it daily. That is the part we do.

So put the file up if you have the time. Just be honest with yourself about which of the two jobs is actually going to get you quoted.