What is llms.txt?

llms.txt is a plain Markdown file you put on your website, usually at /llms.txt. It gives AI agents a short, curated map of the pages that matter. The proposal's author, Jeremy Howard, describes it as "A proposal to standardise on using an /llms.txt file to provide information to help agents use a website." (llmstxt.org)

The key word is proposal. llms.txt isn't an IETF or W3C standard. We haven't found any search engine or AI company saying it ranks or cites sites because they have one. Treat it as a cheap, useful courtesy to agents, not a traffic lever.

Why it exists

The proposal's reasoning: "web pages are built for people. An HTML page wraps its information in navigation, ads, and JavaScript, and converting it back into clean text is difficult and imprecise." Context windows are limited, so "Agents are best served by concise, expert-level information gathered in a single, accessible location." (llmstxt.org) It's used most for software documentation. The spec says coding agents follow it "to find API references and tutorials". It also suggests businesses, personal CV sites and schools.

The format, in order (v2)

From the spec's Format section (llmstxt.org):

1. An optional byte-order mark (BOM)
2. An H1 with the name of the project or site. This is the only required section.
3. A blockquote with a short summary
4. Zero or more paragraphs or lists (no headings) with more detail
5. Zero or more H2 sections, each containing a "file list": - [name](url): optional notes

So a file that is just # My Site is technically valid. A useful file has a summary and links.

What changed in v2 (August 10, 2026)

Per the Changes page:

• Files can live under a path. /docs/llms.txt covers everything under /docs/, and "the most specific file applies."
• Discovery links. rel="describedby" points from a page to the llms.txt covering it, and rel="alternate" type="text/markdown" points to a page's Markdown version. Both can be set as HTML link elements or as an HTTP Link: header.
• Two Markdown URL styles are allowed: page.html.md or page.md.
• "Optional" lost its special meaning. It's still a fine convention for secondary links, but it "no longer carr[ies] mechanical semantics."
• The llms_txt2ctx context-expansion tool is no longer part of the proposal.

Who uses it?

The v2 text says "thousands of sites publish an llms.txt file" (the author's claim; we haven't counted). It lists Mintlify, GitBook, Yoast SEO, AIOSEO and Wix as tools that generate one. Real files we fetched include llmstxt.org/llms.txt and the docs files of Anthropic and Stripe (see /llms-txt-examples). Chrome's Lighthouse includes an llms.txt audit in its experimental Agentic Browsing category (see /lighthouse-llms-txt).

Best practices (from the spec, plus our notes)

The spec's own guidance: "Use concise, clear language", give links "brief, informative descriptions", "Avoid ambiguous terms or unexplained jargon", and "Test your file by asking an agent questions about your content, giving it only your llms.txt as a starting point." Our additions (opinion, not spec):

• Link to pages an agent can actually read. The spec recommends Markdown versions where you have them.
• Keep it small enough to fit in context, and put detail behind links.
• Use absolute https URLs so the file works wherever it's copied.

Check or make yours

• Already have one? Paste it into the llms.txt checker.
• Starting from scratch? Use the llms.txt generator.

Leaflint is independent and isn't affiliated with, sponsored by or endorsed by any company, product or project named on this page. Names are used only to identify them; trademarks belong to their owners.

Guide draft: not indexed until Legal clears it.