llms.txt: What the File Is, Who Uses It and What It Demonstrably Does

What is llms.txt?

llms.txt is a Markdown file that gives language models and AI agents a curated overview of a website. It sits at /llms.txt or in a subpath such as /docs/llms.txt. It contains a short summary and links to the most important content. The short version is in our glossary entry on llms.txt.

Jeremy Howard published the proposal on llmstxt.org on 3 September 2024. His reasoning: HTML pages contain navigation, ads and JavaScript, and context windows are too small for entire websites. Agents work better with concise information in one place.

The file complements existing standards. robots.txt controls crawler access, and the sitemap lists all URLs for search engines. llms.txt selects instead and explains how to read the linked content.

Version 2 of the proposal adds two points. Pages should offer a clean Markdown version at the same URL with a .md extension. Link relations in the HTML or HTTP header (rel="alternate" and rel="describedby") help agents find these files.

How is an llms.txt file structured?

Only the H1 with the name of the site is required. All other parts are optional, but their order is fixed:

1. an H1 with the name of the project or site,

2. a blockquote with a short summary,

3. any paragraphs or lists without headings that add detail,

4. H2 sections with link lists in the format [name](URL), optionally with a note,

5. an “Optional” section with links an agent may skip when context is short.

# Example Software

> Example Software is accounting software for small businesses in the EU.

All prices exclude VAT and are in euros. The API is REST-based.

## Documentation

- [Getting started](https://example.com/docs/start.md): create an account and send a first invoice
- [API reference](https://example.com/docs/api.md): endpoints, authentication, limits

## Optional

- [Changelog](https://example.com/changelog.md)

Who uses llms.txt?

Developer documentation providers are the main adopters. According to version 2 of the proposal, “thousands” of sites publish an llms.txt file, and documentation platforms generate one automatically. TechTafel retrieved the following files on 13 November 2026:

• OpenAI for its developer documentation (developers.openai.com),

• Anthropic for the Claude documentation (docs.claude.com),

• Google for the Gemini API documentation (ai.google.dev),

• Stripe, Cloudflare, Vercel and Perplexity for their documentation.

Even so, the file is not widespread. SE Ranking found it on 10.13 percent of almost 300,000 domains analysed. Among sites with more than 100,000 visits, the share was slightly lower at 8.27 percent.

The files show how differently the format is used. Cloudflare’s llms.txt points to separate files per product, each listing the complete documentation. Stripe gives agents direct instructions, such as checking the npm registry for the current version before pinning a package version. There, the file works as a manual for coding agents rather than as a showcase for search engines.

What does llms.txt demonstrably do?

For Google Search, llms.txt does nothing. Google’s guide to generative AI features says site owners do not need machine-readable files, AI text files or Markdown, because Google Search does not use them. Maintaining llms.txt for other services is fine. It has no effect on visibility in Google Search.

There is no evidence for more AI citations. SE Ranking analysed almost 300,000 domains using Spearman correlation and an XGBoost model. No link appeared between llms.txt and how often a domain is cited by AI systems. Removing the variable even made the model more accurate. The authors stress that the results depend on the dataset. The article does not detail which AI systems were queried.

Chrome now checks the file. The Lighthouse audit in the Agentic Browsing category only flags server errors on retrieval. If the file is missing, Lighthouse rates the audit as not applicable and deducts nothing.

Who should create the file?

In our editorial view, llms.txt pays off mainly where agents read with a purpose: software documentation, API references and knowledge bases. There the file matches the purpose of the proposal, and the effort stays small (an hour is often enough).

For a company website or a blog there is no proven benefit. If you want to be visible in AI search, check the basics first: crawlability, robots.txt, clear answers in the text and structured data. Our article What is GEO? sets out the wider picture.

How to proceed:

1. Pick the 10 to 30 most important pages, such as product, pricing, documentation and contact.

2. Write a one-sentence note per page that says what it contains.

3. Place the file at /llms.txt following the format and serve it as UTF-8.

4. Watch the server logs to see whether bots fetch the file at all.

Only a measurement over several weeks shows whether the file changes your visibility in AI answers. Our comparison of AI visibility tools covers suitable products. Record a baseline before you publish the file, otherwise no effect can be detected.