What is llms.txt, and does it help SEO?
llms.txt is a proposed file that gives AI agents a short, hand-picked guide to your website. It is not an official standard, and Google says you do not need AI text files to appear in its AI features. Here is what it is, what goes in it and when it is worth adding.
llms.txt is a proposed standard: a Markdown file at the root of a website, /llms.txt, that gives AI agents a short, hand-picked guide to the site and links to clean versions of its most useful pages. It is not an official web standard. Google says you do not need AI text files to appear in its AI features, so it is not an SEO lever for Google. For documentation-heavy sites it is cheap to add and can help the agents that do read it.
Where llms.txt comes from
The idea was proposed by Jeremy Howard in a document first published on September 3, 2024, at llmstxt.org. The page now carries version 2 of the proposal, modified August 10, 2026. The problem it addresses is practical: web pages are built for people, wrapped in navigation, ads and scripts, and an AI agent wastes time and context trying to pull the useful text back out.
According to the v2 proposal, thousands of sites now publish one, documentation platforms generate them automatically, and AI labs publish them for their own developer docs. OpenAI's developer site, for example, links to its own llms.txt. That is adoption by publishers. It is not the same as search engines using the file to rank anything.
What goes in an llms.txt file
The proposal sets out a simple structure, in this order: a heading with the name of the site (the only required part), a one-paragraph summary in a blockquote, optional notes, then sections of links to the pages that matter, each with a short description. A section named Optional holds links an agent can skip when it needs to save space. A small business version might look like this:
# Example Plumbing Co. > Licensed plumbers serving Orange County, CA. > Emergency repairs, water heaters and repiping. ## Services - [Water heaters](https://example.com/water-heaters.md): repair and replacement - [Repiping](https://example.com/repiping.md): whole-home copper and PEX ## Optional - [About us](https://example.com/about.md)
The proposal also suggests offering clean Markdown versions of key pages at the same address with .md added, so the links in the file lead to text an agent can read without the clutter.
llms.txt examples
Real files are the quickest way to understand the format. These were all live on September 13, 2026, and notice what they have in common: almost all of them are documentation for developers.
- OpenAI Developers: an index of its API and product documentation.
- Anthropic's developer documentation: a long overview of its docs.
- Stripe's documentation: guidance for developers and agents building with Stripe.
- Vercel: a short summary of the platform with links to its docs.
- FastHTML's documentation: the example the proposal itself points to, placed under /docs/ to cover only the documentation.
How to add an llms.txt file
- Decide what it covers. The root (/llms.txt) covers the whole site; a file under a path, such as /docs/llms.txt, covers only that section.
- Write the top. A heading with the site's name, then a short summary of what the site or business is.
- List the pages that matter, each as a link with one line saying what it covers. Put secondary pages under a section called Optional.
- Link to clean text where you can. The proposal suggests Markdown versions of pages at the same address with .md added.
- Publish it as plain text at the chosen path, then open the address in a browser to check it loads.
- Put a reminder in the calendar to update it whenever key pages change.
llms.txt, robots.txt and sitemaps
The three files sit side by side at the root of a site but do different jobs:
| File | What it does | Who uses it |
|---|---|---|
| robots.txt | Tells crawlers which parts of a site they may access. | Search engines and AI crawlers, as documented by each. |
| sitemap.xml | Lists the URLs you want search engines to know about. | Search engines. |
| llms.txt | Gives AI agents a short, hand-picked guide with links to clean text. | Agents and tools that choose to read it. A proposal, not a standard. |
If you are setting up the classic files for the first time, our guide to sitemap generators covers the sitemap side.
Does llms.txt help SEO?
For Google, no. Its page on AI features and your website says you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode, and that there is no special schema to add. What gets a page into those features is the same SEO foundation as the rest of Search.
For other AI search products, the documentation we checked on September 13, 2026 does not list llms.txt as a way in. OpenAI's crawler overview points to robots.txt settings for OAI-SearchBot, and Perplexity's crawler guide points to robots.txt and its published IP ranges for PerplexityBot.
So treat llms.txt as a courtesy to agents, not a ranking tactic. Anyone selling it as the key to AI visibility is selling the wrong thing.
When it is worth adding
- Documentation-heavy sites: software products, APIs and technical manuals, where coding agents are the main readers the proposal describes.
- Sites with a lot of structure: where a short, hand-picked map genuinely helps an agent find the right page.
- Anyone who can keep it current: a stale llms.txt points agents at old information.
For most small business websites, the better investment is the SEO foundation that every AI search product relies on: pages that can be crawled, clear answers and accurate facts. An llms.txt file generator can draft one in minutes, but read and edit what it produces; a generated list of every URL defeats the point of a hand-picked guide. Our AI search optimization work starts with those fundamentals.
Frequently asked questions
What is llms.txt?
llms.txt is a proposed Markdown file placed at a website's root, /llms.txt, that gives AI agents a short, hand-picked overview of the site and links to clean versions of its key pages. Jeremy Howard proposed it in September 2024; it is a proposal, not an official web standard.
Will an llms.txt file help my SEO?
Not with Google. Google says you do not need new AI text files to appear in AI Overviews or AI Mode. It can help AI agents that choose to read it, especially on documentation sites, but it is not a ranking lever.
What should I do with an llms.txt file?
If you publish one, place it at /llms.txt, start with an H1 naming the site and a short summary, then list the pages that matter most with a line of description each. Link to clean Markdown versions if you can, and update it when your site changes.
Is llms.txt an official standard?
No. It is a proposal published at llmstxt.org, now in version 2. Many sites and documentation platforms have adopted it, but it has not been adopted as an official web standard, and Google documents no role for it in its AI features.
Sources
Checked September 13, 2026. The proposal and the platforms' documentation change. The linked page is always the final word.
- llmstxt.org: The /llms.txt file, v2 (Jeremy Howard; published September 3, 2024; modified August 10, 2026): the proposal, its format and adoption
- Google: AI features and your website (last updated December 10, 2025): no AI text files needed for Google's AI features
- OpenAI: Overview of OpenAI crawlers: how ChatGPT search access works
- OpenAI Developers: llms.txt: an example of a published llms.txt
- Anthropic, Stripe, Vercel and FastHTML llms.txt files: further examples, each linked above
- Perplexity: Perplexity crawlers: how Perplexity search access works
Put the AI search question to our founders.
Tell us what you want AI tools to understand about your business. We will discuss what is worth doing first and what can wait.
Talk to our founders