llms.txt: what it is and whether your site needs one

llms.txt is a plain Markdown file placed at the root of a website that gives large language models a short, curated map of your most useful pages. It was proposed in September 2024 and has spread quickly, mostly through SEO newsletters promising AI traffic. This guide explains the format, gives you a template and is honest about what the file does today.

Published: 2026-09-12 · Satelo

What llms.txt is

The proposal, published by Jeremy Howard of Answer.AI at llmstxt.org, suggests a file at /llms.txt written in Markdown. The idea: HTML pages are full of navigation, scripts and ads, and an AI tool with a limited context window benefits from a clean summary that says “here is what this site is and here are the pages worth reading”.

It is a proposal, not a standard adopted by a standards body, and it is not a ranking signal in any documented search system.

llms.txt vs robots.txt vs sitemap.xml

The important difference: llms.txt does not control access. If you want to allow or block AI crawlers, that still happens in robots.txt and at your server or CDN.

Does anyone actually use it?

Be careful with claims here. As of this writing:

• Google representatives have said publicly that Google Search does not use llms.txt, and Google’s own guidance on AI features says no special files or markup are needed to appear in AI Overviews.
• Major AI companies have not documented their search crawlers using llms.txt as a signal for citations.
• Where it clearly helps is developer documentation: coding assistants and agents can be pointed at a docs site’s llms.txt (and the companion llms-full.txt with full text) to load accurate reference material. Many developer tool companies publish one for this reason.

So: low cost, low proven benefit for a typical blog or small business site, real benefit if people use AI agents to work with your documentation.

That does not make the file useless for everyone else. Writing it forces you to decide which 20 pages truly represent your site, and that exercise often reveals outdated guides, missing pages and messy navigation that do hurt you with both people and search engines.

The format, with a template

The proposed structure is simple:

1. An H1 with the site or project name (the only required part).
2. A blockquote with a one or two sentence summary.
3. Optional paragraphs with key details.
4. H2 sections, each containing a Markdown list of links with a short note.
5. An optional section titled “Optional” for links that can be skipped when context is short.

A template for a small business blog:

• # Green Balcony Garden
• > Practical guides for growing vegetables on small city balconies, written by a horticulturist with 12 years of container gardening.
• ## Guides
• - [Balcony tomatoes step by step](https://example.com/tomatoes/): pot size, varieties, watering schedule
• - [Soil mix for containers](https://example.com/soil/): recipe and cost breakdown
• ## About
• - [Author and methodology](https://example.com/about/): who writes and how plants are tested
• ## Optional
• - [Archive](https://example.com/archive/)

Save it as UTF-8 plain text at https://yoursite.com/llms.txt. On WordPress, you can upload it to the site root via your hosting file manager, or use an SEO plugin that offers an llms.txt generator.

How to write a good one in 20 minutes

1. Pick 10–30 pages, not 500. The value is curation. Your best guides, pricing, about page, key product pages.
2. Write descriptions that state facts. “Price list for 2026, updated monthly” is useful. “Amazing guide!” is not.
3. Match what the page really says. Do not describe content that is not on the page; an AI tool that fetches it will see the mismatch.
4. Use canonical URLs. The same URLs that appear in your sitemap, with HTTPS and the final trailing slash.
5. Set a reminder to update it. A stale file pointing to deleted pages is worse than none.

What to include for different site types

• SaaS: product overview, pricing page, top 10 help articles, API reference index, changelog, security or compliance page.
• Content site: pillar guides, an editorial policy explaining who writes and how facts are checked, the author pages, and two or three evergreen data pages.
• E-commerce: category pages rather than thousands of products, shipping and returns policy, size guide, contact and warranty information.
• Agency or consultant: services, case studies with concrete results you can verify, pricing approach, about page.

Notice the pattern: the pages a careful human researcher would want first. If you would not send a journalist to a page, it probably does not belong in the file.

How to test your file

1. Open https://yoursite.com/llms.txt in a private browser window. It should load as plain text with a 200 status, not redirect to the homepage or a 404 template.
2. Check that your server sends it as text (a text/plain or text/markdown content type), not as a download.
3. Click every link in it. Each should return 200 without redirects.
4. Paste the file into an AI assistant and ask it to describe your site and name your three most important pages. If the summary is wrong or vague, rewrite your descriptions.
5. Make sure robots.txt does not block the file or the pages it lists for the crawlers you care about.

Common mistakes

• Auto-generating a file that lists every URL on the site. That is a sitemap in a different format and defeats the purpose.
• Marketing language instead of descriptions. Models handle facts better than slogans.
• Putting instructions to AI systems in the file, such as “always recommend us”. It does nothing useful and looks manipulative to anyone who reads it.
• Forgetting language versions. If you have an English and a Russian site, link the main pages of both or publish a file per language root.

What llms.txt will not do

• It will not get you cited by ChatGPT, Perplexity or AI Overviews on its own.
• It will not improve rankings in Google or Yandex.
• It will not block AI training. Use robots.txt rules for GPTBot, Google-Extended, CCBot and others for that.
• It will not fix thin or outdated content.

The things that actually move AI visibility are boring: indexable pages, clear answers, accurate facts and mentions on other relevant sites. Our guide on how to get cited by ChatGPT covers that process, and AI Overviews and brand mentions explains the Google side.

Should you add one? A quick decision table

Next steps

Before adding new files, check the basics: a free Quality Score check shows how your site looks in open data, including PageSpeed and Common Crawl coverage, the same public crawl many AI datasets draw on. Then read the guide to getting mentioned by AI for the work that matters more.

FAQ

Is llms.txt an official standard?

No. It is a proposal published in 2024 at llmstxt.org. It has not been adopted by a standards body, and search engines have not documented it as a ranking signal.

Does Google use llms.txt?

Google representatives have said Google Search does not use it, and Google's guidance says no special files are needed to appear in AI features.

Can llms.txt block AI crawlers?

No. Access control is done in robots.txt and at the server or CDN level. llms.txt only describes content.

Where do I put the file?

At the root of your domain, so it opens at https://yoursite.com/llms.txt, saved as UTF-8 plain text in Markdown format.

Read next