Quick answer: llms.txt is a proposed Markdown file, placed at yourdomain.com/llms.txt, that gives AI tools a short, curated map of a website’s most useful content. It’s easy to create and does no harm, but there’s no evidence that it improves visibility in AI search today. Google has said it isn’t needed for its AI features, and no major AI search engine has confirmed using it for ranking or citations. Treat it as an optional extra, not an SEO lever.
Few files have had as much hype in SEO circles as llms.txt. Plugins add it automatically, agencies sell it as an “AI SEO” service, and some guides present it as the robots.txt of the AI era. The reality is more modest. This guide explains what the file is, what the major platforms have actually said about it, and whether it’s worth ten minutes of your time.
What is llms.txt?
llms.txt was proposed by Jeremy Howard of Answer.AI in September 2024. The idea is simple: web pages are built for people, full of navigation, scripts and ads, which makes them awkward for language models to read. An llms.txt file offers a clean, Markdown-formatted entry point that points AI agents to the pages that matter, ideally in plain Markdown versions as well.
According to the specification, the file is meant mainly for inference, when an AI agent is helping a user with a question, rather than for training. It’s designed to sit alongside robots.txt and your XML sitemap, not replace them.
| File | What it does | Who reads it |
|---|---|---|
| robots.txt | Tells crawlers what they may and may not access | Search engines and most AI crawlers |
| sitemap.xml | Lists the URLs you want discovered and indexed | Search engines |
| llms.txt | Gives AI tools a curated, readable summary of your key content | No major AI search engine has confirmed using it |
What goes in an llms.txt file
The format is plain Markdown in a fixed order. Only the first element is required:
- An H1 title with the name of your site or project (required).
- A blockquote with a one- or two-sentence summary (optional).
- Short details: paragraphs or lists explaining what the site covers, with no headings (optional).
- H2 sections of links, each a list in the form
[Page name](URL): short note(optional). - An “Optional” section for secondary links an AI tool can skip when it needs less context.
A simple version for a content site might look like this:
# PostDune
> Guides to AI search, SEO and technology, plus AI news and analysis.
## Core guides
- [Complete guide to AI search](https://postdune.com/complete-guide-ai-search-2026/): how AI search engines find and cite sources
- [Generative engine optimisation](https://postdune.com/generative-engine-optimization-guide/): how to get cited in AI answers
## Tools
- [Free SEO tools](https://postdune.com/tools/): robots.txt, meta tag, redirect and sitemap checkers
## Optional
- [Contact PostDune](https://postdune.com/contact-us/)The proposal also suggests offering Markdown copies of important pages at the same URL with .md added, for example page.html.md. Most sites skip this part, and it’s the most work to maintain.
What Google and AI companies have said
This is where expectations need adjusting.
- Google: In June 2025, Google’s John Mueller wrote that “no AI system currently uses llms.txt.” In January 2026, asked whether an llms.txt file on a Google developer site meant Google endorsed the format, he answered: “to be direct, no.” Reporting on Google’s 2026 guidance for AI search features says llms.txt is not needed to appear in AI Overviews or AI Mode.
- Other Google teams: Some Google developer sites, such as ai.google.dev, have published llms.txt files for their documentation. Mueller has said this doesn’t signal a Search endorsement.
- AI search engines: OpenAI, Anthropic and Perplexity have published detailed documentation on how their crawlers work and how to control them with robots.txt. None of them, as of this writing, documents llms.txt as a factor in whether a site is crawled, ranked or cited in their search answers.
In short, the file is mainly useful where AI agents are reading documentation, such as developer docs that a coding assistant may look up. For an ordinary blog or business site, there is no documented benefit for search visibility.
Should you create one?
A sensible way to decide:
| Your site | Recommendation |
|---|---|
| Developer docs, APIs, software products | Worth doing. AI coding assistants and agents may read it, and it helps keep their answers about your product accurate. |
| SaaS or product sites with complex documentation | Reasonable, low-cost extra if you can keep it up to date. |
| Blogs, news and content sites | Optional. It won’t hurt, but don’t expect any measurable traffic gain. |
| Local businesses and small brochure sites | Low priority. Your time is better spent on your Google Business Profile, reviews and clear service pages. |
If you do add one, keep it short, link only to pages that are live and indexable, and update it when your key pages change. An out-of-date llms.txt that points to deleted or noindexed pages is worse than none.
How to add llms.txt to a WordPress site
- Write the file in a plain text editor using the structure above and save it as
llms.txt. - Upload it to your site’s root folder (the same place as robots.txt) using your host’s file manager or FTP. Some SEO plugins can also generate one for you; check what they include before turning it on.
- Open
yourdomain.com/llms.txtin a browser to confirm it loads as plain text. - Check that every link in it returns a 200 status. Our free HTTP Status Checker makes this quick.
What actually helps AI search visibility
If your goal is to be cited by ChatGPT, Perplexity, Claude or Google’s AI features, these steps have far more evidence behind them than llms.txt:
- Let AI search crawlers in. Make sure robots.txt allows OAI-SearchBot, Claude-SearchBot and PerplexityBot. Our guide to allowing or blocking AI crawlers lists the exact lines.
- Get indexed by Google and Bing. Several AI tools draw on traditional search indexes, so basic indexability still matters.
- Answer questions directly. Clear headings, short direct answers and tables are easier for AI tools to quote. See our answer engine optimisation guide.
- Build real authority. Original information, named authors and mentions on trusted sites matter more than any file. Our generative engine optimisation guide covers this in depth.
Frequently asked questions
Does llms.txt improve rankings in ChatGPT or Google AI Overviews?
There’s no evidence that it does. Google has said its AI features don’t need it, and no major AI search engine documents it as a ranking or citation factor.
Is llms.txt an official web standard?
No. It’s a community proposal from Answer.AI, not a standard from a body like the IETF or W3C.
Can llms.txt block AI crawlers?
No. It only suggests content for AI tools to read. To allow or block crawlers, use robots.txt.
Can llms.txt hurt my SEO?
Not directly. The main risk is wasted effort, or an outdated file that points AI tools to old or broken pages.
What is llms-full.txt?
Some sites also publish an llms-full.txt containing the full text of their key pages in one file. It isn’t part of the core proposal, and the same lack of evidence for search benefits applies.
Sources
- llmstxt.org: The /llms.txt file (specification)
- Answer.AI: /llms.txt, a proposal to provide information to help LLMs use websites
- Search Engine Roundtable: Google says no AI system currently uses llms.txt (June 2025)
- Search Engine Roundtable: Google Search team does not endorse llms.txt (January 2026)
