Glossary · Glossary
llms.txt
llms.txt is a community Markdown file that maps useful pages for AI agents. It is not a robots.txt replacement or a Google ranking signal.
llms.txt is a proposed Markdown file, usually published at /llms.txt, that lists the pages a publisher wants an AI agent to treat as the useful map of a site. The community specification asks for a project title and then optional sections that point to documentation, guides, or other high-value URLs.
It is a curated on-demand map, not a crawl-control file and not a search ranking system. A team can publish it, skip it, or keep a short version. None of those choices, by itself, proves that Google, ChatGPT, or any other answer surface will read the file or cite the listed pages.
Why it matters
Operators keep hearing that they should “add llms.txt for AI SEO.” The useful question is narrower: do you need a maintained Markdown map, or do you need access rules and citable pages?
If public guides are blocked in robots.txt, or the HTML never becomes crawlable, a curated list cannot repair that. If the listed pages are thin or undated, the file only advertises weak sources.
The file can still be useful when a site has many URLs and a human editor can point agents at a short set of durable pages. That is an editorial job. It is not a substitute for crawling, indexing, or citation work.
How it differs
Robots.txt tells compliant crawlers which URL paths they may request. It is an access file. llms.txt does not grant or deny crawl permission.
A sitemap lists indexable URLs for discovery. llms.txt is supposed to be smaller and curated. A sitemap can include every public guide. An llms.txt that repeats the whole sitemap is no longer a map.
Meta robots and noindex are page-level indexing controls. They are not Markdown maps.
Example
# Example Lighting Co.
> Field guides for export lighting buyers.
## Guides
- [Category demand](https://www.example.com/guides/category-demand/): When a category can support a page-asset system.
- [Page assets](https://www.example.com/guides/page-assets/): What makes a URL worth citing.
## Optional
- [Contact](https://www.example.com/contact/): Sales path, not a source page.This example points agents at two operator pages and labels the contact URL as a next step, not a source. The file is only as good as those destinations.
How teams use it
Teams consider llms.txt when they already have a small set of durable public pages and want a human-readable index for agents. A practical review asks:
| Check | Question |
|---|---|
| Maintenance | Will someone update the file when URLs move? |
| Curation | Does it list the pages you would hand a new analyst, or every URL? |
| Access | Are those URLs allowed in robots.txt and reachable? |
| Claim | Are you treating the file as a map, or as a ranking lever? |
If the team cannot maintain the list, skip the file. A stale map is worse than no map.
Common misunderstanding
Publishing llms.txt is not proof that an assistant will retrieve, quote, or cite those URLs. It is also not a Google-endorsed citation or ranking control. Access, training collection, and user-triggered fetches are separate jobs, usually handled by crawler tokens and robots rules, not by this Markdown file.
Read next
Use these glossary paths to move from the definition into adjacent concepts, topic clusters, and operator guides.