What is llms.txt, and do you need one?

llms.txt is a proposed standard: a Markdown file at /llms.txt that gives AI models a short description of your site and links to its most useful pages. It is cheap to add and harmless, but no major AI search engine has confirmed using it, and Google says AI text files are not needed for AI Overviews. Treat it as optional housekeeping, not a ranking factor.

Updated · Citable

What llms.txt is

The proposal at llmstxt.org describes a file in Markdown, served at the root of a site, that helps language models use the site at inference time: when a model or agent is asked about the site and needs to find the right pages quickly, without parsing navigation, ads and scripts.

A related convention is offering a plain Markdown version of each page (for example page.md next to page), and an llms-full.txt with the full text of the docs in one file.

The format

llms.txt (shortened)
# Citable

> Press release distribution to 500+ news sites and directory launches
> on up to 100 startup directories, each paid once.

## Product
- [Pricing](https://citable.sh/pricing): both packages and what each includes

## Guides
- [How to get cited by ChatGPT](https://citable.sh/guides/how-to-get-cited-by-chatgpt): crawler access, indexing and mentions

## Optional
- [Glossary](https://citable.sh/glossary): AI search, PR and SEO terms

Rules from the proposal: one H1 (required), an optional blockquote summary, optional free text, then H2 sections containing lists of [name](url): notes. A section called "Optional" marks links that can be skipped when context is short. You can see Citable's full file at /llms.txt.

Does anyone use it?

Adoption on the publishing side is broad, especially among developer tools and documentation platforms, many of which generate it automatically. Use on the reading side is less clear:

  • Google says you do not need to create new machine-readable files or AI text files to appear in AI Overviews or AI Mode, and Google representatives have said its systems do not use llms.txt.
  • OpenAI, Anthropic and Perplexity document their crawlers and robots.txt rules, but none of their crawler documentation lists llms.txt as an input.
  • Coding assistants and agents are the clearest users: a developer can point an agent at a docs site's llms.txt to load the right pages.

Should you add one?

If you run documentation or an API, yes: it helps agents and developers, and costs an hour. If you are a marketing site, it is fine to add and unlikely to change your AI visibility on its own. Spend the time on what assistants demonstrably use: crawler access, pages that rank, quotable content and third-party mentions.

Questions people ask

Is llms.txt like robots.txt?

No. robots.txt tells crawlers what they may fetch. llms.txt is a reading guide for models, with no access rules.

Will llms.txt get me into ChatGPT answers?

There is no evidence that it does. ChatGPT search finds pages through its crawler and search providers.

Sources

  1. llmstxt.org: The /llms.txt file proposal
  2. Google Search Central: AI features and your website

Your brand on AP and USA Today, this week.

220M+ potential reach on 500+ news sites, indexed by search engines and ready for AI citations. Pay once, write it or let AI write it, and submit. Live within 48 hours of submitting, or your money back.