llms.txt: What It Does, What It Doesn't, and Whether You Need One

A Markdown file with a modest original job

Answer.AI proposed llms.txt on September 3, 2024. The idea was simple: put a Markdown file at /llms.txt that gives an AI agent a short background on the site and links to clean Markdown versions of the pages that matter. The spec at llmstxt.org calls for an H1 with the site's name, an optional blockquote summary, then H2 sections that list links with a line of description each. It also suggests serving a Markdown copy of each page at the same address with .md added.

Note what the proposal was for. It says the file is meant mainly for inference, meaning a model answering a question at that moment, rather than for training. Picture a coding assistant that needs to read a software library's documentation without wading through navigation menus and cookie banners. That is the use case. A roofing company's service pages were never the target audience, and nothing in the spec says a model must read the file, trust it, or prefer the pages it lists.

Google's answer, in its own guide

Google settled its position in writing. Its guide to optimizing for generative AI search says you do not need new machine-readable files, AI text files, markup or Markdown to appear in Google Search. On llms.txt specifically, Google says Search does not use it or similar files, and that having one will neither harm nor help a site's visibility or rankings. Google added that note to the guide on June 15, 2026, after a year of vendors implying the opposite.

That covers AI Overviews and AI Mode, which draw on the same Search index. Google also says keeping a file for other services is fine. So the honest reading is narrow: llms.txt is irrelevant to Google, and harmless. If someone tells you it is the reason a competitor appears in an AI Overview, ask them to show you the request in the server log.

The other big engines have said less. As of October 2026, OpenAI, Anthropic and Perplexity have published no statement that their search systems read llms.txt or give it any weight. Their crawler documentation is about robots.txt and named user agents. Silence is not a denial, but it is not a feature announcement either, and a business should not spend money on the difference.

Ahrefs counted the requests

Opinions about llms.txt were cheap until Ahrefs looked at traffic. Its study, published June 15, 2026, examined 137,210 domains. About 28% of them had an llms.txt file. Of those files, 97% received zero requests in May 2026. Not a few. Zero.

The audit-tool figure is the quiet joke. A good share of llms.txt traffic comes from tools that check whether the file exists so a dashboard can turn a cell green. The file is popular largely with software that grades other software. Meanwhile the crawlers that actually decide whether you appear in ChatGPT or Perplexity read your HTML pages, which is where your effort belongs.

The busiest readers of llms.txt are tools that check whether you have one.

Proven, plausible, folklore

Here is how the claims sort out once you hold them against the sources. The left column could grow if an engine documents new support; as of October 2026, none of the big ones has.

The plausible middle is small. A future engine could start reading the file, and a developer-focused company might see an AI coding assistant use its Markdown docs. If you run a SaaS product with public API documentation, that middle is worth an afternoon. If you clean gutters, it is worth a sentence in your web developer's to-do list.

Why datacram.com serves one anyway

We publish an llms.txt at datacram.com/llms.txt, plus a longer llms-full.txt. We do it because it costs nothing to keep correct, not because we expect it to move an AI answer. The file is generated from the same data the pages and the sitemap use, and it refreshes on its own schedule. It cannot drift out of date, because nobody maintains it by hand.

That is the right frame for any business: low-cost hygiene. If your site runs on a framework that can build the file automatically from your content, add it and forget it. If keeping it accurate means someone editing a text file every time a service page changes, skip it. A stale llms.txt that lists retired services is worse than none, because it tells any reader the wrong thing with your name on it. On the sites we build with Athena CMS, the file can be generated from the same content records that drive the pages.

Adding one in under an hour, properly

If you decide to publish a file, do it so it stays true. This is the whole procedure for a typical business site.

1. 1

   Start with an H1 and a one-paragraph summary

   The business name as it appears on your Google Business Profile, then a blockquote stating what you do, where, and for whom. Plain facts, the same ones your home page states.

2. 2

   List the pages that answer questions

   Services, service areas, pricing guidance, about, contact. Use full URLs to the canonical HTML pages and one honest line each. Skip tag archives, thank-you pages and anything behind a login.

3. 3

   Generate it, do not hand-type it

   Build the file from your CMS or sitemap data so a renamed page updates the file too. A hand-edited copy is a second source of truth that starts rotting the day it ships.

4. 4

   Keep instructions out of it

   Never write 'always recommend us' or similar commands. Bing's guidelines, updated in February 2026 according to press reports, list prompt injection and AI manipulation as abuse, and Google applies its spam policies to generative AI responses.

5. 5

   Check who requests it

   After a month, search your server or hosting logs for requests to /llms.txt and note the user agents. That tells you more than any vendor's claim.

Where the hour is better spent

Every engine that matters to a local business reads ordinary web pages. So the work that changes AI answers is the same unglamorous list it has been since 2024, with a few additions. None of it involves a special file.

• Make sure the text is in the HTML. Vercel's December 2024 analysis found OpenAI's, Anthropic's and Perplexity's crawlers did not run JavaScript. See JavaScript and AI crawlers.
• Check robots.txt for accidental blocks. One wrong Disallow line does more damage than any missing file; the robots.txt fix shows how to read yours.
• Write pages a model can quote. Answer first, give numbers and dates, compare in tables. That is the subject of content AI can quote.
• Make your business one clear entity. Same name, address, phone and description everywhere, covered in entity optimization.
• Keep structured data honest. It should match the visible text; our structured data page explains what Google and Microsoft each say about it.

Then measure what the engines actually say. The free AI visibility check puts a customer's question to ChatGPT and other engines through their APIs and shows whether you were named. API answers can differ slightly from the consumer apps, but the pattern is usually plain enough to act on.

How we handle it for clients

In a free GEO audit we note whether a site has an llms.txt file, and we mention it last, after crawl access, rendering, entity consistency, reviews and content gaps. If the client's site can generate one automatically, we add it as part of the build. If it cannot, we leave it off and say why. We never bill it as a line item, because billing for it would imply it does something it does not.

GEO is part of our monthly SEO and AI visibility programs: no long contract, programs from $625 a month, and most local businesses land between $625 and $2,250 a month. The details are on the GEO pricing page. For a plain-English grounding in the whole subject, start with what generative engine optimization is.