Research · Phase 1 · benchmark v1.0.0 · run 6 September 2026
What llms.txt does to the cost of reading a website
We sent a small retrieval agent to 20 websites with five questions each and counted the tokens it burned before it could answer. Four routes: the pages the conventional way, the sitemap, llms.txt, and llms-full.txt.
A small llms.txt made a site roughly twice as cheap to read and more likely to answer at all. A large one made it nine times more expensive than having none.
By route, taken flat
Median tokens to answer, over attempts that produced an answer. From the benchmark's own baseline file, not retyped.
| Route | Applicable | Answered | Median tokens |
|---|---|---|---|
| Conventional website | 100 | 55 | 2,294 |
| Sitemap | 100 | 44 | 21,850 |
| llms.txt | 50 | 29 | 21,196 |
| llms-full.txt | 10 | 0 | — |
Read flat, llms.txt costs about eight times more than crawling the site. That headline is real and it is not the finding. It hides one variable.
The finding: file size decides the outcome
The ten sites that publish an llms.txt, ordered by the size of that file. Median tokens to answer on questions both routes answered.
| Site | llms.txt size | Website route | llms.txt route | Cheaper |
|---|---|---|---|---|
| anchorbrowser.io | 72 | 2,414 | 72 | llms.txt |
| answer.ai | 173 | 3,067 | 173 | llms.txt |
| annoto.net | 997 | 3,625 | 1,498 | llms.txt |
| agno.com | 1,446 | 2,798 | 1,446 | llms.txt |
| amplemarket.com | 4,333 | 6,945 | 4,333 | llms.txt |
| alexop.dev | 8,636 | — | — | no shared answer |
| activepieces.com | 21,196 | 2,409 | 21,196 | website |
| akool.com | 23,337 | 4,842 | 23,494 | website |
| ably.com | 41,015 | 1,145 | 41,761 | website |
| abstractapi.com | 42,771 | 1,167 | 42,771 | website |
llms.txt under 10,000 tokens · n=13
2.06x cheaper than crawling
llms.txt over 10,000 tokens · n=16
0.11x, about 9x more expensive
Perfectly monotonic. Every file under roughly 5,000 tokens wins. Every file over roughly 21,000 loses. Nothing crosses. The websites cost about the same in both groups (median 3,124 tokens against 2,409), so the file is doing the work, not the homepage.
The mechanism is plain. The llms.txt route answers in a median of one request at 100% useful content: it reads the file and the file contains the answer. Its cost is therefore the file size. A 42,000-token llms.txt is not an index. It is a second copy of the website.
What else the run showed
- No route bought cheapness with wrong answers. Zero incorrect answers on any route; llms.txt was correct on 28 of 29 answers it produced.
- llms.txt found the price more often. "How much does it cost?" was answered from llms.txt 60% of the time against 45% from the site, the question websites hide most.
- llms-full.txt failed completely. Only two of the twenty sites publish one, and both exceeded the 200,000-token budget outright (209,694 and 331,575 tokens). Reading either costs more than reading the whole site it describes. Two files is not a sample; it is a warning.
- Sitemaps mostly could not answer. 44 of 100 sitemap attempts stopped because nothing in the sitemap looked like the answer. The site listed its URLs, but not usefully.
Defects in this run, disclosed
- The sample is 20 sites that all begin with "a". The registry had scanned only the start of its alphabetical seed list, and the set skews to AI and developer SaaS. Adequate for finding methodology faults. Not a sample anything may be generalised from. Phase 2 is a matched set of 166 domains across eleven verticals, 87 with an llms.txt and 79 without, every one verified live. Selected, not yet run.
- The judge is too strict on action questions. "How do I get started?" and "How do I contact support?" are answered by a button or a form, not a quotable sentence, and the judge refused them. Those two rows are not quoted here. The main finding is a within-question ratio between two routes measured by the same judge, so it cancels out.
- The judge has not been calibrated against human labels. So the efficiency figures stand, and the answer-quality parity claim is provisional until that line carries a number.
- One attempt of 400 tokenised a PNG as text. It stopped at the token limit and never answered, so no figure above was inflated. Fixed in v1.1.0.
- The last domain may have hit API quota exhaustion. It is a control site with no llms.txt, so it does not touch the size-band finding. Its row is treated as unreliable.
- The first site tested ran against our own thesis. ably.com's 41,015-token llms.txt lost to its homepage. Two pilot sites pointed at the conclusion "llms.txt does not reduce retrieval cost". It took the other eighteen to see the variable that actually mattered.
What this does not say
- That llms.txt improves AI rankings or visibility.
- That ChatGPT, Claude, Gemini or any other assistant reads llms.txt.
- That our token counts equal the internal cost incurred by any AI company.
- That fewer tokens is automatically better, absent answer quality.
- A single unexplained 0 to 100 "AI readiness" score derived from these metrics.
This list is copied from the methodology the benchmark ships with. Retrieval here is our own agent-like system; results describe it, not any provider's crawler.
Cite this
llmstxt.studio (2026). AI Read Cost, phase 1: what llms.txt does to agent token spend. 20 domains, 400 attempts, benchmark v1.0.0, run 6 September 2026. https://llmstxt.studio/research/ai-read-cost
Methodology v1.1.0 is published in full and rendered from the same file the engine ships with. Phase 2 (166 matched domains, per-question judging under v1.1.0) will not be numerically comparable with phase 1; that is what versioning is for. It has been selected and verified but not yet run; this page will say when it has.