What Is llms.txt?
llms.txt is a plain text file in markdown format that sits at your website's root -- yourdomain.com/llms.txt. It tells large language models who you are, what your site is about, and which pages matter most.
Think of it as your website's resume for AI. A human visits your site and browses around. AI visits your site and reads your llms.txt. Same purpose -- different audience.
The specification was created by Jeremy Howard, co-founder of Answer.AI and fast.ai. It's published at llmstxt.org and is an open standard anyone can implement. No proprietary lock-in. No vendor dependency. Just a markdown file.
# Portland Sourdough Co.
> Family-owned artisan bakery in Portland, Oregon. Specializing in naturally leavened sourdough breads and seasonal pastries since 1987.
## Products
- [Sourdough Breads](https://portlandsourdough.com/breads): Our signature loaves -- country, rye, whole wheat, and seasonal varieties
- [Pastries](https://portlandsourdough.com/pastries): Fresh-baked croissants, danish, and morning buns daily
- [Wholesale](https://portlandsourdough.com/wholesale): Bulk ordering for restaurants and cafes
## About
- [Our Story](https://portlandsourdough.com/about): Three generations of bakers, same starter since 1987
- [Locations](https://portlandsourdough.com/locations): Two Portland locations -- NW 23rd and SE Division
## Optional
- [Blog](https://portlandsourdough.com/blog): Baking tips, recipes, and seasonal updates
- [Catering](https://portlandsourdough.com/catering): Event catering and custom bread programsA complete llms.txt file for a local bakery. Any AI tool that reads the file gets the products, locations and history in one place, without parsing the site.
How AI Discoverability Actually Works
Traditional search is simple: Google crawls your site, indexes your pages, and ranks them for keywords. You've spent years learning that game. AI search is different.
AI search engines -- ChatGPT, Claude, and Gemini -- don't return a list of links. They synthesize an answer. They read multiple sources, understand the content, and compose a response. Then they cite the sources they used.
Your job is to be one of those cited sources. Here's how the pipeline works:
User Asks a Question
"What's the best sourdough bakery in Portland?" This could be asked on ChatGPT, Claude, Gemini, or any AI assistant.
AI Searches and Crawls
The AI system uses web crawlers (GPTBot, ClaudeBot, PerplexityBot) to find and read relevant websites. It looks for structured information it can parse quickly.
llms.txt Gets Read (If the Tool Supports It)
If your site has an llms.txt, an AI tool that supports the standard can find your site summary, your product list, your location, and your history in one place. This is the step nobody has confirmed for ChatGPT, Claude or Gemini search. Independent server-log studies in 2026 found AI crawlers fetching robots.txt hundreds of times for every fetch of llms.txt.
AI Composes an Answer
The AI synthesizes information from multiple sources into a coherent response. Sites with clear structure and accurate information are more likely to be included and cited.
You Get Cited (or You Don't)
The answer includes citations -- links to the sources it used. Clear, accurate, reachable pages are what earn a place in that list. There is no published evidence that the presence of an llms.txt changes it.
Why Traditional SEO Isn't Enough
SEO got you on Google. That's still important. But AI search is a parallel channel that's growing fast, and the rules are different.
Google rewards backlinks and domain authority. AI rewards clarity, structure, and accurate information. You need both strategies. Read our AI SEO guide for the full breakdown.
llms.txt vs. robots.txt vs. sitemap.xml
These three files serve different purposes. You need all of them. Here's how they fit together.
robots.txt -- The Bouncer
Controls access. Tells crawlers which pages they're allowed to visit. If you block GPTBot here, AI can't read your site at all. It's the gate -- nothing else matters if this is locked.
sitemap.xml -- The Map
Lists all your pages with metadata like last-modified dates. Tells crawlers what exists on your site. It's comprehensive but provides no context about what each page is about or which pages matter most.
llms.txt -- The Guide
Provides context and priority. Tells AI systems what your site is about, which pages are most important, and what each page contains. It's curated, descriptive, and designed specifically for how LLMs process information.
The analogy: robots.txt lets AI through the door. sitemap.xml gives AI a floor plan. llms.txt gives AI a personal tour of the highlights. You need all three, but llms.txt is the one most sites are missing.
What Happens When You Deploy One
Here is the practical timeline, limited to what can be shown.
Immediately
Your llms.txt is live at your site root. Any tool that asks for it gets it. Our registry records the file and every later change to it, so there is a public record of when it appeared.
What is measured
Our read-cost benchmark asks an agent the same questions about a site with and without its llms.txt. A file under roughly 10,000 tokens made the site about twice as cheap to read. A file over that made it about nine times more expensive. Size is the whole story.
What is not shown
That ChatGPT, Claude, Gemini or Perplexity search reads the file, or that it changes whether you are cited. No provider has said so, and independent studies of AI-citation data have found no correlation. You can measure your own citations with an AI Citation Check; do not expect the file alone to move them.
How to Get Started
Three steps. Under 5 minutes total. No technical knowledge required.
Generate
Enter your URL on llmstxt.studio. We scan your sitemap and build a spec-compliant file with AI-powered descriptions. Free. 30 seconds.
Deploy
Download the file and upload it to your site root. Works with WordPress, Shopify, Squarespace, custom sites -- anything. Read the full creation guide.
Verify
Run our free AI Readiness Audit to confirm your file is live, valid and small enough to help.
See Who Already Has One
Over 840 companies have deployed llms.txt files. Anthropic, Stripe, Cloudflare, Hugging Face among them. Across a ranked sample of the wider web, our registry finds well under one site in ten publishes one. Browse our examples directory to see how companies in your industry structure their files.
For deeper context, read What is llms.txt? and our Visibility Index to understand how AI readiness is measured.
Make Your Website AI-Discoverable
AI is already answering questions about your industry. Make sure it can reach your site, find what matters, and read it cheaply.