When llms.txt launched in 2025, the hype was immediate: "publish this file and AI search will cite you." Two years of data later, the reality is clearer — and it's not what the hype promised.
The short version: AI search crawlers mostly ignore llms.txt. Coding agents and MCP servers actively use it. That distinction changes where you invest.
What the data says
Two independent studies define the 2026 picture:
- SE Ranking analyzed 300,000 domains and found 10.13% adoption — roughly one in ten sites has the file. Their machine-learning analysis found no statistical correlation between llms.txt and AI citation frequency.
- Limy tracked 500M+ LLM bot traffic events and found search-oriented crawlers (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot) almost never request
/llms.txt— they parse HTML directly. Google's Gary Illyes confirmed Google does not support the file.
Adoption is real; the causal link to citations is not. That isn't a failure of the format — it's a mismatch between who reads it and what people hoped it would do.
The B2A shift: where llms.txt actually earns its keep
The genuine growth story is Business-to-Agent (B2A) infrastructure. Agentic coding tools and MCP (Model Context Protocol) servers use /llms.txt and /llms-full.txt as a standard routing mechanism:
- Cursor, Claude Code, Windsurf, and similar fetch the file to load lightweight markdown documentation instead of bloating context windows with noisy HTML.
- MCP servers (like LangChain's
mcpdoc) rely on it to expose docs to agents. - For developer tools, APIs, and technical SaaS, this is real infrastructure — your docs team's life gets easier when agents can consume your API reference without a browser.
If your buyers are developers using coding agents, llms.txt is worth doing — not for SEO, but for agentic adoption of your product.
Who should still publish it
Publish or keep llms.txt when:
- You're a dev tool, API, or infrastructure product whose users run coding agents
- Your docs are already clean markdown (the file points to canonical pages, not 404s)
- You want to guide agent behavior — attribution, priority pages, deprecations
Skip the ceremony if you're a consumer SaaS that doesn't live in developer workflows. Our earlier llms.txt guide has the format and example structure — read it before deciding.
What actually drives AI citations instead
If llms.txt doesn't move ChatGPT citations, what does? The evidence points to:
- Third-party consensus — Reddit threads, G2/Capterra reviews, expert listicles, independent case studies
- Comparison pages — structured, quotable "Needle vs X" content AI assistants extract shortlists from
- Forum presence — real human conversations where your brand is recommended by other people
- Entity consistency — same name, domain, description, and Organization JSON-LD everywhere
That's the organic community GEO playbook — and it's the same conclusion as our AI visibility audits post: measure, fix what you control, and keep your public footprint accurate.
Decision checklist for startup sites
- If dev tool / API → publish llms.txt (agents) + verify docs links work
- If not → skip or keep existing file; don't add engineering time
- Never treat llms.txt as an SEO tactic — no citation correlation
- Re-check robots.txt: allow GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot for marketing/docs paths
- Invest citation effort in comparisons + community presence instead
GEO & LLM Site Analyzer · Browse free Trending Problems · Pricing