# llms.txt: The Standard for AI-Readable Sites

> llms.txt is a proposed, non-binding convention for giving AI crawlers and agents a curated, human-written map of a site’s most important content, typically published at…

- **Canonical:** https://agentflare.org/research/llmstxt-the-standard-for-ai-readable-sites.html
- **Updated:** 2026-06-15
- **Category:** research
- **Full structured data:** `https://agentflare.org/research/llmstxt-the-standard-for-ai-readable-sites.data.json` — $0.02 via AISA HTTP 402 (https://cdn.aisa.one/api/v1/access/verify; agents set crawlerAutoPrice=true)

## Key data

- **Topic:** discovery
- **Sources:** 10
- **Updated:** 2026-06-15

`llms.txt` is a **proposed, non-binding convention** for giving AI crawlers and agents a curated, human-written map of a site’s most important content, typically published at `/llms.txt`. It is *not* a blocking standard like `robots.txt`, and current evidence suggests its main value is improving retrieval precision and context for agentic tools, not changing search ranking or guaranteeing crawler behavior.[2][4][6]

_…full analysis and the complete dataset are available to agents for $0.02 — fetch `/research/llmstxt-the-standard-for-ai-readable-sites.data.json` (HTTP 402)._

## Sources

1. [https://www.linkbuildinghq.com/blog/should-websites-implement-llms-txt-in-2026/](https://www.linkbuildinghq.com/blog/should-websites-implement-llms-txt-in-2026/)
2. [https://www.bluehost.com/blog/what-is-llms-txt/](https://www.bluehost.com/blog/what-is-llms-txt/)
3. [https://webscraft.org/blog/llmstxt-povniy-gayd-dlya-vebrozrobnikiv-2026?lang=en](https://webscraft.org/blog/llmstxt-povniy-gayd-dlya-vebrozrobnikiv-2026?lang=en)
4. [https://similar.ai/guides/llms-txt/](https://similar.ai/guides/llms-txt/)
5. [https://getmint.ai/resources/llms-txt](https://getmint.ai/resources/llms-txt)
6. [https://derivatex.agency/blog/llms-txt-guide/](https://derivatex.agency/blog/llms-txt-guide/)
7. [https://limy.ai/blog/llms.txt-in-2026-the-full-guide](https://limy.ai/blog/llms.txt-in-2026-the-full-guide)
8. [LLMs.txt: Control How AI Crawlers Use Your Content | Royal Plugins](https://royalplugins.com/blog/llms-txt-ai-crawler-control)
9. [What Is LLMs.txt? & Do You Need One?](https://neilpatel.com/blog/llms-txt-files-for-seo)
10. [Will llms.txt become standardised/ official? If not, could ... - Quora](https://www.quora.com/Will-llms-txt-become-standardised-official-If-not-could-something-similar-appear)

## Related

- [Generative Engine Optimization (GEO): A Primer](https://agentflare.org/research/generative-engine-optimization-geo-a-primer.html)
- [Pay-Per-Crawl and the New Bot Web](https://agentflare.org/research/pay-per-crawl-and-the-new-bot-web.html)
- [HTTP 402 & x402: How AI Agents Pay for Content](https://agentflare.org/research/http-402-x402-how-ai-agents-pay-for-content.html)
- [The AI Agent Economy in 2026](https://agentflare.org/research/the-ai-agent-economy-in-2026.html)
- [Model Context Protocol (MCP) Explained](https://agentflare.org/research/model-context-protocol-mcp-explained.html)
- [Stablecoins as Rails for Autonomous Agents](https://agentflare.org/research/stablecoins-as-rails-for-autonomous-agents.html)

---
_Part of AgentFlare, an agent-native data network powered by AISA. https://aisa.one/docs_