Short answer: llms.txt is a plain-text file placed at the root of your website (yourdomain.com/llms.txt) that tells AI language models what your site contains, which pages are most useful, and how you want your content represented in AI-generated answers. It is an emerging convention — not a ratified web standard — and support varies across AI systems. That said, publishing one costs almost nothing and carries no downside risk, so if your business cares about AI-era visibility it is worth doing now.
The Problem llms.txt Is Trying to Solve
When a large language model is trained or retrieves content from the web, it processes your pages the way a browser does: HTML, scripts, navigation chrome, cookie banners, footer links, and the actual content all land in the same text stream. The model has to figure out what is signal and what is noise.
For a human reader, your website's visual hierarchy makes this obvious. For an LLM processing raw text, it is much less clear. A 2,000-word blog post buried after 800 words of navigation, sidebar, and promotional banners looks very different from a clean Markdown document that starts with your main point.
llms.txt addresses this by giving models a curated index of your most important content in clean Markdown — essentially a structured reading list. Rather than the model discovering your pages through a crawl and inferring their relative importance from link structure, you tell it directly: these are my core pages, here is what each one is for, and here is a summary of what my business does.
This post goes deeper than the overview in How to Get Your Business Found in ChatGPT and AI Search, which covers the concept briefly. Here we look at the format in detail, show you exactly what to write, and give an honest account of what the file can and cannot do.
Where Did llms.txt Come From?
The llms.txt convention was proposed by Jeremy Howard (fast.ai) in 2024 as a practical solution to the problem described above. It was never submitted to a standards body like the IETF or W3C, and it does not have the same status as robots.txt (which has formal RFC backing). Some AI systems and documentation tools have begun honouring it; others ignore it entirely.
This is worth being clear about: publishing llms.txt does not guarantee that ChatGPT, Perplexity, Claude, or any other AI system will read it. The file is a convention that makes sense if models choose to look for it — and increasingly, some do. It is best thought of as a low-cost signal that improves your odds rather than a guaranteed channel.
What Does an llms.txt File Look Like?
The format is Markdown. The convention defines a small set of sections:
# NexaEx
> NexaEx is a senior software engineering studio registered in Erode, Tamil Nadu. We build custom web apps, APIs, and data systems for Indian businesses. Remote-first, fixed-price, senior engineers only.
## Core pages
- [Services](https://nexaex.in/services): What we build and how we work — web apps, APIs, database design, cloud infrastructure.
- [Pricing](https://nexaex.in/pricing): Fixed-price project ranges by scope. No retainers, no lock-in.
- [Blog](https://nexaex.in/blog): Practical guides on software development, AI integration, and digital strategy for Indian businesses.
- [Contact](https://nexaex.in/contact): How to start a project conversation.
## About
NexaEx works exclusively with senior engineers — no juniors, no outsourcing. Projects are quoted at a fixed price in writing before any work begins. Clients receive a working build every week and own all code and assets at handover. We do not lock clients into ongoing contracts.
## Optional: llms-full.txt
For AI systems that want the full site content in a single document, see: https://nexaex.in/llms-full.txt
The first-level heading is your site name. The blockquote under it is a concise summary — think of it as a system prompt about your business. The ## Core pages section is a curated list of your most important pages with brief descriptions. You can add an ## Optional section pointing to llms-full.txt, a longer document containing full page content for models that want to load everything at once.
NexaEx publishes its own llms.txt at nexaex.in/llms.txt — the file follows this structure and is updated whenever our core pages change.
llms-full.txt: When and Why
The full-content companion file llms-full.txt is useful for documentation-heavy sites, SaaS products with API docs, or any business where detailed technical content is the primary asset. It concatenates your key pages — stripped of HTML, menus, and scripts — into a single clean document.
Most AI coding tools and retrieval-augmented generation (RAG) systems that index external content prefer this format because it avoids repeated crawl overhead. If you are building a developer tool, running a technical blog, or publishing a product with documentation, llms-full.txt is worth adding.
For a typical business marketing site, start with llms.txt alone. If you see AI tools quoting stale or inaccurate information about your business, the full file can help because it gives AI systems an authoritative, up-to-date source that is explicitly yours.
What llms.txt Cannot Do
It cannot override an AI system's existing training data. If a model was trained six months ago and learned incorrect information about your business, llms.txt only helps AI systems that perform live retrieval at inference time — systems like Perplexity, ChatGPT with browsing enabled, and some versions of Claude. It does not retroactively correct base training.
It cannot replace robots.txt. If you have blocked AI crawlers in robots.txt, those crawlers will not reach your llms.txt either. The two files serve different purposes: robots.txt controls crawl access, while llms.txt improves how well an AI system understands what it has been allowed to crawl. Both are necessary; neither replaces the other.
It also cannot make up for thin, low-quality content. An llms.txt pointing to empty pages gives AI systems nothing useful to work with. The file is most powerful when it points to genuinely informative, well-structured content.
robots.txt and AI Crawler Policies: The Foundation
Before llms.txt is useful, you need an explicit AI crawler policy. Here is what a current, complete robots.txt looks like for a business that wants full AI visibility:
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: CCBot
Allow: /
User-agent: *
Allow: /
If you want to block specific bots while allowing others, change Allow to Disallow selectively. The important thing is to be explicit — ambiguous or missing entries mean different crawlers make different assumptions.
How to Write Your Own llms.txt
| Element | What to write | Length |
|---|---|---|
| Site name (H1) | Your trading name, as it appears on your site | 1–5 words |
| Blockquote summary | Who you are, what you do, who you serve, where you operate | 2–4 sentences |
| Core pages | The 4–8 pages an AI should know exist, with accurate one-line descriptions | 1 line each |
| About section | Your differentiators in plain prose, without marketing superlatives | 3–6 sentences |
| Optional pointers | Links to llms-full.txt or other machine-readable assets | As needed |
Write the summary blockquote as if you are briefing someone who has never heard of your business and needs to describe it accurately to a third party. Avoid adjectives like "leading", "innovative", or "world-class" — AI systems trained on web text have learned to discount these as low-information marketing language.
Should Your Business Publish llms.txt Now?
The case for publishing: it takes under an hour for a typical business site, costs nothing to host, carries no SEO penalty, and positions your site correctly for AI systems that do honour the convention. If you have a clear AI visibility strategy, llms.txt is one of several low-cost, high-signal moves worth making in 2026.
The case for waiting: if your core pages are thin, inconsistent, or out of date, publishing llms.txt that points AI systems to that content may do more harm than good — you will be directing AI systems to accurate information about your inaccuracies. Fix your content first.
The honest summary is: publish llms.txt once your core pages are solid. It is not a strategy on its own, but it is a sensible component of a broader SEO and AEO strategy for 2026.
Implementation Alongside Schema and Other Signals
llms.txt works alongside, not instead of, structured data. JSON-LD schema markup (see our schema guide for Indian businesses) gives AI systems machine-readable facts about your entity. llms.txt gives them a curated content index. Together they create two reinforcing layers of machine-readable information about your business.
The full signal stack for AI-era visibility:
robots.txtwith explicit AI crawler permissions- JSON-LD schema (Organization, LocalBusiness, Service, FAQPage)
llms.txtwith accurate site summary and page list- Answer-first content structure on core pages
- Consistent off-site entity signals (LinkedIn, GitHub, directory listings)
Items 1–3 can typically be implemented in a single technical engagement. Items 4–5 are ongoing content and distribution work. NexaEx has implemented all five on this site and can handle the technical build for your site too.
Want help publishing a correct, maintained llms.txt alongside a full AI visibility setup? Contact NexaEx — we build it as code, not an afterthought.
Frequently asked questions
Is llms.txt an official web standard?
No. llms.txt is an emerging community convention proposed in 2024, not a ratified standard from the IETF, W3C, or any other body. It does not have the formal backing that robots.txt has. Some AI systems and developer tools honour it; others ignore it. This does not make it useless — low-cost signals with no downside risk are worth publishing — but you should understand it is not a guaranteed channel.
Will publishing llms.txt fix incorrect AI answers about my business?
Only partially. llms.txt helps AI systems that perform live retrieval at inference time — tools like Perplexity or ChatGPT with browsing enabled. It does not retroactively correct base training data in models that were trained before they crawled your file. For base training corrections, the more powerful levers are consistent off-site entity signals, updated structured data, and getting accurate information cited in authoritative third-party sources.
How long should my llms.txt file be?
For a typical business site, the core llms.txt file should be concise — under 500 words. The goal is a quick, accurate briefing for an AI system, not a full content dump. If you want to provide comprehensive content, create a separate llms-full.txt and link to it from the main file. Keep the summary blockquote to two to four sentences and the page list to your eight most important pages at most.
Does llms.txt replace robots.txt?
No — they serve entirely different purposes. robots.txt controls whether a crawler is allowed to access your pages at all. llms.txt helps an AI system understand what your site contains once it has been allowed to crawl. You need both: robots.txt to grant access to the AI crawlers you want, and llms.txt to give those crawlers an accurate, curated view of your most important content.