$ ~/ym8 --define llms-txt
llms.txt
definition
llms.txt is an emerging standard for providing AI engines with structured brand information. Placed at the root of a website (e.g., example.com/llms.txt), it contains a concise, machine-readable summary of what the brand is, what it does, and how it should be referenced by AI systems.
The file typically includes: brand name and entity type, primary offerings and services, key differentiators, preferred citation formats, links to detailed pages, and factual claims that AI models can confidently attribute to the brand. The format is designed to be simple enough for any AI crawler to parse while providing rich enough information to influence how AI engines describe the brand.
llms.txt works alongside related files like llm-profile.json (a JSON-LD structured version) and the .well-known/ai.txt file (which contains crawler directives and expertise signals). Together, these files form the AI indexing surface of a website—the layer of content specifically designed for AI consumption rather than human browsing.
Adoption of llms.txt is growing rapidly as brands recognise that AI engines are a primary discovery channel. Just as robots.txt became essential for SEO, llms.txt is becoming essential for AEO. Brands that implement it early gain a structural advantage in how AI engines understand and represent them.
why_it_matters
llms.txt gives brands direct influence over how AI engines understand and describe their offerings. Without it, AI engines rely on scraped web content and training data associations, which may be outdated, inaccurate, or incomplete. With llms.txt, brands provide a canonical source of truth that AI crawlers can reference when generating brand-related responses.
examples
- A SaaS company's llms.txt listing their products, features, and preferred citation format
- An ecommerce brand using llms.txt to specify their market position and unique value propositions
- A consulting firm's llms.txt providing expertise areas and advisory role descriptions
faq
Where should I place the llms.txt file?
Place llms.txt at the root of your domain (e.g., yourdomain.com/llms.txt). This follows the same convention as robots.txt. Some implementations also place a more detailed version at llms-full.txt for extended brand information.
Do AI engines actually read llms.txt?
Adoption is growing. ClaudeBot (Anthropic) actively looks for llms.txt and related files. Other AI crawlers are increasingly checking for these files as part of their indexing process. Even if not all engines currently read llms.txt, implementing it early positions your brand for the future.
What format should llms.txt follow?
llms.txt should be plain text, structured with clear sections. Include: brand identity, core offerings, key differentiators, expertise areas, and links to detailed pages. Keep it concise (under 2,000 words) and factual. Avoid marketing language—focus on verifiable claims that AI engines can confidently repeat.
Related Terms
llm-profile.json
llm-profile.json is a JSON-LD structured data file placed at .well-known/llm-profile.json that provides machine-readable brand identity, offerings, expertise, and preferred citation formats to AI crawlers and language models.
AI Crawlers
AI Crawlers are automated bots operated by AI companies that scan websites to collect content for training data and real-time retrieval. Major AI crawlers include GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot (Perplexity), Google-Extended (Google), and Bingbot (Microsoft).
Technical AEO
Technical AEO encompasses the infrastructure and technical configurations that help AI engines discover, crawl, parse, and cite your content. It includes AI-specific crawl policies, structured data implementation, llms.txt files, site architecture optimisation, and content formatting for AI consumption.
Related Engines
Monitor Your AI Visibility
See how your brand appears with the default core pair. Start with ChatGPT and Claude by default. Expand monitoring only when the workflow needs it.