Quick answer: If you’re asking “should I create an llms.txt file?” hoping it’ll get you cited more by ChatGPT or show up in AI Overviews, the honest answer is probably not yet – not before you’ve handled the fundamentals. If you run developer tools, an API, or documentation that coding agents actually read, it’s a different and much easier yes.
That’s not the vague “it depends” you’ve already read on ten other blogs. Here’s the actual evidence behind that answer, and how to tell which camp you’re in.
What an llms.txt File Actually Does (Quick Refresher)
You’ve clearly already run into this term, so here’s the one-paragraph version: an llms.txt file is a plain txt file you place in your site’s root directory — the same spot as robots.txt — that hands AI models a curated, markdown-formatted summary of your most important pages instead of making them parse your full HTML.
You’ll sometimes see it searched as llm.txt (singular); same idea, but llms.txt is the file name every major spec and platform actually looks for. If you want the full walk-through of the format, that’s a different post – you’re here for the verdict.
The Honest Limitation: No Confirmed Adoption
Google is the loudest voice on llms.txt, so it’s worth starting there – but llms.txt itself was never pitched as a Google-specific tool. It was pitched as infrastructure meant to help any LLM read a website, from ChatGPT to Claude to Gemini. Google’s own guide to AI features in Search says flatly that “you don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search,” and that publishing an llms.txt file “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” Google’s John Mueller went further in a Reddit reply that made the SEO trade press, likening llms.txt to the long-abandoned keywords meta tag. The comparison holds up mechanically, not just rhetorically: robots.txt works because every major crawler agreed, years ago, to honor it, and there’s an enforcement side to that handshake. Llms.txt has no such agreement. It’s a file site owners write about their own content, with nothing checking it against the actual pages. Google Search is just one company’s product, though, not a verdict on how the rest of the AI ecosystem treats llms.txt.
The same low-adoption pattern for llms.txt shows up even in research that has nothing to do with Google. Ahrefs crawled over 137,000 domains and, unlike most studies, broke down exactly which companies’ bots were actually requesting llms.txt files. OpenAI’s GPTBot was the single biggest fetcher of llms.txt files, requesting roughly 4.5% of the ones Ahrefs checked; Anthropic’s ClaudeBot picked up a much smaller share; and retrieval-focused bots like OpenAI’s OAI-SearchBot and Perplexity’s crawler, combined, made only a couple hundred llms.txt fetches total across the entire dataset. The one crawler that meaningfully used llms.txt files was Claude Code, Anthropic’s coding agent, which alone out-fetched every AI search bot, assistant, and training crawler in the study combined.
This points to the central split in llms.txt usage: coding and agent tools, regardless of which company built them, actually use llms.txt files; general-purpose AI search and assistant traffic, from OpenAI, Anthropic, or anyone else, mostly doesn’t.
A separate look at roughly 300,000 domains by SE Ranking backs up that low-usage finding from a different angle: about 10% of domains had an llms.txt file, and there was no statistical link between having one and how often a domain gets cited in AI answers overall – dropping llms.txt from SE Ranking’s prediction model didn’t cost it any accuracy. And when OtterlyAI monitored a live site’s aggregate AI bot traffic for 90 days, only 84 of its 62,000+ AI bot visits went to that site’s llms.txt file, versus roughly 265 visits to an average content page over the same stretch.
Three separate research teams, three different methods, and the same conclusion each time: llms.txt sees limited adoption, and even where it exists, AI crawlers mostly skip it.
Low adoption doesn’t mean llms.txt is useless everywhere, though. OpenAI publishes its own llms.txt and llms-full.txt for its Agentic Commerce Protocol, Stripe ships an llms.txt file loaded with instructions for AI agents, and Anthropic, Vercel, Cloudflare, GitHub, and Adobe all maintain their own. Since May 2026, Chrome’s Lighthouse tool has even added an “Agentic Browsing” audit that checks whether a site has an llms.txt file and flags it for a missing H1, thin content, or no links – proof that tooling is forming around llms.txt even while search ranking systems ignore it entirely. Splitting the use case in two makes the picture clear: search visibility and agent tooling are different problems, and today llms.txt only really solves the agent-tooling problem – and it solves that problem the same way whether the agent belongs to OpenAI, Anthropic, or anyone else.
We’re not going to tell you an llms.txt file is your ticket to getting cited by ChatGPT tomorrow. That kind of promise is exactly the black-box, trust-us marketing we built this agency to avoid.
llms.txt: Low Effort, Low Risk, Uncertain Payoff
Creating an llms.txt file takes maybe twenty minutes once you know your priority pages. An llms.txt file won’t hurt your site, slow down crawling, or conflict with your robots.txt file in any meaningful way.
So the real question about llms.txt isn’t “will this hurt me?” It’s “is this the best use of my time right now?” Companies like Stripe, Vercel, and Cloudflare ship an llms.txt file anyway, but they’re building for agentic browsers and coding assistants that read documentation directly – not chasing a search ranking boost. For most SaaS and B2B teams without that same agent-facing use case, the answer depends on where you are in your broader AI-search strategy. If you haven’t nailed the fundamentals yet, an llms.txt file is a distraction from work that actually moves the needle.
When You Should Build an llms.txt File
Build an llms.txt file when:
- You’ve already covered the fundamentals: clean structured data, clear entity definitions, and authoritative content that answers real questions
- Your site serves developer tools, APIs, or documentation that AI coding assistants regularly pull from
- Your site has a lot of pages, and you want to guide AI crawlers toward the ones that matter most
- You can monitor your server logs for AI-agent user agents actually requesting your llms.txt file, so you’re tracking real usage instead of flying blind
When You Should Skip llms.txt
Hold off on creating an llms.txt file when:
- Your core content still needs work — schema markup, clear headings, and well-structured answers will do far more for AI visibility right now
- You have no way to check whether it’s making a difference
- You’re looking for a quick win instead of a real strategy
- You’d rather spend that time on link building, brand mentions, technical SEO, or content that actually earns citations
If you’re unsure which category you fall into, that’s usually a sign your foundational SEO work needs attention before you chase newer tactics like llms.txt.
How to Create an llms.txt File, If You Decide to Move Forward
Create a txt file named llms.txt and place it in your root directory — the same location as your robots.txt file.
Keep it short and factual:
| # Your Company Name > One or two sentence summary of what your company does and who you serve. ## Docs – [Getting Started](https://yoursite.com/docs/getting-started): Brief description of the page – [API Reference](https://yoursite.com/docs/api): Brief description of the page ## Blog – [Your Best Post](https://yoursite.com/blog/post-slug): Why this page matters |
Machines don’t need persuasion, they need clarity. An llms.txt file isn’t the place for marketing copy.
Update the file whenever you publish something genuinely important. There’s no benefit to listing every page on your site, so stick to the content that best represents your expertise.
FAQ
For most sites chasing visibility in Google Search or consumer AI search tools, no – Google has said plainly that llms.txt files don’t affect Search rankings, and independent studies have found little evidence that mainstream AI crawlers read them at scale. For sites serving developer tools, APIs, or coding agents, llms.txt files are worth the roughly twenty minutes it takes to create one, since agentic tools are the audience actually requesting them.
No. An llms.txt file isn’t required for any AI system, search engine, or LLM to find, crawl, or cite your website – nothing about how AI tools access your site depends on it. It’s an optional file, and skipping it won’t hurt your visibility anywhere. It mainly helps in one specific case: AI coding agents and developer tools built to look for it.
Markdown is generally considered the best data format for LLMs, since it keeps a document’s structure — headings, lists, links — without the tag overhead of raw HTML, which burns tokens an AI model would otherwise spend parsing markup instead of understanding content. Plain, clean text works nearly as well for straightforward prose, while structured formats like JSON suit data an AI agent needs to parse programmatically rather than read. That’s exactly why an llms.txt file uses markdown: it’s a stripped-down, LLM-friendly summary of a site instead of raw HTML.
A well-formed llms.txt file opens with an H1 heading naming your site or company, followed by a one- or two-sentence blockquote summarizing what you do. Below that, H2 sections (like Docs or Blog) organize markdown links to your most important pages, each with a short, factual description rather than marketing copy. Keep the file focused on the pages that best represent your site or documentation — there’s no benefit to listing every page you have.
Where llms.txt Fits Into Your Bigger AI Search Strategy
An llms.txt file is one small piece of a much larger AI search puzzle. On its own, it won’t move your visibility numbers with Google or most consumer AI search tools. Paired with structured data, strong entity signals, and content built to answer real questions, it becomes part of a more complete approach – and it earns its place for sites where agentic tools are a real referrer. If you haven’t tackled those fundamentals yet, our LLM SEO guide covers the strategy that actually moves AI search visibility, with llms.txt as one small piece of it rather than the centerpiece.
The Bottom Line: Should You Create an llms.txt File?
Should you create an llms.txt file? If your fundamentals are solid, you serve developer or agent-facing traffic, and you have time to experiment, go ahead – it costs little and carries minimal risk. But if you’re still building out your core AI-search strategy and your traffic isn’t coming from coding agents or AI browsers, your time is better spent elsewhere. Structured data, clear content, and genuine authority will always matter more than an unproven file format.
