Title: What Is llms.txt and How to Create One for Your Site Author: Entexis Team Category: SEO, GEO & AEO Read time: 11 min URL: https://entexis.in/what-is-llms-txt-and-how-to-create-one-for-your-site Published: 2026-08-13 --- llms.txt is a small text file you put at the root of your website to tell AI models which pages on your site are worth reading first. The idea is simple: instead of letting ChatGPT, Claude, or Perplexity guess at what is most important on your site by crawling everything, you point them directly at the pages you want them to read carefully. The file lives at your-site.com/llms.txt, takes about 5 minutes to write well, costs nothing to deliver, and is a real lever in AI search visibility for sites that build it with discipline. The convention was proposed in September 2024 by Jeremy Howard. It is not yet officially endorsed by every AI engine, but adoption is real, growing, and one-sided in your favor: the file is dead simple, harmless when it is ignored, and helpful where it is read. Anthropic, Mintlify, Cloudflare, and a growing list of others now publish one. We have delivered llms.txt on our own site after running our own RAG-grounded chatbot, and the honest answer is that it works as a low-cost, low-risk signal that gets stronger every month as more engines start to read it. Below is what goes inside an llms.txt, what a good one looks like next to a bad one, the 5 sections to never leave out, the 3 ways people commonly mess it up, and the 5-step path from an empty root directory to a working file the answer layer can actually use. File at the root of your site, named llms.txt, where the convention puts it. 4Sections a well-formed llms.txt actually needs; everything else dilutes the signal. 5 minRealistic time to write a clean llms.txt once you know which 4 sections to fill. 0Setup cost, ongoing fee, or authentication needed to deliver it. Just a text file at the root. You will see the structure that holds up, the good-vs-bad example, the 5 sections to include, and the practical path from blank file to live on your site. ## What Goes Inside an llms.txt, by Priority of Attention An llms.txt is short and structured by design. The first lines carry the most weight, because that is what an AI model reads when it skims the file before deciding how much further to read. The structure below is the priority order. The higher a section sits, the more attention it gets and the more carefully it should be written. *[Diagram: 4 Sections of llms.txt, in the Order the Model Reads Them]* Tier 2, Context for the Reader A Short Paragraph on Who Your Content Is For An optional but heavily recommended paragraph just below the description. Who reads your site, what kinds of questions they bring, what kind of expertise your content carries. The model uses this to decide whether your pages are worth retrieving for a given query, so vagueness here costs you retrieval coverage. Write the paragraph in plain language and skip the marketing voice. Tier 3, The Pages You Most Want Read A Sectioned List of Your Core Content An H2 section with bullet links to the pages that carry your strongest first-party content. Not every page on the site. The 10 to 30 pages that say something only you can say. Each link should have a short note next to it telling the model what the page covers. This is the section that decides which of your content the model picks up first when it reaches your site. Tier 4, Supporting Content An Optional Section for Less Critical Resources An H2 section called "Optional" with links to documentation, references, FAQs, and other useful but lower-priority content. The model treats this section as background reading rather than primary source material, which means it is fine to include things here that you want available but not central to the model's understanding of the site. Below the File, Commodity Default Everything Not Listed in llms.txt Every page on your site that is not listed in llms.txt is treated as default crawl-and-discover content. The model may still read it, but it does not get the priority lift the file provides. This is fine for transactional pages, glossary entries, and routine docs that do not need to be lifted. The whole point of llms.txt is to separate the signal from the floor. Tiers 1 and 3 Carry Most of the Weight The site identity (Tier 1) is what the model quotes when summarizing your site. The core content section (Tier 3) is what the model retrieves when answering a query you cover. Get those 2 tight and the rest of the file does its job. Spend most of the writing time there. Once the structure is clear, writing the file is mostly mechanical. The judgment is in the 1-line description (Tier 1) and the curated list of pages (Tier 3). Those 2 are where a good llms.txt diverges from a mediocre one, and where it is worth taking a real pass rather than letting an off-shelf generator fill them in for you. ## A Good llms.txt Next to a Bad One The cleanest way to internalize the structure is to read a clean example and a stuffed one side by side. Both are real-shape examples, simplified for clarity. The difference is what gets included and what does not, and the gap is the whole reason llms.txt is worth writing carefully rather than auto-generating. *[Diagram: The Same Site Could Deliver Either Version. Only One Helps.]* Stuffed, Auto-Generated and Diluted # Bright Smile Dental Pune Best Dental Clinic Pune Affordable Dentistry > Premier dental clinic serving Pune and surrounding areas with quality care. ## All Pages - /home - /about - /contact - /services - /services/cleaning - /services/whitening - /services/braces - /services/aligners - /services/implants - /services/extraction - /blog - /blog/category/tips - /blog/category/news - /privacy-policy - /terms-of-service - /sitemap.xml - (plus 47 more URLs) Keyword-stuffed identity line, generic description, every page listed without curation or notes. The model treats this as crawl-everything noise and ignores the file as a meaningful signal. The Identity Line and the Curated List Are the File Almost every llms.txt failure mode comes from 1 of 2 mistakes: keyword-stuffed identity lines that look like the homepage title, and uncurated link dumps that include every page on the site. Avoid both and the file does its job, no matter how simple the rest of it looks. The judgment is small but real. Write the identity line in plain language. Pick the 10 to 30 pages that actually carry first-party content. Skip the rest. That single decision is what makes a clean llms.txt useful and what makes a stuffed one invisible. ## 5 Sections You Should Never Leave Out If the file feels short, that is a sign the structure is working. These are the 5 sections that actually need to be present, written with discipline, and refreshed when your content shifts. Everything else in the spec is optional and most of it dilutes rather than helps. A One-Line Blockquote DescriptionDirectly under the H1, a blockquote (starts with a greater-than sign) that says what the site does in 1 to 2 sentences. Specific, concrete, rooted in what your business actually publishes. This is the line the model most often quotes when summarizing what your site is for, so the writing matters far more than the section length. A Plain-Language Audience ParagraphA short paragraph that names who reads your site and what kind of expertise the content carries. The model uses this to decide whether your pages are worth retrieving for a given query. Vague audience descriptions cost you retrieval coverage; specific ones lift it. "For solo and small-clinic owners" is much more useful to the model than "for everyone interested in dentistry." A Curated Core Content SectionAn H2 section with bullet links to the 10 to 30 pages on your site that carry actual first-party content. Each link gets a short explanatory note. Not every blog post. Not every product page. The pages that say something only you can say. This is the section that decides which of your content the answer layer picks up first, so spend most of the writing time here. An Optional Section for Background ResourcesAn H2 called "Optional" with links to references, FAQs, and other background content. The model treats this as supporting material rather than primary source, which is the right behavior for documentation and routine reference content. Skipping this section is fine; including it gives the model context without diluting the core section above. None of the 5 needs to be long. Each one needs to be clear. The whole file fits on a screen for almost every business that is not running a 10,000-page documentation site, and the discipline of keeping it short is most of why it works at all. ## 3 Ways Sites Mess Up llms.txt The same 3 mistakes show up across most sites that deliver a file and see no benefit from it. Each one is fixable in 10 minutes and the lift from fixing them is real. Writing the Identity Line Like a Page TitleThe H1 of your llms.txt is not your homepage title and not the SEO meta description. Sites that write it like one fill the line with keywords ("Best Dental Clinic Pune Affordable Family Dentistry") instead of a real name ("Bright Smile Dental Pune"). The model reads the keyword version as marketing noise and quotes the plain-language version as identity. Use a name, not a title. Delivering It Once and Never Updating ItAn llms.txt that was written 18 months ago points the model at content that may no longer be your best work. Sites that win consistently treat the file as a quarterly review item: update the curated list when new first-party content delivers, drop pages that are no longer relevant, and refresh the description when the business shifts. A stale file is worse than no file, because it actively steers the model away from your current best content. > **The Forward Read:** llms.txt adoption is moving in 1 clear direction. The convention is being read by more crawlers every quarter, the spec is stabilizing, and the cost of delivering a file is so low that the only real reason not to do it is forgetting. By 2027 the file will be a standard expectation, like robots.txt is now, and sites that have been iterating on theirs for 12 months will have a quietly significant head start. The 5-minute work is worth doing now, and the quarterly refresh is what turns it from a one-time chore into a slowly compounding signal. ## 5 Questions Before You Publish Your llms.txt Before the file goes live at your site root, these 5 questions filter out the most common reasons it ends up underperforming. Ask them at the draft stage, not after. Are the Core Section Pages Actually First-Party Content?Look at the bullet list. Every entry should be a page that says something only your business can say. If the list is half about-pages, service overviews, and recycled topic explainers, the file is steering the model toward content that does not earn citations. Cut anything that is not genuinely first-party and keep the section short. Does Every Link Have a Short Explanatory Note?Each bullet should have a colon and a short note explaining what the page covers. Bare URLs are crawler bait and read as noise. A 10-word note next to each link tells the model what to expect on the page, which lifts retrieval accuracy for queries the page actually answers. Did Anyone Senior Actually Read the File?llms.txt is short enough that a founder, head of marketing, or content lead can read it in 90 seconds and catch the worst mistakes. Have someone senior actually read the file before it goes live. The most common gap is that the file was treated as technical configuration and never crossed a real reader's desk. Is It on the Quarterly Review Calendar?A file that delivers and is never touched again decays as the content underneath shifts. Put llms.txt on the same review cycle as your sitemap and your structured data: every 90 days, walk through the file, drop dead links, add new first-party content, refresh the description if anything material has changed. The file gets stronger with iteration. ## From Blank File to Live on Your Site The shortest practical path from no llms.txt to a working one fits in 5 steps. Done in order, the work takes about an hour for most sites the first time and 10 minutes per quarterly refresh after that. The roadmap below is what we run for our own site and for clients building this for the first time. *[Diagram: From Blank File to a Working llms.txt at Your Site Root]* 2 Write Identity Draft the H1 and Blockquote Write the site name as a person would say it and the 1 to 2 sentence description in plain language. The first 3 lines of the file. 3 Group Pages Split Into 2 or 3 Sections Group the curated pages into 2 or 3 themed H2 sections (research, case notes, references). Add the explanatory note next to each bullet. 4 Format as Markdown Save as a Plain Text File Save the file as llms.txt with Markdown formatting (H1, blockquote, H2, bullet links). UTF-8 plain text. No HTML, no fancy structure. 5 Upload + Schedule Put It at the Site Root Upload to your-site.com/llms.txt. Confirm it serves as plain text. Put the quarterly refresh on the calendar so the file stays current. Steps 1 and 3 Are the Real Work The curation (Step 1) and the grouping (Step 3) are the judgment work that decides whether the file lifts your visibility or sits inert. The formatting and uploading are mechanical. Sites that spend 90 percent of the time on Steps 1 and 3 deliver a file that works; sites that skip straight to formatting deliver a file that does nothing. The path is the same whether you are a 2-person clinic site, a mid-market SaaS, or a hundred-page brand site. The number of curated pages changes; the structure does not. Build the first version, refresh it quarterly, and the file becomes a quietly compounding signal across every engine that reads it. ## Frequently Asked Questions Is llms.txt actually read by ChatGPT, Claude, and Perplexity yet?Adoption is partial and growing. The convention was proposed in September 2024 and has been picked up by a meaningful subset of AI crawlers and engines, with more added every quarter. ChatGPT, Claude, and Google have not officially announced that they read it; some indie engines and tooling already do. The honest answer is that llms.txt is in the early phase where the file is increasingly read but not yet universally honored. The reason to deliver it now is the asymmetric cost-benefit: it takes about 5 minutes to write a clean file, costs nothing, and the upside grows every month as adoption expands. Skipping it is the only choice with no upside; delivering it locks in the benefit for whenever the engines you care about start reading. Where exactly does the llms.txt file go on my site?At the root of your domain, served as plain text. The full URL is your-site.com/llms.txt. The same place sitemap.xml and robots.txt live. On most static sites this means putting the file in the public or root folder; on most CMS platforms you can upload it through the file manager. After uploading, open the URL in your browser to confirm the file serves as plain text. If it returns a 404, a redirect, or HTML, the upload location is wrong. The serving has to be clean for crawlers to read it. Is llms.txt worth doing for a small business or solo practice?Yes, and it is the right move for small businesses. The file does not require a content team or a budget. A 2-person clinic, a solo consultant, or a small online shop can deliver a clean llms.txt in an hour the first time. The audience is honest: a small site with 5 curated first-party pages benefits more than a large site that auto-generated a list of 5,000 URLs. Size is not the variable. Curation is. Will llms.txt hurt my standard search rankings if I do it wrong?No. llms.txt is a separate signal from your standard search optimization, and crawlers that do not read it simply ignore the file. The worst-case scenario for a poorly written llms.txt is that it does nothing; it does not penalize you in Google Search, AI Overviews, or any other engine. The only practical risk is opportunity cost: a stuffed or auto-generated file fails to lift your AI search visibility, which means the work was spent without payback. There is no scenario where the file actively harms rankings. Do I need llms-full.txt as well, or is llms.txt enough?For most sites, llms.txt alone is enough to start. llms-full.txt is a separate convention that delivers the full content of your curated pages as a single concatenated text file, intended for engines that want to ingest your content in bulk rather than crawl page by page. The bigger file is useful for documentation-heavy sites, technical product sites, and reference sources where the full text matters. For a typical clinic, small SaaS, or e-commerce business, the bulk file is overkill in 2026; deliver the simpler llms.txt first and revisit llms-full.txt once the simpler file is settled and crawled. How often should I refresh the file once it is live?Quarterly is the right cadence for most sites. The first version takes about an hour to write; the quarterly refresh takes 10 to 20 minutes. Walk through the curated section, drop pages that are no longer relevant, add new first-party content that has delivered since the last refresh, and confirm the description still matches the business. Sites that put llms.txt on the same review cycle as their sitemap and structured data keep the file aligned with the content underneath. Sites that deliver once and never touch it again decay slowly as the content shifts and the file stops pointing at the strongest pages. Can Entexis create and maintain the llms.txt for our site?Yes, and it is the kind of work that pays back well as part of a broader AI search content engagement. We deliver llms.txt as part of the technical layer for clients, write it for the audience and the content they actually have, and put the quarterly refresh on a recurring schedule so the file stays current as the site shifts. We run llms.txt on our own site as well, so the work is something we already practice rather than something we describe. If your team can write the curated list yourselves, the file takes about an hour the first time and 20 minutes a quarter to maintain; if you want it done as part of an end-to-end AI search content engagement, we handle the curation, the writing, the upload, and the recurring refresh as part of the broader stack. For the broader thesis behind this, why first-party data is the AI search moat and why a curated file like llms.txt only works if the pages it points to are worth citing, the anchor piece is here: [Why First-Party Data Is the AI Search Moat](/why-first-party-data-is-the-ai-search-moat). For the citation side of the same problem, how AI engines actually pick what to quote, see: [How to Get Cited by ChatGPT, Claude, and Perplexity](/how-to-get-cited-by-chatgpt-claude-and-perplexity). For the measurement side, how to track AI search traffic when standard analytics tools cannot, see: [How to Track Traffic from ChatGPT, Claude, and Perplexity](/how-to-track-traffic-from-chatgpt-claude-and-perplexity). The most important thing to take from this is that llms.txt is not a hack and not a trick. It is a small piece of plumbing that lets you tell AI models which of your pages are worth reading first. The work is real but small: pick the 10 to 30 pages that say something only you can say, write a clean identity line and audience paragraph, group the pages into themed sections, and upload the file to your site root. The discipline is in the curation, not the formatting. Sites that launch this version get a quietly compounding signal across every engine that reads the file, and that list grows every quarter. > **Want llms.txt Done Right as Part of a Real AI Search Stack?:** At Entexis, we deliver llms.txt as part of the technical layer for clients building AI search visibility from the ground up. We curate the list of first-party pages, write the identity line and audience paragraph in plain language, group the content into themed sections, and put the quarterly refresh on the calendar so the file stays current. We run llms.txt on our own site too, so the work is something we already practice rather than something we describe. If your AI search content is starting to deliver but the technical layer underneath is half-built, the answer is probably not another plug-in. It is the small files and structured signals that the answer layer actually reads. Start the conversation with Entexis.