1. Home
  2. Guides
  3. AI search
  4. llms.txt: What It Is, a Full Example and How to Add One
AI search guide

llms.txt: What It Is, a Full Example and How to Add One

llms.txt is a short Markdown file that tells AI tools what your site is about and which pages matter most. This guide covers what the proposal actually says, a complete example, how to add one on WordPress, and an honest view of what it will and will not do for you.

By the AI Marketing Services team · Updated · 11 min read

llms.txt is a proposed Markdown file, placed at the root of a website as /llms.txt, that gives large language models a plain summary of the site and a curated list of its most useful pages. Jeremy Howard of Answer.AI published the proposal in September 2024. It is a suggestion, not an official web standard.

Key takeaways

  • llms.txt is a Markdown file at the root of your site that points AI tools to your most useful pages.
  • It has a fixed shape: an H1 name, a one-line summary in a blockquote, then H2 sections of links with short notes.
  • It is a proposal, not a standard. No major AI company has said it relies on it, so treat it as cheap and low-risk.
  • It does not replace robots.txt or your XML sitemap. Each file does a different job.

What the llms.txt file is and where it came from

Web pages are built for people with browsers. A typical page carries navigation menus, cookie banners, scripts, footers and tracking code around a few hundred words of useful text. When an AI tool fetches that page while answering a question, most of what it downloads is noise, and the context window it can work with is limited.

Jeremy Howard, co-founder of Answer.AI, proposed a fix in September 2024: put a small Markdown file at /llms.txt that describes the site in plain text and points to the pages worth reading. Markdown was chosen because both people and language models read it easily, and because it has enough structure for a simple program to parse.

The parts of the format

The proposal sets out a fixed order:

  1. An H1 title with the name of the site or project. This is the only required part.
  2. A blockquote summary, one or two sentences that explain what the site is and who it is for.
  3. Optional free text, such as short paragraphs or bullet points with extra context. No headings in this part.
  4. H2 sections that list links. Each line is a Markdown link followed by a colon and a short note about what the page contains.

One H2 name has a special meaning. A section called Optional holds links that a tool can skip when it needs a shorter version. The proposal also suggests offering a clean Markdown copy of important pages at the same address with .md added, so a tool can fetch the text without the page furniture.

A companion file, llms-full.txt, has become a common convention on documentation sites. Instead of links, it holds the full text of the key pages in one long file. It is optional, and for most business websites the short file is plenty.

Anatomy of an llms.txt file

# Your CompanyH1: the site or project name (required)
> One-line summary of what you doBlockquote: a short description
Optional paragraphs of contextPlain text, no headings
## ServicesH2 section
- [Page name](https://…): one-line noteLink list items
## OptionalLinks an AI tool can skip if short on space
The four parts of the format, top to bottom.

A full llms.txt example

Here is a complete llms.txt example for a made-up payroll software company. The domain is a placeholder, so swap in your own pages.

# Harbor Payroll

> Harbor Payroll is payroll and HR software for US companies with 10 to 200 employees. This file lists the pages that best explain the product, pricing and help documentation.

Harbor Payroll runs payroll in all 50 states, files federal and state payroll taxes, and connects to QuickBooks Online and Xero. Pricing is per employee per month. The company sells only to US employers.

## Product

- [Payroll features](https://www.example.com/payroll/): what the product does, supported pay schedules and contractor payments
- [Pricing](https://www.example.com/pricing/): plans, per-employee prices and what each plan includes
- [Integrations](https://www.example.com/integrations/): accounting, time tracking and benefits connections

## Help docs

- [Getting started](https://www.example.com/docs/getting-started.md): account setup for a new employer
- [Running your first payroll](https://www.example.com/docs/first-payroll.md): the steps, with deadlines by state
- [Payroll tax filing](https://www.example.com/docs/tax-filing.md): which forms are filed and when

## Company

- [About Harbor Payroll](https://www.example.com/about/): founders, headquarters and contact details
- [Security](https://www.example.com/security/): how employee data is stored and protected

## Optional

- [Blog](https://www.example.com/blog/): payroll guides and product news
- [Changelog](https://www.example.com/changelog/): release notes by month

A few things are doing the work here. The blockquote says who the product is for in one breath, which is the fact a model most often gets wrong about smaller companies. The free text adds facts that help an assistant answer questions such as "does Harbor Payroll work with Xero" without opening another page. Every link has a note, so a tool can decide what to fetch. The blog sits under Optional because it is useful background but not essential.

How to write your llms.txt file, step by step

  1. Pick the pages that answer real questions. Start with the pages a new customer would need: what you sell, who it is for, pricing, how it works, support docs and contact details. For a typical business site, 10 to 40 links is a sensible range. A file listing all 900 blog posts defeats the purpose.
  2. Write the summary as if for a stranger. Name the product or service, the type of customer and the market. "A dental group with four clinics in Melbourne offering general, cosmetic and emergency dentistry" beats "your smile is our priority".
  3. Group links under plain H2 headings. Product, Services, Locations, Docs, Policies and Company are all fine. Use the words your customers use, not internal team names.
  4. Add a short, factual note to each link. Say what the page contains, not why it is great. Keep notes to one line.
  5. Move the nice-to-have pages under Optional. Case studies, older articles and press pages usually belong there.
  6. Save it as plain UTF-8 text named llms.txt and publish it at the root of your domain, so it loads at https://www.yourdomain.com/llms.txt.

Check the facts in the file against your site before you publish. If your pricing page says one thing and llms.txt says another, you have created a contradiction for any tool that reads both. Set a reminder to review the file whenever you launch a product, change prices or restructure URLs.

llms.txt vs robots.txt vs sitemap.xml

People often assume llms.txt is a new robots.txt for AI. It is not. The three files do different jobs, and only one of them controls access.

llms.txtrobots.txtsitemap.xml
Main jobSummarize the site and point to the best pagesTell crawlers which paths they may or may not crawlList the URLs you want search engines to discover
FormatMarkdownPlain text directivesXML
Written forLanguage models and the people building AI toolsCrawlers such as Googlebot, Bingbot and OAI-SearchBotSearch engine crawlers
StatusA community proposal from 2024Long-standing convention, published as RFC 9309 in 2022Long-standing protocol supported by the major search engines
Blocks or allows botsNoYes, for crawlers that respect itNo
Includes descriptionsYes, a summary and a note per linkNoNo, only URLs and optional dates

The practical point: if you want to stop or allow an AI crawler, the place to do that is robots.txt (and your firewall rules). Our guide on how to rank in ChatGPT walks through the OpenAI crawlers and what each one does. An llms.txt file is closer to a short, annotated table of contents.

Which file does what

  1. robots.txtSays which crawlers may fetch which paths
  2. XML sitemapLists every URL you want crawled and indexed
  3. llms.txtHand-picks the pages an AI tool should read first
robots.txt controls access, the sitemap lists URLs, llms.txt curates.

Does llms.txt help? An honest status check

The llms.txt file is a proposal, not a standard. No standards body has adopted it, and the major AI companies have not publicly committed to reading it when they build search results or answers. Google has not announced any use of it in Search or AI Overviews. Some AI and developer tool companies publish llms.txt files for their own documentation, but publishing one is not the same as their crawlers relying on yours.

So treat it as cheap and low-risk, not as a ranking factor. Writing a good file takes an hour or two. It will not hurt your Google rankings, and it will not get you cited in ChatGPT on its own.

Where it is more likely to be useful

  • Developer documentation. Coding assistants and AI agents are often pointed at a docs site directly, and a clean index with Markdown copies of pages saves them effort.
  • Products with facts that models keep getting wrong. A clear summary of who you serve and what you charge costs little to maintain and may help any tool that does fetch it.
  • Sites where a user or agent fetches pages on demand. When someone asks an assistant to read your site, a short index is easier to work with than a heavy homepage.
  • Your own AI projects. If you build a chatbot or internal assistant on your content, an llms.txt file is a tidy starting point for it.

If you want to know whether anything reads your file, check your server or CDN logs for requests to /llms.txt and look at the user agents. That tells you more than any opinion online. The work that moves AI visibility is still the core of generative engine optimization: crawlable pages, clear answers, consistent facts about your business and mentions on the sites AI tools cite.

How to add llms.txt on WordPress

WordPress does not create an llms.txt file on its own. There are three common ways to add one.

Option 1: upload a real file

This is the simplest route and the one we suggest for most sites. Open your host's file manager or connect by SFTP, go to the folder that holds wp-config.php (often public_html), and upload your llms.txt there. The web server serves a real file before WordPress handles the request, so no plugin is involved.

Option 2: use an SEO plugin

Some popular SEO plugins have added an llms.txt generator. Look in your plugin's settings. Generated files tend to list recent posts automatically, so review the output and edit the summary and link notes by hand. An automatic list of your latest 20 articles is rarely the best map of your business.

Option 3: serve it from code

If you cannot upload to the root, a small snippet in a site-specific plugin can serve a file stored in your theme folder:

add_action( 'init', function () {
    $path = parse_url( $_SERVER['REQUEST_URI'], PHP_URL_PATH );
    $file = get_stylesheet_directory() . '/llms.txt';
    if ( '/llms.txt' === $path && is_readable( $file ) ) {
        header( 'Content-Type: text/plain; charset=utf-8' );
        readfile( $file );
        exit;
    }
} );

Test it after publishing

  • Open /llms.txt in a private browser window. You should see plain text, not your theme or a 404 page.
  • Run curl -I https://www.yourdomain.com/llms.txt and confirm a 200 status with a text/plain content type.
  • Clear any page cache or CDN cache if an older version shows.
  • Check that a security plugin or firewall rule is not redirecting the request.

If you are reviewing your wider setup for AI search, an AI visibility audit checks crawler access, structured data and how assistants describe you today, which tells you more about where to spend effort than llms.txt alone.

Frequently asked questions

Is an llms.txt file required?

No. llms.txt is a voluntary proposal, not a requirement from Google, OpenAI or any standards body. Your site will be crawled and indexed without it. Many businesses add one because it is quick to write and low-risk, but it should come after basics such as crawlable pages, clear content and a correct robots.txt file.

Does llms.txt improve Google rankings?

There is no evidence that it does, and Google has not announced any use of llms.txt in Search or AI Overviews. Treat it as a tidy summary of your site for AI tools that choose to read it, not as a ranking factor. Rankings still depend on content quality, technical health and links.

Can llms.txt block AI crawlers from my site?

No. llms.txt has no allow or disallow rules and does not control access. To block or allow crawlers such as GPTBot, OAI-SearchBot or PerplexityBot, use robots.txt, and check that firewall or CDN bot settings match what you intend. The two files do different jobs and can sit side by side.

What is the difference between llms.txt and llms-full.txt?

llms.txt is a short index: a title, a summary and links to key pages with brief notes. llms-full.txt is a common companion convention that holds the full text of those pages in one file, mostly used by documentation sites. A typical business website only needs the short llms.txt file.

How often should I update my llms.txt file?

Update it whenever the facts in it change, for example new products, new prices, new locations or changed URLs. A quick review every quarter is a sensible habit. An outdated file that contradicts your website is worse than having no file, because it gives AI tools two different versions of the truth.

Related services: generative engine optimization services and AI SEO Services and AI Visibility Audit.

Want help getting your site ready for AI search?

Tell us a little about your business and what you want to change. We reply by email with honest next steps, and a call if it would help.

✉ info@serphike.com 🌏 Working with businesses in the US, UK, Australia, Canada, Singapore and the UAE

We use your details only to reply to you. See our privacy policy.