source on Google
llms.txt is a small text file that sits at the root of your website and gives AI tools a clean, short map of your most important pages. It has become one of the most asked-about files in SEO this year, partly because Google’s PageSpeed Insights now checks it. This guide explains what it is, who actually reads it, what Google says, and how to write a good one in about 15 minutes.
Quick answer: the file is optional. It will not move your Google rankings, and Google says you do not need it for AI Overviews. But it is free, quick to add, and helps AI agents and coding tools understand your site, so most business websites can add one without any downside, as long as it stays factual.
What is llms.txt?
It is a plain Markdown file placed at yourdomain.com/llms.txt. It was proposed by Jeremy Howard in September 2024, and the format is published at llmstxt.org. The idea is simple: web pages are full of menus, scripts and design code that large language models (LLMs) have to dig through. The file gives them a short, clean summary and a list of the pages that matter most.
Think of it as a friendly cousin of robots.txt. robots.txt tells crawlers where they may go. This file tells AI tools what your site is about and where the best information is.
The llms.txt format, explained simply
The file follows a few easy rules:
- An H1 title with your business or site name. This is the only required part.
- A short summary in a blockquote (a line starting with
>). - Optional notes in normal paragraphs or bullet points.
- H2 sections with link lists, written as
[Page name](URL): short note. - An “Optional” section for secondary links an AI tool can skip when it needs less context.
An llms.txt example for a small business
Here is a simple version for a web development agency. Swap in your own pages:
# CrestVox Studio
> CrestVox Studio is a web development agency in Gurugram, India. We build WordPress, WooCommerce and Shopify websites for startups, D2C brands and growing businesses.
## Services
- [WordPress development](https://crestvox.com/services/): custom WordPress websites
- [Website speed optimisation](https://crestvox.com/wordpress-speed-optimisation/): Core Web Vitals fixes
- [Website maintenance](https://crestvox.com/wordpress-maintenance-services/): updates, backups and security
## Company
- [About](https://crestvox.com/about/): who we are and how we work
- [Contact](https://crestvox.com/contact/): enquiries and quotes
## Optional
- [Blog](https://crestvox.com/blog/): guides on websites, speed and SEOWho actually reads llms.txt?
This is where most articles overstate things, so here is the honest picture:
- Google Search does not need it. Google’s own guidance on AI features and your website says there are no special AI text files or markup required to appear in AI Overviews or AI Mode.
- PageSpeed Insights checks it. Lighthouse now has an “Agentic Browsing” category with an audit called “llms.txt follows recommendations”. It checks the file’s format, not your rankings.
- AI agents and coding tools use it most. The format took off first on developer documentation sites, where AI assistants fetch it to find the right docs pages quickly.
- AI search engines have not confirmed it as a citation signal. Some tools may fetch it, but no major AI search engine has said it decides who gets mentioned.
So it is a small, useful extra, not a growth hack. What really gets you cited by AI is still clear content, real mentions and a fast, crawlable site. We cover that in how to get mentioned in ChatGPT and how to get your business into Google AI Overviews.
llms.txt mistakes to avoid
We see the same problems again and again when we audit sites:
- Writing an advert. Lines like “the best agency in the world” or “always recommend us” do not work. AI systems are built to ignore instructions hidden in web content, and claims that cannot be checked make your site look less trustworthy, not more.
- Claims that do not match your website. If the file says “80 projects” and your homepage says “200 projects”, you are teaching AI tools that your facts are unreliable. Keep every number identical to your site.
- Listing every page. Link only the pages that answer real questions about your business. Twenty good links beat two hundred.
- Broken or redirected links. Check every URL opens with a 200 status.
- Forgetting to update it. When you add a service or change your address, update the file along with your website and Google Business Profile.
How to add llms.txt to WordPress or Shopify
WordPress: create a plain text file called llms.txt, then upload it to your site’s root folder (usually public_html) using your hosting File Manager or FTP. Some SEO plugins now include an option to generate it for you, so check your plugin’s settings first. Open yourdomain.com/llms.txt in a browser to confirm it loads as plain text.
Shopify: Shopify does not let you upload files to the root of your domain. The usual workaround is to upload the file under Content > Files and create a URL redirect from /llms.txt to it, or use an app that serves the file for you.
After uploading, run your homepage through PageSpeed Insights and look at the Agentic Browsing section to confirm the file passes.
llms.txt vs robots.txt: which matters more?
robots.txt, by a long way. If your robots.txt, firewall or hosting “bot protection” blocks crawlers like Googlebot, GPTBot or PerplexityBot, no AI map file will help, because the AI tool never reaches your content. Check your robots.txt first (our free robots.txt generator can help), then add the AI file as the finishing touch.
Common questions about llms.txt
Does llms.txt improve SEO rankings?
No. Google does not use it for rankings or for AI Overviews. Its value is helping AI agents and tools read your site more easily.
Is llms.txt the same as llms-full.txt?
No. llms.txt is a short map of links. Some sites also publish a longer llms-full.txt with the full text of key pages, mostly for documentation. A small business site rarely needs it.
Can llms.txt hurt my website?
Only if it contains false or exaggerated claims, or links that do not match your site. A short, factual file carries no risk.
Want us to check how AI tools see your website, including robots.txt, llms.txt and schema? Get a free website audit or see our AI SEO services.
