llms.txt – What it is, and why WordPress site owners should care

If you run a WordPress site, you’ve probably heard of robots.txt – the little text file that tells search engines what they can and can’t crawl. Well, there’s a new kid on the block: llms.txt.

The idea is simple. Instead of web crawlers, this file is aimed at large language models (LLMs) like ChatGPT, Claude, or Gemini. These models need huge amounts of training data, and they scrape websites to get it. The llms.txt file is an attempt to give site owners a say in that process – letting you allow, restrict, or guide how your content is used by AI tools.

How does llms.txt work?

Much like robots.txt, llms.txt sits in the root of your WordPress site (so https://yoursite.com/llms.txt). Inside, you can define rules such as:

User-agent: GPTBot
Disallow: /

That would tell OpenAI’s crawler to steer clear of your site entirely. Or you could get more specific and only block certain directories.

Why llms.txt matters for WordPress

Content protection – If your site publishes original blogs, images, or resources, you might not want them fed into AI training models without permission.

Traffic considerations – LLM scrapers can hit your site with a surprising amount of requests. Blocking them can reduce server load.

Brand control – You get to decide if your words show up in the dataset that trains tomorrow’s AI.

Adding llms.txt in WordPress

The simplest way is to create a plain text file called llms.txt, upload it to your WordPress root directory, and add your rules. If you’re already familiar with editing robots.txt, it’s the same process. For those less comfy with FTP, plugins like Virtual Robots.txt could easily be adapted to handle an llms.txt file too.

Should you use llms.txt in WordPress?

That depends. Right now, support for llms.txt is still evolving – not every AI company respects it. But it’s worth setting up if you want to signal your preferences early. At worst, it’s ignored. At best, it gives you a little more control over how your WordPress content is used in the age of AI.