Add llms.txt and llms-full.txt support for AI agent readiness - Part 1

AI agents (ChatGPT, Claude, Perplexity, Gemini) are increasingly how people find businesses and services — and they need a different signal than Google does. The emerging standard is llms.txt and llms-full.txt, two plain Markdown files served from your web root that give AI agents a clean, structured map of your site — what it covers, your main categories, key pages. Think of it as robots.txt but for LLMs. The spec is documented at llmstxt.org.

Cloudflare recently launched a free AI readiness scanner at isitagentready.com that scores your site 0–100 across five dimensions. The llms.txt files directly impact the Content score. Without them, AI agents have to scrape HTML and often skip the site entirely.

The request:

1. Add a dedicated AI Identity section in the eDirectory admin panel where directory owners can provide:

   • Directory name and tagline

   • Purpose and description of the directory

   • Who the directory is for (target audience)

   • What regions, industries, or niches it covers

   • When the directory was founded/launched

   • Primary contact name and email

   • Owner/operator website or social links

2. Auto-generate llms.txt and llms-full.txt by combining that AI Identity information with the directory's live category structure, key pages, and site description — similar to how sitemaps are already generated

3. Serve both files from the web root with correct Content-Type: text/plain headers and keep them updated as categories and pages change

The AI Identity section is important because auto-generating from categories alone gives AI agents the structure of the directory but not the context — who runs it, why it exists, who it serves. That context is what helps an AI agent decide whether your directory is the right source to cite for a given query.

For sites already on Cloudflare, this would also push scores significantly on isitagentready.com. Worth noting that full marks on the Content dimension also requires serving Markdown on Accept: text/markdown requests, which could be a natural follow-on feature.  See Part 2

This is a quick win that puts eDirectory sites ahead of the vast majority of the web right now. The window where this is a differentiator rather than table stakes won't be open long.

Aaron