LLMs.txt Tools - Agentic Documentation Discovery

Read an llms.txt index with LLMsTxtTools, then fetch only the documentation pages the agent decides are relevant.

Use an llms.txt index to discover documentation pages before choosing which ones to fetch.

llms_txt_tools.py
"""
LLMs.txt Tools - Agentic Documentation Discovery
=============================

Demonstrates how to use LLMsTxtTools in agentic mode where the agent:
1. Reads the llms.txt index to discover available documentation pages
2. Decides which pages are relevant to the user's question
3. Fetches only the specific pages it needs

The llms.txt format (https://llmstxt.org) is a standardized way for websites
to provide LLM-friendly documentation indexes.
"""

from agno.agent import Agent
from agno.models.openai import OpenAIResponses
from agno.tools.llms_txt import LLMsTxtTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

agent = Agent(
    model=OpenAIResponses(id="gpt-5.4"),
    tools=[LLMsTxtTools()],
    instructions=[
        "You can read llms.txt files to discover documentation for any project.",
        "First use get_llms_txt_index to see what pages are available.",
        "Then use read_llms_txt_url to fetch only the pages relevant to the user's question.",
    ],
    markdown=True,
)

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    agent.print_response(
        "Using the llms.txt at https://docs.agno.com/llms.txt, "
        "find and read the documentation about how to create an agent with tools",
        markdown=True,
        stream=True,
    )

Without a knowledge base, the toolkit exposes two tools: index discovery and direct URL reading. max_urls does not cap the index or the number of calls the agent can make in this mode. A linked sub-index is another document to inspect; recursive expansion is not automatic. The fetched text is used for the current run and is not stored in a vector database.

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno beautifulsoup4 openai

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as llms_txt_tools.py, then run:

python llms_txt_tools.py

Full source: cookbook/91_tools/llms_txt_tools.py