Readers
Readers
Convert files, URLs, and text into searchable documents.
Readers transform raw content into Document objects that can be chunked, embedded, and stored in your knowledge base. Each reader handles a specific format (PDF, CSV, Markdown, etc.) and extracts text and metadata.
from agno.knowledge.reader.pdf_reader import PDFReader
reader = PDFReader(chunk=True, chunk_size=5000)
documents = reader.read("company_handbook.pdf")How Readers Work
- Parse: Read the raw content using format-specific logic
- Extract: Pull out text and metadata (page numbers, authors, etc.)
- Chunk: Split large content into smaller pieces (if enabled)
- Return: Provide a list of
Documentobjects ready for embedding
# Output structure
Document(
content="The extracted text...",
id="unique_id",
name="document_name",
meta_data={"page": 1, "source": "handbook.pdf"},
)Supported Readers
| Reader | Description |
|---|---|
PDFReader | Extract text from PDF files |
DoclingReader | Process multiple formats via Docling |
TextReader | Plain text files |
MarkdownReader | Markdown files |
CSVReader | CSV files (rows become documents) |
FieldLabeledCSVReader | CSV rows as field-labeled text |
JSONReader | JSON files |
DocxReader | Microsoft Word documents |
ExcelReader | Excel workbooks (.xlsx and .xls) |
PPTXReader | PowerPoint presentations |
ArxivReader | Academic papers from arXiv |
WikipediaReader | Wikipedia articles |
YouTubeReader | YouTube transcripts |
WebsiteReader | Crawl websites recursively |
WebSearchReader | Web search results |
FirecrawlReader | Web scraping via Firecrawl API |
TavilyReader | Extract URL content via Tavily API |
LLMsTxtReader | Read llms.txt files |
SitemapReader | Discover and read pages from XML sitemaps |
Using Readers with Knowledge
Pass a reader to knowledge.insert() to override automatic format detection:
from agno.knowledge.knowledge import Knowledge
from agno.knowledge.reader.pdf_reader import PDFReader
knowledge = Knowledge(vector_db=vector_db)
# Use custom reader configuration
reader = PDFReader(chunk_size=3000, split_on_pages=True)
knowledge.insert(path="documents/", reader=reader)Auto-Selection
Agno automatically selects the right reader based on file extension or URL:
from agno.knowledge.reader.reader_factory import ReaderFactory
# By file extension
reader = ReaderFactory.get_reader_for_extension(".pdf") # PDFReader
reader = ReaderFactory.get_reader_for_extension(".csv") # CSVReader
# By URL
reader = ReaderFactory.get_reader_for_url("https://youtube.com/watch?v=...") # YouTubeReaderWhen using knowledge.insert(), this happens automatically.
Configuration
Chunking
reader = PDFReader(
chunk=True, # Enable chunking (default: True)
chunk_size=5000, # Characters per chunk
)Format-Specific Options
# PDF with encryption
reader = PDFReader(
password="secret",
split_on_pages=True, # One document per page
)
# PDF with OCR for images
reader = PDFImageReader(
password="secret",
)
# CSV with custom encoding
reader = CSVReader(
encoding="latin-1",
)
# Text with encoding override
reader = TextReader(
encoding="utf-8",
)Runtime Options
Override settings when calling read():
documents = reader.read(
"file.pdf",
name="custom_document_name", # Override default naming
password="runtime_password", # Password at read time
)Async Processing
Readers expose async interfaces, but how they perform parsing and I/O varies. An async method does not guarantee that every operation is nonblocking:
import asyncio
# Single file
documents = await reader.async_read("file.pdf")
# Batch processing
tasks = [reader.async_read(file) for file in files]
all_documents = await asyncio.gather(*tasks)Custom Chunking Strategy
Override the default chunking behavior:
from agno.knowledge.chunking.semantic import SemanticChunking
reader = PDFReader(
chunk=True,
chunking_strategy=SemanticChunking(),
)See Chunking for available strategies.
Restricting URL Fetches
For readers that support it, allowed_hosts restricts their own URL fetching to a fixed hostname allowlist. This parameter is not a policy over every Knowledge ingestion fetch; see the Docling ingestion caveat. URLs outside the list are skipped and return no documents. Matching is case-insensitive and applies to the whole hostname, so list every subdomain you want to permit.
reader = WebsiteReader(allowed_hosts=["docs.agno.com"])WebsiteReader, WebSearchReader, and LLMsTxtReader also re-check the allowlist on each redirect, so an allowed host can't redirect to a blocked one. FirecrawlReader and DoclingReader validate the initial URL only.
Error Handling
Most readers return an empty list when processing fails. JSONReader raises instead. Check logs for debugging information:
documents = reader.read("corrupted.pdf")
if not documents:
print("Failed to read file, check logs for details")