Firecrawl Reader
Parameters and methods of FirecrawlReader, which scrapes or crawls websites into documents using Firecrawl.
FirecrawlReader reads data from websites using Firecrawl.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
api_key | Optional[str] | None | Firecrawl API key for authentication |
params | Optional[Dict] | None | Additional parameters passed to the Firecrawl API |
mode | Literal["scrape", "crawl"] | "scrape" | "scrape" reads a single page, "crawl" follows links to read multiple pages |
chunk | bool | True | Chunk documents after reading |
chunk_size | int | 5000 | Size of each chunk when chunking is enabled |
chunking_strategy | Optional[ChunkingStrategy] | SemanticChunking() | Strategy used to chunk documents |
name | Optional[str] | None | Name of the reader |
description | Optional[str] | None | Description of the reader |
allowed_hosts | Optional[List[str]] | None | Allowed hostnames for the initial URL sent to Firecrawl. This local check does not constrain redirects or the provider's remote crawl; configure the Firecrawl crawl scope separately. |
Methods
read()
Scrapes or crawls the given URL, depending on mode, and returns a list of documents.
| Parameter | Type | Default | Description |
|---|---|---|---|
url | str | Required | URL of the website to scrape or crawl |
name | Optional[str] | None | Name override for the resulting documents |
async_read()
Async variant of read(). Accepts the same parameters.
Setup
Install the reader and chunking dependencies:
uv pip install -U agno firecrawl-py chonkie openaiConfigure FIRECRAWL_API_KEY or pass api_key. The default semantic chunker uses OpenAI embeddings and requires OPENAI_API_KEY when chunking. Set an explicit non-semantic chunking_strategy or chunk=False to avoid embedding calls; Chonkie is still required when importing the reader.