The Firecrawl Reader uses the Firecrawl API to scrape and crawl web content, converting it into documents for your knowledge base.

​
Code

examples/concepts/knowledge/readers/firecrawl_reader.py
import os

from agno.knowledge.reader.firecrawl_reader import FirecrawlReader

api_key = os.getenv("FIRECRAWL_API_KEY")

reader = FirecrawlReader(
    api_key=api_key,
    mode="scrape",
    chunk=True,
    # for crawling
    # params={
    #     'limit': 5,
    #     'scrapeOptions': {'formats': ['markdown']}
    # }
    # for scraping
    params={"formats": ["markdown"]},
)

try:
    print("Starting scrape...")
    documents = reader.read("https://github.com/agno-agi/agno")

    if documents:
        for doc in documents:
            print(doc.name)
            print(doc.content)
            print(f"Content length: {len(doc.content)}")
            print("-" * 80)
    else:
        print("No documents were returned")

except Exception as e:
    print(f"Error type: {type(e)}")
    print(f"Error occurred: {str(e)}")

​
Usage

1

Create a virtual environment

Open the Terminal and create a python virtual environment.
python3 -m venv .venv
source .venv/bin/activate
2

Install libraries

pip install -U firecrawl-py agno
3

Set API Key

export FIRECRAWL_API_KEY="your-firecrawl-api-key"
4

Run Agent

python examples/concepts/knowledge/readers/firecrawl_reader.py

​
Params

ParameterTypeDefaultDescription
api_keyOptional[str]NoneFirecrawl API key for authentication
paramsOptional[Dict]NoneAdditional parameters to pass to the Firecrawl API
modeLiteral["scrape", "crawl"]"scrape"Mode of operation - "scrape" for single page, "crawl" for multiple pages
urlstrRequiredURL of the website to scrape or crawl