Firecrawl
Use Firecrawl with Agno to scrape and crawl the web.
FirecrawlTools enable an Agent to perform web crawling and scraping tasks.
Prerequisites
The following example requires the firecrawl-py library and an API key which can be obtained from Firecrawl.
uv pip install -U agno firecrawl-py openaiexport FIRECRAWL_API_KEY=***Example
The following agent will crawl https://finance.yahoo.com/ and return a summary of the content:
from agno.agent import Agent
from agno.tools.firecrawl import FirecrawlTools
agent = Agent(tools=[FirecrawlTools(enable_scrape=False, enable_crawl=True)], markdown=True)
agent.print_response("Summarize this https://finance.yahoo.com/")Toolkit Params
| Parameter | Type | Default | Description |
|---|---|---|---|
api_key | str | None | API key for authentication. Uses FIRECRAWL_API_KEY env var if not provided |
enable_scrape | bool | True | Enables website scraping functionality |
enable_crawl | bool | False | Enables website crawling functionality |
enable_mapping | bool | False | Enables website mapping functionality |
enable_search | bool | False | Enables web search functionality |
all | bool | False | Enables all functionality when set to True |
formats | List[str] | None | List of formats to be used for operations |
limit | int | 10 | Maximum number of items to retrieve |
poll_interval | int | 30 | Interval in seconds between polling for results |
search_params | Dict[str, Any] | None | Parameters for search operations |
api_url | str | https://api.firecrawl.dev | API URL to use for the Firecrawl app |
Toolkit Functions
| Function | Description |
|---|---|
scrape_website | Scrapes a website using Firecrawl. Parameters include url to specify the URL to scrape. The function supports optional formats if specified. Returns the results of the scraping in JSON format. |
crawl_website | Crawls a website using Firecrawl. Parameters include url to specify the URL to crawl. An optional per-call limit overrides the toolkit-level limit. The function supports optional formats and returns the crawling results in JSON format. |
map_website | Maps a website structure using Firecrawl. Parameters include url to specify the URL to map. Returns the mapping results in JSON format. |
search_web | Performs a web search using Firecrawl. Parameters include query for the search term. An optional per-call limit overrides search_params["limit"], which overrides the toolkit limit. Returns search results in JSON format. |