> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agno.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Trafilatura

> TrafilaturaTools wraps the trafilatura library to expose extract_text, extract_metadata_only, html_to_text, extract_batch, and crawl_website functions.

**TrafilaturaTools** enable an Agent to extract text and metadata from web pages using the Trafilatura library.

## Prerequisites

The following example requires the `trafilatura` library.

```shell theme={null}
uv pip install -U trafilatura openai
```

## Example

The following agent can extract and analyze web content:

```python theme={null}
from agno.agent import Agent
from agno.tools.trafilatura import TrafilaturaTools

agent = Agent(
    instructions=[
        "You are a web content extraction specialist",
        "Extract clean text and structured data from web pages",
        "Provide detailed analysis of web content and metadata",
        "Help with content research and web data collection",
    ],
    tools=[TrafilaturaTools()],
)

agent.print_response("Extract the main content from https://example.com/article", stream=True)
```

## Toolkit Params

| Parameter                      | Type            | Default  | Description                                                          |
| ------------------------------ | --------------- | -------- | -------------------------------------------------------------------- |
| `output_format`                | `str`           | `"txt"`  | Default output format (txt, json, xml, markdown, csv, html, xmltei). |
| `include_comments`             | `bool`          | `True`   | Whether to extract comments along with main text.                    |
| `include_tables`               | `bool`          | `True`   | Whether to include table content.                                    |
| `include_images`               | `bool`          | `False`  | Whether to include image information (experimental).                 |
| `include_formatting`           | `bool`          | `False`  | Whether to preserve text formatting.                                 |
| `include_links`                | `bool`          | `False`  | Whether to preserve links (experimental).                            |
| `with_metadata`                | `bool`          | `False`  | Whether to include metadata in extractions.                          |
| `favor_precision`              | `bool`          | `False`  | Whether to prefer precision over recall.                             |
| `favor_recall`                 | `bool`          | `False`  | Whether to prefer recall over precision.                             |
| `target_language`              | `Optional[str]` | `None`   | Target language filter (ISO 639-1 format).                           |
| `deduplicate`                  | `bool`          | `False`  | Whether to remove duplicate segments.                                |
| `max_tree_size`                | `Optional[int]` | `None`   | Maximum tree size for processing.                                    |
| `max_crawl_urls`               | `int`           | `10`     | Maximum number of URLs to crawl per website.                         |
| `max_known_urls`               | `int`           | `100000` | Maximum number of known URLs during crawling.                        |
| `enable_extract_text`          | `bool`          | `True`   | Whether to extract text content.                                     |
| `enable_extract_metadata_only` | `bool`          | `True`   | Whether to extract metadata information.                             |
| `enable_html_to_text`          | `bool`          | `True`   | Whether to convert HTML content to clean text.                       |
| `enable_extract_batch`         | `bool`          | `True`   | Whether to extract content from multiple URLs in batch.              |
| `enable_crawl_website`         | `bool`          | `True`   | Whether to crawl a website and extract content from multiple pages.  |
| `all`                          | `bool`          | `False`  | Enable all tools.                                                    |

## Toolkit Functions

| Function                | Description                                              |
| ----------------------- | -------------------------------------------------------- |
| `extract_text`          | Extract main text content from a URL.                    |
| `extract_metadata_only` | Extract metadata information from web pages.             |
| `html_to_text`          | Convert HTML content to clean text.                      |
| `crawl_website`         | Crawl a website and extract content from multiple pages. |
| `extract_batch`         | Extract content from multiple URLs in batch.             |

## Developer Resources

* [Tools Source](https://github.com/agno-agi/agno/blob/main/libs/agno/agno/tools/trafilatura.py)
* [Trafilatura Documentation](https://trafilatura.readthedocs.io/)
* [Web Scraping Best Practices](https://trafilatura.readthedocs.io/en/latest/corefunctions.html)