Website Reader

WebsiteReader reads data from websites.

Parameters

Parameter	Type	Default	Description
`chunking_strategy`	`Optional[ChunkingStrategy]`	`FixedSizeChunking()`	Strategy used to chunk documents
`max_depth`	`int`	`3`	Maximum depth level for crawling links
`max_links`	`int`	`10`	Maximum number of links to crawl
`timeout`	`int`	`10`	Request timeout in seconds
`proxy`	`Optional[str]`	`None`	Proxy URL used for requests
`allowed_hosts`	`Optional[List[str]]`	`None`	Hostnames the reader is allowed to fetch from. See Restricting URL Fetches.

WebsiteReader also accepts the base Reader constructor parameters.

Crawls a website starting from the given URL and returns a list of documents.

Parameter	Type	Default	Description
`url`	`str`	Required	URL of the website to crawl and read
`name`	`Optional[str]`	`None`	Name override for the resulting documents

Async variant of read(). Accepts the same parameters.

⌘I