WebSearchReader is a reader class that allows you to read data from web search results.
ParameterTypeDefaultDescription
search_timeoutint10Timeout for search operations in seconds
request_timeoutint30Timeout for HTTP requests in seconds
delay_between_requestsfloat2.0Delay between requests in seconds
max_retriesint3Maximum number of retries for failed requests
user_agentstr"Mozilla/5.0..."User agent string for HTTP requests
search_engineLiteral["duckduckgo", "google"]"duckduckgo"Search engine to use
search_delayfloat3.0Delay between search requests in seconds
max_search_retriesint2Maximum retries for search operations
rate_limit_delayfloat5.0Delay when rate limited in seconds
exponential_backoffboolTrueWhether to use exponential backoff for retries
chunking_strategyOptional[ChunkingStrategy]SemanticChunking()Strategy for chunking content