HuggingFace

The HuggingFace model provides access to models hosted on the HuggingFace Hub.

Parameters

Parameter	Type	Default	Description
`id`	`str`	`"meta-llama/Meta-Llama-3-8B-Instruct"`	The id of the Hugging Face model to use
`name`	`str`	`"HuggingFace"`	The name of the model
`provider`	`str`	`"HuggingFace"`	The provider of the model
`store`	`Optional[bool]`	`None`	Declared but rejected by the current `chat_completion` method when set; leave `None`.
`frequency_penalty`	`Optional[float]`	`None`	Penalizes new tokens based on their frequency in the text so far
`logit_bias`	`Optional[Any]`	`None`	Modifies the likelihood of specified tokens appearing
`logprobs`	`Optional[bool]`	`None`	Return log probabilities of the output tokens
`max_tokens`	`Optional[int]`	`None`	Maximum number of tokens to generate
`presence_penalty`	`Optional[float]`	`None`	Penalizes new tokens based on whether they appear in the text so far
`seed`	`Optional[int]`	`None`	Random seed for deterministic sampling
`stop`	`Optional[Union[str, List[str]]]`	`None`	Sequences where the API stops generating further tokens
`temperature`	`Optional[float]`	`None`	Controls randomness in the model's output
`top_logprobs`	`Optional[int]`	`None`	Number of most likely tokens to return log probabilities for
`top_p`	`Optional[float]`	`None`	Controls diversity via nucleus sampling
`request_params`	`Optional[Dict[str, Any]]`	`None`	Additional parameters to include in the request
`api_key`	`Optional[str]`	`None`	The API key for Hugging Face (defaults to HF_TOKEN env var)
`base_url`	`Optional[Union[str, httpx.URL]]`	`None`	The base URL for the inference endpoint
`timeout`	`Optional[float]`	`None`	Request timeout in seconds
`max_retries`	`Optional[int]`	`None`	Declared but rejected by the current Hugging Face client constructor when set; leave `None`.
`default_headers`	`Optional[Any]`	`None`	Declared but rejected by the current Hugging Face client constructor when set; leave `None`.
`default_query`	`Optional[Any]`	`None`	Declared but rejected by the current Hugging Face client constructor when set; leave `None`.
`client_params`	`Optional[Dict[str, Any]]`	`None`	Additional parameters for client configuration
`client`	`Optional[InferenceClient]`	`None`	Pre-configured HuggingFace inference client
`async_client`	`Optional[AsyncInferenceClient]`	`None`	Pre-configured async HuggingFace inference client
`model_type`	`ModelType`	`ModelType.MODEL`	Functional role of this model (e.g. `MODEL`, `OUTPUT_MODEL`, `PARSER_MODEL`). Set by the agent
`supports_native_structured_outputs`	`bool`	`False`	Whether the model supports structured outputs natively
`supports_json_schema_outputs`	`bool`	`False`	Whether the model uses JSON schema for structured outputs
`system_prompt`	`Optional[str]`	`None`	System prompt from the model added to the agent
`instructions`	`Optional[List[str]]`	`None`	Instructions from the model added to the agent
`tool_message_role`	`str`	`"tool"`	Role of tool messages
`assistant_message_role`	`str`	`"assistant"`	Role of assistant messages
`cache_response`	`bool`	`False`	Cache model responses to avoid redundant API calls during development
`cache_ttl`	`Optional[int]`	`None`	Time-to-live for cached responses, in seconds
`cache_dir`	`Optional[str]`	`None`	Directory for the response cache. Defaults to `~/.agno/cache/model_responses`
`retries`	`int`	`0`	Number of retries to attempt before raising a ModelProviderError
`delay_between_retries`	`int`	`1`	Delay between retries, in seconds
`exponential_backoff`	`bool`	`False`	If True, the delay between retries is doubled each time
`retry_with_guidance`	`bool`	`True`	Retry a failed model invocation once with a guidance message appended
`retry_with_guidance_limit`	`int`	`1`	Number of times to retry the model invocation with guidance
With the current Hugging Face inference clients, use `client_params={"headers": {...}}` for custom headers or inject configured clients. The incompatible fields above must remain unset; adding `client_params` does not remove them.

Parameters

On this page