HuggingFace
Use Hugging Face models with Agno agents.
Hugging Face hosts model repositories and routes inference through its Inference Providers.
Agno's HuggingFace model connects to this API using the huggingface_hub client library.
See the Inference Providers documentation for model/provider routing and availability.
Authentication
Create a Hugging Face token with Make calls to Inference Providers permission and set HF_TOKEN. Ensure your account has access and sufficient credits for a provider serving the chosen model. A model repository alone does not guarantee hosted inference availability.
export HF_TOKEN="your_value_here"Example
Install the huggingface_hub package:
uv pip install -U huggingface_hub agnoUse HuggingFace with your Agent:
from agno.agent import Agent
from agno.models.huggingface import HuggingFace
agent = Agent(
model=HuggingFace(
id="meta-llama/Meta-Llama-3-8B-Instruct",
max_tokens=4096,
),
markdown=True
)
# Print the response in the terminal
agent.print_response("Share a 2 sentence horror story.") View more examples here.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
id | str | "meta-llama/Meta-Llama-3-8B-Instruct" | The id of the Hugging Face model to use |
name | str | "HuggingFace" | The name of the model |
provider | str | "HuggingFace" | The provider of the model |
api_key | Optional[str] | None | The API key for Hugging Face (defaults to HF_TOKEN env var) |
base_url | Optional[Union[str, httpx.URL]] | None | The base URL for the inference endpoint |
max_tokens | Optional[int] | None | Maximum number of tokens to generate |
temperature | Optional[float] | None | Controls randomness in the model's output |
top_p | Optional[float] | None | Controls diversity via nucleus sampling |
timeout | Optional[float] | None | Request timeout in seconds |
HuggingFace is a subclass of the Model class and has access to the same params.