LiteLLM | Agno

LiteLLM provides a unified interface for various LLM providers, allowing you to use different models with the same code.

Agno integrates with LiteLLM in two ways:

Direct SDK integration - Using the LiteLLM Python SDK
Proxy Server integration - Using LiteLLM as an OpenAI-compatible proxy

Prerequisites

For both integration methods, you'll need:

# Install required packages
uv pip install "agno[litellm]"

Set up your API key: Agno checks LITELLM_API_KEY first. If it's not set, it falls back to the provider-specific key for the model you're using, e.g. OPENAI_API_KEY for an OpenAI model.

export LITELLM_API_KEY="your_openai_api_key"

SDK Integration

The LiteLLM class provides direct integration with the LiteLLM Python SDK.

Basic Usage

from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="gpt-5-mini",  # Model ID to use
        name="LiteLLM",  # Optional display name
        temperature=None,
        top_p=None,
    ),
    markdown=True,
)

# Get a response
agent.print_response("Share a 2 sentence horror story")

GPT-5 Mini does not accept the adapter's default sampling settings. Setting temperature=None and top_p=None omits them from the request. Other models may support explicit values.

Using Hugging Face Models

LiteLLM can also work with Hugging Face models. Before this example, replace the OpenAI credential with a Hugging Face token that has Make calls to Inference Providers permission:

export LITELLM_API_KEY="your_huggingface_token"

LITELLM_API_KEY overrides provider-specific environment variables. Change it when switching backends, or unset it and configure each provider's own credential. Select a model with an available Hugging Face inference provider.

from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="huggingface/mistralai/Mistral-7B-Instruct-v0.2",
        top_p=0.95,
    ),
    markdown=True,
)

agent.print_response("What's happening in France?")

Configuration Options

The LiteLLM class accepts the following parameters:

Parameter	Type	Description	Default
`id`	str	Model identifier (e.g., "gpt-5-mini" or "huggingface/mistralai/Mistral-7B-Instruct-v0.2")	"gpt-4o"
`name`	str	Display name for the model	"LiteLLM"
`provider`	str	Provider name	"LiteLLM"
`api_key`	Optional[str]	API key (falls back to LITELLM_API_KEY environment variable)	None
`api_base`	Optional[str]	Base URL for API requests	None
`max_tokens`	Optional[int]	Maximum tokens in the response	None
`temperature`	float	Sampling temperature	0.7
`top_p`	float	Top-p sampling value	1.0
`request_params`	Optional[Dict[str, Any]]	Additional request parameters	None

Examples

View more examples here.