LiteLLM

LiteLLM provides a unified interface for various LLM providers, allowing you to use different models with the same code. Agno integrates with LiteLLM in two ways:

Direct SDK integration - Using the LiteLLM Python SDK
Proxy Server integration - Using LiteLLM as an OpenAI-compatible proxy

Prerequisites

For both integration methods, you’ll need:

# Install required packages
uv pip install agno litellm

Set up your API key: Agno checks LITELLM_API_KEY first. If it’s not set, it falls back to the provider-specific key for the model you’re using, e.g. OPENAI_API_KEY for an OpenAI model.

export LITELLM_API_KEY=your_api_key_here

SDK Integration

The LiteLLM class provides direct integration with the LiteLLM Python SDK.

Basic Usage

from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="gpt-5-mini",  # Model ID to use
        name="LiteLLM",  # Optional display name
    ),
    markdown=True,
)

# Get a response
agent.print_response("Share a 2 sentence horror story")

Using Hugging Face Models

LiteLLM can also work with Hugging Face models:

from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="huggingface/mistralai/Mistral-7B-Instruct-v0.2",
        top_p=0.95,
    ),
    markdown=True,
)

agent.print_response("What's happening in France?")

Configuration Options

The LiteLLM class accepts the following parameters:

Parameter	Type	Description	Default
`id`	str	Model identifier (e.g., “gpt-5-mini” or “huggingface/mistralai/Mistral-7B-Instruct-v0.2”)	“gpt-4o”
`name`	str	Display name for the model	”LiteLLM”
`provider`	str	Provider name	”LiteLLM”
`api_key`	Optional[str]	API key (falls back to LITELLM_API_KEY environment variable)	None
`api_base`	Optional[str]	Base URL for API requests	None
`max_tokens`	Optional[int]	Maximum tokens in the response	None
`temperature`	float	Sampling temperature	0.7
`top_p`	float	Top-p sampling value	1.0
`request_params`	Optional[Dict[str, Any]]	Additional request parameters	None

Examples

View more examples here.

Get Started

Basics

Advanced

Production

Providers

Other

Additional Resources

Prerequisites

SDK Integration

Basic Usage

Using Hugging Face Models

Configuration Options

Examples

​Prerequisites

​SDK Integration

​Basic Usage

​Using Hugging Face Models

​Configuration Options

​Examples

Prerequisites

SDK Integration

Basic Usage

Using Hugging Face Models

Configuration Options

Examples