# LiteLLM (/models/providers/gateways/litellm/overview)



[LiteLLM](https://docs.litellm.ai/docs/) provides a unified interface for various LLM providers, allowing you to use different models with the same code.

Agno integrates with LiteLLM in two ways:

1. **Direct SDK integration** - Using the LiteLLM Python SDK
2. **[Proxy Server integration](/models/providers/gateways/litellm-openai/overview)** - Using LiteLLM as an OpenAI-compatible proxy

## Prerequisites [#prerequisites]

For both integration methods, you'll need:

```shell
# Install required packages
uv pip install "agno[litellm]"
```

Set up your API key:
Agno checks `LITELLM_API_KEY` first. If it's not set, it falls back to the provider-specific key for the model you're using, e.g. `OPENAI_API_KEY` for an OpenAI model.

```shell
export LITELLM_API_KEY="your_openai_api_key"
```

## SDK Integration [#sdk-integration]

The `LiteLLM` class provides direct integration with the LiteLLM Python SDK.

### Basic Usage [#basic-usage]

```python
from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="gpt-5-mini",  # Model ID to use
        name="LiteLLM",  # Optional display name
        temperature=None,
        top_p=None,
    ),
    markdown=True,
)

# Get a response
agent.print_response("Share a 2 sentence horror story")
```

GPT-5 Mini does not accept the adapter's default sampling settings. Setting `temperature=None` and `top_p=None` omits them from the request. Other models may support explicit values.

### Using Hugging Face Models [#using-hugging-face-models]

LiteLLM can also work with Hugging Face models. Before this example, replace the OpenAI credential with a Hugging Face token that has **Make calls to Inference Providers** permission:

```shell
export LITELLM_API_KEY="your_huggingface_token"
```

`LITELLM_API_KEY` overrides provider-specific environment variables. Change it when switching backends, or unset it and configure each provider's own credential. Select a model with an available [Hugging Face inference provider](/models/providers/gateways/huggingface/overview).

```python
from agno.agent import Agent
from agno.models.litellm import LiteLLM

agent = Agent(
    model=LiteLLM(
        id="huggingface/mistralai/Mistral-7B-Instruct-v0.2",
        top_p=0.95,
    ),
    markdown=True,
)

agent.print_response("What's happening in France?")
```

### Configuration Options [#configuration-options]

The `LiteLLM` class accepts the following parameters:

| Parameter        | Type                       | Description                                                                               | Default   |
| ---------------- | -------------------------- | ----------------------------------------------------------------------------------------- | --------- |
| `id`             | str                        | Model identifier (e.g., "gpt-5-mini" or "huggingface/mistralai/Mistral-7B-Instruct-v0.2") | "gpt-4o"  |
| `name`           | str                        | Display name for the model                                                                | "LiteLLM" |
| `provider`       | str                        | Provider name                                                                             | "LiteLLM" |
| `api_key`        | Optional\[str]             | API key (falls back to LITELLM\_API\_KEY environment variable)                            | None      |
| `api_base`       | Optional\[str]             | Base URL for API requests                                                                 | None      |
| `max_tokens`     | Optional\[int]             | Maximum tokens in the response                                                            | None      |
| `temperature`    | float                      | Sampling temperature                                                                      | 0.7       |
| `top_p`          | float                      | Top-p sampling value                                                                      | 1.0       |
| `request_params` | Optional\[Dict\[str, Any]] | Additional request parameters                                                             | None      |

### Examples [#examples]

<Note>
   View more examples 

  [here](/models/providers/gateways/litellm/usage/basic-stream)

  . 
</Note>