LiteLLM OpenAI

Proxy Server Integration

LiteLLM can also be used as an OpenAI-compatible proxy server, allowing you to route requests to different models through a unified API.

Starting the Proxy Server

First, install LiteLLM with proxy support and the openai package, which LiteLLMOpenAI uses as its client:

uv pip install 'litellm[proxy]' openai

Start the proxy server:

litellm --model gpt-5-mini --host 127.0.0.1 --port 4000

Set Your API Key

LiteLLMOpenAI requires the LITELLM_API_KEY environment variable to authenticate requests to the proxy server:

export LITELLM_API_KEY=xxx

Using the Proxy

The LiteLLMOpenAI class connects to the LiteLLM proxy using an OpenAI-compatible interface:

from agno.agent import Agent
from agno.models.litellm import LiteLLMOpenAI

agent = Agent(
    model=LiteLLMOpenAI(
        id="gpt-5-mini",  # Model ID to use
    ),
    markdown=True,
)

agent.print_response("Share a 2 sentence horror story")

Configuration Options

The LiteLLMOpenAI class accepts the following parameters:

Parameter	Type	Description	Default
`id`	str	Model identifier	”gpt-4o”
`name`	str	Display name for the model	”LiteLLMOpenAI”
`provider`	str	Provider name	”LiteLLM”
`api_key`	Optional[str]	API key (falls back to LITELLM_API_KEY environment variable)	None
`base_url`	str	URL of the LiteLLM proxy server	”http://0.0.0.0:4000”

LiteLLMOpenAI is a subclass of the OpenAILike class and has access to the same params.

Examples

View more examples here.

Get Started

Basics

Advanced

Production

Providers

Other

Additional Resources

LiteLLM OpenAI

Proxy Server Integration

Starting the Proxy Server

Set Your API Key

Using the Proxy

Configuration Options

Examples

​Proxy Server Integration

​Starting the Proxy Server

​Set Your API Key

​Using the Proxy

​Configuration Options

​Examples

Proxy Server Integration

Starting the Proxy Server

Set Your API Key

Using the Proxy

Configuration Options

Examples