LiteLLM OpenAI

Proxy Server Integration

LiteLLM can also be used as an OpenAI-compatible proxy server, allowing you to route requests to different models through a unified API.

Starting the Proxy Server

Use Python 3.10 or newer. Install Agno, LiteLLM with proxy support, and the OpenAI client:

uv pip install agno 'litellm[proxy]>=1.83.0' openai

In the terminal that runs the proxy, set the upstream OpenAI credential and start the server:

export OPENAI_API_KEY="your_openai_api_key"
litellm --model gpt-5-mini --host 127.0.0.1 --port 4000

Leave the proxy running. In a second terminal, activate the same environment and set the client credential. This local CLI example has no proxy authentication, so the OpenAI client can use a placeholder:

export LITELLM_API_KEY="local-placeholder"

OPENAI_API_KEY belongs to the proxy process and authorizes its OpenAI requests. LITELLM_API_KEY is sent from Agno to the proxy. For an authenticated proxy, use the key issued by that proxy instead of the placeholder. See the LiteLLM proxy setup.

Using the Proxy

The LiteLLMOpenAI class connects to the LiteLLM proxy using an OpenAI-compatible interface:

from agno.agent import Agent
from agno.models.litellm import LiteLLMOpenAI

agent = Agent(
    model=LiteLLMOpenAI(
        id="gpt-5-mini",  # Model ID to use
        base_url="http://127.0.0.1:4000",
    ),
    markdown=True,
)

agent.print_response("Share a 2 sentence horror story")

Save this example as proxy_agent.py and run python proxy_agent.py in the client terminal.

Configuration Options

The LiteLLMOpenAI class accepts the following parameters:

Parameter	Type	Description	Default
`id`	str	Model identifier	"gpt-4o"
`name`	str	Display name for the model	"LiteLLMOpenAI"
`provider`	str	Provider name	"LiteLLM"
`api_key`	Optional[str]	API key (falls back to LITELLM_API_KEY environment variable)	None
`base_url`	str	URL of the LiteLLM proxy server	"http://0.0.0.0:4000"

LiteLLMOpenAI is a subclass of the OpenAILike class and has access to the same params.

Examples

View more examples here.