Streaming Agent

Stream a LiteLLM proxy agent's response with stream=True.

This example uses a local LiteLLM proxy. Follow the two-terminal setup below; see Proxy Server Integration for the distinction between provider and proxy credentials.

Code

from agno.agent import Agent
from agno.models.litellm import LiteLLMOpenAI

agent = Agent(model=LiteLLMOpenAI(id="gpt-4o", base_url="http://127.0.0.1:4000"), markdown=True)

agent.print_response("Share a 2 sentence horror story", stream=True)

Usage

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Set the proxy process credentials

export OPENAI_API_KEY="your_openai_api_key"

Install dependencies

uv pip install -U 'litellm[proxy]>=1.83.0' openai agno

Start the proxy server

litellm --model gpt-4o --host 127.0.0.1 --port 4000

Run Agent

Leave the proxy running. In a second terminal, activate the same virtual environment and set the local client placeholder. If your proxy requires authentication, use its issued key instead.

export LITELLM_API_KEY="local-placeholder"

Save the code above as basic.py, then run:

python basic.py