Structured Output

Get structured output from a vLLM agent with a Pydantic output_schema.

Code

from typing import List

from agno.agent import Agent
from agno.models.vllm import VLLM
from pydantic import BaseModel, Field

class MovieScript(BaseModel):
    name: str = Field(..., description="Give a name to this movie")
    setting: str = Field(
        ..., description="Provide a nice setting for a blockbuster movie."
    )
    ending: str = Field(
        ...,
        description="Ending of the movie. If not available, provide a happy ending.",
    )
    genre: str = Field(
        ...,
        description="Genre of the movie. If not available, select action, thriller or romantic comedy.",
    )
    characters: List[str] = Field(..., description="Name of characters for this movie.")
    storyline: str = Field(
        ..., description="3 sentence storyline for the movie. Make it exciting!"
    )

agent = Agent(
    model=VLLM(id="Qwen/Qwen2.5-7B-Instruct", top_k=20, enable_thinking=False),
    description="You write movie scripts.",
    output_schema=MovieScript,
)

agent.print_response("Llamas ruling the world")

Usage

Install vLLM in a server environment using the installation guide for your hardware. The GPU commands below require a supported Linux accelerator environment. The Agno client can run in a separate Python environment with agno, openai and the example's other dependencies.

Run vllm serve in a dedicated server terminal with its environment active and keep it running. In a client terminal, activate the environment for your saved Python file, export VLLM_API_KEY, and run the agent. The examples use port 8000; for a server on another host, set the matching base_url on VLLM. A server started without --api-key accepts a placeholder key, but Agno still requires a nonempty VLLM_API_KEY.

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno pydantic openai

Start vLLM server

vllm serve Qwen/Qwen2.5-7B-Instruct \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --dtype float16 \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.9

Set your API key

export VLLM_API_KEY=xxx

Run Agent

Save the code above as structured_output.py, then run:

python structured_output.py