Code Generation

Generate Python code with a vLLM agent running a code model.

Code

from agno.agent import Agent
from agno.models.vllm import VLLM

agent = Agent(
    model=VLLM(id="deepseek-ai/deepseek-coder-6.7b-instruct"),
    description="You are an expert Python developer.",
    markdown=True,
)

agent.print_response(
    "Write a Python function that returns the nth Fibonacci number using dynamic programming."
)

Usage

Install vLLM in a server environment using the installation guide for your hardware. The GPU commands below require a supported Linux accelerator environment. The Agno client can run in a separate Python environment with agno, openai and the example's other dependencies.

Run vllm serve in a dedicated server terminal with its environment active and keep it running. In a client terminal, activate the environment for your saved Python file, export VLLM_API_KEY, and run the agent. The examples use port 8000; for a server on another host, set the matching base_url on VLLM. A server started without --api-key accepts a placeholder key, but Agno still requires a nonempty VLLM_API_KEY.

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno openai

Start the vLLM server in a separate terminal

source .venv/bin/activate
vllm serve deepseek-ai/deepseek-coder-6.7b-instruct \
    --dtype float32 \
    --tool-call-parser pythonic

Set your API key

export VLLM_API_KEY=xxx

Save the example

Save the code above as vllm_code_generation.py.

Run Agent

python vllm_code_generation.py