Code Generation

Code

vllm_code_generation.py

from agno.agent import Agent
from agno.models.vllm import VLLM

agent = Agent(
    model=VLLM(id="deepseek-ai/deepseek-coder-6.7b-instruct"),
    description="You are an expert Python developer.",
    markdown=True,
)

agent.print_response(
    "Write a Python function that returns the nth Fibonacci number using dynamic programming."
)

Usage

vLLM installation depends on your operating system and accelerator. Check the official installation guide for supported NVIDIA CUDA, AMD ROCm, Intel XPU, CPU, and Apple Silicon configurations.

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

uv venv --python 3.12
.venv\Scripts\activate

Install dependencies

uv pip install -U agno openai vllm

Start the vLLM server in a separate terminal

source .venv/bin/activate
vllm serve deepseek-ai/deepseek-coder-6.7b-instruct \
    --dtype float32 \
    --tool-call-parser pythonic

Set your API key

export VLLM_API_KEY=xxx

Save the example

Save the code above as vllm_code_generation.py.

Run Agent

python vllm_code_generation.py

Async Agent with Streaming Agent with Storage

⌘I

Get Started

Basics

Advanced

Production

Providers

Other

Additional Resources

Code Generation

Code

Usage

​Code

​Usage

Code

Usage