from agno.agent import Agent
from agno.models.vllm import vLLM
from agno.tools.duckduckgo import DuckDuckGoTools
agent = Agent(
model=vLLM(
id="NousResearch/Nous-Hermes-2-Mistral-7B-DPO", top_k=20, enable_thinking=False
),
tools=[DuckDuckGoTools()],
show_tool_calls=True,
markdown=True,
)
agent.print_response("Whats happening in France?", stream=True)
Create a virtual environment
Terminal
and create a python virtual environment.python3 -m venv .venv
source .venv/bin/activate
Install Libraries
pip install -U agno openai vllm duckduckgo-search
Start vLLM server
vllm serve NousResearch/Nous-Hermes-2-Mistral-7B-DPO \
--enable-auto-tool-choice \
--tool-call-parser hermes \
--dtype float16 \
--max-model-len 8192 \
--gpu-memory-utilization 0.9
Run Agent
python cookbook/models/vllm/tool_use.py
Was this page helpful?