Agentic RAG with Reranking

This example combines three retrieval techniques:

Agentic RAG: Agent decides when to search the knowledge base
Hybrid search: Combines vector similarity with keyword matching
Reranking: Reorders results using a dedicated ranking model

from agno.agent import Agent
from agno.knowledge.knowledge import Knowledge
from agno.knowledge.embedder.openai import OpenAIEmbedder
from agno.knowledge.reranker.cohere import CohereReranker
from agno.models.anthropic import Claude
from agno.vectordb.lancedb import LanceDb, SearchType

knowledge = Knowledge(
    vector_db=LanceDb(
        uri="tmp/lancedb",
        table_name="docs",
        search_type=SearchType.hybrid,
        embedder=OpenAIEmbedder(id="text-embedding-3-small"),
    ),
    reranker=CohereReranker(model="rerank-v3.5"),
)

agent = Agent(
    model=Claude(id="claude-sonnet-4-5"),
    knowledge=knowledge,
    search_knowledge=True,
)

Why Combine These Techniques

Technique	What It Does
Agentic RAG	The agent searches when needed and can reformulate queries
Hybrid search	Catches both semantic matches and exact terms
Reranking	Uses a dedicated model to reorder results by relevance

The three techniques cover complementary retrieval and ranking stages.

How Reranking Works

After hybrid search returns a candidate pool, the reranker on Knowledge:

Takes the query and candidate documents
Scores each document for relevance using a cross-encoder model
Reorders results so the most relevant appear first

Cohere's rerank-v3.5 is trained specifically for this task.

Example

agentic_rag.py

import asyncio

from agno.agent import Agent
from agno.knowledge.embedder.openai import OpenAIEmbedder
from agno.knowledge.knowledge import Knowledge
from agno.knowledge.reranker.cohere import CohereReranker
from agno.models.anthropic import Claude
from agno.vectordb.lancedb import LanceDb, SearchType

# Create knowledge base with hybrid search and reranking
knowledge = Knowledge(
    vector_db=LanceDb(
        uri="tmp/lancedb",
        table_name="agno_docs",
        search_type=SearchType.hybrid,
        embedder=OpenAIEmbedder(id="text-embedding-3-small"),
    ),
    reranker=CohereReranker(model="rerank-v3.5"),
)

# Load content
asyncio.run(
    knowledge.ainsert(url="https://docs.agno.com/agents/overview.md")
)

# Create agent with knowledge
agent = Agent(
    model=Claude(id="claude-sonnet-4-5"),
    knowledge=knowledge,
    search_knowledge=True,
    instructions=[
        "Search your knowledge before answering.",
        "Include sources in your response.",
    ],
    markdown=True,
)

agent.print_response("What are Agents?", stream=True)

Usage

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno anthropic beautifulsoup4 cohere lancedb openai sqlalchemy

Export your API keys

export ANTHROPIC_API_KEY=your_anthropic_api_key_here
export CO_API_KEY=your_cohere_api_key_here
export OPENAI_API_KEY=your_openai_api_key_here

Run Agent

python agentic_rag.py

Configuration Options

Different Rerankers

Choose one reranker:

from agno.knowledge.reranker.cohere import CohereReranker

reranker = CohereReranker(model="rerank-v3.5")

# uv pip install -U sentence-transformers
from agno.knowledge.reranker.sentence_transformer import SentenceTransformerReranker

reranker = SentenceTransformerReranker(model="BAAI/bge-reranker-v2-m3")

from agno.knowledge.reranker.infinity import InfinityReranker

reranker = InfinityReranker(model="BAAI/bge-reranker-base", host="localhost", port=7997)

For the Infinity option, start the reranker server in a separate terminal:

uv pip install -U "infinity-emb[all]"
infinity_emb v2 --model-id BAAI/bge-reranker-base --port 7997

Pass the selected reranker to Knowledge:

from agno.knowledge.embedder.openai import OpenAIEmbedder
from agno.knowledge.knowledge import Knowledge
from agno.vectordb.lancedb import LanceDb, SearchType

knowledge = Knowledge(
    vector_db=LanceDb(
        uri="tmp/lancedb",
        table_name="docs",
        search_type=SearchType.hybrid,
        embedder=OpenAIEmbedder(id="text-embedding-3-small"),
    ),
    reranker=reranker,
)

Adjusting Results

knowledge = Knowledge(
    vector_db=vector_db,
    reranker=reranker,
    max_results=5,  # Number of results to return after reranking (default: 10)
)

Knowledge fetches a wider candidate pool than max_results for the reranker. See Reranking to size the pool.