Agentic RAG with Reranking

This example combines three retrieval techniques:

Agentic RAG: Agent decides when to search the knowledge base
Hybrid search: Combines vector similarity with keyword matching
Reranking: Reorders results using a dedicated ranking model

from agno.agent import Agent
from agno.knowledge.knowledge import Knowledge
from agno.knowledge.embedder.openai import OpenAIEmbedder
from agno.knowledge.reranker.cohere import CohereReranker
from agno.models.anthropic import Claude
from agno.vectordb.lancedb import LanceDb, SearchType

knowledge = Knowledge(
    vector_db=LanceDb(
        uri="tmp/lancedb",
        table_name="docs",
        search_type=SearchType.hybrid,
        embedder=OpenAIEmbedder(id="text-embedding-3-small"),
        reranker=CohereReranker(model="rerank-v3.5"),
    ),
)

agent = Agent(
    model=Claude(id="claude-sonnet-4-5"),
    knowledge=knowledge,
    search_knowledge=True,
)

Why Combine These Techniques

Technique	What It Does
Agentic RAG	The agent searches when needed and can reformulate queries
Hybrid search	Catches both semantic matches and exact terms
Reranking	Uses a dedicated model to reorder results by relevance

The three techniques cover complementary retrieval and ranking stages.

How Reranking Works

After hybrid search returns initial results, the reranker:

Takes the query and candidate documents
Scores each document for relevance using a cross-encoder model
Reorders results so the most relevant appear first

Cohere’s rerank-v3.5 is trained specifically for this task.

Example

agentic_rag.py

import asyncio

from agno.agent import Agent
from agno.knowledge.embedder.openai import OpenAIEmbedder
from agno.knowledge.knowledge import Knowledge
from agno.knowledge.reranker.cohere import CohereReranker
from agno.models.anthropic import Claude
from agno.vectordb.lancedb import LanceDb, SearchType

# Create knowledge base with hybrid search and reranking
knowledge = Knowledge(
    vector_db=LanceDb(
        uri="tmp/lancedb",
        table_name="agno_docs",
        search_type=SearchType.hybrid,
        embedder=OpenAIEmbedder(id="text-embedding-3-small"),
        reranker=CohereReranker(model="rerank-v3.5"),
    ),
)

# Load content
asyncio.run(
    knowledge.ainsert(url="https://docs.agno.com/agents/overview.md")
)

# Create agent with knowledge
agent = Agent(
    model=Claude(id="claude-sonnet-4-5"),
    knowledge=knowledge,
    search_knowledge=True,
    instructions=[
        "Search your knowledge before answering.",
        "Include sources in your response.",
    ],
    markdown=True,
)

agent.print_response("What are Agents?", stream=True)

Usage

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

uv venv --python 3.12
.venv\Scripts\activate

Install dependencies

uv pip install -U agno anthropic beautifulsoup4 cohere lancedb openai sqlalchemy

Export your API keys

export ANTHROPIC_API_KEY=your_anthropic_api_key_here
export CO_API_KEY=your_cohere_api_key_here
export OPENAI_API_KEY=your_openai_api_key_here

Run Agent

python agentic_rag.py

Configuration Options

Different Rerankers

Choose one reranker:

from agno.knowledge.reranker.cohere import CohereReranker

reranker = CohereReranker(model="rerank-v3.5")

# uv pip install -U sentence-transformers
from agno.knowledge.reranker.sentence_transformer import SentenceTransformerReranker

reranker = SentenceTransformerReranker(model="BAAI/bge-reranker-v2-m3")

from agno.knowledge.reranker.infinity import InfinityReranker

reranker = InfinityReranker(model="BAAI/bge-reranker-base", host="localhost", port=7997)

For the Infinity option, start the reranker server in a separate terminal:

uv pip install -U "infinity-emb[all]"
infinity_emb v2 --model-id BAAI/bge-reranker-base --port 7997

Use the selected reranker in LanceDB:

from agno.knowledge.embedder.openai import OpenAIEmbedder
from agno.vectordb.lancedb import LanceDb, SearchType

vector_db = LanceDb(
    uri="tmp/lancedb",
    table_name="docs",
    search_type=SearchType.hybrid,
    embedder=OpenAIEmbedder(id="text-embedding-3-small"),
    reranker=reranker,
)

Adjusting Results

knowledge = Knowledge(
    vector_db=vector_db,
    max_results=5,  # Number of results to return after reranking (default: 10)
)

Get Started

Basics

Advanced

Production

Providers

Other

Additional Resources

Agentic RAG with Reranking

Why Combine These Techniques

How Reranking Works

Example

Usage

Configuration Options

Different Rerankers

Adjusting Results

Next Steps

Hybrid Search

Embedders

​Why Combine These Techniques

​How Reranking Works

​Example

​Usage

​Configuration Options

​Different Rerankers

​Adjusting Results

​Next Steps

Hybrid Search

Embedders

Why Combine These Techniques

How Reranking Works

Example

Usage

Configuration Options

Different Rerankers

Adjusting Results

Next Steps