Router Fallback

Configure Router candidate fallback before streaming starts, with separate startup and idle deadlines.

Router can try another candidate before committing a successful 200 text/event-stream response. Once streaming starts, an upstream failure ends the stream; it cannot switch candidates mid-answer. Invalid or unauthorized requests fail immediately, and exhausting all candidates still fails the request. See Router fallback behavior.

fallback.py
"""
Router Fallback
===============

Cookbook example for `ramp/fallback.py`.

Router serves the first candidate that answers, so a rate limit or an outage at one provider
falls through to the next instead of failing the run. Candidates are the `catalog_id` values
from `GET https://api.router.com/v1/models`, optionally suffixed with a service tier.

`models` replaces `id` rather than adding to it: Router rejects a request that carries both.
"""

from agno.agent import Agent
from agno.models.ramp import RampRouter

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

agent = Agent(
    model=RampRouter(
        models=[
            "openai:gpt-5.6-luna",
            "anthropic:claude-haiku-4-5",
            "openai:gpt-5-nano:flex",
        ],
        # Move on to the next candidate if a provider has not answered in time
        provider_timeout=30,
        # Attribute the spend in the Router dashboard
        metadata={"team": "platform"},
    ),
    markdown=True,
)

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    agent.print_response("Write a haiku about routing", stream=True)

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno openai

Export your API keys

export RAMP_ROUTER_API_KEY="your_ramp_router_api_key_here"

Select models available to your key

Choose exact callable IDs from the authenticated GET https://api.router.com/v1/models catalog, following Router's model selection guide. Select candidates with the tool or structured-output capabilities required by this example. Access is specific to your API key.

Update id or the models list in the saved file as needed. In the basic example, also update the model string: keep its ramp: provider prefix and put the selected callable ID after it. Do not pass both id and models.

Set startup and idle deadlines

Add timeout_before_headers=30 to RampRouter(...) for a streaming startup deadline. Keep provider_timeout=30 as the idle deadline after the stream starts. The preserved source comment conflates these two request fields.

Run the example

Save the code above as fallback.py, then run:

python fallback.py

Full source: cookbook/90_models/ramp/fallback.py