> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agno.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Eval Suite

> Declare a few Cases and run them as a suite with the built-in CLI.

```python suite_basic.py theme={null}
"""
Eval Suite
==========

Declare a few Cases and run them as a suite with the built-in CLI.

python cookbook/09_evals/suite/suite_basic.py                 # run all cases
python cookbook/09_evals/suite/suite_basic.py --list          # list cases
python cookbook/09_evals/suite/suite_basic.py --tag smoke     # run a tagged subset
python cookbook/09_evals/suite/suite_basic.py --json-output tmp/evals.json
"""

import sys

from agno.agent import Agent
from agno.eval import Case, cli
from agno.models.openai import OpenAIResponses
from agno.tools.calculator import CalculatorTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------
agent = Agent(
    id="math-tutor",
    model=OpenAIResponses(id="gpt-5.5"),
    tools=[CalculatorTools()],
    instructions="Use the calculator tools for any arithmetic.",
)

# ---------------------------------------------------------------------------
# Declare Cases
# ---------------------------------------------------------------------------
CASES = (
    Case(
        name="factorial_uses_calculator",
        agent=agent,
        input="What is 10! (ten factorial)?",
        tags=("smoke",),
        criteria="States that 10! equals 3628800.",
        expected_tool_calls=("factorial",),
    ),
    Case(
        name="explains_compound_interest",
        agent=agent,
        input="Explain compound interest in one short paragraph.",
        criteria="Explains that interest is earned on both the principal and previously earned interest.",
    ),
)

# ---------------------------------------------------------------------------
# Run Suite
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    sys.exit(cli(CASES))
```

## Run the Example

<Steps>
  <Snippet file="create-venv-step.mdx" />

  <Step title="Install dependencies">
    ```bash theme={null}
    uv pip install -U agno openai
    ```
  </Step>

  <Step title="Export your OpenAI API key">
    <CodeGroup>
      ```bash Mac/Linux theme={null}
      export OPENAI_API_KEY="your_openai_api_key_here"
      ```

      ```bash Windows theme={null}
      $Env:OPENAI_API_KEY="your_openai_api_key_here"
      ```
    </CodeGroup>
  </Step>

  <Step title="Run the example">
    Save the code above as `suite_basic.py`, then run:

    ```bash theme={null}
    python suite_basic.py
    ```
  </Step>
</Steps>

Full source: [cookbook/09\_evals/suite/suite\_basic.py](https://github.com/agno-agi/agno/blob/main/cookbook/09_evals/suite/suite_basic.py)