# Agent-as-Judge Eval Metrics (/examples/evals/agent-as-judge/agent-as-judge-eval-metrics)



Demonstrates that eval model metrics are accumulated back into the original agent's run\_output when AgentAsJudgeEval is used as a post\_hook.

```python title="agent_as_judge_eval_metrics.py"
"""
Agent-as-Judge Eval Metrics
============================

Demonstrates that eval model metrics are accumulated back into the
original agent's run_output when AgentAsJudgeEval is used as a post_hook.

After the agent runs, the evaluator agent makes its own model call.
Those eval tokens show up under "eval_model" in run_output.metrics.details.
"""

from agno.agent import Agent
from agno.eval.agent_as_judge import AgentAsJudgeEval
from agno.models.openai import OpenAIChat
from rich.pretty import pprint

# ---------------------------------------------------------------------------
# Create eval as a post-hook
# ---------------------------------------------------------------------------
eval_hook = AgentAsJudgeEval(
    name="Quality Check",
    model=OpenAIChat(id="gpt-5.6-luna"),
    criteria="Response should be accurate, clear, and concise",
    scoring_strategy="binary",
)

agent = Agent(
    model=OpenAIChat(id="gpt-5.6-luna"),
    instructions="Answer questions concisely.",
    post_hooks=[eval_hook],
)

# ---------------------------------------------------------------------------
# Run
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    result = agent.run("What is the capital of France?")

    # The run metrics now include both agent model + eval model tokens
    if result.metrics:
        print("Total tokens (agent + eval):", result.metrics.total_tokens)

        if result.metrics.details:
            # Agent's own model call
            if "model" in result.metrics.details:
                agent_tokens = sum(
                    metric.total_tokens for metric in result.metrics.details["model"]
                )
                print("Agent model tokens:", agent_tokens)

            # Eval model call (accumulated from evaluator agent)
            if "eval_model" in result.metrics.details:
                eval_tokens = sum(
                    metric.total_tokens
                    for metric in result.metrics.details["eval_model"]
                )
                print("Eval model tokens:", eval_tokens)
                for metric in result.metrics.details["eval_model"]:
                    print(f"  Evaluator: {metric.id} ({metric.provider})")

            print("\nFull metrics details:")
            pprint(result.metrics.to_dict())
```

## Run the Example [#run-the-example]

<Steps>
    <Step title="Set up your virtual environment">
      <CodeBlockTabs defaultValue="Mac">
        <CodeBlockTabsList>
          <CodeBlockTabsTrigger value="Mac">
            Mac
          </CodeBlockTabsTrigger>

          <CodeBlockTabsTrigger value="Windows">
            Windows
          </CodeBlockTabsTrigger>
        </CodeBlockTabsList>

        <CodeBlockTab value="Mac">
          ```bash
          uv venv --python 3.12
          source .venv/bin/activate
          ```
        </CodeBlockTab>

        <CodeBlockTab value="Windows">
          ```bash
          uv venv --python 3.12
          .venv\Scripts\activate
          ```
        </CodeBlockTab>
      </CodeBlockTabs>
    </Step>

  <Step title="Install dependencies">
    ```bash
    uv pip install -U agno openai
    ```
  </Step>

  <Step title="Export your OpenAI API key">
    <CodeBlockTabs defaultValue="Mac/Linux">
      <CodeBlockTabsList>
        <CodeBlockTabsTrigger value="Mac/Linux">
          Mac/Linux
        </CodeBlockTabsTrigger>

        <CodeBlockTabsTrigger value="Windows">
          Windows
        </CodeBlockTabsTrigger>
      </CodeBlockTabsList>

      <CodeBlockTab value="Mac/Linux">
        ```bash
        export OPENAI_API_KEY="your_openai_api_key_here"
        ```
      </CodeBlockTab>

      <CodeBlockTab value="Windows">
        ```bash
        $Env:OPENAI_API_KEY="your_openai_api_key_here"
        ```
      </CodeBlockTab>
    </CodeBlockTabs>
  </Step>

  <Step title="Run the example">
    Save the code above as `agent_as_judge_eval_metrics.py`, then run:

    ```bash
    python agent_as_judge_eval_metrics.py
    ```
  </Step>
</Steps>

Full source: [cookbook/09\_evals/agent\_as\_judge/agent\_as\_judge\_eval\_metrics.py](https://github.com/agno-agi/agno/blob/8f36eaf2d18e91afa7b327eec66a3cd3685dcb87/cookbook/09_evals/agent_as_judge/agent_as_judge_eval_metrics.py)