AccuracyEval

AccuracyEval runs an agent or team against an input, then scores the response against an expected output with an evaluator agent on a 1-10 scale.

from agno.eval import AccuracyEval, AccuracyResult

Parameters

Parameter	Type	Default	Description
`input`	`Union[str, Callable]`	Required	Input sent to the agent or team. A callable must return the input string.
`expected_output`	`Union[str, Callable]`	Required	Expected answer the evaluator scores against. A callable must return the expected string.
`agent`	`Optional[Agent]`	`None`	Agent to evaluate. Provide exactly one of `agent` or `team`.
`team`	`Optional[Team]`	`None`	Team to evaluate. Provide exactly one of `agent` or `team`.
`name`	`Optional[str]`	`None`	Evaluation name.
`num_iterations`	`int`	`1`	Number of times the input is run and scored.
`model`	`Optional[Model]`	`None`	Model for the default evaluator agent. Falls back to OpenAI `o4-mini` when unset.
`evaluator_agent`	`Optional[Agent]`	`None`	Custom agent that scores the response. Replaces the default evaluator.
`additional_guidelines`	`Optional[Union[str, List[str]]]`	`None`	Guidelines appended to the default evaluator's instructions.
`additional_context`	`Optional[str]`	`None`	Context appended to the default evaluator's instructions.
`print_summary`	`bool`	`False`	Print the summary table with score statistics after the run.
`print_results`	`bool`	`False`	Print the per-iteration results table after the run.
`show_spinner`	`bool`	`True`	Render the progress spinner. Set to `False` when the eval runs inside code that must not write to the console.
`file_path_to_save_results`	`Optional[str]`	`None`	File path where the result is saved.
`debug_mode`	`bool`	`False`	Enable debug logs. Defaults to `True` when the `AGNO_DEBUG` environment variable is `"true"`.
`db`	`Optional[Union[BaseDb, AsyncBaseDb]]`	`None`	Database where the eval run is stored.
`telemetry`	`bool`	`True`	Log minimal telemetry for the eval run.
`result`	`Optional[AccuracyResult]`	`None`	Result of the evaluation. Set after a run completes.

Each execution creates a fresh run_id. The evaluation object’s result holds its latest returned result.

Methods

Method	Return type	Description
`run(*, print_summary=True, print_results=True)`	`Optional[AccuracyResult]`	Run the agent or team `num_iterations` times and score each response. Raises with an async database; use `arun()` instead.
`arun(*, print_summary=True, print_results=True)`	`Optional[AccuracyResult]`	Async version of `run()`.
`run_with_output(*, output, print_summary=True, print_results=True)`	`Optional[AccuracyResult]`	Score the given `output` string instead of running the agent or team.
`arun_with_output(*, output, print_summary=True, print_results=True)`	`Optional[AccuracyResult]`	Async version of `run_with_output()`.

AccuracyResult

Field	Type	Description
`run_id`	`str`	Unique ID generated for this evaluation execution.
`results`	`List[AccuracyEvaluation]`	One entry per scored iteration.
`avg_score`	`Optional[float]`	Average score across iterations. `None` when there are no results.
`mean_score`	`Optional[float]`	Mean score across iterations. Same value as `avg_score`.
`min_score`	`Optional[float]`	Lowest score across iterations.
`max_score`	`Optional[float]`	Highest score across iterations.
`std_dev_score`	`Optional[float]`	Standard deviation of scores. `0` with a single result.

AccuracyResult also exposes print_summary(console=None) and print_results(console=None) to render the statistics and per-iteration tables.

AccuracyEvaluation

Each entry in AccuracyResult.results is an AccuracyEvaluation:

Field	Type	Description
`input`	`str`	Input sent to the agent or team.
`output`	`str`	Response produced by the agent or team.
`expected_output`	`str`	Expected answer the response was scored against.
`score`	`int`	Accuracy score from 1 to 10.
`reason`	`str`	The evaluator's reasoning for the score.

Parameters

Methods

AccuracyResult

AccuracyEvaluation

On this page