# AutoGen Instantiation Performance Evaluation (/examples/evals/performance/comparison/autogen-instantiation)



Demonstrates agent instantiation benchmarking with AutoGen.

This evaluation performs 2,010 callable invocations: 10 warm-ups, 1,000 runtime measurements, and 1,000 memory measurements. It measures construction, not model inference.

`PerformanceEval` runs the callable separately for each enabled metric: warm-ups first, then runtime measurements, then memory measurements. `num_iterations` applies to each metric, and the default is 10 additional warm-up calls. Model retries, tools, delegation, and memory extraction can add provider requests beyond the callable count.

Memory measurements use Python’s `tracemalloc`; they do not measure process RSS, GPU memory, database-server memory, or remote model memory. Record dependency versions, database state, and model settings when comparing results.

```python title="autogen_instantiation.py"
"""
AutoGen Instantiation Performance Evaluation
============================================

Demonstrates agent instantiation benchmarking with AutoGen.
"""

from typing import Literal

from agno.eval.performance import PerformanceEval
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient


# ---------------------------------------------------------------------------
# Create Benchmark Tool
# ---------------------------------------------------------------------------
def get_weather(city: Literal["nyc", "sf"]):
    """Use this to get weather information."""
    if city == "nyc":
        return "It might be cloudy in nyc"
    elif city == "sf":
        return "It's always sunny in sf"
    else:
        raise AssertionError("Unknown city")


tools = [get_weather]


# ---------------------------------------------------------------------------
# Create Benchmark Function
# ---------------------------------------------------------------------------
def instantiate_agent():
    return AssistantAgent(
        name="assistant",
        model_client=OpenAIChatCompletionClient(
            model="gpt-5.6-luna",
            model_info={
                "vision": False,
                "function_calling": True,
                "json_output": False,
                "family": "gpt-5.6-luna",
                "structured_output": True,
            },
        ),
        tools=tools,
    )


# ---------------------------------------------------------------------------
# Create Evaluation
# ---------------------------------------------------------------------------
autogen_instantiation = PerformanceEval(func=instantiate_agent, num_iterations=1000)

# ---------------------------------------------------------------------------
# Run Evaluation
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    autogen_instantiation.run(print_results=True, print_summary=True)
```

## Run the Example [#run-the-example]

<Steps>
    <Step title="Set up your virtual environment">
      <CodeBlockTabs defaultValue="Mac">
        <CodeBlockTabsList>
          <CodeBlockTabsTrigger value="Mac">
            Mac
          </CodeBlockTabsTrigger>

          <CodeBlockTabsTrigger value="Windows">
            Windows
          </CodeBlockTabsTrigger>
        </CodeBlockTabsList>

        <CodeBlockTab value="Mac">
          ```bash
          uv venv --python 3.12
          source .venv/bin/activate
          ```
        </CodeBlockTab>

        <CodeBlockTab value="Windows">
          ```bash
          uv venv --python 3.12
          .venv\Scripts\activate
          ```
        </CodeBlockTab>
      </CodeBlockTabs>
    </Step>

  <Step title="Install dependencies">
    ```bash
    uv pip install -U agno "autogen-ext[openai]" autogen-agentchat
    ```
  </Step>

  <Step title="Export your API keys">
    <CodeBlockTabs defaultValue="Mac/Linux">
      <CodeBlockTabsList>
        <CodeBlockTabsTrigger value="Mac/Linux">
          Mac/Linux
        </CodeBlockTabsTrigger>

        <CodeBlockTabsTrigger value="Windows">
          Windows
        </CodeBlockTabsTrigger>
      </CodeBlockTabsList>

      <CodeBlockTab value="Mac/Linux">
        ```bash
        export OPENAI_API_KEY="your_openai_api_key_here"
        ```
      </CodeBlockTab>

      <CodeBlockTab value="Windows">
        ```bash
        $Env:OPENAI_API_KEY="your_openai_api_key_here"
        ```
      </CodeBlockTab>
    </CodeBlockTabs>
  </Step>

  <Step title="Run the example">
    Save the code above as `autogen_instantiation.py`, then run:

    ```bash
    python autogen_instantiation.py
    ```
  </Step>
</Steps>

Full source: [cookbook/09\_evals/performance/comparison/autogen\_instantiation.py](https://github.com/agno-agi/agno/blob/8f36eaf2d18e91afa7b327eec66a3cd3685dcb87/cookbook/09_evals/performance/comparison/autogen_instantiation.py)