Scale across replicas

Keep PostgreSQL as the durable queue store and add Redis for shared live run events and cancellation. Configure shared credentials and MCP transport behavior before increasing the number of API instances.

Start from a working Render deployment. Complete AgentCore and S3 setup if your platform runs engineers, so their work is independent of API containers.

What each service does

A request can reach one API instance while another instance executes the run. Redis lets the instances coordinate that live activity. Adding replicas increases execution capacity; it does not create a separate queue for every agent.

Add a Redis-compatible service

Create a paid Render Key Value instance in the same workspace and region as the API. Use its internal connection URL and keep external access disabled unless you have a specific need for it. Choose a persistence policy appropriate for retaining live coordination data and monitor memory use.

Save the internal URL as REDIS_URL in .env.production. The URL must be reachable by every API instance.

Add redis to the dependencies in pyproject.toml, then regenerate the template's requirements:

./scripts/generate_requirements.sh

Commit both dependency files with the application changes below. Rebuild the API from the updated requirements.

Share queue coordination

In app/main.py, replace the existing queue argument with:

queue=QueueConfig(durable=True, db=db, redis=getenv("REDIS_URL")),

Keep the other AgentOS arguments. getenv, QueueConfig, and db already exist in this file. Confirm REDIS_URL is set in the production environment; leaving it unset keeps coordination local to each process.

The redis setting wires both the live event stream and distributed cancellation. PostgreSQL remains the durable queue store in this configuration.

Share internal credentials

Generate a random value once:

openssl rand -hex 32

Save it as AGENTOS_INTERNAL_SERVICE_TOKEN in .env.production, and add this argument to the existing AgentOS(...) call:

internal_service_token=getenv("AGENTOS_INTERNAL_SERVICE_TOKEN"),

AGENTOS_INTERNAL_SERVICE_TOKEN is the environment name introduced by this guide; the constructor argument above reads it. Use the same value on every replica. Otherwise, automatically generated scheduler credentials can differ between instances.

Also keep JWT verification settings, MCP_CONNECT_SECRET, AGENTOS_URL, database connections, and sandbox checkpoint settings consistent across replicas. Keep queue timing and retry settings uniform during deployment.

Configure MCP for your clients

For this guide, use stateless MCP transport so legacy transport sessions do not depend on reaching the same API instance. In app/interfaces.py, add stateless=True to the existing MCPConfig(...) inside attach_interfaces():

         default_tools=True,
+        stateless=True,
     )

Keep the existing tools list, which exposes OS Manager, and the authentication configuration in app/main.py. attach_interfaces() supplies the final MCP configuration, so changing only mcp=True in app/main.py would be overwritten.

Modern MCP requests are already sessionless. For legacy clients, stateless mode gives up server-initiated requests and transport-stream resumability; saved AgentOS conversations and runs remain separate from transport sessions. Test the clients you intend to support.

If you need legacy transport sessions, use an affinity strategy compatible with those clients and validate it through your load balancer. Affinity alone does not share live run events or cancellation.

Deploy coordination before adding replicas

Commit and push the code and dependency changes while the API still has one instance. Upload the new settings:

./scripts/render/env-sync.sh

Check startup logs, a background run, streaming, and cancellation. Once the shared configuration works, change the API service's numInstances in render.yaml from 1 to 2, commit, push, and apply the Blueprint update. See Render's Blueprint reference for instance configuration.

Keep one Uvicorn worker per instance for this guide. Account for database connections and model usage as you add instances.

Check behavior across instances

Use disposable work and inspect per-instance logs to confirm requests and execution reach different instances. Sending several requests through the public URL alone does not prove this.

With the default max_attempts=1, a crashed run can end as a visible failure. Enabling retries re-executes a run from its original input and can repeat external actions. Choose retry policies deliberately for tools that change external state.

Check file downloads separately: checkpoint storage is not a public artifact-delivery service.

Continue to Run recurring improvement reviews.