Multi-agent AutoGen systems default to a single heavyweight LLM for every role, which wastes money on tasks a smaller model handles fine. This guide walks through autogen cost optimization cheap models n4n.ai by mapping each agent to a purpose-fit model and routing all calls through one OpenAI-compatible gateway.
1. Audit your agent topology
Before changing code, list every agent in your GroupChat. Most AutoGen deployments evolve from a single assistant into a cast of specialists: an orchestrator that plans steps, a coder that writes and executes Python, a critic that reviews output, and a summarizer that compresses the transcript for the user.
Classify each role by reasoning depth. The orchestrator needs strong instruction following and multi-step planning. The coder benefits from decent code syntax but rarely needs philosophical reasoning. The summarizer mostly extracts key lines from a known context. That gap between roles is exactly where cost hides.
Walk the actual conversation log from a representative task. Tag each message with the agent that produced it and note whether a cheaper model could have generated an equivalent reply. You will usually find that 60–70% of messages are summarization, status, or formatting—work a small model does at a fraction of the price.
2. Assign model tiers per agent
Pick a model per role based on intuition grounded in public capability profiles: heavy for planning, mid for code generation, cheap for extraction. The n4n.ai OpenAI-compatible endpoint addresses 240+ models and provides automatic fallback when a provider is rate-limited, so you can point every agent at the same base URL and just swap the model field.
from autogen import AssistantAgent, UserProxyAgent
GATEWAY = "https://api.n4n.ai/v1"
llm_heavy = {"model": "gpt-4o", "base_url": GATEWAY, "api_key": "KEY"}
llm_mid = {"model": "gpt-3.5-turbo", "base_url": GATEWAY, "api_key": "KEY"}
llm_cheap = {"model": "anthropic/claude-3-haiku-20240307", "base_url": GATEWAY, "api_key": "KEY"}
orchestrator = AssistantAgent("orchestrator", llm_config=llm_heavy)
coder = AssistantAgent("coder", llm_config=llm_mid)
summarizer = AssistantAgent("summarizer", llm_config=llm_cheap)
user = UserProxyAgent("user", human_input_mode="NEVER")Avoid the trap of sharing one config dict across agents. AutoGen reads llm_config per instance, so heterogeneous dicts are first-class. If you mutate a shared dict at runtime, you silently upgrade or downgrade every agent at once.
A practical tiering:
- Orchestrator:
gpt-4oor equivalent frontier model. - Coder:
gpt-3.5-turboormistralai/mixtral-8x7b-instruct. - Critic: mid-tier if it must catch logic bugs; cheap if it only checks style.
- Summarizer:
claude-3-haikuor similar low-cost instruction model.
3. Wire up group chat with heterogeneous agents
GroupChat schedules speakers but does not care which model backs each agent. Instantiate the manager with the strongest model because it synthesizes speaker transitions and resolves deadlocks.
from autogen import GroupChat, GroupChatManager
chat = GroupChat(
agents=[orchestrator, coder, summarizer, user],
messages=[],
max_round=12,
speaker_selection_method="auto",
)
manager = GroupChatManager(chat, llm_config=llm_heavy)
user.initiate_chat(manager, message="Build a CSV parser with tests.")Run this and watch the first three rounds. The orchestrator picks the coder; the coder returns code; the summarizer condenses. Only the orchestrator’s planning and the manager’s routing hit the expensive model. The coder and summarizer rounds stay on cheap tiers.
If you use speaker_selection_method="manual" or a custom function, bake the tier logic into the selector. For example, never let the summarizer speak before the coder has produced a block—that prevents empty summary calls.
4. Forward cache-control and routing directives
Cheap models still accumulate context. When you know a summarizer prompt is reused across rounds, set provider cache hints. The gateway honors client routing directives and forwards provider cache-control hints, so it passes them through to the upstream provider without extra plumbing.
llm_cheap_cached = {
"model": "anthropic/claude-3-haiku-20240307",
"base_url": GATEWAY,
"api_key": "KEY",
"extra_body": {"cache_control": {"type": "ephemeral"}}
}
summarizer_cached = AssistantAgent("summarizer", llm_config=llm_cheap_cached)This matters in group chat where the same system message and task brief repeat on every speaker turn. Cache hits cut token billing on the long prefix, sometimes by half on multi-round tasks.
You can also pin a specific provider via routing hints if one is cheaper in your region. The gateway forwards those hints, so your AutoGen code stays provider-agnostic.
5. Meter usage per agent
Cost control fails without measurement. The gateway returns per-token usage on each response. In AutoGen you can hook the underlying OpenAI client after a run to pull totals:
# inspect last usage for a given agent
usage = orchestrator.client.session.last_response.usage
print(f"prompt:{usage.prompt_tokens} completion:{usage.completion_tokens}")Wrap this in a small loop over your agent registry to build a per-agent cost table:
for agent in [orchestrator, coder, summarizer]:
u = agent.client.session.last_response.usage
print(agent.name, u.prompt_tokens + u.completion_tokens)If the summarizer suddenly consumes more tokens than the coder, your prompt is leaking full transcript context instead of a trimmed slice. Per-token metering lets you spot that within a single debug session rather than on a monthly bill.
Common pitfalls
Latency stacking. Cheap models are faster per token but may need more rounds to converge. In a group chat with max_round=12, a weak summarizer can trigger re-plans from the orchestrator. Set strict max_round and keep the orchestrator strong to cut loops early.
Context window mismatch. Haiku handles 200k tokens; some small open-weight models cap at 4k. If the coder receives a huge file, it truncates silently and returns broken code. Match model limits to agent inputs explicitly.
Over-delegation. Engineers assign a critic agent to every message. A critic that calls a mid-tier model on each round doubles cost. Make critics event-driven: only run when the coder produces a block, not on every summarizer ping.
Silent fallback confusion. Automatic fallback is useful, but if you meter by model name you may see a different model than requested during provider degradation. Log the actual model field from the usage response, not your local config.
Tradeoffs
You trade some peak quality for predictable spend. A haiku summarizer will miss nuance a gpt-4o catches, but for status updates it is fine. Keep the heavy model on path-critical reasoning and push everything else down.
Test the degraded path: temporarily force the coder to the cheap tier and see if the orchestrator compensates. If the task still completes, you can permanently downgrade. If it spirals into retries, keep the mid tier.
For further autogen cost optimization cheap models n4n.ai, revisit the agent list after two weeks of metering. Drop any agent that never changes the final answer to the cheapest tier, and promote any that cause repeated retries. The goal is not the lowest possible model per agent, but the cheapest model that does not increase total rounds.