The multi-agent debate pattern is one of the most practical ways to improve LLM reasoning on complex tasks. Instead of a single model generating an answer, you spin up multiple agents that argue, critique, and refine each other’s output. This tutorial shows you how to implement it with AutoGen and route calls through n4n.ai so you can swap models without rewriting your orchestration logic.
Prerequisites
You need Python 3.10+ and an n4n.ai API key. Install the dependencies:
pip install autogen-agentchat==0.2.0 autogen-ext==0.2.0 openai python-dotenvCreate a .env file with your n4n.ai credentials:
N4N_API_KEY=your_key_here
N4N_BASE_URL=https://api.n4n.ai/v1The n4n.ai endpoint is OpenAI-compatible, so AutoGen’s OpenAIChatCompletionClient works directly. You get access to 240+ models through one base URL, and the gateway handles provider fallback automatically when a model is rate-limited or degraded.
The debate architecture
We’ll build three agents: a Proposer that generates an initial answer, a Critic that finds flaws, and a Judge that synthesizes a final response. They communicate through a RoundRobinGroupChat with a termination condition that stops after a fixed number of rounds or when the judge signals consensus.
# debate_agents.py
import os
from dotenv import load_dotenv
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient
load_dotenv()
model_client = OpenAIChatCompletionClient(
model="gpt-4o-mini",
api_key=os.getenv("N4N_API_KEY"),
base_url=os.getenv("N4N_BASE_URL"),
)
proposer = AssistantAgent(
name="Proposer",
model_client=model_client,
system_message=(
"You are a careful reasoner. Given a question, produce a clear, "
"well-structured answer. Be concise but thorough. Do not hedge "
"excessively; take a position and defend it."
),
)
critic = AssistantAgent(
name="Critic",
model_client=model_client,
system_message=(
"You are a rigorous critic. Your job is to find logical gaps, "
"factual errors, missing edge cases, and unstated assumptions in "
"the Proposer's answer. Be specific. Quote the passage you're "
"critiquing. Then suggest a concrete improvement."
),
)
judge = AssistantAgent(
name="Judge",
model_client=model_client,
system_message=(
"You are a synthesizer. Read the Proposer's answer and the Critic's "
"feedback. Produce a final answer that incorporates valid critiques "
"and resolves disagreements. If the Critic's points are weak, say so "
"and keep the original. End your response with exactly: "
"'CONSENSUS_REACHED' if the answer is solid, or 'NEEDS_MORE_WORK' "
"if another round would help."
),
)
termination = TextMentionTermination("CONSENSUS_REACHED") | MaxMessageTermination(12)
team = RoundRobinGroupChat(
participants=[proposer, critic, judge],
termination_condition=termination,
)Run a test question to verify the pipeline works:
# run_debate.py
import asyncio
from debate_agents import team
async def main():
task = (
"Should a startup optimize for profitability or growth in its first "
"two years? Assume a B2B SaaS company with $500k ARR, 15% monthly "
"churn, and a 3-month sales cycle."
)
async for msg in team.run_stream(task=task):
print(f"[{msg.source}] {msg.content[:200]}...")
if __name__ == "__main__":
asyncio.run(main())Expected output (truncated):
[Proposer] Optimize for growth. At $500k ARR with 15% monthly churn, you're losing $75k/month...
[Critic] The Proposer ignores that 15% monthly churn implies 80% annual logo churn...
[Judge] The Critic correctly identifies the churn math error. Revised answer: ...
CONSENSUS_REACHEDAdding structured output for the judge
Free-text consensus signals are brittle. Let’s give the Judge a tool that emits a structured decision so the termination condition can be deterministic.
# structured_judge.py
from pydantic import BaseModel
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient
import os
from dotenv import load_dotenv
load_dotenv()
class JudgeDecision(BaseModel):
final_answer: str
consensus_reached: bool
critique_summary: str
model_client = OpenAIChatCompletionClient(
model="gpt-4o-mini",
api_key=os.getenv("N4N_API_KEY"),
base_url=os.getenv("N4N_BASE_URL"),
)
judge = AssistantAgent(
name="Judge",
model_client=model_client,
system_message=(
"You are a synthesizer. Read the Proposer's answer and the Critic's "
"feedback. Produce a final answer that incorporates valid critiques. "
"Call the `emit_decision` tool with your structured decision."
),
tools=[
{
"type": "function",
"function": {
"name": "emit_decision",
"description": "Emit the final structured decision",
"parameters": JudgeDecision.model_json_schema(),
},
}
],
)Update the termination condition to watch for the tool call:
# debate_agents_v2.py
from autogen_agentchat.conditions import ToolCallTermination
from structured_judge import judge, proposer, critic
termination = ToolCallTermination("emit_decision") | MaxMessageTermination(12)
team = RoundRobinGroupChat(
participants=[proposer, critic, judge],
termination_condition=termination,
)Now the debate stops cleanly when the Judge calls emit_decision with consensus_reached=true.
Routing directives for model selection
Different debate roles benefit from different models. The Proposer needs creativity, the Critic needs precision, and the Judge needs synthesis. n4n.ai honors client-side routing directives via the model parameter — you can specify any of the 240+ available models per agent.
# routed_agents.py
from autogen_ext.models.openai import OpenAIChatCompletionClient
import os
from dotenv import load_dotenv
load_dotenv()
def make_client(model: str) -> OpenAIChatCompletionClient:
return OpenAIChatCompletionClient(
model=model,
api_key=os.getenv("N4N_API_KEY"),
base_url=os.getenv("N4N_BASE_URL"),
)
proposer_client = make_client("anthropic/claude-3.5-sonnet")
critic_client = make_client("openai/gpt-4o")
judge_client = make_client("google/gemini-1.5-pro")Each agent gets its own client instance. The gateway handles the provider routing, and you get per-token usage metering broken down by model automatically.
Adding a memory buffer for long debates
If you increase the round limit, agents need context from earlier rounds. AutoGen’s ChatCompletionContext handles this, but you can also inject a rolling summary to keep token usage bounded.
# memory_debate.py
from autogen_agentchat.messages import TextMessage
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination
class SummarizingGroupChat(RoundRobinGroupChat):
def __init__(self, *args, summary_every: int = 4, **kwargs):
super().__init__(*args, **kwargs)
self.summary_every = summary_every
self._message_count = 0
async def _run_single_turn(self, *args, **kwargs):
result = await super()._run_single_turn(*args, **kwargs)
self._message_count += 1
if self._message_count % self.summary_every == 0:
await self._inject_summary()
return result
async def _inject_summary(self):
# In practice, call a summarizer agent here and prepend a SystemMessage
passFor production, wire a dedicated summarizer agent that runs every N turns and emits a SystemMessage with the compressed history. This keeps the context window manageable without losing the debate thread.
Evaluating debate quality
You need a way to measure whether the debate actually improves answers. Build a simple eval harness that runs the same question through single-agent and multi-agent pipelines, then scores them with a rubric.
# eval_debate.py
import asyncio
import json
from dataclasses import dataclass
from debate_agents_v2 import team as debate_team
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient
import os
from dotenv import load_dotenv
load_dotenv()
@dataclass
class EvalCase:
question: str
criteria: list[str]
cases = [
EvalCase(
question=(
"Should a startup optimize for profitability or growth in its first "
"two years? Assume a B2B SaaS company with $500k ARR, 15% monthly "
"churn, and a 3-month sales cycle."
),
criteria=[
"Addresses churn math correctly",
"Considers sales cycle impact on cash flow",
"Mentions rule of 40 or relevant benchmarks",
"Gives a clear recommendation with conditions",
],
),
]
single_client = OpenAIChatCompletionClient(
model="gpt-4o-mini",
api_key=os.getenv("N4N_API_KEY"),
base_url=os.getenv("N4N_BASE_URL"),
)
single_agent = AssistantAgent(
name="SingleAgent",
model_client=single_client,
system_message="Answer the question thoroughly.",
)
async def run_single(question: str) -> str:
result = await single_agent.run(task=question)
return result.messages[-1].content
async def run_debate(question: str) -> str:
result = await debate_team.run(task=question)
# Extract the judge's final answer from the tool call
for msg in result.messages:
if hasattr(msg, 'tool_calls') and msg.tool_calls:
for tc in msg.tool_calls:
if tc.get('name') == 'emit_decision':
return json.loads(tc['arguments'])['final_answer']
return result.messages[-1].content
async def main():
for case in cases:
print(f"\n=== {case.question[:60]}... ===")
single_ans = await run_single(case.question)
debate_ans = await run_debate(case.question)
print(f"\nSingle agent ({len(single_ans)} chars):")
print(single_ans[:300])
print(f"\nDebate ({len(debate_ans)} chars):")
print(debate_ans[:300])
if __name__ == "__main__":
asyncio.run(main())Expected output:
=== Should a startup optimize for profitability or growth... ===
Single agent (1,240 chars):
At $500k ARR with 15% monthly churn, the company loses $75k/month...
[misses that 15% monthly = ~80% annual churn]
Debate (1,890 chars):
The Proposer initially recommended growth, but the Critic correctly
calculated that 15% monthly churn implies ~80% annual logo churn...
[incorporates the correction and adds cash flow analysis]Common failure modes and fixes
The Critic agrees too much. Add a temperature=0.3 to the Critic’s client and strengthen the system message: “You are rewarded for finding real flaws. If you cannot find any, say ‘NO_VALID_CRITIQUE’ and explain why the answer holds.”
The Judge hallucinates consensus. Require the Judge to quote the specific Critic points it accepts or rejects in the critique_summary field. The structured tool call makes this enforceable.
Debate loops forever. The MaxMessageTermination is your safety net. Set it to 3 * num_participants as a starting heuristic. Monitor actual round counts in production and adjust.
Token costs explode. Use the per-token metering from n4n.ai to track spend per agent per debate. Route the Proposer to a cheaper model (e.g., meta-llama/llama-3.1-70b-instruct) and keep the Critic and Judge on stronger models.
Production checklist
- Wrap the team in a retry policy with exponential backoff for transient gateway errors
- Log each agent’s model, token usage, and latency to your observability stack
- Add a
TextMentionTermination("NO_VALID_CRITIQUE")so the Critic can end early when the answer is solid - Implement a caching layer for repeated questions — the gateway forwards provider
cache-controlhints you can respect - Set per-agent timeouts; a stuck Critic shouldn’t block the Judge
The debate pattern scales well beyond three agents. You can add a FactChecker with tool access, a Devil’sAdvocate that argues the opposite side, or a DomainExpert that only activates for specific topics. The orchestration stays the same; you just add participants to the RoundRobinGroupChat.