Tracing AutoGen multi-agent conversations is messy because the framework hides message passing behind group-chat loops, agent replies, and internal tool calls. This guide gives you a concrete, end-to-end method to capture every LLM request, response, and inter-agent message with minimal code, so you can debug hangs, runaway costs, and bad handoffs.
Step 1: Stand up a reproducible multi-agent test bed
Install the AgentChat API and a compatible model client. AutoGen 0.4+ exposes autogen.agentchat and autogen.ext.chat_completion_client.
pip install "autogen-agentchat" "autogen-ext[openai]"Create two agents and a round-robin group. Point the model client at an OpenAI-compatible gateway so you get uniform auth and usage metering. n4n.ai provides one OpenAI-compatible endpoint that addresses 240+ models and returns per-token usage, which makes later cost reconciliation trivial.
import os
from autogen.agentchat import AssistantAgent, RoundRobinGroupChat
from autogen.ext.chat_completion_client import OpenAIChatCompletionClient
model_client = OpenAIChatCompletionClient(
model="gpt-4o-mini",
api_key=os.environ["N4N_API_KEY"],
base_url="https://api.n4n.ai/v1",
)
coder = AssistantAgent(
"coder",
model_client=model_client,
system_message="You write concise Python. Respond with code only.",
)
critic = AssistantAgent(
"critic",
model_client=model_client,
system_message="You review code for bugs. One sentence.",
)
group = RoundRobinGroupChat([coder, critic], max_turns=3)Run a task to confirm the wiring works before adding instrumentation.
import asyncio
async def main():
await group.run_stream(task="Write a function to compute fib(n) iteratively.")
asyncio.run(main())Step 2: Wrap the model client to log every LLM call
AutoGen sends each agent turn through its ChatCompletionClient. Subclass the client to emit a structured record per call. This captures prompt tokens, completion tokens, latency, and the raw message list—exactly what you need for tracing AutoGen multi-agent conversations at the token level.
import json, time
from autogen.ext.chat_completion_client import OpenAIChatCompletionClient
class TracingClient(OpenAIChatCompletionClient):
def __init__(self, trace_path: str, **kwargs):
super().__init__(**kwargs)
self.trace_path = trace_path
async def create(self, messages, **kwargs):
start = time.time()
resp = await super().create(messages, **kwargs)
rec = {
"ts": time.time(),
"latency_ms": round((time.time() - start) * 1000, 1),
"req": [m.model_dump() for m in messages],
"resp": resp.model_dump(),
"usage": resp.usage.model_dump() if resp.usage else None,
}
with open(self.trace_path, "a") as f:
f.write(json.dumps(rec) + "\n")
return resp
traced_client = TracingClient(
trace_path="llm_calls.jsonl",
model="gpt-4o-mini",
api_key=os.environ["N4N_API_KEY"],
base_url="https://api.n4n.ai/v1",
)
coder.model_client = traced_client
critic.model_client = traced_clientRe-run the group chat. You now have a JSONL file with one line per LLM invocation. Each line shows the exact message history the agent saw, which is the fastest way to spot context-bleed between agents.
Step 3: Hook group-chat messages for conversation-level traces
LLM logs alone don’t show agent-to-agent text. AutoGen 0.4 exposes a runtime message consumer. Register a callback to capture the public conversation flow.
from autogen import runtime
def log_msg(msg):
with open("conversation.jsonl", "a") as f:
f.write(json.dumps({"ts": time.time(), "msg": msg.model_dump()}) + "\n")
runtime.add_message_consumer(log_msg)If your version lacks runtime.add_message_consumer, fall back to a UserProxyAgent that echoes everything:
from autogen.agentchat import UserProxyAgent
def echo(recipient, messages, sender, silent):
for m in messages:
log_msg(m)
return False, None # do not auto-reply
spy = UserProxyAgent("spy", human_input_mode="NEVER", function_map={})
spy.register_reply(AssistantAgent, reply_func=echo)
group = RoundRobinGroupChat([coder, critic, spy], max_turns=3)Either method produces a clean transcript of who said what, which is the core of tracing AutoGen multi-agent conversations.
Step 4: Export traces to OpenTelemetry for aggregation
Console JSONL is fine for a single run; production debugging wants spans. Wrap the trace emission in an OTel span so you can pipe to Jaeger or Tempo.
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(
SimpleSpanProcessor(ConsoleSpanExporter())
)
tracer = trace.get_tracer("autogen.trace")
# Inside TracingClient.create, replace file write with:
with tracer.start_as_current_span("llm_call") as span:
span.set_attribute("agent.messages", len(messages))
if resp.usage:
span.set_attribute("tokens.total", resp.usage.total_tokens)This gives you per-turn latency histograms and token counts across the whole multi-agent graph.
Step 5: Verify success and common pitfalls
Run the script and check three things:
llm_calls.jsonlhas exactlynum_agents * max_turnslines (minus early terminations).conversation.jsonlshows alternatingcoder/criticmessages with nonullcontent.- Token usage in the LLM records matches the metering from your provider (if using n4n.ai, cross-check the per-token usage field).
wc -l llm_calls.jsonl conversation.jsonl
head -1 llm_calls.jsonl | python -m json.toolIf line counts are off, check max_turns and whether an agent raised an exception inside create. AutoGen swallows some errors into the group chat state—your wrapper will surface them because the await super().create will throw before writing the record.
A frequent mistake is wrapping only one agent’s client. Both coder and critic must share the traced client, or you’ll see half the conversation missing. Another is logging messages by reference and mutating later; always model_dump() immediately as shown.
For streaming agents, override create_stream the same way, buffering chunks into a single record on completion. Otherwise your trace files will be sparse and misleading.
What you get
After these steps, tracing AutoGen multi-agent conversations becomes a cat away. You can reconstruct any run: which agent called the LLM, what context it had, how long it waited, and how many tokens it burned. When a handoff goes wrong, the conversation JSONL shows the exact message that triggered the bad reply, and the LLM JSONL shows the prompt that produced it. That closure is what makes multi-agent systems debuggable instead of magical.