Logging AutoGen agent messages for debugging

Logging AutoGen agent messages is the fastest way to see why a multi-agent workflow went off the rails. This tutorial builds a reusable logging layer around AutoGen 0.2 that captures every agent reply, tool call, and handoff with minimal code, then shows the exact output you should expect at each checkpoint.

Prerequisites

• Python 3.10 or newer
• pyautogen (install with pip install pyautogen==0.2.32 or similar 0.2.x)
• An OpenAI API key, or any OpenAI-compatible endpoint URL and key exported as OPENAI_API_KEY and OPENAI_BASE_URL
• A working knowledge of AssistantAgent and UserProxyAgent

If you route through n4n.ai, its OpenAI-compatible endpoint fronts 240+ models and handles fallback automatically; set OPENAI_BASE_URL to that endpoint and your logs should record which model actually answered.

Step 1: Stand up a minimal two-agent chat

We start with the smallest useful AutoGen program: a user proxy that never asks for input, and an assistant that answers.

import os
from autogen import AssistantAgent, UserProxyAgent

llm_config = {
    "config_list": [{
        "model": "gpt-4o-mini",
        "api_key": os.environ["OPENAI_API_KEY"],
        "base_url": os.environ.get("OPENAI_BASE_URL"),
    }],
    "temperature": 0,
}

assistant = AssistantAgent("assistant", llm_config=llm_config)
user = UserProxyAgent(
    "user",
    human_input_mode="NEVER",
    code_execution_config=False,
    is_termination_msg=lambda x: x.get("content", "").strip().endswith("TERMINATE"),
)

result = user.initiate_chat(assistant, message="List three edge cases for a rate limiter.")
print(result.summary)

Expected output (truncated):

user (to assistant):
List three edge cases for a rate limiter.

assistant (to user):
1. Burst traffic exceeding the limit in a single window...
TERMINATE

Step 2: Post-hoc logging from chat history

AutoGen returns a ChatResult object. The full transcript lives in result.chat_history. Dump it to JSON Lines for later grep-ing.

import json

def dump_history(result, path="chat.jsonl"):
    with open(path, "w") as f:
        for msg in result.chat_history:
            f.write(json.dumps(msg) + "\n")

dump_history(result)

Each line looks like:

{"role": "user", "content": "List three edge cases for a rate limiter.", "name": "user"}
{"role": "assistant", "content": "1. Burst traffic...", "name": "assistant"}

This is logging AutoGen agent messages after the fact. It is enough for batch debugging but useless when the agent hangs.

Step 3: Real-time logging via agent subclass

Subclass the agents and override generate_reply. This captures the exact input and output of every LLM call.

import logging
from autogen import AssistantAgent, UserProxyAgent

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")

class LoggingAssistant(AssistantAgent):
    def generate_reply(self, messages, sender=None, **kwargs):
        logging.info("ASSISTANT IN: %s", [m.get("content") for m in messages])
        reply = super().generate_reply(messages, sender, **kwargs)
        logging.info("ASSISTANT OUT: %s", reply)
        return reply

class LoggingUser(UserProxyAgent):
    def send(self, message, recipient, request_reply=True):
        logging.info("USER SEND: %s", message.get("content") if isinstance(message, dict) else message)
        return super().send(message, recipient, request_reply)

assistant = LoggingAssistant("assistant", llm_config=llm_config)
user = LoggingUser("user", human_input_mode="NEVER", code_execution_config=False)
user.initiate_chat(assistant, message="Explain token bucket vs leaky bucket.")

Checkpoint output:

2024-05-12 10:01:22 INFO ASSISTANT IN: ['Explain token bucket vs leaky bucket.']
2024-05-12 10:01:23 INFO ASSISTANT OUT: Token bucket allows bursts...
2024-05-12 10:01:23 INFO USER SEND: Token bucket allows bursts...

Now logging AutoGen agent messages happens live, so you can attach a debugger or tail the log in another terminal.

Step 4: Structured JSON logs

Plain text is fine for a laptop, but a multi-agent system emits dozens of messages per run. Emit JSON so you can query by agent name or message type.

import logging
import json

class JsonFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "ts": self.formatTime(record),
            "level": record.levelname,
            "event": record.getMessage(),
        })

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logging.getLogger().handlers = [handler]
logging.getLogger().setLevel(logging.INFO)

Rerun Step 3. You now get:

{"ts": "2024-05-12 10:01:22", "level": "INFO", "event": "ASSISTANT IN: ['Explain token bucket vs leaky bucket.']"}

Pipe to jq to filter: jq 'select(.event | contains("OUT"))'.

Step 5: Logging group chats

Group chats add a manager and multiple agents. The GroupChat object keeps messages after each round. Wrap the manager’s run or just poll the list.

from autogen import GroupChat, GroupChatManager

def make_group():
    gpt = LoggingAssistant("gpt", llm_config=llm_config)
    reviewer = LoggingAssistant("reviewer", llm_config=llm_config)
    group = GroupChat(agents=[gpt, reviewer, user], messages=[], max_round=4)
    manager = GroupChatManager(group, llm_config=llm_config)
    return group, manager

group, manager = make_group()
user.initiate_chat(manager, message="Draft a retry policy, then critique it.")

for i, m in enumerate(group.messages):
    logging.info("ROUND %d: %s -> %s", i, m.get("name"), m.get("content")[:50])

Expected log snippet:

INFO ROUND 0: user -> Draft a retry policy, then critique it.
INFO ROUND 1: gpt -> A retry policy should use exponential backoff...
INFO ROUND 2: reviewer -> The policy misses jitter; suggest adding...

This pattern gives you full visibility into handoffs—critical when logging AutoGen agent messages across a team of specialists.

Step 6: Recording model routing

When you use an OpenAI-compatible gateway, the model string in llm_config may not be the model that actually served the token. Capture the response’s model field by wrapping the LLM client. AutoGen passes extra through; if your endpoint returns the resolved model in the usage or response, log it.

orig_create = assistant.client.create

def logged_create(*args, **kwargs):
    resp = orig_create(*args, **kwargs)
    logging.info("MODEL USED: %s", resp.model)
    return resp

assistant.client.create = logged_create

If you point AutoGen at n4n.ai, the gateway honors your routing directives and returns the concrete backend model in the response, so the above line tells you whether fallback kicked in.

Takeaways

• Use chat_history for quick post-mortems; subclass agents for live traces.
• JSON logs survive complex multi-agent runs better than stdout scrapes.
• In group chats, iterate group.messages to reconstruct the round sequence.
• Log the resolved model name whenever a gateway sits between you and the LLM.

Solid logging AutoGen agent messages turns a black-box orchestration into a debuggable pipeline. The code above is drop-in for AutoGen 0.2 and scales to a dozen agents without extra machinery.