Multi-agent LLM workflows fail silently when agents ping-pong messages without converging. debugging infinite loops multi-agent requires treating the system as a distributed process with observable state, not a single prompt. The procedure below gives you concrete steps to trace, isolate, and terminate runaway cycles in production code, with runnable snippets you can drop into a Python orchestrator.
Step 1: Capture every message with a correlation ID
You cannot debug what you cannot see. Most agent frameworks log only the final output; you need the full message graph. Assign a trace_id at the workflow entry point and propagate it through every model call, tool invocation, and inter-agent handoff. Use contextvars so the ID survives async hops.
import uuid, json, logging, time, contextvars
trace_id_var = contextvars.ContextVar("trace_id")
def traced_call(agent, messages, trace_id=None):
tid = trace_id or trace_id_var.get() or uuid.uuid4().hex
trace_id_var.set(tid)
logging.info(json.dumps({
"ts": time.time_ns(),
"trace_id": tid,
"agent": agent.name,
"direction": messages[-1]["role"],
"preview": messages[-1]["content"][:200],
"msg_count": len(messages)
}))
return agent.run(messages, trace_id=tid)Ship these logs to a structured store (JSONLines in S3, or OpenTelemetry). The key is that every line shares the same trace_id so you can reconstruct the sequence later. Without this, debugging infinite loops multi-agent degenerates into guesswork. Add the ts field with nanosecond precision; concurrent agents will otherwise interleave ambiguously.
Step 2: Reconstruct the conversation graph
Pull the logs for a suspect trace_id and build a directed graph of agent transitions. A loop becomes visible when a cycle appears in the edge list. Sort by timestamp to order events correctly across concurrent agents.
from collections import defaultdict
def build_graph(logs):
edges = defaultdict(int)
prev = None
for entry in sorted(logs, key=lambda e: e["ts"]):
if prev:
edges[(prev, entry["agent"])] += 1
prev = entry["agent"]
return edgesFor longer cycles, load into NetworkX and detect simple cycles:
import networkx as nx
G = nx.DiGraph()
for (a, b), w in edges.items():
G.add_edge(a, b, weight=w)
cycles = list(nx.simple_cycles(G))
# cycles like [['planner','coder','reviewer','planner']]If you see the same cycle repeated with increasing msg_count but no change in content, the agents are stuck. This graph is also your postmortem artifact. Export it as JSON for the incident record so reviewers can see exactly which agents formed the closed loop.
Step 3: Identify the loop signature
Not all repetition is a bug. A writer agent iterating on a draft will send similar messages to an editor; that is progress. You need to distinguish productive iteration from a stuck state. Hash the last three messages plus the tool calls. When the hash repeats across two consecutive agent turns, the agents are generating the same output.
import hashlib, json
def state_hash(messages, tools):
payload = json.dumps({
"messages": [m["content"] for m in messages[-3:]],
"tool_calls": [t["id"] for t in tools]
}, sort_keys=True)
return hashlib.sha256(payload.encode()).hexdigest()
# inside tracing:
last_h = None
h = state_hash(current_messages, current_tools)
if h == last_h:
logging.warning("Repeat state %s in trace %s", h[:8], tid)
last_h = hFor semantic loops where wording changes but intent is identical, add a lightweight embedding distance check (cosine > 0.98 on the last agent response). That catches loops where the model paraphrases itself endlessly. A typical captured loop looks like this:
{"trace_id":"a1","agent":"reviewer","content":"This code fails lint."}
{"trace_id":"a1","agent":"coder","content":"Fixed lint."}
{"trace_id":"a1","agent":"reviewer","content":"This code fails lint."}The hash stays constant while the text varies slightly. Detection must cover both exact and near-duplicate states.
Step 4: Bound retries and add explicit termination
The fastest fix is a hard iteration cap at the orchestrator level. Do not rely on the model to emit “done”. Track a counter and force a stop. Also support a consensus stop: if two agents signal completion, exit.
MAX_STEPS = 8
def orchestrate(agents, initial_msg):
tid = uuid.uuid4().hex
msg = initial_msg
stops = 0
for step in range(MAX_STEPS):
agent = select_agent(msg)
msg = traced_call(agent, msg, tid)
if msg.get("stop"):
stops += 1
if stops >= 2:
break
else:
logging.error("Loop bound hit for %s", tid)
msg = {"error": "max_steps_exceeded"}
return msgAdd a stop flag that agents set when they produce a final answer. If the loop bound triggers, you have converted an infinite hang into a manageable error. Tune MAX_STEPS based on the workflow’s normal depth; most task decompositions finish in under five handoffs. For deeply recursive planners, make the cap dynamic: max_steps = 4 + len(task_subgoals). The point is to fail loud, not to guess the perfect constant.
Step 5: Inject provider fallback to avoid retry storms
Infinite loops often start as transient errors. An agent receives a 429 from a provider, retries with the same context, and the retry logic lacks backoff or alternates providers. If you route through a gateway like n4n.ai, the automatic fallback when a provider is rate-limited or degraded removes the need for client-side retry loops that can spin. Regardless of gateway, enforce a single retry with jitter and fail loud:
import time, random
def call_with_backoff(fn, attempts=2):
for i in range(attempts):
try:
return fn()
except RateLimitError:
time.sleep((2 ** i) + random.random())
raise RuntimeError("provider unavailable")This prevents a degraded upstream from turning into a multi-agent ping-pong where each agent blames the other for the missing response. Also honor provider cache-control hints so repeated context isn’t re-sent, reducing the chance of hitting rate limits in the first place. Client routing directives should forward the same trace_id to the fallback provider; otherwise your logs split and the loop becomes invisible again.
Step 6: Verify with a deterministic replay test
Debugging infinite loops multi-agent is not done until you have a test that fails before the fix and passes after. Capture a real looping trace and replay it against your patched orchestrator in a sandbox. Store the fixture as JSON:
{
"trace_id": "fixture_loop",
"steps": [
{"agent": "planner", "content": "Plan task"},
{"agent": "coder", "content": "Write code"},
{"agent": "planner", "content": "Plan task"}
]
}Then assert termination:
def test_loop_terminates():
trace = load_fixture("loop_trace.json")
result = orchestrate_from_log(trace)
assert result.get("error") != "max_steps_exceeded"
assert result.get("stop") is TrueRun it in CI. If you cannot reproduce the loop offline, synthesize one: two agents that echo each other.
class EchoAgent:
name = "echo"
def run(self, msg, **kw):
return {"role": "assistant", "content": msg[-1]["content"], "stop": False}Wire them into orchestrate and confirm the MAX_STEPS guard fires. That proves your bound is active. Additionally, assert that state_hash repeats trigger a warning log; this ensures detection works. A replay test that uses the real fixture is the only proof that the production loop is actually broken.
Step 7: Monitor state divergence in production
After deployment, watch the ratio of trace_ids that hit the step cap. A sudden spike means a prompt change introduced a new cycle. Add an alert:
# count error logs per hour
grep "max_steps_exceeded" app.log | cut -d' ' -f1 | uniq -cIf you use per-token metering, a loop also shows as a flat line of high token burn with no completed workflows. That metric is often the first signal something is wrong before users complain. Set a budget alert at 3x baseline token spend for a single trace. Correlate the alert with the trace_id graph from Step 2 to pinpoint the cycling agents.
Verification checklist
You have successfully debugged the loop when:
- A captured trace shows no repeated
state_hashbeyond the allowed refinement window. - The orchestrator test in Step 6 passes on the real fixture and the synthetic echo test.
- Production dashboards show zero
max_steps_exceedederrors for the patched workflow over 24 hours. - Token metering for representative tasks returns to expected ranges.
Debugging infinite loops multi-agent is mostly about making the invisible state visible and forcing a stop. The code above is minimal but production-grade; drop it into your gateway or agent framework and keep the iteration cap low until you trust the convergence logic. The moment you treat agent cycles like distributed systems bugs, they become tractable.