Debugging LlamaIndex agent workflows: a practical guide

Debugging LlamaIndex agents is rarely about a single broken line; it’s about observing a non-deterministic loop of LLM calls, tool executions, and state mutations. This guide lays out an ordered path to isolating failures in agent workflows, from enabling trace logging to replaying tool outputs in isolation.

1. Enable structured logging before you touch the code

Print statements lie when multiple async tasks interleave. Start by attaching a CallbackManager with LlamaDebugHandler to Settings. This captures LLM prompts, completions, and tool calls in chronological order without you modifying agent logic.

from llama_index.core import Settings
from llama_index.core.callbacks import CallbackManager, LlamaDebugHandler

debug_handler = LlamaDebugHandler()
Settings.callback_manager = CallbackManager([debug_handler])

After a run, inspect the captured events:

events = debug_handler.get_events()
for e in events:
    if e.payload.get("event_type") in ("llm", "function_call", "function_result"):
        print(e.timestamp, e.payload)

If you see a function_call with no following function_result, the tool raised an exception that the agent loop swallowed. That is your first breakpoint. The debug handler also records token usage, which helps spot truncated context windows.

Pitfall: leaving the debug handler on in production. It retains every event in memory. Swap it for a streaming handler that writes to stdout or OpenTelemetry in prod, and never attach it globally via Settings outside local dev.

2. Stream workflow events live

Agent workflows in LlamaIndex are event-driven. The AgentWorkflow.run() call returns a handler that exposes stream_events(). Use it to see exact transition points instead of waiting for the final answer.

handler = workflow.run(input="Calculate (21 * 2) + 5")
async for ev in handler.stream_events():
    name = type(ev).__name__
    if name == "ToolCall":
        print(f"-> calling {ev.tool_name} with {ev.tool_kwargs}")
    elif name == "ToolResult":
        print(f"<- result: {ev.tool_output}")
    elif name == "AgentOutput":
        print("agent decided:", ev.response)

If the stream stops after a ToolCall and never emits ToolResult, the tool either hung or raised. Wrap tool bodies in try/except that returns a string error; LlamaIndex will treat that as a result and continue, which is better than silent death.

Tradeoff: streaming adds minor latency. For debugging locally it is worth it; for high-throughput serving, sample 5% of sessions or use the callback approach from step 1.

3. Validate input and output schemas explicitly

Most agent failures come from the LLM producing arguments that don’t match the tool’s type hints. FunctionTool.from_defaults uses Python introspection to build the schema, but LLMs still hallucinate extra keys or wrong types.

from llama_index.core.tools import FunctionTool

def add(a: int, b: int) -> int:
    return a + b

tool = FunctionTool.from_defaults(fn=add)
tool.metadata.fn_schema_strict = True  # enforce strict JSON schema if model supports it

When debugging llamaindex agents, print ev.tool_kwargs from the event stream and compare against tool.metadata.get_parameters_dict(). A missing required key means the prompt needs stricter instruction, not a code fix. If the model sends "b": "two" instead of 2, the schema validation will reject it before your function ever runs—and the agent will see the error and retry.

Pitfall: assuming the LLM respects types because you used Python hints. Always log the raw kwargs during debugging.

4. Replay tool calls outside the agent loop

Once you have the exact kwargs from a failed run, call the tool synchronously in a REPL or a small script. This removes LLM noise and tests your function in isolation.

# extracted from event log
kwargs = {"a": 21, "b": "two"}  # note b is str
try:
    print(tool.fn(**kwargs))
except TypeError as e:
    print("Schema violation:", e)

If the tool itself is buggy, you’ll see it immediately. If it works, the problem is upstream: the agent’s planning prompt or the model’s reasoning. A common pitfall is tools that mutate global state. Replaying them twice changes results, making the agent’s memory inconsistent. Make tools pure or explicitly document side effects.

For stateful tools (database writes), build a fake fixture and replay against that.

5. Handle retries and provider degradation

Transient 429s from a model provider look like agent stupidity. If you route LLM calls through a unified inference gateway such as n4n.ai, automatic fallback to a secondary provider removes that class of error from your logs. Otherwise, set explicit retry logic in your LLM client.

from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-4o-mini", max_retries=3, timeout=30)

For agent workflows, a single retry at the LLM layer is usually enough; the workflow itself does not retry tool calls unless you code it. Decide deliberately: retrying a write tool (send_email) is dangerous. Make idempotent tools or mark them non-retryable by catching ToolRetryError and returning a stable message.

Tradeoff: more retries hide real model-quality issues. Cap them and alert when fallback triggers.

6. Trace state across workflow steps

LlamaIndex workflows carry a Context with mutable state. Agents store conversation history there. Dump it after each step to confirm memory persists.

from llama_index.core.workflow import Context

async def dump_state(ctx: Context):
    state = await ctx.get("state", default={})
    print("STATE:", state)

If the agent repeats a tool call it already made, the history wasn’t persisted or was truncated. Check Settings.context_window and the memory buffer. A frequent bug: using a fresh Context per call instead of reusing the persisted one across turns.

# correct: reuse ctx across user turns
ctx = Context(workflow)
handler = workflow.run(input="first", ctx=ctx)
await handler
handler2 = workflow.run(input="follow-up", ctx=ctx)

When debugging llamaindex agents, verify the ctx object is the same instance between calls. A new context silently drops prior tool results.

7. Common pitfalls and tradeoffs

• Swallowing exceptions in tools. Returning a generic “error” string hides the stack trace. Log locally, return specific message.
• Assuming deterministic ordering. Workflows can branch; don’t assert event order strictly in tests.
• Over-instrumenting. Every event logged to disk slows the loop. Use sampling.
• Tight coupling to one model. Pin the model version. A provider silent upgrade changes token counts and breaks your assumptions.
• Ignoring provider cache hints. If you forward cache_control and the gateway honors it, repeated prompts cost less and run faster; missing it makes debugging slower and noisier.

Debugging llamaindex agents is mostly about visibility. Build the logging first, replay the failing slice, and only then change agent logic. The workflow is a distributed system where the LLM is the flakiest node—treat it like one.