When an agent calls the wrong tool, hallucinates parameters, or silently fails, you need visibility into the reasoning loop — not guesswork. This llamaindex debug agent tool calls tutorial walks through instrumenting an agent, capturing every tool invocation, and diagnosing the most common failure modes. You’ll finish with a reusable debugging setup you can drop into any LlamaIndex project.
Step 1: Enable verbose logging and capture the reasoning trace
LlamaIndex agents expose a verbose flag, but it only prints to stdout. For real debugging, redirect that output to a structured logger and capture the AgentChatResponse objects that contain the full reasoning trace.
import logging
import json
from llama_index.core.agent import ReActAgent
from llama_index.core.tools import FunctionTool
from llama_index.llms.openai import OpenAI
# Configure structured logging
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s %(levelname)s %(name)s: %(message)s"
)
logger = logging.getLogger("llamaindex.debug")
def get_weather(location: str) -> str:
"""Get current weather for a location."""
return f"Weather in {location}: 72°F, sunny"
def calculate_mortgage(principal: float, rate: float, years: int) -> str:
"""Calculate monthly mortgage payment."""
monthly_rate = rate / 12 / 100
months = years * 12
payment = principal * monthly_rate / (1 - (1 + monthly_rate) ** -months)
return f"Monthly payment: ${payment:,.2f}"
tools = [
FunctionTool.from_defaults(fn=get_weather),
FunctionTool.from_defaults(fn=calculate_mortgage),
]
llm = OpenAI(model="gpt-4o-mini", temperature=0)
agent = ReActAgent.from_tools(tools, llm=llm, verbose=True)Run a test query and observe the raw output:
response = agent.chat("What's the weather in Tokyo and what's a 30-year mortgage on $400k at 6.5%?")
print(response)Verify success: You see the agent’s thought process, tool selections, and tool outputs printed to the console. Each step shows Thought:, Action:, Action Input:, and Observation: — this is the ReAct loop in action.
Step 2: Wrap the agent with a callback handler for programmatic access
Printing to stdout doesn’t scale. LlamaIndex’s callback system lets you intercept every event — tool calls, LLM calls, errors — and write them to a file, database, or observability platform.
from llama_index.core.callbacks import CallbackManager, BaseCallbackHandler
from llama_index.core.callbacks.schema import CBEventType, EventPayload
from typing import Any, Dict, Optional
import uuid
class ToolCallLogger(BaseCallbackHandler):
"""Captures every tool invocation with inputs and outputs."""
def __init__(self):
super().__init__(
event_starts_to_ignore=[],
event_ends_to_ignore=[],
)
self.tool_calls = []
def on_event_start(
self,
event_type: CBEventType,
payload: Optional[Dict[str, Any]] = None,
event_id: str = "",
**kwargs: Any,
) -> str:
if event_type == CBEventType.FUNCTION_CALL:
tool_name = payload.get(EventPayload.FUNCTION_CALL, "unknown")
tool_input = payload.get(EventPayload.FUNCTION_CALL_ARGS, {})
self.tool_calls.append({
"event_id": event_id,
"tool": tool_name,
"input": tool_input,
"status": "started",
})
logger.debug(f"TOOL START: {tool_name} with {tool_input}")
return event_id
def on_event_end(
self,
event_type: CBEventType,
payload: Optional[Dict[str, Any]] = None,
event_id: str = "",
**kwargs: Any,
) -> None:
if event_type == CBEventType.FUNCTION_CALL:
tool_output = payload.get(EventPayload.FUNCTION_CALL_RESULT, "")
# Find and update the matching start event
for call in self.tool_calls:
if call["event_id"] == event_id:
call["output"] = str(tool_output)
call["status"] = "completed"
break
logger.debug(f"TOOL END: {event_id} -> {tool_output}")
# Attach the callback manager
callback_manager = CallbackManager([ToolCallLogger()])
agent = ReActAgent.from_tools(
tools,
llm=llm,
verbose=True,
callback_manager=callback_manager
)
response = agent.chat("Calculate a mortgage for $300k at 5% for 15 years")Verify success: After the call completes, callback_manager.handlers[0].tool_calls contains a list of dicts with tool name, input arguments, and output. You can serialize this to JSON for later analysis or feed it into a trace viewer.
Step 3: Inspect the agent’s internal state with a custom agent worker
For deeper inspection — seeing the raw LLM prompt, the parsed action, and the tool selection logic — subclass AgentWorker and override get_next_step. This reveals exactly what the model generated before parsing.
from llama_index.core.agent.react import ReActAgentWorker
from llama_index.core.agent.types import BaseAgentWorker
from llama_index.core.tools import ToolSelection, ToolOutput
from llama_index.core.llms import ChatMessage
from typing import List, Tuple, Any
class DebugReActAgentWorker(ReActAgentWorker):
"""Agent worker that logs every reasoning step in detail."""
def get_next_step(
self,
input: str,
chat_history: List[ChatMessage],
**kwargs: Any,
) -> Tuple[ToolSelection, str]:
# Log the full prompt being sent to the LLM
prompt = self._get_prompt(input, chat_history)
logger.debug(f"FULL PROMPT:\n{prompt}")
# Get the tool selection from parent
tool_selection, reasoning = super().get_next_step(input, chat_history, **kwargs)
# Log what the model actually chose
logger.debug(f"REASONING: {reasoning}")
logger.debug(f"SELECTED TOOL: {tool_selection.tool_name}")
logger.debug(f"TOOL INPUT: {tool_selection.tool_kwargs}")
return tool_selection, reasoning
# Use the debug worker
debug_worker = DebugReActAgentWorker.from_tools(
tools=tools,
llm=llm,
verbose=True,
callback_manager=callback_manager,
)
agent = debug_worker.as_agent()Verify success: The logs now show the complete prompt template with all few-shot examples, the model’s raw completion, and the parsed tool selection. When the agent picks the wrong tool, you can see exactly why — whether the prompt was ambiguous, the model misunderstood the tool description, or the parsing failed.
Step 4: Catch and diagnose common failure modes
Three failure patterns cover most tool-calling bugs. Add explicit handling for each.
4.1: Parameter validation errors
The model passes arguments that don’t match the function signature. LlamaIndex raises a ToolException — catch it and log the mismatch.
from llama_index.core.tools import ToolException
def strict_calculate(principal: float, rate: float, years: int) -> str:
if principal <= 0:
raise ToolException("Principal must be positive")
if rate <= 0 or rate > 100:
raise ToolException("Rate must be between 0 and 100")
if years <= 0:
raise ToolException("Years must be positive")
monthly_rate = rate / 12 / 100
months = years * 12
payment = principal * monthly_rate / (1 - (1 + monthly_rate) ** -months)
return f"Monthly payment: ${payment:,.2f}"
strict_tool = FunctionTool.from_defaults(fn=strict_calculate)
# Test with bad input
agent = ReActAgent.from_tools([strict_tool], llm=llm, verbose=True)
try:
response = agent.chat("Calculate mortgage for -$100k at 5% for 30 years")
except ToolException as e:
logger.error(f"Tool validation failed: {e}")
# The agent should retry with corrected parametersVerify success: The agent catches the exception, feeds it back to the LLM as an observation, and retries with valid parameters. Your logs show the failed attempt, the error message, and the corrected retry.
4.2: Tool not found or ambiguous selection
When the model hallucinates a tool name or picks between similar tools, add a fallback handler.
from llama_index.core.agent import AgentRunner
from llama_index.core.tools import ToolOutput
class SafeAgentRunner(AgentRunner):
"""Runner that catches unknown tool calls."""
def _get_tool_output(self, tool_selection: ToolSelection, **kwargs) -> ToolOutput:
try:
return super()._get_tool_output(tool_selection, **kwargs)
except ValueError as e:
if "not found" in str(e).lower():
available = [t.metadata.name for t in self._tools]
return ToolOutput(
content=f"Tool '{tool_selection.tool_name}' not found. Available: {available}",
tool_name=tool_selection.tool_name,
raw_input=tool_selection.tool_kwargs,
raw_output=str(e),
is_error=True,
)
raise
agent = SafeAgentRunner(debug_worker, callback_manager=callback_manager)
response = agent.chat("Use the 'calculate_loan' tool for $200k at 4% for 20 years")Verify success: Instead of crashing, the agent receives a structured error observation listing available tools. The LLM then corrects itself and calls the right tool (calculate_mortgage).
4.3: Silent failures from empty or malformed tool output
A tool returns None, an empty string, or raises an exception that gets swallowed. Wrap tools to guarantee a usable observation.
from functools import wraps
def safe_tool(fn):
@wraps(fn)
def wrapper(*args, **kwargs):
try:
result = fn(*args, **kwargs)
if result is None:
return "Tool executed successfully but returned no output."
if isinstance(result, str) and not result.strip():
return "Tool returned empty string."
return result
except Exception as e:
return f"Tool error: {type(e).__name__}: {e}"
return wrapper
@safe_tool
def flaky_api_call(query: str) -> str:
# Simulates an API that sometimes fails
import random
if random.random() < 0.3:
raise ConnectionError("API timeout")
return f"Result for: {query}"
flaky_tool = FunctionTool.from_defaults(fn=flaky_api_call)
agent = ReActAgent.from_tools([flaky_tool], llm=llm, verbose=True)
response = agent.chat("Call the flaky API with 'test query'")Verify success: Every tool invocation produces a non-empty string observation. The agent never sees None or an unhandled exception — it always gets text it can reason over.
Step 5: Add a replay harness for regression testing
Once you’ve diagnosed a bug, capture the failing conversation and turn it into a deterministic test case.
import json
from dataclasses import dataclass, asdict
from typing import List
@dataclass
class ConversationTurn:
user: str
expected_tool: str
expected_args: dict
# Record a failing case
failing_case = ConversationTurn(
user="What's a 30-year mortgage on $400k at 6.5%?",
expected_tool="calculate_mortgage",
expected_args={"principal": 400000, "rate": 6.5, "years": 30}
)
def test_agent_tool_selection(agent: ReActAgent, cases: List[ConversationTurn]):
"""Verify the agent selects the right tool with correct arguments."""
logger = ToolCallLogger()
cb_manager = CallbackManager([logger])
agent.callback_manager = cb_manager
for case in cases:
logger.tool_calls.clear()
response = agent.chat(case.user)
assert len(logger.tool_calls) > 0, f"No tool called for: {case.user}"
call = logger.tool_calls[0]
assert call["tool"] == case.expected_tool, \
f"Expected {case.expected_tool}, got {call['tool']}"
assert call["input"] == case.expected_args, \
f"Expected args {case.expected_args}, got {call['input']}"
print(f"✓ PASS: {case.user}")
# Run the test
test_agent_tool_selection(agent, [failing_case])Verify success: The test passes when the agent calls exactly the expected tool with exactly the expected arguments. Add this to your CI pipeline to catch regressions when you update prompts, swap models, or modify tool descriptions.
Step 6: Export traces for team debugging
When a bug only reproduces in production, you need a portable trace format. Serialize the full callback history to JSON Lines — one line per event — so teammates can replay it locally.
def export_trace(callback_manager: CallbackManager, filepath: str):
"""Export all captured events to JSONL for sharing."""
handler = callback_manager.handlers[0]
with open(filepath, "w") as f:
for call in handler.tool_calls:
f.write(json.dumps(call) + "\n")
def replay_trace(filepath: str) -> List[dict]:
"""Load a trace file for analysis."""
events = []
with open(filepath) as f:
for line in f:
events.append(json.loads(line))
return events
# Export after a debugging session
export_trace(callback_manager, "debug_trace.jsonl")
# Later: analyze the trace
trace = replay_trace("debug_trace.jsonl")
for event in trace:
print(f"{event['tool']}({event['input']}) -> {event.get('output', 'N/A')[:80]}")Verify success: The JSONL file contains every tool call with inputs and outputs. A colleague can load it, replay the conversation against a fixed agent version, and confirm the fix without access to your environment.
Step 7: Integrate with an LLM gateway for production observability
In production, you’re likely routing through multiple model providers. If you use an OpenAI-compatible gateway like n4n.ai, you get automatic fallback when a provider degrades, per-token usage metering, and provider cache-control hints forwarded from the upstream — all without changing your LlamaIndex code. Point your OpenAI client at the gateway endpoint and the same callback handlers capture tool calls regardless of which backing model serves the request.
from llama_index.llms.openai import OpenAI
# Gateway endpoint with automatic fallback and usage metering
gateway_llm = OpenAI(
model="gpt-4o-mini",
api_base="https://api.n4n.ai/v1", # Example gateway endpoint
api_key="your-gateway-key",
temperature=0,
)
agent = ReActAgent.from_tools(tools, llm=gateway_llm, callback_manager=callback_manager)Verify success: Your debugging callbacks work identically. The gateway handles provider routing transparently, and your tool-call traces show which model actually executed each step via the response headers.
Summary checklist
- Enable verbose logging and capture
AgentChatResponseobjects - Attach a
CallbackManagerwith a customBaseCallbackHandlerto record tool calls programmatically - Subclass
ReActAgentWorkerto log the full prompt, raw model output, and parsed tool selection - Handle the three common failure modes: parameter validation, unknown tools, and silent empty outputs
- Build a replay harness with recorded conversation turns for regression testing
- Export traces to JSONL for sharing and offline analysis
- Route through an LLM gateway to maintain observability across provider failovers
With this setup, every tool call is visible, testable, and reproducible. The next time an agent behaves unexpectedly, you’ll have the full trace — prompt, reasoning, tool choice, parameters, and output — ready to inspect in seconds.