Production agents break quietly when a tool call goes wrong. To debug failed function calls effectively, you need the exact transcript of what the model requested, what your code executed, and what came back—not a postmortem guess from aggregated latency graphs. This article gives you an end-to-end procedure to find the root cause and close the loop.
Step 1: Capture the raw request and response cycle
Most teams log only the final assistant message or the user-facing answer. That is useless when a tool invocation fails. You must log the full round trip: the tools schema sent to the model, the tool_calls block the model emitted, the arguments as raw strings, and the JSON you returned as tool role content.
Wrap your client so every call hits your logger before it hits your dispatcher.
import json
import logging
from openai import OpenAI
logger = logging.getLogger("fn_calls")
client = OpenAI() # or any OpenAI-compatible base_url
def chat_with_logging(messages, tools, model="gpt-4o-mini"):
resp = client.chat.completions.create(
model=model, messages=messages, tools=tools
)
msg = resp.choices[0].message
if msg.tool_calls:
for tc in msg.tool_calls:
logger.error("MODEL_TOOL_CALL", extra={
"run_id": resp.id,
"tool": tc.function.name,
"arguments_raw": tc.function.arguments,
})
return resp
def execute_and_log(name, args_raw):
try:
parsed = json.loads(args_raw)
result = dispatch(name, parsed)
status = "ok"
except Exception as e:
result = {"error": str(e)}
status = "error"
logger.error("TOOL_RESULT", extra={
"tool": name, "status": status, "result": result
})
return resultShip these logs to a searchable store with a stable run_id. When you later debug failed function calls, you can reconstruct the exact state the model saw.
Step 2: Classify the failure as model-side or tool-side
A failure is either the model emitting something your schema rejects, or your code throwing on input that is technically valid. Misclassifying wastes hours.
Validate the raw arguments against your expected shape before any business logic runs.
from pydantic import BaseModel, ValidationError
class ChargeCardArgs(BaseModel):
amount_cents: int
currency: str
idempotency_key: str
def parse_model_call(tc):
try:
args = ChargeCardArgs(**json.loads(tc.function.arguments))
return args, None
except ValidationError as e:
return None, eIf ValidationError fires, the model hallucinated a field or used a string where you wanted an int. That is model-side. If validation passes but dispatch raises ConnectionError or a KeyError on a downstream response, it is tool-side.
Common model-side bugs:
- Missing required fields because the system prompt under-specified the tool.
- Wrong enum values (e.g.,
"USD "with trailing space). - Nested objects flattened into strings.
Common tool-side bugs:
- Upstream API returns 502 under load.
- Auth token expired in the middle of a long session.
- Your parser assumed a field that the vendor deprecated.
When you debug failed function calls at scale, tag each log entry with failure_domain: model|tool. That single field drives your remediation path.
Step 3: Reproduce the exact call in a sandbox
Never trust a one-off. Pull the messages array and tools definition from the log, set temperature=0, and replay.
def replay(messages, tools, model="gpt-4o-mini"):
return client.chat.completions.create(
model=model, messages=messages, tools=tools, temperature=0
)If the model now emits correct arguments, you are dealing with nondeterminism—tighten the prompt or enable strict mode. If it still emits the bad shape, you have a reproducible schema comprehension bug.
For tool-side failures, call the tool directly with the parsed arguments:
curl -s -o /dev/null -w "%{http_code}\n" \
-X POST https://api.internal/charge \
-H "Content-Type: application/json" \
-d '{"amount_cents": 1999, "currency": "USD", "idempotency_key": "abc"}'Run it from the same network namespace as production. A 200 locally but 403 from the worker node means your secrets mount is wrong. Reproducing in isolation removes the model from the equation so you can point the finger accurately.
Step 4: Enforce schemas and wrap tools with error contracts
After you debug failed function calls, prevent the same class from recurring. Turn on strict function calling if your provider supports it. In the OpenAI schema that means adding "strict": True and providing a JSON Schema with no optionals.
tools = [{
"type": "function",
"function": {
"name": "charge_card",
"description": "Charge a card",
"strict": True,
"parameters": {
"type": "object",
"properties": {
"amount_cents": {"type": "integer"},
"currency": {"type": "string"},
"idempotency_key": {"type": "string"}
},
"required": ["amount_cents", "currency", "idempotency_key"],
"additionalProperties": False
}
}
}]Strict mode forces the model to emit conforming JSON or nothing. It does not stop your tool from crashing on a valid-but-unexpected value (e.g., a currency your processor rejects).
Wrap every tool in a uniform envelope so the model gets a string it can reason about:
def safe_dispatch(name, args_raw):
try:
args = json.loads(args_raw)
except json.JSONDecodeError as e:
return {"ok": False, "error": f"bad_json:{e}"}
try:
data = dispatch(name, args)
return {"ok": True, "data": data}
except Exception as e:
return {"ok": False, "error": f"{type(e).__name__}:{e}"}Return that dict as the content of the tool message. The model sees "ok": false and can retry with corrected input instead of fabricating a success.
If you route through a gateway such as n4n.ai, the automatic fallback to a secondary provider when the primary is degraded removes a class of transient tool-side timeouts that masquerade as model mistakes, and the per-token usage logs give you an audit trail correlated to each call.
Step 5: Propagate correlation IDs and capture token metrics
A single user action can spawn five tool round trips. Without a trace_id you cannot tie a failure to a session. Generate one at the entrypoint and inject it into every log line.
import uuid, logging
trace_id = uuid.uuid4().hex
logger.error("MODEL_TOOL_CALL", extra={"trace_id": trace_id, "tool": name})If your gateway honors client routing directives, pass the trace_id in a header so the provider metadata echoes it back. Count tokens per failed attempt. A tool that burns 3,000 tokens in retries before succeeding is a latent incident even if it “works.”
Track two counters: model_side_errors and tool_side_errors. Graph them separately. A spike in model_side_errors after a model version bump tells you to roll back the model pin, not your code.
Step 6: Aggregate and alert on failure modes
Structured logs are only useful if you query them. Push the TOOL_RESULT events into a columnar store and run a daily top-offenders query.
SELECT
tool,
status,
error_type,
count(*) AS n
FROM tool_logs
WHERE status = 'error'
GROUP BY tool, status, error_type
ORDER BY n DESC
LIMIT 20;Set a hard alert: if any single tool exceeds a 5% error rate over a 10-minute window, page the owning team. For model-side failures, alert on a sudden change in validation_error rate per model version.
Add a replay CI job. Every night, take the 100 most recent production messages blobs and run them against the staged model. If a previously passing conversation now fails validation, block the deploy.
Verify success
Deploy the logging wrapper, strict schemas, and safe_dispatch envelope. In staging, force a bad argument (e.g., pass amount_cents: "twenty") and confirm the log shows failure_domain: model and the model recovers on the next turn. Run your replay script against the last 24 hours of production transcripts and assert zero uncaught exceptions in execute_and_log. If your dashboard shows no TOOL_EXECUTION_ERROR entries for a full day under real traffic, you have debugged the failure class and built a guard against its return.