Function calling turns an LLM into an action-taking component, but most production incidents trace back to a handful of repeated errors. This guide lays out a practical, ordered path for common function calling mistakes fix, drawn from shipping tool-augmented systems under real load. We’ll move from schema definition through execution safety and cross-model behavior.
1. Define the Schema as a Contract, Not a Hint
A repeated mistake is pasting a rough JSON Schema with vague descriptions and assuming the model will infer intent. It won’t. Weak description fields and missing required arrays produce silent argument omissions that surface later as runtime errors or, worse, as wrong-side-effect calls.
Treat the schema like an API contract. Use explicit type, enumerate enum where the domain is closed, and set required for any field the function cannot default safely. Every property description should state the unit, format, and failure mode.
{
"name": "create_refund",
"description": "Issue a refund to a customer order. Requires order_id and reason.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "UUID of the order, e.g. 'ord_8f3a'. Never guess."
},
"amount_cents": {
"type": "integer",
"description": "Refund amount in cents. Must be <= captured amount."
},
"reason": {
"type": "string",
"enum": ["duplicate", "fraud", "customer_request"],
"description": "Category required for compliance logging."
}
},
"required": ["order_id", "amount_cents", "reason"]
}
}Pitfall: over-specifying with deeply nested objects the model rarely fills correctly. Keep the parameter shape flat. If you need complexity, split into multiple tools with clear verbs (get_order, refund_order). Naming matters: use imperative prefixes so the model disambiguates intent.
2. Stop Forcing tool_choice on Every Turn
A common function calling mistakes fix is removing tool_choice: "required" from your default config. Forcing a call makes the model invent arguments when none are needed, triggering spurious side effects like sending emails or writing rows.
Let the model decide. Use "auto" unless you are in a strict state machine where a call is mandatory.
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What's the weather?"}],
tools=[weather_tool],
tool_choice="auto", # default, but be explicit
)
msg = resp.choices[0].message
if msg.tool_calls:
handle_calls(msg)
else:
print(msg.content) # normal reply, no tool invokedTradeoff: with auto, you must handle the no-call branch. That branch is cheaper than cleaning up unintended database writes.
Parallel Calls Need Dedup
Models may emit multiple tool_calls in one message. Process them concurrently but key each by id to avoid double execution. Some providers stream partial tool_calls; buffer until the stream closes before dispatching.
3. Validate Arguments Before Any Side Effect
The LLM emits strings. Even with a schema, amount_cents might arrive as "50" (string) or -5. Never pass tool_call.function.arguments straight to your business layer. This common function calling mistakes fix step saves more production incidents than any prompt tweak.
Use a validator. Pydantic is enough.
from pydantic import BaseModel, PositiveInt, validator
import json
class RefundArgs(BaseModel):
order_id: str
amount_cents: PositiveInt
reason: str
@validator("order_id")
def must_be_uuid(cls, v):
if not v.startswith("ord_"):
raise ValueError("order_id must start with ord_")
return v
def execute(call):
try:
args = RefundArgs(**json.loads(call.function.arguments))
except Exception as e:
return {"error": f"invalid_args: {e}"}
# safe to chargeCoerce types explicitly if your model tends to emit numbers as strings. But reject out-of-range values; do not silently clamp.
4. Make Execution Idempotent
Retries happen. The gateway may resend, or your orchestration loop may call the model twice. If create_refund runs twice, you issued two refunds.
Attach an idempotency key derived from tool_call.id. Your backend should reject repeats.
def handle_tool(call, request_nonce):
key = f"tool:{request_nonce}:{call.id}"
if redis.exists(key):
return redis.get(key) # cached prior result
result = run_refund(call)
redis.setex(key, 86400, result)
return resultPitfall: tool_call.id is per-model-output, not globally unique across providers. Prefix with a request nonce or model name. For distributed workers, use a lock with short TTL around the execution, not just a cache check.
5. Return Errors as Structured Content, Not Exceptions
Swallowing Exception and returning {"success": false} starves the model of recovery context. The fix: return the error as a tool message content with enough detail for the model to self-correct. This is a core common function calling mistakes fix: treat the tool round-trip as dialogue, not RPC.
{
"role": "tool",
"tool_call_id": "call_abc",
"content": "{\"error\": \"order_not_found\", \"detail\": \"ord_8f3a does not exist in region us-east\"}"
}The model can then ask the user for the correct ID or suggest alternatives. If the error is transient (timeout), say so; the model may retry with different params. Truncate overly long error dumps—send the salient field, not a 50-line stack trace.
6. Trim Tool Results Before They Hit Context
A get_order response with 200 lines of JSON will blow your context and inflate cost. Summarize or select fields before returning.
def trim_order(raw):
return {
"id": raw["id"],
"status": raw["status"],
"total_cents": raw["total_cents"],
}When using a gateway that forwards provider cache-control hints, mark static portions (like schema definitions) as cached to avoid re-billing. n4n.ai honors client routing directives and forwards provider cache-control hints, so you can keep large tool schemas pinned in cache across turns without extra tokens.
Tradeoff: trimming loses data the model might later need. Keep a short ID to refetch details rather than dumping everything upfront. Measure context growth per turn; if a tool result exceeds ~1K tokens, you are doing it wrong.
7. Test Across Models, Not Just One
OpenAI’s tool parsing is not Anthropic’s. Some models emit arguments as a JSON string, others as an object; some support parallel calls, some don’t. A common function calling mistakes fix is building a normalization layer and running the same prompt against multiple endpoints.
A gateway such as n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models, with automatic fallback when a provider is rate-limited or degraded. That lets you run a nightly matrix test without writing per-vendor adapters.
curl https://api.n4n.ai/v1/chat/completions \
-H "Authorization: Bearer $KEY" \
-d '{"model":"anthropic/claude-3.5-sonnet","tools":[...],"messages":[...]}'Even without a gateway, abstract the response shape:
def get_calls(msg):
if hasattr(msg, "tool_calls"): return msg.tool_calls
if isinstance(msg, dict) and "tool_calls" in msg: return msg["tool_calls"]
return []Run at least one small model and one large model in CI. Small models drop required args more often; your validation layer will catch it.
Ordered Path to Common Function Calling Mistakes Fix
- Write tight schemas with required fields, enums, and verb-named tools.
- Default to
tool_choice: "auto"; handle the no-call branch explicitly. - Validate arguments with a typed model; never trust raw strings from the model.
- Key executions by
tool_call.idplus a request nonce for idempotency. - Return structured errors as tool content so the model can recover.
- Trim and cache tool outputs to control context and cost.
- Normalize across models and test on a matrix, not a single vendor.
Following this sequence removes the vast majority of production incidents we’ve seen. The common function calling mistakes fix is less about clever prompts and more about treating tools as untrusted, stateful I/O with the same rigor you’d apply to any external API.