Most engineers hit a wall when they try to make an LLM act on the world instead of just chatting. This LLM function calling guide walks through the concrete steps to define tools, invoke them through an OpenAI-compatible API, and execute the returned calls safely in your own runtime.
Function calling is not magic. The model emits a structured payload that matches a schema you declared; your code runs the real function and feeds the result back. The following ordered path takes you from zero to a working tool loop.
1. Model the tool as a strict JSON schema
The model cannot call arbitrary Python. You give it a JSON Schema describing each function’s name, purpose, and parameters. Precision here determines success rate.
Keep descriptions imperative and unambiguous. The parameters object should declare every field with types and required arrays. Avoid additionalProperties: true unless you genuinely accept unknown keys.
{
"type": "function",
"function": {
"name": "charge_credit_card",
"description": "Charge a customer card for a given amount in USD",
"parameters": {
"type": "object",
"properties": {
"customer_id": {"type": "string"},
"amount_cents": {"type": "integer", "minimum": 50},
"idempotency_key": {"type": "string"}
},
"required": ["customer_id", "amount_cents", "idempotency_key"]
}
}
}A common mistake in any LLM function calling guide is letting the schema be too loose. If you accept "amount": "twenty dollars", the model will happily emit a string and your parser will crash. Enforce types and minimums at the schema level. Treat the schema as a contract, not a suggestion.
Smaller models especially benefit from tight constraints. A 7B-class model will stray from loose schemas far more often than a frontier model, so the schema is also your primary lever for reliability when cost forces you down a model tier.
2. Send the schema alongside the conversation
OpenAI-compatible APIs accept a tools array in the chat completion request. The model decides whether to call a tool based on the user prompt and the supplied descriptions.
from openai import OpenAI
client = OpenAI() # defaults to OpenAI; swap base_url for other gateways
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Charge customer cust_123 $19.99"}],
tools=[{
"type": "function",
"function": {
"name": "charge_credit_card",
"description": "Charge a customer card for a given amount in USD",
"parameters": {
"type": "object",
"properties": {
"customer_id": {"type": "string"},
"amount_cents": {"type": "integer"},
"idempotency_key": {"type": "string"}
},
"required": ["customer_id", "amount_cents", "idempotency_key"]
}
}
}],
tool_choice="auto"
)
msg = resp.choices[0].message
print(msg.tool_calls)The response contains tool_calls only if the model opted to invoke. With tool_choice="auto" the model may also reply with plain text. For forced calls, set tool_choice={"type":"function","function":{"name":"..."}}. Use tool_choice="none" to temporarily disable tools for a turn—handy when you want the model to summarize prior tool results without triggering more calls.
Tool choice modes
auto: model decides. Best for agents.none: model must reply normally. Use after gathering tool data.- forced function: guarantees a call. Useful for extraction pipelines where you always want structured output.
3. Detect and execute the call in your runtime
Never trust the model to run side effects. Your server maps tool_calls to local functions, validates arguments, and executes. Wrap execution in try/except so a broken tool doesn’t kill the loop.
import json
def charge_credit_card(customer_id, amount_cents, idempotency_key):
# integration with payment provider
return {"status": "ok", "txn": "txn_987", "charged_cents": amount_cents}
if msg.tool_calls:
for call in msg.tool_calls:
if call.function.name == "charge_credit_card":
try:
args = json.loads(call.function.arguments)
# pydantic validation recommended here
result = charge_credit_card(**args)
tool_result = json.dumps(result)
except Exception as e:
tool_result = json.dumps({"error": str(e)})
# store for next stepValidate before executing
Use a validation library like Pydantic to parse args into a typed model. The model may omit a required field despite the schema; the API only gates malformed JSON, not semantic completeness. Reject and return a tool error message so the model can self-correct.
Concurrent execution
If the model emits multiple tool_calls in one turn, run them in parallel with asyncio.gather. They are independent by construction. Serial execution needlessly doubles latency for read-only fetches.
4. Return results and continue the loop
Feed the tool result back as a tool role message referencing the tool_call_id. The model then synthesizes a user-facing answer or issues another call.
followup = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Charge customer cust_123 $19.99"},
msg,
{"role": "tool", "tool_call_id": call.id, "content": tool_result}
]
)
print(followup.choices[0].message.content)This request-response-tool triangle is the atomic unit of agentic behavior. Loop it until the model returns no tool_calls or you hit a max iteration guard.
Conversation state management
Each loop iteration resends the growing message list. Long system prompts and verbose tool outputs inflate token spend. Trim old tool messages or summarize them after they are consumed. Keep the schema static—prefix caching on many providers will reuse it across calls.
5. Handle partial and ambiguous calls
Models sometimes emit arguments that are syntactically valid but semantically incomplete—e.g., "amount_cents": 1999 but missing idempotency_key. Your schema said it was required, but the sampler isn’t a validator.
Set a max iteration bound
Unbounded loops burn tokens and can cascade errors. Cap at 5–10 steps and return a fallback message. Log the transcript; most failures repeat across similar prompts.
Prefer explicit enums over free text
If a parameter has known values, use "enum". It shrinks the search space and reduces miscalls. This LLM function calling guide recommends enums for any categorical input.
Separate read-only from write tools
Expose get_* functions freely; gate create_*, delete_*, charge_* behind a human confirmation step or a deterministic policy check. The model should not unilaterally mutate external state. A simple allowlist in your execution layer beats prompt-based safeguards.
6. Scale across providers without rewriting
When you move past a single vendor, you’ll face rate limits and regional outages. An OpenAI-compatible gateway such as n4n.ai lets you keep the exact tools payload while routing to 240+ models, with automatic fallback when a provider is degraded and per-token metering for cost tracking. You change only base_url and api_key.
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="your-key"
)
# same tools, same call patternThis decoupling matters because tool-calling behavior varies slightly across model families. Test each candidate on a fixed set of prompts before promoting it to production. A model that misses required enum values in 20% of calls will cost you more in validation retries than the per-token savings.
Common pitfalls and tradeoffs
- Over-broad descriptions. A vague
descriptionmakes the model guess when to call. Write precise triggers. - Ignoring latency. Each tool round trip adds 1–2 seconds plus model inference. Batch independent calls when the API supports parallel
tool_calls. - Silent schema drift. If you change a function signature, update the schema simultaneously. Mismatches produce runtime errors that look like model hallucinations.
- Cost leakage. Long system prompts repeated every loop iteration inflate bills. Trim conversation history or summarize prior steps.
- Security blind spots. Tool outputs often contain raw DB rows or API responses. Sanitize before returning to the model to avoid prompt injection from downstream systems.
- No fallback on tool failure. Returning a raw exception string can confuse the model. Return a structured
{"error": "...", "retryable": true}so it can adapt.
Function calling turns an LLM into a router for your code. The discipline is in the schema, the validation, and the loop guardrails—not in the model itself. Build those three well and the agent will do useful work instead of hallucinating JSON.
This LLM function calling guide skipped framework hype for the raw API surface because that’s what breaks in production. Start with one read-only tool, add a write tool behind a check, and measure where the model actually fails.