Function calling is a constrained generation protocol where a model emits a structured payload describing an external function and its arguments, rather than producing free-form text. Understanding how function calling works in GPT-4o and Claude shows that both systems implement it as schema-guided token prediction: the model selects a tool and produces JSON conforming to a supplied schema, and your code executes the actual side effect.
What function calling actually is
At its core, function calling is not “calling” anything. The LLM never imports your code, opens a socket, or touches a database. It outputs a deterministic-structured suggestion—usually a JSON object—that names a function and supplies argument values inferred from the conversation. Your application loop intercepts that suggestion, runs the real function, and feeds the result back as another message.
The training objective for these models includes a supervised signal that teaches them to emit well-formed tool invocations when the prompt context includes a tool catalog. That catalog is a list of function signatures plus natural-language descriptions. The model uses the descriptions to decide whether to call, and the schema to decide how to shape the arguments.
This matters because it converts an unpredictable natural-language interface into a typed boundary. Instead of parsing “please check the weather in San Francisco” with regex, you receive {"location": "San Francisco"} against a known contract.
How the protocol works under the hood
Both GPT-4o and Claude treat function calling as a structured output mode layered on top of the standard autoregressive generation. The client sends a list of available tools. The model generates a response that either contains normal text or a special tool-invocation block. The API response separates that block from chat text so your parser doesn’t need to scrape strings.
GPT-4o’s implementation
OpenAI exposes tools via the tools parameter in the Chat Completions API. Each tool is typed as "function" and carries a parameters object that is a JSON Schema draft. GPT-4o can emit zero, one, or many tool_calls on a single assistant message. Each call has a function.name and a function.arguments string (JSON-encoded).
{
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Fetch current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
}
]
}The response looks like:
{
"choices": [
{
"message": {
"role": "assistant",
"tool_calls": [
{
"id": "call_abc",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\": \"San Francisco\"}"
}
}
]
}
}
]
}GPT-4o also supports parallel_tool_calls and a strict mode that enforces schema adherence more aggressively.
Claude’s implementation
Anthropic’s Messages API uses a tools array where each entry has name, description, and input_schema (a JSON Schema). Claude returns tool_use blocks inside the content array, each with an id, name, and input object (already parsed, not a string).
{
"tools": [
{
"name": "get_weather",
"description": "Fetch current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
]
}Response content:
{
"content": [
{
"type": "tool_use",
"id": "toolu_123",
"name": "get_weather",
"input": {"location": "San Francisco"}
}
]
}The structural difference is cosmetic: OpenAI wraps arguments as a JSON string inside function.arguments; Claude parses them into input. Both require you to echo a tool_result (OpenAI: role: "tool", tool_call_id; Claude: role: "user" with tool_result block) on the next turn.
Why it matters for agent builders
Without function calling, every integration with external state requires fragile prompt engineering and output parsing. With it, you get:
- Type safety at the boundary. The schema defines required fields; the model fills them.
- Composability. Tools are declarative; you can swap or add capabilities without retraining.
- Auditability. Each invocation is explicit, logged, and replayable.
- Multi-step control. You own the loop, so you can enforce budgets, permissions, and fallbacks.
Knowing how function calling works lets you design agents where the model proposes, but your code disposes. That separation is what keeps a typo in a model’s output from deleting a production table.
Concrete example: a weather agent
Below are minimal clients for both providers hitting the same logical tool. The business logic (get_weather) is identical; only the transport differs.
# shared stub
def get_weather(location: str) -> str:
return f"Sunny in {location}, 72F"
# GPT-4o
from openai import OpenAI
oai = OpenAI()
oai_resp = oai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Weather in SF?"}],
tools=[{
"type": "function",
"function": {
"name": "get_weather",
"parameters": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"]
}
}
}]
)
tc = oai_resp.choices[0].message.tool_calls[0]
args = json.loads(tc.function.arguments)
result = get_weather(args["location"])
# send result back:
oai.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "user", "content": "Weather in SF?"},
oai_resp.choices[0].message,
{"role": "tool", "tool_call_id": tc.id, "content": result}
]
)# Claude
import anthropic, json
cl = anthropic.Anthropic()
cl_resp = cl.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
tools=[{
"name": "get_weather",
"input_schema": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"]
}
}],
messages=[{"role": "user", "content": "Weather in SF?"}]
)
tool_block = next(b for b in cl_resp.content if b.type == "tool_use")
result = get_weather(tool_block.input["location"])
cl.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
tools=[{"name": "get_weather", "input_schema": {"type":"object","properties":{"location":{"type":"string"}},"required":["location"]}}],
messages=[
{"role": "user", "content": "Weather in SF?"},
{"role": "assistant", "content": cl_resp.content},
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": tool_block.id, "content": result}]}
]
)The pattern is identical: detect tool request, execute, return result. How function calling works in practice is this round trip, repeated until the model emits a final answer with no tool blocks.
Common misconceptions
“The model runs my code.” False. It emits a name and arguments. If you don’t write the executor, nothing happens.
“Schemas are optional.” They are not. Omit required and you will get intermittent missing fields. Both providers validate against the schema only loosely at inference time; GPT-4o strict mode helps but still expects you to handle malformed edges.
“Function calling is an OpenAI-only feature.” Claude, Google, and open-weight models support equivalent tool-use protocols. The wire format differs; the concept is portable.
“Parallel calls mean transactional guarantees.” GPT-4o may emit three tool calls in one turn. Your code must execute them with whatever consistency you need; the model does not coordinate side effects.
“Streaming makes tool calls hard.” You accumulate tool_calls deltas and concatenate function.arguments fragments before parsing. Claude streams tool_use with incremental input objects. It is bookkeeping, not magic.
“Once a tool is called, the model remembers the result automatically.” You must append the result message. Forgetting this yields infinite loops or ignored data.
Running the same tools across models
If you serve traffic across GPT-4o and Claude for redundancy, normalize the tool schema once and translate per provider. An inference gateway such as n4n.ai that exposes a single OpenAI-compatible endpoint across 240+ models can forward the same tools definition and apply automatic fallback when one provider is rate-limited, without rewriting your client loop. The model-specific quirks (argument string vs parsed input) get handled at the boundary, letting you reason about how function calling works at the protocol level rather than per vendor.
The takeaway: function calling is a disciplined handshake between sampling and execution. You define the contract; the model proposes; your code enforces reality.