Function calling vs JSON mode: what's the difference?

The function calling vs json mode difference is often blurred in docs, but the two solve distinct problems. Function calling is a request/response protocol for the model to select and parameterize external tools; JSON mode is a response constraint that forces the output to be parseable JSON. Understanding both saves you from building fragile parsing loops or over-engineering a tool router when you only needed structured text.

What each mechanism actually does

Function calling (tool calls)

In the OpenAI API, you send a tools array describing functions with JSON schemas. The model returns a tool_calls block instead of (or in addition to) content. Your code executes the function and returns the result in a tool message, then calls the model again to continue.

from openai import OpenAI
client = OpenAI()

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role":"user","content":"What's the weather in Berlin?"}],
    tools=[{
        "type":"function",
        "function":{
            "name":"get_weather",
            "description":"Get current weather for a city",
            "parameters":{
                "type":"object",
                "properties":{"city":{"type":"string"}},
                "required":["city"]
            }
        }
    }]
)
print(resp.choices[0].message.tool_calls)

The model decides whether to call get_weather and with what arguments. It does not execute anything.

JSON mode

JSON mode is enabled by setting response_format={"type":"json_object"}. The model must output a valid JSON object, but you define the shape via prompt or, with newer structured outputs, via a schema. No tool loop exists.

import json
from openai import OpenAI
client = OpenAI()

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role":"user","content":"Return JSON with fields: city, temp_c"}],
    response_format={"type":"json_object"}
)
data = json.loads(resp.choices[0].message.content)

The output is just a string you parse.

Head-to-head comparison

Capabilities

Function calling shines when the model must choose among multiple actions or call several in parallel. It can return three tool calls in one response; you map them to executors. JSON mode cannot express “call this function” — it can only describe an action in data, which you then interpret.

For pure data extraction (e.g., pull name, date, amount from a contract), JSON mode with a strict schema is enough. You don’t need the model to “decide” to call a function; you dictate the shape.

A subtle capability gap: function calling lets the model omit calls entirely when the query is unrelated. JSON mode always returns an object, so you must handle empty or default fields.

Cost model

Both are billed per token at the same rate from the provider. The function calling vs json mode difference in cost appears in context overhead. Tool schemas are injected into every request, adding hundreds of tokens per call if you have many tools. JSON mode may need a schema in the prompt, but you can keep it terse.

If you actually execute tools and send results back, function calling multiplies tokens: initial call + result message + second call. For high-volume extraction, that extra round trip is pure cost. JSON mode finishes in one shot.

When routing through a gateway like n4n.ai, per-token metering is identical, but the gateway forwards provider cache-control hints, so repeated tool schemas can hit prompt caches and soften the overhead.

Latency and throughput

Single-shot JSON mode typically wins latency: one network round trip, no server-side tool validation beyond JSON parse. Function calling adds a second trip if you invoke the tool. However, modern models parse tool calls with negligible extra compute; the bottleneck is your own function execution.

Throughput at scale is similar. If you batch many extraction jobs, JSON mode’s one-shot nature simplifies queueing. Function calling pipelines need state machines to track tool_call_id and message history, which raises engineering cost more than wall-clock time.

Ergonomics

OpenAI’s Python SDK gives you message.tool_calls as typed objects. You iterate, dispatch, append results. It’s clean for agents.

JSON mode forces you to json.loads(content) and validate with pydantic or jsonschema. With structured outputs, the SDK can return parsed objects, but you still wrote the schema twice (prompt + code).

For a quick script that scrapes fields, JSON mode is ten lines. For an agent that books flights, sends email, and queries DB, function calling removes hand-rolled intent detection.

Ecosystem and portability

Function calling (under names “tools”, “function_declarations”) exists in OpenAI, Anthropic, Google, and Mistral. An OpenAI-compatible endpoint that addresses 240+ models, such as n4n.ai, passes tool definitions through and honors client routing directives, so the same tools array works across providers with fallback when one is degraded.

JSON mode began as OpenAI-specific. Many open-weight models now support response_format via inference servers, but behavior varies: some only guarantee valid JSON, not your schema. If you target multiple providers, function calling has broader consistent semantics.

Limits and caveats

• Tool count limits: OpenAI allows up to 128 tools; large schemas can hit context limits.
• JSON mode requires the word “JSON” in the prompt or it may refuse; structured outputs require strict schema (no additional properties).
• Function calling does not validate arguments beyond type; a model can still pass "city": 123 if your schema allows it (use enum/strict).
• Both can hallucinate fields; neither replaces application-level validation.

Which to choose

Single structured extraction

Use JSON mode (or structured outputs). You want one object, no side effects. Lower latency, cheaper, simpler.

Multi-step agentic workflows

Use function calling. The model orchestrates tools; you avoid writing a custom intent classifier.

Constrained generation across providers

Prefer function calling for portability. If you must use JSON mode, pin to models with verified schema support and keep schemas lenient.

Cheap and simple parsing

If you control the prompt and the task is trivial, JSON mode with a pydantic guard is the fastest path. Reserve function calling for when the model needs to choose actions, not just fill a template.

The function calling vs json mode difference is not about “newer vs older” but about control flow. Pick the one that matches who decides what runs: you (JSON) or the model (tools).