OpenAI JSON mode vs Anthropic tool use for structured output

Getting reliable structured data from LLMs usually comes down to two patterns: OpenAI’s JSON mode and Anthropic’s tool use. The JSON mode vs tool use decision shapes your parsing code, error handling, and even token spend, so it deserves a concrete comparison rather than a gut call. Both approaches solve the same problem—turning free-form model text into typed fields—but they sit at different layers of the stack.

Capabilities

OpenAI JSON mode

JSON mode constrains the model to emit a valid JSON object. With the strict variant (available on recent models like gpt-4o-2024-08-06), the output also conforms to a supplied JSON schema. You get a single object in message.content that you json.loads directly. The service rejects or retries generations that violate the schema mid-flight, so client-side validation becomes a backstop rather than a necessity.

from openai import OpenAI
import json
client = OpenAI()

resp = client.chat.completions.create(
    model="gpt-4o-2024-08-06",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "user",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"}
                },
                "required": ["name", "age"],
                "additionalProperties": False
            }
        }
    },
    messages=[{"role": "user", "content": "Extract: Jane is 30"}]
)
data = json.loads(resp.choices[0].message.content)

The model cannot call external functions; it only fills the schema. This is ideal for extraction, classification, and form filling where the set of fields is known upfront.

Anthropic tool use

Anthropic’s Claude uses tool definitions as a structured-output mechanism. You pass a tools array describing inputs, and force invocation via tool_choice. The model returns a tool_use block with input already parsed as a Python dict by the SDK. The same interface that produces structured data can trigger real side effects if you choose to execute the tool.

import anthropic
client = anthropic.Anthropic()

resp = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    tools=[{
        "name": "extract_user",
        "description": "Return user data",
        "input_schema": {
            "type": "object",
            "properties": {
                "name": {"type": "string"},
                "age": {"type": "integer"}
            },
            "required": ["name", "age"]
        }
    }],
    tool_choice={"type": "tool", "name": "extract_user"},
    messages=[{"role": "user", "content": "Extract: Jane is 30"}]
)

tool_block = next(b for b in resp.content if b.type == "tool_use")
data = tool_block.input

Tool use doubles as agentic action: the same interface triggers real API calls. For pure structured output, you ignore the “action” semantics and treat it as schema-bound generation. Unlike JSON mode, you can define several tools and let the model pick the closest match, which is useful for ambiguous inputs.

Price / cost model

Neither provider charges extra for the structuring mechanism itself; you pay per input and output token. The JSON mode vs tool use difference is in incidental token cost:

• OpenAI JSON mode: the schema lives in response_format (input tokens) and the output is compact JSON. Strict mode may add a few tokens of validator overhead server-side, not billed.
• Anthropic tool use: the tool definition lives in the tools array (input tokens) and the model echoes the tool name and a delimiter in its response (minor output overhead). For high-volume small extracts, the delta is negligible but present.

Both providers meter by token, not by request. If you route through a gateway such as n4n.ai, per-token usage metering is forwarded unchanged, so cost analysis stays provider-native. Prompt caching applies to both: prefix your system prompt with stable schema text to hit cache hits on repeated calls.

Latency / throughput

Measured end-to-end, both patterns add minimal overhead beyond base generation. JSON mode parses after the fact; tool use requires the model to emit a specific tool_use stop sequence. In practice, latency tracks the underlying model’s time-to-first-token and total completion length.

Streaming works for both. OpenAI streams JSON text chunks; Anthropic streams tool input incrementally. If you need to abort on schema violation, JSON mode with strict schema fails faster because the service validates during generation, while Anthropic validates only at tool boundary. Throughput under batch loads is equivalent; the bottleneck is model inference, not the framing.

Ergonomics

JSON mode is simpler if your stack already speaks OpenAI. You swap response_format and add a json.loads. Strict mode removes the need for retry loops on malformed JSON. Error handling is straightforward:

try:
    data = json.loads(resp.choices[0].message.content)
except json.JSONDecodeError:
    # strict mode makes this rare
    pass

Tool use forces you to map business objects to input_schema and inspect tool_use blocks. The payoff is native support for multiple alternative schemas (define several tools, let the model pick). JSON mode selects one schema per call.

# Multiple tools, model chooses
resp = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    tools=[schema_a, schema_b],
    tool_choice={"type": "any"},  # force any tool
    messages=[{"role": "user", "content": "Ambiguous input"}]
)

OpenAI’s json_schema strict mode supports multiple schemas only via separate calls or function-calling (which is distinct from JSON mode). That distinction matters when you compare JSON mode vs tool use head-to-head: tool use is function calling with a forced choice, JSON mode is constrained decoding.

Ecosystem

OpenAI’s format is replicated by many open-weight models (Mixtral, Llama via vLLM) and gateways. For example, n4n.ai provides one OpenAI-compatible endpoint that addresses 240+ models and automatic fallback when a provider is rate-limited or degraded; JSON mode requests translate cleanly across those backends. Anthropic’s tool use is Claude-specific, though Amazon Bedrock exposes a similar shape with translation.

For fallback, a gateway that honors client routing directives can shift a JSON-mode request from OpenAI to a compatible provider when rate-limited. The same gateway forwards provider cache-control hints, so prompt prefixes with schemas can be cached identically. Tool use requests cannot be transparently migrated because the tool schema dialect differs. If you already standardize on OpenAI SDK, JSON mode keeps your code portable; if you live in Claude toolchains, tool use avoids an adaptation layer.

Limits

OpenAI JSON mode strict requires additionalProperties: false and full required lists; recursive schemas are supported but constrained by model reasoning. Max output tokens still bind the response. The model may still produce semantically wrong data even if syntactically valid.

Anthropic tool use limits: input_schema must be JSON Schema draft-07 subset; nested arrays depth is fine but very large schemas eat context. Tool choice any forces a tool but not a specific one, while tool forces a named tool. Both fail gracefully only if you handle refusal or partial output. JSON mode can still produce semantically wrong data; tool use can return a tool call with missing fields if you omit required.

Comparison table

Which to choose

Use OpenAI JSON mode when:

• You need a single predictable object and already use OpenAI SDK.
• Portability across models behind one OpenAI-compatible endpoint matters.
• You want service-side schema validation to cut retry code.
• Extraction, NER, or scoring tasks dominate and you never need model-driven branching.

Use Anthropic tool use when:

• You are building an agent that may later invoke real functions.
• You need the model to choose among several output shapes in one pass.
• You are already on Claude and want to avoid a second abstraction layer.
• You want tool descriptions to guide extraction contextually and leverage tool_choice routing.

Hybrid / gateway pattern: Standardize on JSON mode at the edge and translate to tool use only for Claude-specific routes that need agentic steps. That keeps 80% of your code path provider-agnostic while preserving Anthropic’s richer tool routing for complex agents. The JSON mode vs tool use split is not about right or wrong; it is about whether structured output is a destination (decode and move on) or a waypoint (model acts next). Pick the primitive that matches that trajectory.