JSON mode across providers: OpenAI, Anthropic, and Gemini

The phrase “json mode openai anthropic gemini” hides three distinct mechanisms for forcing language models to emit parseable JSON. OpenAI gives you a response_format flag, Anthropic pushes you toward tool use, and Gemini exposes a responseSchema field. Pick wrong and you’ll ship a parser that breaks on every provider swap.

Capabilities

OpenAI: response_format and Structured Outputs

OpenAI ships two flavors. The older json_object mode only guarantees syntactically valid JSON, not schema adherence. You still need to tell the model to produce JSON in the prompt. The newer Structured Outputs (json_schema) enforces a supplied schema strictly on supported models (gpt-4o, gpt-4o-mini, and dated snapshots).

from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o-2024-08-06",
    messages=[{"role":"user","content":"Extract name and age from: John is 30"}],
    response_format={
        "type":"json_schema",
        "json_schema":{
            "name":"person",
            "schema":{
                "type":"object",
                "properties":{"name":{"type":"string"},"age":{"type":"integer"}},
                "required":["name","age"],
                "additionalProperties": False
            }
        }
    }
)
print(resp.choices[0].message.content)

The output is guaranteed to match the schema or the call fails validation server-side.

Anthropic: tool use as JSON mode

Anthropic has no json mode openai anthropic gemini equivalent flag. The supported path is defining a tool with an input_schema and letting the model emit a tool_use block. Force it with tool_choice to avoid text preamble.

import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=256,
    tools=[{
        "name":"extract_person",
        "description":"Extract person data",
        "input_schema":{
            "type":"object",
            "properties":{"name":{"type":"string"},"age":{"type":"integer"}},
            "required":["name","age"]
        }
    }],
    tool_choice={"type":"tool","name":"extract_person"},
    messages=[{"role":"user","content":"Extract name and age from: John is 30"}]
)
block = next(b for b in resp.content if b.type == "tool_use")
print(block.input)

Schema adherence is strong but not mathematically guaranteed; the model can still hallucinate extra keys unless you validate downstream.

Gemini: responseMimeType and responseSchema

Gemini uses responseMimeType: "application/json" plus a responseSchema object that follows Google’s own type enum (OBJECT, STRING, INTEGER, NUMBER, BOOLEAN, ARRAY). It works on Gemini 1.5 and later.

import google.generativeai as genai
genai.configure(api_key="...")
model = genai.GenerativeModel("gemini-1.5-pro")
resp = model.generate_content(
    "Extract name and age from: John is 30",
    generation_config={
        "response_mime_type":"application/json",
        "response_schema":{
            "type":"OBJECT",
            "properties":{
                "name":{"type":"STRING"},
                "age":{"type":"INTEGER"}
            },
            "required":["name","age"]
        }
    }
)
print(resp.text)

This yields strict JSON, but the schema language is a subset—no pattern, format, or anyOf.

Cost model

All three providers bill per input/output token with no separate surcharge for structured output. OpenAI Structured Outputs may reflect the schema in the prompt, adding a few hundred input tokens on complex definitions. Anthropic tool use places the schema in the tools array, inflating context similarly. Gemini sends schema in the generation_config, same minor overhead. If you route through a gateway such as n4n.ai, per-token usage metering remains transparent across all three, so you can compare effective cost without manual accounting.

Latency and throughput

JSON mode openai anthropic gemini adds single-digit milliseconds to time-to-first-token on well-provisioned instances. OpenAI Structured Outputs can retry on validation failure, which occasionally spikes tail latency on deeply nested schemas. Anthropic tool use streams normal text then emits the tool call, adding one parse step; forcing tool_choice removes ambiguity but still requires the model to fill the block. Gemini validates server-side before returning, keeping latency flat. None of these are throughput bottlenecks relative to base model inference.

Ergonomics

OpenAI is the cleanest: one parameter, standard SDK. Gemini is close but the upper-case type names (STRING vs string) cause porting bugs. Anthropic demands tool scaffolding and block parsing—more lines, but it composes with agent loops.

// OpenAI
const res = await openai.chat.completions.create({
  model: "gpt-4o",
  response_format: { type: "json_object" },
  messages
});
// Anthropic
const msg = await anthropic.messages.create({
  model: "claude-3-5-sonnet",
  tools: [{ name: "extract", input_schema: {...} }],
  tool_choice: { type: "tool", name: "extract" },
  messages
});
// Gemini
const gen = await model.generateContent({
  contents,
  generationConfig: { responseMimeType: "application/json", responseSchema: {...} }
});

Streaming JSON is awkward on all three: OpenAI streams partial JSON strings, Anthropic streams tool input incrementally, Gemini returns the whole object at the end unless you use streaming beta.

Ecosystem and tooling

OpenAI’s response_format is mimicked by open-weight models (via Outlines, llama-cpp) and proxies. Anthropic’s tool use is the backbone of Claude agent frameworks; LangChain and Anthropic’s own SDKs abstract it. Gemini’s schema ties you to Google’s SDK or raw REST. If you need to switch providers at runtime, a unified OpenAI-compatible endpoint that addresses 240+ models—like n4n.ai—lets you send the same response_format and get automatic fallback when a provider is rate-limited, while forwarding cache-control hints to cut repeat costs.

Limits and caveats

OpenAI Structured Outputs reject schemas with unbounded additionalProperties and cap recursion depth (typically 5 levels). Anthropic tool use can still emit malformed JSON in edge cases if you omit tool_choice. Gemini’s schema subset omits nullable and anyOf patterns, forcing you to flatten unions. All three require a prompt instruction to produce the target data; the flag alone may not suffice for json_object mode on OpenAI legacy models, and Gemini will error if the schema contradicts the prompt.

Comparison table

Which to choose

Strict extraction on OpenAI stack

Use json_schema Structured Outputs. You get guarantees, simple code, and easy testing. Avoid deep nesting beyond five levels.

Claude-only pipeline

Define a tool, force tool_choice, and treat tool_use.input as your JSON. It’s the only first-class path for json mode openai anthropic gemini on Anthropic, and it doubles as your agent action.

Gemini data pipeline

Set responseMimeType and responseSchema. Avoid complex unions; flatten schema to fit Google’s subset. Good for high-volume server-side extraction.

Multi-provider with fallback

If you must support json mode openai anthropic gemini behind one interface, standardize on the OpenAI response_format shape and route through a gateway that translates to Anthropic tool use and Gemini schema. That removes per-provider branching from your codebase and gives you automatic degradation when one provider is throttled.

Pick based on where your models run, not on which API looks nicest in a snippet.