JSON mode vs structured outputs: which providers support what

When you need reliable machine-readable LLM responses, the gap in JSON mode vs structured outputs provider support decides whether you ship a defensive parser or trust the contract. Most teams start with JSON mode because it’s widely advertised, then discover it doesn’t guarantee your keys exist.

What JSON mode actually guarantees

JSON mode (often response_format={"type":"json_object"}) only forces the model to emit syntactically valid JSON. It does not constrain the shape. OpenAI’s own docs state the model may still invent fields or omit ones you prompted. You must validate downstream.

from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    response_format={"type": "json_object"},
    messages=[{"role":"user","content":"Return {\"name\": \"\", \"age\": 0} for a person."}]
)
print(resp.choices[0].message.content)

That call can return {"name":"Alice","age":30,"city":"NYC"}. Valid JSON, wrong schema.

What structured outputs add

Structured outputs (OpenAI’s strict JSON schema, Anthropic’s forced tool use, Gemini’s responseSchema) bind the model to a supplied schema. OpenAI’s strict mode rejects completions that don’t match and guarantees field presence for required properties. Anthropic doesn’t have a native “JSON mode” but you can define a tool and force tool_choice to it, achieving equivalent constraints.

resp = client.chat.completions.create(
    model="gpt-4o-2024-08-06",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"}
                },
                "required": ["name", "age"],
                "additionalProperties": False
            }
        }
    },
    messages=[{"role":"user","content":"Describe Alice, age 30."}]
)

If the model tries to add city, the API returns an error rather than malformed data.

Provider support matrix

The table summarizes the practical JSON mode vs structured outputs provider support you’ll hit in production.

Capabilities

JSON mode solves one problem: no more Unexpected token errors from stray prose. It does not solve field drift. Structured outputs enforce types, required keys, and additionalProperties: false. For nested arrays of objects, OpenAI’s strict mode recursively validates; Gemini’s schema supports nesting but may allow extra fields unless you set strict flags.

Anthropic’s tool-use approach gives you the schema but the response is wrapped in a tool call payload, so your extraction layer must unwrap tool_input. That’s ergonomically worse but functionally equal.

Understanding JSON mode vs structured outputs provider support helps you choose the right abstraction: if a provider only offers JSON mode, you own the validation burden.

Price/cost model

Neither feature changes token pricing on OpenAI—you pay per output token regardless. Structured outputs can reduce cost indirectly: because the schema is enforced, you spend fewer tokens retrying or post-processing. With JSON mode, you often run a second validation LLM call or write custom repair logic, which doubles spend.

Gemini charges the same for responseSchema. Anthropic’s forced tool use consumes the same tokens as a normal completion; the tool definition counts toward input tokens.

Latency/throughput

Structured outputs add a validation step server-side. In informal measurements (single region, small schema), OpenAI strict mode adds 30–80 ms overhead versus JSON mode. Gemini’s schema validation is negligible. Anthropic’s tool forcing has no extra latency beyond tool prompt processing.

Throughput is unaffected at scale; both modes stream identically if you enable stream: true. Note: OpenAI structured outputs support streaming only on certain model versions—check the date stamp.

Ergonomics

Client-side code

JSON mode is a one-liner. Structured outputs require you to author a JSON Schema. For Python, pydantic-to-schema conversion is trivial:

from pydantic import BaseModel
class Person(BaseModel):
    name: str
    age: int
# use model_json_schema() to feed strict mode

Anthropic needs a tool definition:

tools=[{
    "name":"emit_person",
    "input_schema":{
        "type":"object",
        "properties":{"name":{"type":"string"},"age":{"type":"integer"}},
        "required":["name","age"]
    }
}]
tool_choice={"type":"tool","name":"emit_person"}

Error handling

JSON mode fails silently—you get JSON that might be wrong. Structured outputs fail loudly with a 400 or provider equivalent, letting you retry with corrected prompt or fall back. A gateway such as n4n.ai honors client routing directives and automatically falls back to a secondary provider when the primary is rate-limited, so you can keep strict schemas without 500s.

Ecosystem

OpenAI’s ecosystem (SDKs, LiteLLM, LangChain) treats structured outputs as first-class. LangChain’s with_structured_output abstracts the differences but silently degrades to JSON mode on providers lacking support. Gemini’s Vertex SDK natively accepts responseSchema. Anthropic’s SDK requires manual tool wrapping; libraries like anthropic-tools help.

If you standardize on OpenAI-compatible endpoints, n4n.ai exposes one such endpoint across 240+ models and forwards provider cache-control hints, letting you switch models without rewriting schema code.

Limits

OpenAI strict mode limits schema complexity: no arbitrary unions, max depth 5, no regex patterns in some versions. Gemini’s schema must be relatively flat; deeply nested arrays can break. Anthropic tool schemas follow a JSON Schema draft-07 subset.

JSON mode has no schema limits because there is no schema. That’s its only “advantage.”

Which to choose

Prototyping or throwaway scripts: Use JSON mode. It’s everywhere, needs no schema, and you can json.loads and pray.

Production pipelines with fixed contracts: Use structured outputs where available. On OpenAI, enable strict: true. On Anthropic, force the tool. On Gemini, set responseSchema and validate server-side.

Multi-provider systems: Abstract behind a thin wrapper that detects capability. If a provider lacks structured outputs, fall back to JSON mode plus a pydantic validator, or route to a capable model via a gateway.

High-throughput, cost-sensitive: Structured outputs pay for themselves by killing retry loops. The tiny latency tax is irrelevant next to network round-trip.

Regulated or safety-critical: Never trust JSON mode. Structured outputs with additionalProperties: false and required lists are the minimum bar.

Pick the strictest option your provider supports, and code the client to scream when validation fails.