Structured outputs for extraction: model comparison

Choosing a model for document extraction is not just about benchmark scores; it’s about whether the model emits valid JSON that survives your validation layer. This structured outputs extraction model comparison puts four production-grade options side by side—OpenAI GPT-4o, Anthropic Claude 3.5 Sonnet, Google Gemini 1.5 Pro, and Meta Llama 3.1 70B—so you can see where each wins when the task is turning messy text into typed objects.

The contenders

OpenAI GPT-4o exposes a dedicated Structured Outputs API that enforces a supplied JSON Schema at the decoder level. Anthropic Claude 3.5 Sonnet has no JSON mode, but its tool-use path is robust enough that most teams treat function schemas as the extraction contract. Google Gemini 1.5 Pro supports responseSchema with constrained generation, and Llama 3.1 70B relies on third-party inference platforms that wrap it with grammar-constrained decoding or tool calling. In practice, all four can be made to output parseable JSON; the difference is how much of that guarantee is mathematical versus cultural.

Capabilities: schema adherence

Strictness matters. GPT-4o’s structured outputs guarantee the response matches the schema exactly (types, required fields, enum values) when you set "strict": true. That removes the need for post-hoc repair. Claude’s tool use is not mathematically guaranteed, but in practice it rarely drifts if you give descriptive tool names and avoid deeply nested unions. Gemini’s responseSchema works well for flat and moderately nested structures, but complex anyOf patterns can fall back to best-effort. Llama 3.1 via a gateway with grammar support (e.g., outlines) can enforce context-free grammars, though schema recursion beyond a few levels sometimes truncates.

This structured outputs extraction model comparison shows that the strictness gap is narrowing, but only GPT-4o and grammar-backed Llama give you hard guarantees. For extraction of repeated entities (invoices, resumes, security logs), guaranteed structure means you can skip the retry loop.

Example GPT-4o request:

from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role":"user","content":"Extract invoice fields from: ..."}],
    response_format={
        "type":"json_schema",
        "json_schema":{
            "name":"invoice",
            "strict":True,
            "schema":{
                "type":"object",
                "properties":{
                    "invoice_id":{"type":"string"},
                    "total":{"type":"number"},
                    "line_items":{"type":"array","items":{"type":"string"}}
                },
                "required":["invoice_id","total","line_items"]
            }
        }
    }
)

Claude uses a tool shape:

{
  "name": "extract_invoice",
  "input_schema": {
    "type": "object",
    "properties": {
      "invoice_id": {"type": "string"},
      "total": {"type": "number"}
    },
    "required": ["invoice_id", "total"]
  }
}

Gemini config:

generation_config = {
    "response_mime_type": "application/json",
    "response_schema": {
        "type": "OBJECT",
        "properties": {
            "invoice_id": {"type": "STRING"},
            "total": {"type": "NUMBER"}
        }
    }
}

Price and cost model

Public list prices (mid-2024) give a rough ordering. GPT-4o runs about $5 per million input tokens and $15 per million output. Claude 3.5 Sonnet is $3 in / $15 out. Gemini 1.5 Pro is $3.50 in / $10.50 out for prompts under 128k, doubling beyond. Llama 3.1 70B on hosted platforms typically lands near $0.90 per million input and output, sometimes less with batch. If you are extracting from 10-page PDFs at scale, the gap between GPT-4o and Llama is 5–10x. None of these include the hidden cost of re-parsing invalid JSON; strict models cut that line item to near zero.

In this structured outputs extraction model comparison, cost asymmetry is the loudest signal for bulk workloads. A million documents a month at 2k output tokens each is $30k on GPT-4o versus ~$1.8k on Llama. The accuracy trade-off must justify the difference.

Latency and throughput

Measured anecdotally in production: GPT-4o streams the first token fast (often <400ms on short prompts) and sustains high tokens/sec. Claude 3.5 Sonnet has slightly higher time-to-first-token on long system prompts but strong throughput once generating. Gemini 1.5 Pro handles massive batches with high aggregate throughput, though single-request TTFT varies. Llama 70B on commodity GPUs is slower per request unless served on high-end rigs; expect 30–60 tokens/sec. For synchronous user-facing extraction, GPT-4o or Claude; for offline bulk jobs, Gemini or Llama with batch.

Streaming structured output is still immature across the board—you cannot reliably parse partial JSON until the stream closes. If you need progressive UI, buffer then validate.

Ergonomics

OpenAI’s SDK returns parsed JSON directly when using response_format. Anthropic requires you to parse the tool input field. Gemini returns a string you json.loads. With Llama, you depend on the platform’s grammar wrapper. Pydantic remains the lingua franca: generate schemas from models and validate after the fact. n4n.ai collapses these differences by exposing one OpenAI-compatible endpoint that forwards provider cache-control hints and honors routing directives, so the same Pydantic schema can be tried across all four without rewriting HTTP calls.

from pydantic import BaseModel
class Invoice(BaseModel):
    invoice_id: str
    total: float
    line_items: list[str]
# feed Invoice.model_json_schema() to any of the above

Error modes differ: GPT-4o throws a server error if the schema is unsatisfiable; Claude silently ignores unknown keys; Gemini may emit empty strings for missing numerics; Llama can halt mid-grammar if context overflows. Build a normalization layer regardless.

Ecosystem and limits

GPT-4o: 128k context, hard limit on schema size (a few hundred properties). Claude: 200k context, tool schema limited but generous. Gemini: 1M+ context, ideal for whole-document extraction without chunking. Llama 3.1: 128k context, grammar limits depend on serving stack. All have rate limits that bite at scale; GPT and Claude tier slowly, Gemini offers provisioned capacity. Open-source tooling (LangChain, LlamaIndex) has first-class support for OpenAI and Anthropic, partial for Gemini, and fragmented for self-hosted Llama.

Comparison table

Which to choose

High-stakes extraction (legal, finance): Use GPT-4o or Claude 3.5 Sonnet. Strict schema adherence or near-strict tool use minimizes manual review. Claude’s longer context helps when documents span 100k tokens.

Cost-driven high volume: Llama 3.1 70B or Gemini 1.5 Pro. If you can tolerate best-effort validation, the 5–10x savings compound quickly. Gemini’s million-token window also kills chunking logic.

Low-latency interactive UX: GPT-4o leads. Stream validated partial objects if your client supports it.

Multimodal extraction (PDF scans, images): Gemini 1.5 Pro natively ingests images alongside text; Claude can too but Gemini’s schema path is simpler for mixed input.

Heterogeneous pipeline: If you want to A/B without vendor lock, route through a gateway that normalizes the API. That’s the only place n4n.ai fits—one endpoint, automatic fallback when a provider is degraded, per-token metering, so your structured outputs extraction model comparison becomes a config change rather than a refactor.

Pick based on the constraint that hurts most: accuracy, dollars, milliseconds, or context length. The models are close enough that switching is a week of work, not a quarter.