Getting reliable Pydantic structured outputs across LLM APIs means dealing with three different constraint philosophies. OpenAI enforces a JSON schema via strict response formats, Anthropic coerces structure through tool definitions, and Gemini uses declarative response schemas. Your business logic should validate against one Pydantic model regardless of which backend produced the bytes.
Step 1: Define a single Pydantic model as the contract
Start by writing the schema your application actually needs. Everything else adapts to this.
from pydantic import BaseModel, Field
class InvoiceLineItem(BaseModel):
quantity: int = Field(..., ge=1)
unit_price_cents: int = Field(..., ge=0)
class Invoice(BaseModel):
vendor: str
line_items: list[InvoiceLineItem]
total_cents: intExport the JSON schema once and reuse it for every provider:
schema = Invoice.model_json_schema()Keep the model free of Python-only validators that can’t be expressed in JSON Schema. If you need post-processing, do it after parsing. Strip Pydantic-specific extensions (like title if you don’t want them) because some providers reject unknown keys in the schema object. A small sanitizer helps:
def clean_schema(s: dict) -> dict:
s.pop("title", None)
return s
wire_schema = clean_schema(schema)Step 2: Implement OpenAI strict mode
OpenAI’s response_format with json_schema and strict: true forces the model to emit conforming JSON. The Python SDK accepts the schema directly.
from openai import OpenAI
client = OpenAI() # uses OPENAI_API_KEY
resp = client.chat.completions.create(
model="gpt-4o-2024-08-06",
messages=[{"role": "user", "content": "Extract invoice from: ACME 2x widget @ 500 cents"}],
response_format={
"type": "json_schema",
"json_schema": {"name": "Invoice", "schema": wire_schema, "strict": True}
}
)
data = Invoice.model_validate_json(resp.choices[0].message.content)Strict mode rejects schemas with arbitrary additional properties, so the Pydantic export must set additionalProperties: false implicitly—the SDK handles that. One caveat: strict only works on models trained for it (gpt-4o and later mini variants). On older models the API returns an error rather than degrading gracefully.
Streaming with OpenAI
If you need tokens incrementally, pass stream=True and parse the concatenated delta.content yourself before validation. The partial JSON will not validate until complete, so buffer it.
Step 3: Drive Anthropic via tool calls
Anthropic Claude does not have a native structured output endpoint. You simulate it by defining a single tool whose input schema is your Pydantic model, then forcing tool use.
import anthropic
client = anthropic.Anthropic()
tool = {
"name": "emit_invoice",
"description": "Return extracted invoice",
"input_schema": wire_schema
}
resp = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
tools=[tool],
tool_choice={"type": "tool", "name": "emit_invoice"},
messages=[{"role": "user", "content": "Extract invoice from: ACME 2x widget @ 500 cents"}]
)
tool_use = next(b for b in resp.content if b.type == "tool_use")
data = Invoice.model_validate(tool_use.input)The model returns a tool invocation rather than raw JSON. You lose streaming of partial fields, but you gain guaranteed schema adherence. If you omit tool_choice, the model may answer in natural language and you’ll get a validation error downstream—always force it in extraction pipelines.
Step 4: Configure Gemini constrained response
Gemini’s generateContent API accepts a responseSchema and responseMimeType. It performs constrained decoding server-side.
import google.generativeai as genai
genai.configure(api_key="...")
model = genai.GenerativeModel("gemini-1.5-pro")
resp = model.generate_content(
"Extract invoice from: ACME 2x widget @ 500 cents",
generation_config={
"response_mime_type": "application/json",
"response_schema": wire_schema
}
)
data = Invoice.model_validate_json(resp.text)Gemini’s schema support currently omits some Pydantic features like minimum/maximum on integers in certain SDK versions; test edge cases. Also set max_output_tokens high enough—Gemini truncates silently if the JSON exceeds the budget.
Differences in failure modes
- OpenAI strict mode throws an API error if the schema is invalid.
- Anthropic silently passes unvalidated dicts if you forget
tool_choice. - Gemini may truncate if
max_output_tokensis too low.
Step 5: Abstract the backends behind one function
Write a thin dispatcher so callers never see provider quirks. Implement the three helpers explicitly:
from enum import Enum
class Provider(str, Enum):
OPENAI = "openai"
ANTHROPIC = "anthropic"
GEMINI = "gemini"
def _from_openai(text: str) -> Invoice:
resp = client.chat.completions.create(
model="gpt-4o-2024-08-06",
messages=[{"role": "user", "content": text}],
response_format={"type": "json_schema", "json_schema": {"name": "Invoice", "schema": wire_schema, "strict": True}}
)
return Invoice.model_validate_json(resp.choices[0].message.content)
def _from_anthropic(text: str) -> Invoice:
resp = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
tools=[{"name": "emit_invoice", "description": "Return extracted invoice", "input_schema": wire_schema}],
tool_choice={"type": "tool", "name": "emit_invoice"},
messages=[{"role": "user", "content": text}]
)
tool_use = next(b for b in resp.content if b.type == "tool_use")
return Invoice.model_validate(tool_use.input)
def _from_gemini(text: str) -> Invoice:
resp = model.generate_content(
text,
generation_config={"response_mime_type": "application/json", "response_schema": wire_schema}
)
return Invoice.model_validate_json(resp.text)
def extract_invoice(text: str, provider: Provider) -> Invoice:
return {
Provider.OPENAI: _from_openai,
Provider.ANTHROPIC: _from_anthropic,
Provider.GEMINI: _from_gemini,
}[provider](text)Each helper wraps the SDK call from Steps 2–4 and returns a validated Invoice. This keeps Pydantic structured outputs across LLM APIs consistent in your service layer.
Step 6: Collapse the matrix with an OpenAI-compatible gateway
Maintaining three SDKs is tolerable until you add Mistral, Groq, or self-hosted vLLM. An OpenAI-compatible gateway such as n4n.ai fronts 240+ models behind one endpoint and forwards provider cache-control hints, so the same strict-mode request works without branching.
from openai import OpenAI
gw = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="YOUR_KEY"
)
resp = gw.chat.completions.create(
model="anthropic/claude-3-5-sonnet", # or "openai/gpt-4o", "google/gemini-1.5-pro"
messages=[{"role": "user", "content": "Extract invoice: ACME 2x widget @ 500 cents"}],
response_format={"type": "json_schema", "json_schema": {"name": "Invoice", "schema": wire_schema, "strict": True}}
)
data = Invoice.model_validate_json(resp.choices[0].message.content)The gateway translates the OpenAI-shaped request to the target provider’s native structured output mechanism. You keep one code path and get automatic fallback when a provider is rate-limited. Client routing directives (e.g., model="anthropic/claude-3-5-sonnet") let you pin a vendor without changing the call shape.
Step 7: Verify success
Validation must be automated. Write a pytest case that runs the dispatcher against each provider (or the gateway with explicit routing) and asserts the contract.
def test_invoice_extraction():
for prov in Provider:
inv = extract_invoice("ACME 2x widget @ 500 cents", prov)
assert isinstance(inv, Invoice)
assert inv.total_cents == 1000
assert inv.line_items[0].quantity == 2Run it:
pytest -q test_invoice.pyA green run proves the Pydantic model parses and the arithmetic holds. For the gateway path, check resp.usage to confirm per-token metering and send cache_control in the request to verify the hint is forwarded.
Operational notes
- Log the raw response before validation; when a provider changes its schema translation, you’ll see it immediately.
- Pin model versions. Structured output behavior is a model capability, not a platform constant.
- If you stream, only OpenAI and Gemini support incremental JSON; Anthropic tool calls arrive whole.
- Add a schema-drift test: compare
wire_schemaagainst the provider’s documented limits quarterly.
Building Pydantic structured outputs across LLM APIs this way costs you one Pydantic model and a small dispatcher. Everything else is a transport detail.