Structured output in LangChain with Pydantic and n4n.ai

Getting reliable JSON from an LLM is half the battle when you build agents or data pipelines. This langchain structured output pydantic tutorial walks through a concrete pattern for binding Pydantic schemas to LangChain chat models, validating responses, and degrading gracefully when a provider stalls. You will end up with code that turns messy model text into typed Python objects.

Why bind Pydantic to your LLM calls

LangChain’s with_structured_output exists because prompt engineering alone produces inconsistent text. Pydantic gives you a compile-time contract and runtime validation. If the model returns a string where you expect a float, you learn about it at the boundary, not three modules deep.

The approach also lets you swap models without rewriting parsing logic. The same schema drives function-calling prompts for GPT-class models and JSON mode for open-weight ones.

Environment and dependencies

Install the OpenAI integration and Pydantic:

pip install langchain-openai pydantic

Set your API key as an environment variable. If you target a gateway, set OPENAI_API_BASE or pass base_url explicitly in code.

Define the output contract

Start with a narrow schema. Smaller models respect fewer fields better. Use Field(description=...) liberally; LangChain forwards these as parameter descriptions to the model.

from pydantic import BaseModel, Field

class Product(BaseModel):
    name: str = Field(description="Product display name")
    price_usd: float = Field(description="Price in US dollars, numeric")
    in_stock: bool = Field(description="True if available to ship")

For one-to-many extraction, wrap in a list:

from typing import List

class Catalog(BaseModel):
    products: List[Product]

Attach the schema to the model

with_structured_output returns a new runnable bound to your class. Under the hood it uses function calling when available, otherwise JSON mode.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0,
)
structured_llm = llm.with_structured_output(Product)

result = structured_llm.invoke(
    "Extract: Apple AirPods Pro 2, $249, available"
)
print(result.name, result.price_usd, result.in_stock)

This langchain structured output pydantic tutorial assumes you run a recent LangChain (0.2+). Older versions used PydanticOutputParser and a manual prompt; avoid that path unless you need fine-grained prompt control.

Choosing the binding method

LangChain supports two underlying mechanisms: function_calling and json_mode. Force the method when you know the model’s capabilities:

structured_llm = llm.with_structured_output(Product, method="json_mode")

Function calling is more reliable on GPT-4 class models. JSON mode works on Mistral and Llama variants but requires the schema to be injected into the prompt, which increases failure risk on complex nests.

Handling nested and list responses

Pass the wrapping model to the method:

catalog_llm = llm.with_structured_output(Catalog)
catalog = catalog_llm.invoke(
    "Items: Banana $0.5 yes; Laptop $999 no"
)
for p in catalog.products:
    assert isinstance(p.price_usd, float)

If the model drops a field, Pydantic raises ValidationError. Catch it and retry with a stricter system message or a stronger model.

Enums and constrained values

Closed sets cut validation errors. Use Python Enum for categorical fields:

from enum import Enum

class Category(str, Enum):
    ELECTRONICS = "electronics"
    FOOD = "food"

class Product(BaseModel):
    name: str
    category: Category
    price_usd: float = Field(ge=0)

The model sees an allowed list and is less likely to hallucinate a value.

Validation and self-correction

with_structured_output does not auto-retry by default. Wrap it:

from pydantic import ValidationError

def extract_product(text: str) -> Product | None:
    try:
        return structured_llm.invoke(text)
    except ValidationError:
        # fallback to a larger model or json_mode
        return None

A better pattern is to chain a second LLM call that receives the validation error and the original text. LangChain’s RunnableRetry can encode this, but keep the loop bounded to two attempts to control cost.

Streaming tradeoffs

Structured output and token streaming do not mix well today. with_structured_output waits for the full payload before parsing. If you need incremental UI updates, stream raw text with llm.stream and parse at the end, or use a partial JSON parser at your own risk.

Testing without burning tokens

Lock parsing logic in unit tests with a fake chat model:

from langchain_core.chat_models.fake import FakeMessagesListChatModel
from langchain_core.messages import AIMessage

fake = FakeMessagesListChatModel(
    messages=[AIMessage(content='{"name":"Test","price_usd":1.0,"in_stock":true}')]
)
fake_struct = fake.with_structured_output(Product)
print(fake_struct.invoke("anything"))

This catches schema drift when you upgrade LangChain or Pydantic.

Routing across providers without rewriting code

Point the same ChatOpenAI client at an OpenAI-compatible gateway such as n4n.ai, which fronts 240+ models and automatically fails over when a backend is rate-limited or degraded, while honoring your routing directives and cache-control hints. Your with_structured_output call stays identical; you just change base_url and model name.

llm = ChatOpenAI(
    model="anthropic/claude-3-haiku",
    base_url="https://api.n4n.ai/v1",
    api_key="your-gateway-key",
)

This is the cheapest way to add resilience: no custom fallback code, per-token metering handled at the gateway.

Common pitfalls

Model capability mismatch

Not every model supports function calling. If you pass a schema to a model that only does chat, LangChain falls back to JSON mode, which is weaker. Check the model card.

Pydantic version drift

LangChain 0.2 expects Pydantic v2. If your project pins v1, import from langchain_core.pydantic_v1 to avoid silent breakage.

Overly strict types

float for prices invites rounding errors and model confusion. Sometimes Decimal or str with a regex is safer. Use Field(ge=0) to reject negative prices.

Missing descriptions

The description in Field is not documentation; it becomes the prompt. Omit it and the model guesses.

Latency from large schemas

Each field adds tokens to the function spec. Keep schemas under ~20 fields for sub-second responses on small models.

A complete minimal script

import os
from typing import List
from pydantic import BaseModel, Field, ValidationError
from langchain_openai import ChatOpenAI

class Product(BaseModel):
    name: str = Field(description="Product name")
    price_usd: float = Field(description="Price in USD", ge=0)
    in_stock: bool

class Catalog(BaseModel):
    products: List[Product]

llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0,
    base_url=os.environ.get("OPENAI_API_BASE"),
)
catalog_llm = llm.with_structured_output(Catalog)

text = "Sell: Widget $10 yes, Gadget $20 no"
try:
    cat = catalog_llm.invoke(text)
    for p in cat.products:
        print(p.model_dump())
except ValidationError as e:
    print("Schema mismatch:", e)

This langchain structured output pydantic tutorial deliberately avoids agent frameworks. Structured output is a primitive; once it is solid, you can build retrieval or tool use on top without fighting the parser.

When to skip structured output

If your output is free-form text or a single classification label, a raw completion with a small post-filter is faster. Structured output shines when downstream code needs typed fields, especially across many calls in a batch job. Measure parse failure rate; if it exceeds a few percent, tighten the schema or move up a model tier.

Production error budget

Set a threshold: if 5% of calls fail validation, alert. Use gateway metering to attribute cost per model. Treat the Pydantic model as an API between the LLM and your system—version it, test it with recorded responses, and keep it minimal. The patterns here scale from a weekend script to a production gateway without changing the core binding.