Most LLM integrations fall apart when the model returns loosely shaped JSON. With python instructor pydantic function calling you get typed, validated arguments straight from the model, letting you dispatch to real Python functions without hand-rolled parsers.
Prerequisites
You need Python 3.10 or newer. The code below uses instructor (the patching library that turns OpenAI completions into structured outputs), pydantic v2 for schema definition, and the official openai SDK.
pip install "instructor" "pydantic>=2.0" "openai>=1.0"You also need an API key for an OpenAI-compatible endpoint. Set it in the environment:
export OPENAI_API_KEY="sk-..."If you later point the client at a gateway, the same variable name works.
Setting up the patched client
instructor.patch wraps the OpenAI client so that chat.completions.create accepts a response_model argument. Under the hood it uses the provider’s function-calling or JSON-mode capability and validates the result against your Pydantic class.
from openai import OpenAI
import instructor
client = instructor.patch(OpenAI())That is the entire setup. If you want to route through a single OpenAI-compatible endpoint that addresses 240+ models with automatic fallback when a provider is degraded, pass base_url="https://api.n4n.ai/v1" to OpenAI() and the rest of the code is unchanged.
Defining function schemas with Pydantic
Each callable tool is a Pydantic model. Include a literal function field so the model tells you which tool it picked. This avoids a separate discriminator step.
from pydantic import BaseModel, Field
from typing import Literal, Union
class GetWeatherArgs(BaseModel):
function: Literal["get_weather"]
location: str = Field(description="City name, e.g. 'Tokyo'")
unit: Literal["celsius", "fahrenheit"] = "celsius"
class SendEmailArgs(BaseModel):
function: Literal["send_email"]
recipient: str
subject: str
body: str
ToolCall = Union[GetWeatherArgs, SendEmailArgs]The description on location is forwarded to the model as the parameter schema, which improves accuracy. Pydantic constraints (Literal, min_length, etc.) become part of the generated schema.
Extracting a single function call
Ask the model to pick a tool and fill arguments. response_model=ToolCall makes instructor validate and coerce the response.
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Email Alice at alice@example.com about the Q3 report, subject 'Update'"}
],
response_model=ToolCall,
)
print(resp)Expected output:
function='send_email' recipient='alice@example.com' subject='Update' body='...'The exact body text varies, but the fields are guaranteed present and typed. If the model tries to return an unknown function name, Pydantic raises before your code touches it.
Dispatching to real Python functions
Once you have a validated object, dispatch is a plain if/else or a dict lookup. No json.loads, no key checking.
def get_weather(location: str, unit: str) -> str:
# stubbed external call
return f"22 {unit} and clear in {location}"
def send_email(recipient: str, subject: str, body: str) -> str:
# stubbed SMTP
return f"Sent '{subject}' to {recipient}"
def dispatch(call: ToolCall) -> str:
if isinstance(call, GetWeatherArgs):
return get_weather(call.location, call.unit)
if isinstance(call, SendEmailArgs):
return send_email(call.recipient, call.subject, call.body)
raise ValueError(f"Unhandled call: {call}")
result = dispatch(resp)
print(result)Running the email example prints something like:
Sent 'Update' to alice@example.comThis is the core loop of python instructor pydantic function calling: schema → extraction → dispatch.
Handling multiple function calls
Real agents often need to call several tools from one prompt. Instructor supports list[ToolCall] as a response model, which maps to parallel tool calls.
multi = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Get weather for Berlin in celsius and email bob@test.com saying 'done'"}
],
response_model=list[ToolCall],
)
for call in multi:
print(dispatch(call))Output order is not guaranteed, but each item is a fully validated instance:
22 celsius and clear in Berlin
Sent 'done' to bob@test.comIf the model emits an empty list, you get []—not None—so iteration is safe.
Validation and error handling
Pydantic validates before you dispatch. To catch model mistakes cleanly, wrap the call:
from pydantic import ValidationError
try:
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Do something undefined"}],
response_model=ToolCall,
)
except ValidationError as e:
print("Model returned an invalid tool call:", e)Instructor retries once by default on validation failure, passing the error back to the model as feedback. For production, set max_retries=3 on instructor.patch and log the ValidationError cause.
Add field constraints to reduce retries:
class SendEmailArgs(BaseModel):
function: Literal["send_email"]
recipient: str = Field(pattern=r"^[^@]+@[^@]+\.[^@]+$")
subject: str = Field(min_length=1, max_length=200)
body: strNow an malformed email address is rejected at the schema layer.
Streaming partial arguments
For long forms, stream the extraction:
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Weather in Madrid fahrenheit"}],
response_model=GetWeatherArgs,
stream=True,
)
for partial in stream:
print(partial)Each partial is a partially filled GetWeatherArgs. The final iteration has all fields set. This is useful when you want to show progressive UI or short-circuit on bad location.
Why this beats raw function calling
OpenAI’s native tools parameter returns a dict you must parse and cast yourself. Using python instructor pydantic function calling moves that parsing into a single validated boundary. Your business logic receives a Python object, not a dict with optional keys.
You also keep full control of the prompt. Instructor does not hide the messages; you can inspect client.chat.completions.create(..., response_model=None) to see the raw function schema it generated.
Closing notes on routing
If you serve multiple models, keep the ToolCall schemas in a shared module and swap model= per request. Because instructor emits standard OpenAI function schemas, any compliant gateway forwards them correctly. When a provider is rate-limited, a gateway that honors client routing directives and forwards provider cache-control hints will degrade without code changes on your side.
That is the complete pattern: define Pydantic models, patch the client, extract, validate, dispatch.