When you build LLM features that call your own functions, you need confidence that the plumbing works without burning tokens on every run. A solid approach to testing function calling python pytest separates the model’s output shape from your execution logic, so you can validate argument handling and dispatch deterministically.
Step 1: Define a minimal function registry and schema
Start by declaring the Python callables you want the model to invoke, plus their JSON schemas. Keeping the schema next to the function prevents drift between what the model sees and what your code accepts.
import json
def get_weather(lat: float, lon: float) -> dict:
# stub implementation
return {"temp_c": 21.0, "lat": lat, "lon": lon}
TOOL_SCHEMAS = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current temperature for a coordinate",
"parameters": {
"type": "object",
"properties": {
"lat": {"type": "number"},
"lon": {"type": "number"}
},
"required": ["lat", "lon"]
}
}
}
]
REGISTRY = {"get_weather": get_weather}The registry is a plain dict mapping names to callables. The schema is what you send to the model’s tools parameter. In a larger system you might generate the schema from type hints or pydantic models, but the explicit form above is easiest to test.
Step 2: Build a dispatcher that executes model-requested calls
Write a function that takes the tool_calls list from a chat completion and invokes the matching registry entry. This is pure logic you can test without network access.
def dispatch_tool_calls(tool_calls: list, registry: dict, suppress: bool = False) -> list:
results = []
for call in tool_calls:
fn_name = call.function.name
args = json.loads(call.function.arguments)
if fn_name not in registry:
if suppress:
results.append({"error": f"Unknown tool: {fn_name}"})
continue
raise ValueError(f"Unknown tool: {fn_name}")
try:
results.append(registry[fn_name](**args))
except Exception as e:
if suppress:
results.append({"error": str(e)})
else:
raise
return resultsThe suppress flag lets you choose whether a failing tool call aborts the batch or returns an error object. That decision is a product requirement; make it explicit and test both paths.
Step 3: Unit test argument validation and schema conformance
Use pytest to confirm the dispatcher rejects missing args and unknown tools. You don’t need the real OpenAI SDK objects—a tiny fake is enough.
import pytest
class FakeCall:
def __init__(self, name, args):
self.function = type("F", (), {"name": name, "arguments": args})()
def test_dispatch_happy_path():
calls = [FakeCall("get_weather", '{"lat": 40.7, "lon": -74.0}')]
out = dispatch_tool_calls(calls, REGISTRY)
assert out[0]["temp_c"] == 21.0
def test_dispatch_unknown_tool_raises():
calls = [FakeCall("delete_db", '{}')]
with pytest.raises(ValueError):
dispatch_tool_calls(calls, REGISTRY)
def test_dispatch_unknown_tool_suppressed():
calls = [FakeCall("delete_db", '{}')]
out = dispatch_tool_calls(calls, REGISTRY, suppress=True)
assert "error" in out[0]If you use pydantic for argument binding, add a test that malformed JSON or wrong types raise before the function executes. That keeps your testing function calling python pytest suite focused on contract boundaries.
Step 4: Mock the LLM response to test the happy path
Patch the client’s chat.completions.create to return a canned response containing tool_calls. This isolates your agent loop from network variability and model nondeterminism.
from unittest.mock import patch
def test_agent_requests_weather():
fake_resp = type("R", (), {
"choices": [type("C", (), {
"message": type("M", (), {
"tool_calls": [FakeCall("get_weather", '{"lat": 1.0, "lon": 2.0}')]
})()
})()]
})()
with patch("openai.OpenAI") as mock_client:
mock_client.return_value.chat.completions.create.return_value = fake_resp
from my_agent import run_agent_step
result = run_agent_step("What's the weather at 1,2?")
assert result["temp_c"] == 21.0run_agent_step should call the mocked client, inspect message.tool_calls, and pass them to dispatch_tool_calls. Because the client is mocked, this test runs in milliseconds and never spends a token.
Step 5: Test error handling and partial failures
Real functions throw. Decide whether your dispatcher should swallow or propagate, then test both modes with a flaky callable.
def flaky(**kwargs):
raise RuntimeError("upstream down")
def test_dispatch_propagates_by_default():
reg = {"flaky": flaky}
calls = [FakeCall("flaky", '{}')]
with pytest.raises(RuntimeError):
dispatch_tool_calls(calls, reg)
def test_dispatch_swallows_when_suppress():
reg = {"flaky": flaky}
calls = [FakeCall("flaky", '{}')]
out = dispatch_tool_calls(calls, reg, suppress=True)
assert out[0]["error"] == "upstream down"If your agent retries failed calls, parametrize the retry count and assert the callable is invoked the expected number of times using unittest.mock.MagicMock. This is where testing function calling python pytest pays off: you catch retry storms before they hit a paid API.
Step 6: Integration test against an OpenAI-compatible endpoint
Unit tests catch logic bugs; you still need one live call to confirm the wire format. Point your client at any compliant server. If you route through a gateway such as n4n.ai, which provides a single OpenAI-compatible endpoint covering 240+ models with automatic fallback when a provider is degraded, the same test client works without code changes. Its per-token metering and cache-control forwarding let you assert on usage fields in the response.
Mark the test so it only runs in CI with credentials:
import os
import pytest
@pytest.mark.integration
def test_live_function_call():
from openai import OpenAI
client = OpenAI(
api_key=os.environ["API_KEY"],
base_url=os.environ.get("BASE_URL") # set to gateway URL if used
)
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Call get_weather for lat 40.7 lon -74.0"}],
tools=TOOL_SCHEMAS,
tool_choice="auto"
)
tool_calls = resp.choices[0].message.tool_calls
assert tool_calls, "model did not request a tool"
assert tool_calls[0].function.name == "get_weather"
args = json.loads(tool_calls[0].function.arguments)
assert args["lat"] == 40.7Run it with pytest -m integration only when secrets are present. The assertion confirms the model produced a call matching your schema, not just that the request succeeded.
Step 7: Run the suite and verify success
Execute the full suite with coverage to confirm both mocked and live paths behave:
pytest --cov=my_agent -m "not integration"
pytest -m integration # only in CISuccess criteria for testing function calling python pytest:
- All mocked unit tests pass in under a second.
- The dispatcher tests cover happy path, unknown tool, and error suppression.
- The integration test (when run) returns a
tool_callsentry whose name and arguments matchTOOL_SCHEMAS. - Coverage of
dispatch_tool_callsand the agent loop exceeds 90%.
Add a pre-commit hook or GitHub Action that runs the non-integration suite on every push. Keep the live call in a nightly job to catch schema drift from provider updates. With this split, you get fast feedback on your logic and periodic confidence that the real model still speaks the same protocol.