Testing multi-step tool calling Claude Opus 4.5 demands more than a single assertion on the final text. You need a deterministic harness that replays tool results and verifies the exact sequence of tool invocations the model makes across turns.
Prerequisites
- Python 3.11 or newer
anthropicPython SDK (pip install anthropic)pytestfor the test runner- An API key for Claude Opus 4.5 in
ANTHROPIC_API_KEY(or route through an OpenAI-compatible gateway) - No live external services—we mock every tool
If you route through a single OpenAI-compatible endpoint like n4n.ai, the loop below changes only in client construction; the trajectory assertions stay identical.
The scenario
We will test a tiny travel assistant. Given “What’s the weather in New York?”, it must:
- Call
geocodeto turn the city into coordinates. - Call
get_weatherwith those coordinates. - Return a human-readable temperature.
That is a two-step dependency chain. If the model calls get_weather before geocode, the test fails. This is the core of testing multi-step tool calling Claude Opus 4.5: enforcing order on dependent calls.
Define tools and mocks
Anthropic expects tools as a list of JSON-Schema-like definitions. We pair each with a fixed mock return value.
TOOLS = [
{
"name": "geocode",
"description": "Return latitude and longitude for a city.",
"input_schema": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
{
"name": "get_weather",
"description": "Return temperature in Celsius for coordinates.",
"input_schema": {
"type": "object",
"properties": {"lat": {"type": "number"}, "lon": {"type": "number"}},
"required": ["lat", "lon"],
},
},
]
MOCK_GEOCODE = {"New York": {"lat": 40.71, "lon": -74.01}}
MOCK_WEATHER = {(40.71, -74.01): {"temp_c": 12}}Keep the mock dictionary at module scope. Tests should never reach the network.
Conversation driver
The driver sends the user message, then loops. On each assistant turn, it inspects stop_reason. If the model emitted tool_use blocks, we execute our mocks and feed results back as a user message containing tool_result blocks. Otherwise we return the final text.
from anthropic import Anthropic
client = Anthropic()
def run_agent(user_msg: str, max_steps: int = 5):
messages = [{"role": "user", "content": user_msg}]
for _ in range(max_steps):
resp = client.messages.create(
model="claude-opus-4-5",
max_tokens=1024,
temperature=0,
tools=TOOLS,
messages=messages,
)
if resp.stop_reason != "tool_use":
return resp.content[0].text, messages
messages.append({"role": "assistant", "content": resp.content})
tool_results = []
for block in resp.content:
if block.type != "tool_use":
continue
if block.name == "geocode":
city = block.input["city"]
data = MOCK_GEOCODE.get(city, {"lat": 0, "lon": 0})
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": str(data),
})
elif block.name == "get_weather":
lat = block.input["lat"]
lon = block.input["lon"]
data = MOCK_WEATHER.get((lat, lon), {"temp_c": 0})
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": str(data),
})
messages.append({"role": "user", "content": tool_results})
raise RuntimeError("Exceeded max_steps")Key detail: the assistant message must contain the exact resp.content list (including text and tool_use blocks). Stripping the text blocks breaks the model’s context.
Write the trajectory test
We do not assert on the final string alone. We walk the captured messages and collect tool names in call order.
def test_multi_step_trajectory():
final, trajectory = run_agent("What's the weather in New York?")
calls = []
for msg in trajectory:
if msg["role"] != "assistant":
continue
for block in msg["content"]:
if getattr(block, "type", None) == "tool_use":
calls.append(block.name)
assert calls == ["geocode", "get_weather"], f"Got {calls}"
assert "12" in finalRun it:
pytest -q test_agent.pyExpected pass output:
.
1 passed in 2.34sIf Claude Opus 4.5 reverses the order (rare at temperature=0 but possible on schema drift), the assertion fires:
AssertionError: Got ['get_weather', 'geocode']Checkpoint: inspect the raw trajectory
During development, print the sequence to confirm the loop attaches tool results correctly.
if __name__ == "__main__":
text, traj = run_agent("What's the weather in New York?")
for m in traj:
print(m["role"])
for b in m["content"]:
t = getattr(b, "type", None)
if t == "tool_use":
print(" ->", b.name, b.input)
elif t == "tool_result":
print(" <-", b.content)
elif t == "text":
print(" text:", b.text[:60])Sample output:
user
assistant
-> geocode {'city': 'New York'}
user
<- {'lat': 40.71, 'lon': -74.01}
assistant
-> get_weather {'lat': 40.71, 'lon': -74.01}
user
<- {'temp_c': 12}
assistant
text: The temperature in New York is 12°C.Handling parallel tool calls
Claude Opus 4.5 can emit multiple tool_use blocks in one assistant message when calls are independent. Our loop already iterates resp.content, so both execute in the same turn. For independent calls, assert on a set, not a list:
assert set(calls) == {"geocode", "get_weather"}For the dependent chain in this tutorial, keep the ordered list assertion. Mixing the two patterns by accident is a common bug in agent test suites.
Testing error paths
Production tools fail. Simulate a missing city by returning None from the mock and mark the tool result as an error:
MOCK_GEOCODE = {"Atlantis": None}
# inside the loop:
if data is None:
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": "city not found",
"is_error": True,
})Then assert the model either asks for clarification or returns a graceful message without calling get_weather. This branch is where testing multi-step tool calling Claude Opus 4.5 earns its keep—silent error propagation is the default failure mode.
Dealing with non-determinism
Even at temperature=0, provider updates can shift argument phrasing. Hard-assert on argument values only when your mock keys require it. For geocode, assert city == "New York"; for get_weather, assert the lat/lon match the mocked coordinates within a tolerance, because the model may pass 40.710 instead of 40.71.
If you need hermetic unit tests without network calls, swap client.messages.create for a recorded fixture that yields the same block shapes. The trajectory assertions then run in milliseconds and catch regressions in your loop logic, not in the model.
Why trajectory assertions matter
Testing multi-step tool calling Claude Opus 4.5 without order checks hides real bugs. An agent that fetches weather before coordinates will fail in production when the weather API rejects missing lat/lon. Your eval should fail locally, not at 3 a.m.
Swapping the endpoint
The driver above uses the Anthropic SDK. If you prefer an OpenAI-compatible chat loop, point the client at a gateway that fronts Claude Opus 4.5 and iterate on message.tool_calls instead of tool_use blocks. The test logic—collect names in order, assert dependency chain—does not change.
Final checklist
- Mock every external tool; never call live APIs in the test suite.
- Capture the full message list, not just the last turn.
- Assert tool call order and critical arguments.
- Keep
temperature=0for reproducible runs. - Run a fast fixture-based test in CI and a weekly live integration test.
That is the core of testing multi-step tool calling Claude Opus 4.5 with a deterministic harness. Expand the mock dictionary and add more steps as your agent grows; the loop and assertions scale linearly.