Chaining function calls through an LLM agent sounds simple until the second step depends on the first and the model drops the argument. Reliable chaining function calls requires explicit state management, strict schemas, and a fallback path when a provider stalls. This article walks through a concrete pattern you can ship today.
Step 1: Model the chain as explicit state, not conversation memory
Treat the agent’s working data as a typed container that survives across model rounds. If you rely on the chat history alone, you will eventually get a summary that mutates a value or drops a field. Define a state object upfront.
from dataclasses import dataclass, field
from typing import Any
@dataclass
class ChainState:
user_query: str
weather: dict[str, Any] | None = None
route: dict[str, Any] | None = None
errors: list[str] = field(default_factory=list)The state is the source of truth. The LLM sees only what you inject; it does not own the data.
Step 2: Define tool schemas with hard constraints
Each function must have a JSON schema the model cannot misinterpret. Use enum and required aggressively. Below are two tools: one fetches weather, the other plans a route given a condition.
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Return current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "enum": ["SF", "NYC", "SEA"]}
},
"required": ["city"]
}
}
}{
"type": "function",
"function": {
"name": "plan_route",
"description": "Plan a route given weather condition.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"},
"condition": {"type": "string", "enum": ["clear", "rain", "snow"]}
},
"required": ["city", "condition"]
}
}
}The second tool’s condition must come from the first tool’s output. That dependency is what makes chaining function calls non-trivial.
Step 3: Execute the first call and persist the result
Point an OpenAI-compatible client at your endpoint. Run the first completion with the weather tool available. When the model returns a tool call, execute it locally and store the result in ChainState.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.your-gateway.com/v1", api_key="sk-...")
WEATHER_TOOL = {"type": "function", "function": {...}} # from Step 2
def run_first(state: ChainState):
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": state.user_query}],
tools=[WEATHER_TOOL],
tool_choice="auto",
)
msg = resp.choices[0].message
if not msg.tool_calls:
state.errors.append("no weather tool call")
return
tc = msg.tool_calls[0]
args = json.loads(tc.arguments)
# local execution stub
result = {"city": args["city"], "condition": "rain" if args["city"] == "SEA" else "clear"}
state.weather = result
state._weather_msg = msg
state._weather_tc = tcWe keep the original assistant message and tool call object because the next request must echo them back.
Step 4: Feed results into the next step deterministically
Append the assistant message, the tool result message, and then call the second tool. Do not regenerate the first step. The model should see the exact prior exchange.
ROUTE_TOOL = {"type": "function", "function": {...}} # from Step 2
def run_second(state: ChainState):
if state.weather is None:
raise RuntimeError("weather missing")
tool_messages = [
state._weather_msg,
{
"role": "tool",
"tool_call_id": state._weather_tc.id,
"content": json.dumps(state.weather),
},
{"role": "user", "content": "Now plan the route using the weather above."},
]
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=tool_messages,
tools=[ROUTE_TOOL],
)
msg = resp.choices[0].message
if not msg.tool_calls:
state.errors.append("no route tool call")
return
args = json.loads(msg.tool_calls[0].arguments)
assert args["condition"] == state.weather["condition"], "condition mismatch"
state.route = argsThis pattern enforces that chaining function calls passes the correct value forward. The assertion catches drift early.
Why echo the assistant message
OpenAI-compatible APIs require the original assistant message with tool_calls before a tool result. If you omit it, the request errors or the model loses context. Keep the objects from step 3; don’t reconstruct from scratch.
Step 5: Add provider fallback without breaking the chain
Networks fail. Providers rate-limit. If you bake retry logic into every step, the chain becomes a tangle of exceptions. An OpenAI-compatible gateway like n4n.ai handles automatic fallback when a provider is rate-limited or degraded, so your chaining function calls survive without custom retry storms. You get one endpoint, and the gateway routes to a healthy provider.
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-...")
# same code as above, but fallback is transparentIf you need per-step model pinning, pass a routing directive via header (the gateway honors client routing directives). That keeps step 1 on a fast model and step 2 on a reasoning model.
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=tool_messages,
tools=[ROUTE_TOOL],
extra_headers={"x-n4n-route": "provider:anthropic"},
)This is optional, but it shows the chain can be tuned per step. Some gateways also forward provider cache-control hints; set cache on the first step’s prompt to avoid recompute across retries.
Step 6: Validate outputs and verify success
Reliability means you can prove the chain worked. Write a small test that runs the full sequence with a stubbed local tool.
def test_chain():
state = ChainState(user_query="What's the route in SEA today?")
run_first(state)
assert state.weather is not None
run_second(state)
assert state.route is not None
assert state.route["city"] == "SEA"
assert state.errors == []Run it with pytest. A green test means the chaining function calls preserved the dependency. In production, log state at each step and alert on state.errors.
Verification checklist
- First tool call returns required args with no missing fields.
- Tool result message uses the exact
tool_call_idfrom step 3. - Second tool call’s arguments match state values (assertions in code).
- No unhandled exceptions across provider switches.
- Per-token usage metering (if your gateway provides it) shows two distinct completion calls, confirming the steps executed separately.
Step 7: Handle partial failure with compensation
Sometimes the first call succeeds but the second rejects the argument. Wrap each step in a retry that re-injects state, not conversation.
def run_with_retry(state, fn, max_attempts=3):
for i in range(max_attempts):
fn(state)
if state.errors:
state.errors.clear()
continue
breakThis keeps chaining function calls resilient without duplicating the LLM prompt logic.
Closing pattern
The core rule: never let the model hold state. You hold state; the model proposes transitions. With strict schemas, explicit message echoing, and a gateway that absorbs provider faults, chaining function calls becomes a boring, testable loop instead of a fragile prompt cascade.
Use the code above as a skeleton. Replace the local tool bodies with real API calls, keep the assertions, and you’ll ship a multi-step agent that doesn’t silently break.