GPT-5's tool-calling improvements for multi-step agents

Multi-step agents break down when the model emits malformed arguments or serializes calls that should run concurrently. gpt-5 tool calling tightens schema adherence and adds native parallel invocation, but the surrounding loop still needs engineering discipline to avoid cascading failures.

1. Define tools with strict, minimal schemas

The model performs best when each function has a single purpose and parameters are constrained with enums, patterns, and required fields. Loose object types with free-form properties invite hallucinations and silent validation errors downstream. gpt-5 tool calling uses the description text and parameter constraints together to decide when to invoke, so vague schemas degrade routing accuracy.

tools = [
  {
    "type": "function",
    "function": {
      "name": "get_order_status",
      "description": "Fetch shipping status for a single order id",
      "parameters": {
        "type": "object",
        "properties": {
          "order_id": {"type": "string", "pattern": "^ORD-[0-9]+$"}
        },
        "required": ["order_id"]
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "refund_order",
      "description": "Issue refund for a verified order",
      "parameters": {
        "type": "object",
        "properties": {
          "order_id": {"type": "string", "pattern": "^ORD-[0-9]+$"},
          "reason": {"type": "string", "enum": ["duplicate", "fraud", "customer_request"]}
        },
        "required": ["order_id", "reason"]
      }
    }
  }
]

Avoid polymorphic parameters

Do not accept additionalProperties: true or type: ["string", "number"]. The executor will ignore unexpected fields, and the model will quietly fill them with plausible garbage. If you need variant inputs, define separate tools.

Version your tool names

Append _v1 to tool names when you change semantics. gpt-5 caches tool selection patterns across a session; renaming forces a clean re-decision instead of mixing old and new behaviors.

2. Enable parallel tool calls and dispatch concurrently

A single assistant message from gpt-5 can contain multiple tool_calls. If those calls are independent reads, running them sequentially wastes latency and compounds rate limits.

from openai import OpenAI
import json, asyncio

client = OpenAI()  # or point base_url at your gateway

def call_model(messages, tools):
    resp = client.chat.completions.create(
        model="gpt-5",
        messages=messages,
        tools=tools
    )
    return resp.choices[0].message

async def execute(tool_calls, registry):
    async def run_one(tc):
        fn = registry[tc.function.name]
        args = json.loads(tc.function.arguments)
        return await fn(**args)
    return await asyncio.gather(
        *[run_one(tc) for tc in tool_calls],
        return_exceptions=True
    )

Detecting parallel intent

Inspect len(message.tool_calls). If it is greater than one and all called functions are marked read-only in your registry, dispatch concurrently. Otherwise, iterate sequentially to preserve ordering for state mutations.

Idempotency keys

For any parallel read that influences a later write, pass an explicit idempotency_key parameter in the schema. That lets you safely retry individual failures without duplicating side effects.

The tradeoff: parallel dispatch is only safe for idempotent operations. gpt-5 tool calling will respect a parallel_tool_calls: false directive on the request if your client supports it; use it for refund or create paths.

3. Keep step state in an external store

Stuffing every tool result back into the LLM context burns tokens and confuses later steps. Maintain a session-scoped store and inject only condensed facts.

state = {}
state["step_3"] = {"tool": "get_order_status", "result": "shipped"}

# later, inject a summary instead of raw payload
messages.append({
    "role": "system",
    "content": f"Known facts so far: order ORD-123 is shipped."
})

Cache-control hints

When you route through an OpenAI-compatible gateway such as n4n.ai, the same tool schemas work unchanged and provider cache-control hints are forwarded, so you can mark static schema definitions as cached to cut token cost on long runs. Set cache_control: {"type": "ephemeral"} on the system message that carries your tool list if your provider honors it.

Summarization strategy

After every three steps, replace the detailed tool messages with a single system line. Keep the raw data in your store for debugging, but keep the model’s working set under a few thousand tokens.

4. Feed tool errors back as structured content

gpt-5 recovers better from explicit, machine-readable error payloads than from prose exceptions. Return a dict, not a thrown string.

async def safe_refund(order_id, reason):
    try:
        return await billing.refund(order_id, reason)
    except TimeoutError:
        return {"error": "timeout", "retryable": True}

Append the result as a tool message with the matching tool_call_id:

messages.append({
    "role": "tool",
    "tool_call_id": tc.id,
    "content": json.dumps(result)
})

Retry with backoff

If the error payload marks retryable: True, wait with exponential backoff and re-invoke the same tool with identical arguments. The model will usually continue the plan without rewriting the step.

Distinguish retryable vs fatal

A 404 on an order id is fatal; return {"error": "not_found", "retryable": False}. The agent should then pick a different tool or ask the user. Hiding the distinction forces the model to guess.

5. Stream and reconcile long agent runs

For agents that run more than a handful of steps, stream the completion to detect early stops and to accumulate tool calls safely.

stream = client.chat.completions.create(
    model="gpt-5",
    messages=messages,
    tools=tools,
    stream=True
)

tool_calls = []
for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.tool_calls:
        for tc in delta.tool_calls:
            # accumulate index, id, function name, arguments
            ...

Buffering per index

Argument strings arrive in pieces. Buffer them per index and parse only after the stream ends. gpt-5 tool calling emits well-formed JSON at completion, but partial chunks will throw on json.loads.

Handling partial tool failures

If one of N parallel streams fails mid-flight, you already have return_exceptions=True from the gather. Map exceptions back to their tool_call_id and return an error dict for each failed slot so the model sees consistent structure.

6. Common pitfalls in production multi-step agents

Context rot. Leaving full tool payloads in the message list past step ten causes the model to lose the original goal. Trim or summarize aggressively.

Unbounded recursion. A tool that calls another tool via the model without a max-step guard will loop until you hit a limit. Enforce a hard step count in your orchestrator.

Side-effect storms. Parallel dispatch is tempting but dangerous for writes. Gate mutating tools behind a sequential queue.

Over-broad schemas. Accepting free-form objects lets the model invent fields that your executor ignores, producing silent no-ops.

Ignoring provider degradation. When a model provider is rate-limited, your agent should fall back to a comparable model rather than blocking. If you use a gateway with automatic fallback, set routing directives per step criticality so non-critical reads degrade gracefully.

7. Testing your agent loop

Golden step traces

Record a known multi-step task (e.g., “refund the latest duplicate order”) and assert the sequence of tool names called. gpt-5 tool calling is deterministic enough at temperature 0 to make this a useful regression test.

Fuzz the arguments

Write a unit test that mutates the arguments JSON with missing required fields and confirms your executor returns a structured error rather than raising.

8. A minimal ordered checklist

1. Write tight JSON schemas with required fields and enums; version tool names.
2. Enable parallel calls only for read-only tools; use parallel_tool_calls: false for writes.
3. Store raw results externally; inject summaries into context and cache the schema.
4. Return structured errors with retry hints; back off on retryable failures.
5. Stream long runs and buffer tool-call arguments fully before parsing.
6. Cap steps, trim context, and isolate mutations behind a queue.
7. Add golden trace tests and argument fuzzing before shipping.

Follow that path and gpt-5 tool calling will hold up across dozens of steps without custom validation layers. The model handles the orchestration; your job is to keep the inputs clean and the side effects controlled.