Streaming tool calls: partial arguments across providers

Engineers building agentic UIs quickly discover that the shape of streaming tool calls partial arguments providers emit is anything but standardized. OpenAI, Anthropic, and Google each serialize incremental function invocations differently, and that divergence dictates how you write client-side accumulators, when you can show a loading skeleton, and whether you can cancel a bad call early.

Why partial arguments matter

A tool call is just a structured request from the model to your runtime: “call search with query='weather in'”. If you wait for the full completion before rendering anything, you add hundreds of milliseconds of dead time per hop. Streaming the arguments as they generate lets you pre-fill form fields, start a speculative backend fetch, or show a “calling search…” chip that updates live.

The catch: partial JSON is not valid JSON. You receive a fragment like {"loc and must decide what to do with it. The providers that support this feature disagree on the envelope, the granularity, and the failure modes.

Provider behavior in practice

OpenAI: delta tool_calls arrays

OpenAI’s Chat Completions streaming protocol sends tool_calls inside choices[0].delta. The first chunk carries id and function.name; subsequent chunks carry incremental arguments strings. The arguments are concatenated raw JSON text, not objects.

{
  "choices": [
    {
      "delta": {
        "tool_calls": [
          {
            "index": 0,
            "id": "call_01",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"loc"
            }
          }
        ]
      }
    }
  ]
}

{
  "choices": [
    {
      "delta": {
        "tool_calls": [
          {
            "index": 0,
            "function": { "arguments": "ation\":\"SF\"}" }
          }
        ]
      }
    }
  ]
}

The index field lets you handle parallel tool calls. You must buffer per index. There is no signal that the JSON is complete until the stream ends or you see a finish_reason: "tool_calls".

Anthropic: input_json_delta

Anthropic’s Messages API streams content_block_start for a tool_use block, then a series of content_block_delta events with delta.type: "input_json_delta" and a partial_json string. The partial_json is also raw JSON text, but it is scoped to the tool input only—no wrapper.

{
  "type": "content_block_delta",
  "index": 1,
  "delta": {
    "type": "input_json_delta",
    "partial_json": "{\"loc"
  }
}

{
  "type": "content_block_delta",
  "index": 1,
  "delta": {
    "type": "input_json_delta",
    "partial_json": "ation\":\"SF\"}"
  }
}

Anthropic includes a tool_use block with id and name up front in content_block_start, which makes UI labeling easier than OpenAI’s lazy name delivery.

Google: all or nothing

Gemini’s GenerateContent streaming does not emit partial arguments. A functionCall part arrives as a single complete object inside a chunk. You get zero incremental signal.

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "functionCall": {
              "name": "get_weather",
              "args": { "location": "SF" }
            }
          }
        ]
      }
    }
  ]
}

If you need progressive rendering on Gemini, you must fake it or fall back to a model that streams partials. This is a hard constraint when you target multiple backends.

Parallel calls and index management

Real agents rarely make one tool call. OpenAI can interleave two tool_calls in the same stream; chunk A adds arguments for index:0, chunk B for index:1. Your reducer must key buffers by index, not by name. Anthropic assigns a content_block index per tool_use; map that to the same slot. If you key by name, parallel calls to the same function will collide.

A correct accumulator merges deltas by index and only emits a “complete” event when the stream terminates. Until then, treat each buffer as mutable text.

Failure modes and timeouts

Streams die. A provider may 429 mid-tool-call, or TCP may drop after {"loc. Your client must enforce a timeout: if no delta arrives in N seconds, discard the buffer and surface an error. OpenAI may send a final chunk with finish_reason: "tool_calls"; if that never comes, you hold orphaned JSON.

import asyncio, json

class ToolCallBuffer:
    def __init__(self):
        self.buf = {}
    def ingest(self, chunk):
        for tc in chunk.get("choices", [{}])[0].get("delta", {}).get("tool_calls", []):
            idx = tc["index"]
            self.buf.setdefault(idx, {"id": None, "name": None, "args": ""})
            if tc.get("id"): self.buf[idx]["id"] = tc["id"]
            if tc.get("function", {}).get("name"):
                self.buf[idx]["name"] = tc["function"]["name"]
            self.buf[idx]["args"] += tc.get("function", {}).get("arguments", "")
    def finalize(self):
        out = {}
        for idx, v in self.buf.items():
            try:
                out[idx] = json.loads(v["args"])
            except json.JSONDecodeError:
                out[idx] = None  # incomplete
        return out

async def with_timeout(coro, secs=10):
    return await asyncio.wait_for(coro, timeout=secs)

The mismatch between streaming tool calls partial arguments providers supply means a naive passthrough to your frontend will break the moment you switch models.

Building a tolerant accumulator

The only safe client design is a provider-normalizing reducer that buffers strings and attempts best-effort parsing. Below is a TypeScript sketch for OpenAI-style deltas; Anthropic maps cleanly with a small adapter.

type Acc = { id?: string; name?: string; buf: string };

function onOpenAIDelta(acc: Acc[], chunk: any): Acc[] {
  const calls = chunk.choices?.[0]?.delta?.tool_calls;
  if (!calls) return acc;
  for (const c of calls) {
    acc[c.index] ??= { buf: "" };
    const a = acc[c.index];
    if (c.id) a.id = c.id;
    if (c.function?.name) a.name = c.function.name;
    if (c.function?.arguments) a.buf += c.function.arguments;
  }
  return acc;
}

function parsePartial(buf: string): unknown | null {
  try {
    return JSON.parse(buf);
  } catch {
    return null;
  }
}

For Anthropic, translate input_json_delta into the same buf keyed by index. For Gemini, push the full args object as a completed buffer in one shot.

The key rule: never validate against your JSON schema until finish_reason (or stream end). Partial fragments will fail required checks and waste cycles.

Schema validation without blocking

Use Zod or Pydantic only on the finalized object. During streaming, optionally run a lenient check: count braces, confirm the string starts with {. If you show a “location” field prematurely, pull it via a regex or a streaming JSON parser, but do not assert types. Premature strict validation causes flicker when the model corrects a value mid-stream.

The normalization tradeoff

If you route through a gateway such as n4n.ai, which provides one OpenAI-compatible endpoint covering 240+ models and automatically falls back when a provider is degraded, the chunk shape follows the OpenAI format even for Anthropic-backed models. That hides the input_json_delta nuance but also strips Anthropic’s early name signaling if the gateway doesn’t emulate it. You gain uniformity; you lose access to native partial-event semantics unless you drop to raw provider mode.

This matters because streaming tool calls partial arguments providers produce are the single biggest source of UI flicker in multi-model apps. A normalized stream means one code path, but you must still handle the Gemini case where the “partial” is actually complete—your accumulator should treat a single-shot args object as instantly valid.

Honest tradeoffs

Pros of consuming partials:

Perceived latency drops. Users see the tool name immediately.
You can pre-validate argument types (e.g., “location looks like a string”) and abort a runaway call.
Parallel tool calls are observable as they stream, not after the fact.

Cons:

You must write a JSON fragment buffer. Off-the-shelf JSON.parse throws; you need a tolerant parser or a “wait for close brace” heuristic.
Provider outages mid-stream leave you with orphaned buffers. OpenAI may cut a stream after {"loc; you must timeout and discard.
Gemini forces a synchronous wait, breaking any UI assumption of progress.

Normalizing streaming tool calls partial arguments providers produce is the core job of any inference gateway, yet the client still owns the buffer. Don’t trust a partial object to be well-typed.

Recommendations

Pick a single internal representation: an array of { id, name, buf, done }. Write adapters for each provider’s wire format. Attempt JSON.parse on every delta; if it fails, render a raw “constructing…” state. On stream end, parse strictly and validate with your schema (Zod, Pydantic, etc.).

If you support more than two providers, put a normalization layer in front. A gateway that honors client routing directives and forwards provider cache-control hints can shield you from envelope changes, but verify it preserves the index field and argument concatenation order—some gateways reorder chunks under load.

Decisive takeaway

Stream partial tool arguments only if you are willing to maintain a per-index string buffer and a lenient parser; otherwise, treat tool calls as atomic and render them post-hoc. For multi-provider apps, standardize on the OpenAI delta shape internally and adapt Anthropic and Gemini at the edge. The responsiveness win is real, but the engineering cost is a small, non-negotiable accumulator—skipping it guarantees broken UIs the first time you route to a different backend.