Streaming function calls: API support compared

Building agents that call tools in real time demands a reliable streaming function calls API. The major model providers have each implemented tool use over streams differently, and those differences bite when you write parsing logic. This post puts the native APIs of OpenAI, Anthropic, and Google Gemini side by side so you can pick the right backend—or normalize them behind a gateway such as n4n.ai that exposes one OpenAI-compatible endpoint for 240+ models.

The contenders

We compare the three APIs that engineers actually hit when shipping agents today:

• OpenAI Chat Completions (stream=True, tools)
• Anthropic Messages (messages.stream, tools)
• Google Gemini (generate_content_stream, tools)

All three support tool/function calling, but only OpenAI and Anthropic stream the argument JSON token-by-token. Gemini emits the function call as a discrete object after (or interleaved with) text generation.

Capabilities

OpenAI

Parallel tool calls are first-class. The stream sends tool_calls deltas with an index so you can assemble multiple calls concurrently. Arguments arrive as a fragmented JSON string.

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Book a flight and a hotel"}],
    tools=[{"type": "function", "function": {"name": "book_flight", "parameters": {...}}}],
    stream=True,
)
for chunk in stream:
    dc = chunk.choices[0].delta
    if dc.tool_calls:
        for tc in dc.tool_calls:
            # tc.index, tc.function.name, tc.function.arguments (partial JSON)
            print(tc.index, tc.function.name, tc.function.arguments)

Anthropic

Tool use streams as content_block_delta events of type input_json_delta. You get partial_json strings per tool block. Multiple tools appear as separate content blocks.

with client.messages.stream(
    model="claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "Book a flight and a hotel"}],
    tools=[{"name": "book_flight", "input_schema": {...}}],
) as stream:
    for event in stream:
        if event.type == "content_block_delta" and event.delta.type == "input_json_delta":
            print(event.index, event.delta.partial_json)

Gemini

Function calls are delivered as function_call parts. The arguments are a complete proto struct—not incrementally streamed. You still get a stream of text chunks before the call.

for chunk in model.generate_content_stream(
    contents=["Book a flight and a hotel"],
    tools=[{"function_declarations": [{"name": "book_flight", "parameters": {...}}]}],
):
    for part in chunk.parts:
        if part.function_call:
            print(part.function_call.name, dict(part.function_call.args))

Price/cost model

None of the three charges a premium for streaming. You pay per token (or per character for some Gemini tiers, but token metering is now standard). Tool schemas and prompts count as input tokens. Generated tool-call JSON counts as output tokens.

• OpenAI: input + output token metering; tool definitions billed as input.
• Anthropic: input + output tokens; tool input schema included in input.
• Gemini: token-based on latest 1.5/2.0 pricing; function declarations counted in prompt tokens.

There is no separate “streaming function calls API” surcharge. If you route through a gateway, per-token usage metering is forwarded unchanged.

Latency/throughput

Concrete numbers depend on model size and region; none of the providers publishes streaming-specific tool-call benchmarks. Practically:

• OpenAI and Anthropic show low time-to-first-token (TTFT) on small models; tool-call deltas start after the model commits to a call.
• Gemini streams text fast but the function call object appears as a single event, so perceived tool latency is similar but you lose incremental argument rendering.

Streaming helps UX because you can show partial tool inputs, but it does not reduce end-to-end agent step time.

Ergonomics

OpenAI’s shape is the de facto standard; many frameworks assume it. Anthropic forces a separate system parameter and a different tool schema (input_schema instead of parameters). Gemini uses function_declarations wrapped in a tools list and returns proto objects.

Parsing differences:

• OpenAI: concatenate arguments per index.
• Anthropic: concatenate partial_json per index, then json.loads.
• Gemini: await the function_call part; no concatenation needed.

If you already use an OpenAI-compatible client, a unified streaming function calls API saves rewriting three parsers.

Ecosystem

• OpenAI: largest third-party ecosystem (LangChain, LlamaIndex, Vercel AI SDK). Most inference gateways mirror it.
• Anthropic: first-party Python/TS SDKs, good docs, growing community.
• Gemini: Vertex AI integration, AI Studio, SDKs for Python/JS/Go.

When you need to swap models without touching agent code, an OpenAI-compatible gateway that honors client routing directives lets you keep one parser.

Limits

Documented constraints shift frequently; treat these as directional:

• OpenAI: supports many tools per request (tens), parallel calls via tool_calls index.
• Anthropic: multiple tool blocks per response; payload size limits apply to large schemas.
• Gemini: caps on number of function declarations per tools call; nested schemas allowed but complex arrays increase latency.

All three enforce max output token limits that bound tool-call size.

Comparison table

Which to choose

Use OpenAI if you need maximal compatibility, parallel tool calls, and incremental argument streaming today. Most agent frameworks assume its shape.

Use Anthropic if your evaluation shows better reasoning with tools for your task and you can handle SSE event parsing. The input_json_delta model is clean for building UIs that show tool inputs forming.

Use Gemini if you are already on Google Cloud, need multimodal tool triggers, and can accept non-streamed function arguments. The single-event call is simpler to parse but less interactive.

Use a unified gateway when you want to avoid vendor lock or need to fall back automatically when a provider is rate-limited. A single OpenAI-compatible streaming function calls API that forwards provider cache-control hints and meters per token lets you switch backends by changing a model string rather than rewriting stream handling. For teams shipping agents across many models, that normalization is worth more than any single provider’s marginal streaming feature.