n4nAI
Open console →
Benchmarks & performanceComparisonStreaming Latency Consistency

Time to first token: GPT-4o vs Gemini 2.5 Pro vs Claude

Head-to-head time to first token benchmark of GPT-4o, Gemini 2.5 Pro, and Claude: latency, cost, ergonomics, and which to use per streaming use case.

n4n Team4 min read867 words

Audio narration

Coming soon — every post will get a voice note here.

A time to first token benchmark is the only latency metric that matters when your users watch a response type itself out. We ran GPT-4o, Gemini 2.5 Pro, and Claude under identical streaming loads to see which model starts talking first, how stable that latency stays, and what you pay for the privilege.

Why TTFT dominates perceived performance

Throughput in tokens per second is irrelevant if the user waits three seconds before anything appears. Human perception treats the first paint as the loading bar. In a CLI or chat UI, a sub-500ms first token feels instant; anything past 1.5s reads as a hang.

Streaming architectures amplify this. Your frontend renders deltas, your cancellation logic keys off early errors, and your timeout budgets are set around that first byte. Comparing models purely on aggregate benchmark scores hides the one number that shapes product feel.

How we measured

We used the OpenAI-compatible chat completions streaming endpoint for all three models. Same prompt shape (2k-token system + 200-token user turn), same region, same client. Measurement starts at request send and stops at first non-empty delta.

import time, openai

client = openai.OpenAI(
    base_url="https://api.openai.com/v1",  # swap for provider or gateway
    api_key="YOUR_KEY"
)

start = time.perf_counter()
stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a terse helper."},
        {"role": "user", "content": "Explain TTL in networking."}
    ],
    stream=True
)

ttft = None
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        ttft = time.perf_counter() - start
        break
print(f"Time to first token: {ttft*1000:.0f}ms")

The same loop works against Gemini 2.5 Pro and Claude via their OpenAI-compatible shims. We collected p50 and p99 across 200 runs each, discarding cold-start outliers from separate containers.

Head-to-head dimensions

Capabilities

GPT-4o is natively multimodal and tuned for low-latency interactive voice and vision. Gemini 2.5 Pro brings a 1M-token context and explicit reasoning modes that precondition before emitting. Claude (3.5 Sonnet class) leads on code-structured output and long-form coherence but offers no native real-time multimodal in the base chat endpoint.

Price/cost model

Public list pricing as of mid-2025: GPT-4o sits at $2.50/1M input, $10/1M output. Gemini 2.5 Pro lists $1.25/1M input (≤200k context) rising to $2.50 beyond, with $10/1M output. Claude 3.5 Sonnet is $3/1M input, $15/1M output. None of these include cached prompt discounts, which all three support via cache-control headers.

Latency and throughput

This is where the time to first token benchmark gets interesting. GPT-4o returns first token fastest in our short-prompt runs; its architecture clearly prioritizes interactive feel. Gemini 2.5 Pro adds measurable prep time, especially when the reasoning flag is on—first token arrives later but the subsequent stream is dense. Claude lands between, stable but never as snappy as GPT-4o for tiny prompts.

Under long context (100k+ tokens), the ordering shifts: Gemini’s attention overhead grows linearly, GPT-4o stays flat, Claude degrades gently. Throughput after TTFT favors Gemini for bulk generation, GPT-4o for mixed short turns.

Ergonomics

All three expose OpenAI-style messages, but field names differ. Gemini uses system_instruction out-of-band; Claude expects system as a top-level param or first message. Tool calling schemas are similar but Claude is stricter on required fields. Streaming chunks are SSE in all cases; Gemini occasionally batches deltas.

Ecosystem

GPT-4o has the deepest third-party tooling and SDK maturity. Gemini integrates with Google Cloud Vertex and offers native search grounding. Claude’s ecosystem is strongest in agentic coding frameworks and Anthropic’s own prompt caching docs.

Limits

GPT-4o caps at 128k context for the standard endpoint. Gemini 2.5 Pro advertises 1M but rate limits on long prompts are aggressive. Claude’s 200k context is solid, but output token caps are lower per request than Gemini’s.

Comparison table

Dimension GPT-4o Gemini 2.5 Pro Claude (3.5 Sonnet)
Multimodal Native vision/audio Text + vision via separate API Text only (chat)
Context window 128k 1M (rate-limited) 200k
List price (in/out per 1M) $2.50 / $10 $1.25–$2.50 / $10 $3 / $15
TTFT short prompt Lowest Highest (reasoning prep) Mid
TTFT long context Flat Scales with length Gentle degradation
Streaming ergonomics OpenAI-native SSE, batched deltas OpenAI-like, strict schema
Tool calling Mature Beta-quality Strict, reliable

Streaming latency consistency

A single time to first token benchmark number lies if it’s only p50. What kills UX is p99 variance: one request in twenty stalls. GPT-4o shows tight p99 bands in our runs—usually within 2× p50. Gemini 2.5 Pro’s p99 spreads wider when the scheduler packs long-context jobs. Claude stays predictable but with a higher floor.

If you front these models with an OpenAI-compatible gateway such as n4n.ai, you can issue one request shape and get automatic fallback when a provider’s TTFT spikes, while per-token metering stays accurate. That hides provider instability from your users without rewriting clients.

To keep your own p99 honest, set a client-side first-token deadline:

import asyncio, openai

async def stream_with_timeout(model, messages, max_ttft=1.5):
    client = openai.AsyncOpenAI()
    start = asyncio.get_event_loop().time()
    stream = await client.chat.completions.create(
        model=model, messages=messages, stream=True
    )
    async for chunk in stream:
        if chunk.choices[0].delta.content:
            if asyncio.get_event_loop().time() - start > max_ttft:
                raise TimeoutError("TTFT exceeded budget")
            return chunk

Which to choose

Interactive chat and voice assistants: GPT-4o. Its time to first token benchmark leads, multimodal is built in, and p99 stays tight. Use it when the user is waiting on every keystroke.

Long-document RAG with rare queries: Gemini 2.5 Pro. Pay the TTFT penalty for 1M context and dense follow-up throughput. Disable reasoning mode if you need faster first token on simple lookups.

Code generation and agentic workflows: Claude. Slightly slower first token than GPT-4o, but stricter tool schemas and coherent long outputs justify the latency. Use streaming with a 1s TTFT budget.

Cost-sensitive high-volume micro-tasks: Gemini 2.5 Pro at low context, or GPT-4o with prompt caching. Both beat Claude on price per token at scale.

Resilient production systems: Route across all three. Honor client routing directives, forward cache-control hints, and measure TTFT per provider. When one degrades, fall back without breaking the stream contract.

Tagstime-to-first-tokengpt-4ogemini-2-5-proclaudebenchmark

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Benchmarks & performanceGPT-4o vs Claude Sonnet 4.5: streaming latency comparedA practical head-to-head of GPT-4o vs Claude streaming latency: measuring TTFT, throughput, cost, and ergonomics to help engineers pick the right model.Benchmarks & performanceHow provider load affects streaming latency varianceAnalyzes how backend utilization drives streaming latency variance in LLM providers, with measurement code and routing strategies to reduce jitter.Benchmarks & performanceTracking streaming latency drift over a week of requestsTrack streaming latency drift over time with continuous measurement; point-in-time benchmarks hide weekly variance from provider load, routing, and fallbacks.Benchmarks & performanceWhy streaming latency varies more than average latencyStreaming latency variance vs average latency reveals why tail behavior dominates UX. We break down causes, measurement traps, and mitigation steps.Benchmarks & performanceBenchmarking streaming latency for real-time chat appsA practical guide to benchmarking streaming latency for chat applications: build a test harness, capture timestamps, avoid pitfalls, and track percentiles.Benchmarks & performancep99 streaming latency across 12 LLM providersA practitioner's analysis of p99 streaming latency across 12 LLM providers: why tail latency breaks UX, how to measure it, and mitigation that works.

All streaming latency consistency posts →