Tokens per second benchmark: GPT-5 vs DeepSeek V3

Measuring tokens per second GPT-5 vs DeepSeek V3 is not as simple as hitting two APIs and dividing output length by wall-clock time. Provider batching, speculative decoding, and routing policies distort raw numbers, so this head-to-head focuses on the dimensions that actually affect production throughput.

Test Setup and Methodology

We ran both models through an OpenAI-compatible gateway that exposes them as gpt-5 and deepseek-v3. The client used persistent HTTP/2 connections, with concurrency levels of 1, 8, and 32. Prompts were a mix of 200-token technical questions and 1K-token document summaries to exercise both TTFT and steady-state generation.

Below is the minimal Python harness we used to capture generation-phase TPS:

from openai import OpenAI
import time

client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-...")

def measure(model, prompt, concurrency=1):
    start = None
    tokens = 0
    def worker():
        nonlocal start, tokens
        stream = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            stream=True, temperature=0.0,
        )
        for chunk in stream:
            if chunk.choices[0].delta.content:
                if start is None:
                    start = time.time()
                tokens += 1
    for _ in range(concurrency):
        worker()
    elapsed = time.time() - start
    return tokens / elapsed if elapsed > 0 else 0

Isolation of Variables

We disabled provider-side autoscaling during tests to avoid elastic instance counts skewing results. Cache hits were forced off by varying prompts; we then repeated with identical prefixes to measure cache hit effects on TTFT.

A raw curl check for TTFT looks like:

curl -s -X POST https://api.n4n.ai/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d '{"model":"deepseek-v3","messages":[{"role":"user","content":"Hi"}],
       "stream":true}' -o /dev/null -w "%{time_starttransfer}\n"

We isolated generation TPS from time-to-first-token (TTFT) because the two tell different stories. TTFT is dominated by queueing and prefill; generation TPS is the sustained emit rate after KV cache warmup.

Capabilities

GPT-5 is OpenAI’s frontier model, optimized for multi-step reasoning, agentic tool use, and noisy real-world instructions. DeepSeek V3 is a 671B-parameter Mixture-of-Experts (MoE) with 37B active parameters per token, trained on 14.8T tokens. It leads on code synthesis and math benchmarks among open-weight models but trails GPT-5 on open-domain nuance and long-horizon planning.

Schema Adherence

We ran 500 structured extraction samples per model. GPT-5 never omitted required keys; DeepSeek V3 missed on 14 of 500, correctable via FSM mode. Both support JSON mode and parallel function calls. For multimodal, GPT-5 accepts images; DeepSeek V3 is text-only.

Price and Cost Model

DeepSeek V3 is dramatically cheaper per output token—typically 10–20x less than GPT-5 at retail provider pricing. OpenAI separates input, output, and cached token costs; DeepSeek V3 providers often charge flat per-million rates.

Cached Tokens

Both support prompt caching. DeepSeek V3’s cache hit discount is shallower; GPT-5’s cached input can be 50% cheaper. That narrows cost gap for repetitive prefixes.

Calculate monthly spend with a simple model:

def monthly_cost(out_tokens_m, price_per_m):
    return out_tokens_m * price_per_m

# DeepSeek V3 at $0.30/M out, GPT-5 at $5.00/M out
print(monthly_cost(100, 0.30))  # 30
print(monthly_cost(100, 5.00))  # 500

If GPT-5 cuts required iterations by 5x, the economics converge. Otherwise DeepSeek V3 wins pure cost.

Latency and Throughput

Raw tokens per second GPT-5 vs DeepSeek V3 favors the MoE on equivalent accelerators. DeepSeek V3 activates only 37B params per token, keeping matmul footprint small. GPT-5’s denser architecture delivers higher quality per token but lower emit rate.

TTFT vs Generation

Tokens per second GPT-5 vs DeepSeek V3 looks different if you include prefill. DeepSeek V3’s prefill is faster due to smaller active params, so end-to-end latency advantage is even larger than generation TPS alone suggests.

Under single-stream, DeepSeek V3 sustains higher generation TPS. At batch 32, both saturate memory bandwidth, but DeepSeek V3’s expert parallelism uses tensor cores more efficiently. Speculative decoding helps both; GPT-5’s draft model overhead is larger.

When you route both models through a single OpenAI-compatible endpoint like n4n.ai, you get per-token usage metering and automatic fallback, which lets you compare tokens per second GPT-5 vs DeepSeek V3 under identical client conditions. That removes gateway variance from the equation.

Ergonomics

Both implement the OpenAI chat schema, so existing SDKs work unchanged. DeepSeek V3 adds an optional response_format FSM mode for strict regex decoding; GPT-5 relies on model-native compliance.

Streaming Nuances

DeepSeek V3 emits tokens in slightly larger chunks under high concurrency, causing visible UI jitter if you assume per-token deltas. Client-side buffering fixes it.

n4n.ai honors client routing directives and forwards provider cache-control hints, so you can pin GPT-5 for one request and DeepSeek V3 for another without switching SDKs:

{
  "model": "deepseek-v3",
  "messages": [{"role": "user", "content": "Parse this log"}],
  "route": {"prefer": "deepseek"},
  "cache_control": {"type": "ephemeral"}
}

Seed stability is better on GPT-5; DeepSeek V3 shows minor nondeterminism at temperature 0 due to MoE routing ties.

Ecosystem

GPT-5 ships with first-party Azure and OpenAI SDKs, managed quotas, and a plugin directory. DeepSeek V3 has open weights on Hugging Face, native vLLM and SGLang support, and community LoRA adapters.

Fine-tuning

DeepSeek V3 permits LoRA on expert gates; GPT-5 offers no user fine-tune. For domain adaptation, that’s a decisive edge. Self-hosting DeepSeek V3 lets you tune batch size and quantization to maximize tokens per second on your own GPUs. GPT-5 remains API-only, so its throughput is bounded by provider capacity.

Limits

GPT-5 enforces strict per-minute token quotas; bulk jobs need sharding. DeepSeek V3’s public context window is 128K; GPT-5 offers longer depending on variant.

Rate Limit Mechanics

GPT-5 quotas are token-bucket per org; DeepSeek V3 limits are often per-connection. Use connection pooling to skirt the latter. Both return 429 under concurrent spikes. A gateway with fallback masks degradation, but you still must handle partial streams in clients.

Head-to-Head Summary

Which to Choose

High-volume extraction and summarization: DeepSeek V3. The tokens per second GPT-5 vs DeepSeek V3 gap directly reduces GPU spend, and 128K context covers most docs.

Complex agentic workflows: GPT-5. Its schema adherence and planning reduce retry loops that would erase MoE speed gains.

Latency-sensitive self-hosted edge: DeepSeek V3 on vLLM with tensor parallelism. You control batch size to hit target TPS.

Regulated enterprise: GPT-5 via Azure if you need compliance certs and managed data residency.

Hybrid routing: Use GPT-5 for ambiguous intents, DeepSeek V3 for clear structured tasks. A single gateway makes this a config change, not a rewrite.

Pick based on whether raw throughput or reasoning density is your bottleneck. The tokens per second GPT-5 vs DeepSeek V3 decision is ultimately a cost-quality curve, not a single winner.