DeepSeek R1 time to first token is the latency metric that breaks teams who treat reasoning models like vanilla chat models. Because R1 performs an internal chain-of-thought before producing visible answer tokens, the gap between request and first byte is dominated by prefill and reasoning overhead, not by token decode speed. If you benchmark it like a normal LLM, you will misallocate engineering effort.
Why TTFB is the wrong lens for decode, but the right lens for prefill
Most latency dashboards track time to first token as a proxy for “how fast the model starts talking.” For dense 7B–70B instruct models, TTFB is usually a few hundred milliseconds and tracks prefill of the input prompt. For DeepSeek R1, the MoE architecture (671B total parameters, 37B active per token) means prefill is expensive in memory bandwidth and all-to-all expert communication, while decode of each token is comparatively cheap because only a fraction of experts activate.
The trap is assuming TTFB measures the same thing across model classes. In R1, the first token you see may be a thinking token or the final answer depending on API configuration. The clock starts at request receipt and stops at the first streamed chunk. That interval includes:
- Network round trip to provider
- Queueing behind other requests on the GPU batch
- Prefill of the full prompt (attention over all input tokens)
- Generation of hidden reasoning tokens if the provider buffers them
- Scheduling of the first decode step
For a 2K-token prompt, prefill on H100s for an MoE of this size is manageable but not free. For 32K-token legal documents, prefill dominates and can push DeepSeek R1 time to first token into double-digit seconds on congested endpoints.
What actually happens on a DeepSeek R1 request
Prefill cost scales with prompt + thinking budget
DeepSeek R1 is trained to reason with a long internal monologue. The model decides when to stop thinking; some providers expose max_thinking_tokens, others cap total completion. The prefill stage processes the prompt, then the model enters autoregressive decode for reasoning tokens. If the provider does not stream thinking, those tokens are computed before the first byte leaves the server. That makes TTFB a function of reasoning length, not just prompt length.
Empirically, a simple math question can trigger hundreds to thousands of reasoning tokens before any answer appears. At a decode rate of 20–50 tokens/sec on shared infrastructure, that is tens of seconds of silent compute. The user sees a stalled UI.
Streaming thinking tokens changes perceived TTFB
DeepSeek’s official API streams reasoning content in a separate field (reasoning_content) before the final content. When you stream, the first chunk arrives after prefill completes and the first thinking token decodes. That first token may appear in 1–3 seconds for short prompts on healthy infrastructure. The key point: DeepSeek R1 time to first token measured at the API boundary is the moment the first thinking token ships, not the first answer token.
If you abort early because you think the model hung, you waste the prefill. Clients must distinguish reasoning_content from content and render a spinner or partial thinking view.
Measuring DeepSeek R1 time to first token correctly
You cannot trust a single curl timing. Use a streaming client that captures the timestamp of the first non-empty delta. Below is a minimal Python snippet using the OpenAI SDK against DeepSeek’s endpoint.
from openai import OpenAI
import time
client = OpenAI(
base_url="https://api.deepseek.com",
api_key="YOUR_KEY",
)
start = time.perf_counter()
stream = client.chat.completions.create(
model="deepseek-reasoner",
messages=[{"role": "user", "content": "Explain why prime numbers are infinite."}],
stream=True,
)
first_ts = None
for chunk in stream:
delta = chunk.choices[0].delta
if delta.reasoning_content or delta.content:
first_ts = time.perf_counter()
break
ttft = first_ts - start
print(f"DeepSeek R1 time to first token: {ttft:.2f}s")Run this across prompt sizes and concurrency levels. You will see variance driven by batch occupancy, not by model weights. A warm cache on the prefix shrinks prefill dramatically.
The impact of prompt caching on TTFB
Prefill recomputes attention for every token in the context. If your system sends a long system prompt or retrieved documents that rarely change, prefix caching avoids that cost. DeepSeek supports prompt cache on its platform; OpenRouter-class gateways expose it via cache-control extensions.
A gateway like n4n.ai forwards provider cache-control hints to upstream DeepSeek endpoints, so a cached prefix skips recomputed prefill and turns a 12-second TTFB into sub-second. The client sends:
{
"model": "deepseek/deepseek-r1",
"messages": [
{"role": "system", "content": "You are a legal analyst. ...", "cache_control": {"type": "ephemeral"}},
{"role": "user", "content": "Summarize clause 4"}
],
"stream": true
}If the provider honors the hint, the system prompt prefix is read from cache. The DeepSeek R1 time to first token then reflects only the new user tokens plus scheduling delay.
Concurrency and batching: the silent TTFB killer
Shared inference servers use continuous batching, but MoE models amplify contention. When many requests with long prompts arrive, the all-to-all expert routing saturates interconnect bandwidth. Your prefill may wait behind a batch that is itself waiting on expert weights. This is invisible to single-request benchmarks.
Measure under load. A simple loop firing N concurrent requests and recording per-request TTFB reveals a fat tail. For example, at 16 concurrent 4K-token prompts, p95 TTFB can be 4–6x the p50. Treat that tail as a capacity planning input, not an anomaly.
Quantization tradeoffs
DeepSeek R1 is served in FP8 on H100-class hardware by some providers, and in lower-precision formats by others. FP8 preserves prefill throughput close to BF16 while halving memory traffic. INT4 variants reduce memory footprint further but can increase kernel launch overhead on MoE routing. The qualitative takeaway: quantization primarily helps prefill-bound metrics like TTFB, not decode-bound metrics. Choose a provider whose quantization matches your latency budget.
Tradeoffs: hide thinking vs stream thinking
Providers face a product choice. Hiding reasoning produces clean answer-only outputs but makes TTFB look terrible because the user waits for the entire thought process. Streaming thinking improves perceived latency and lets clients show progress, but increases output token count and billing.
For internal tooling, stream thinking and render it in a collapsible panel. For customer-facing chat, you may hide it but must set expectations with a “Reasoning…” state backed by a heartbeat. Never poll for completion; stream.
Handling TTFB tails with fallback
The nasty part of DeepSeek R1 time to first token is the tail. A provider with GPU contention or rate limits can stretch prefill to minutes. Naive retries compound the problem: you send a second request that also queues.
Route through a layer that detects degraded upstreams. n4n.ai’s automatic fallback across 240+ models includes DeepSeek R1 endpoints, so a rate-limited provider does not turn into a 30-second TTFB stall; the request reroutes to a healthy one. This only works if your client honors the same conversation format and you treat the first token as cancellable.
Set a client-side TTFB SLO, e.g., abort if no first chunk in 8 seconds, but only after the gateway has had a chance to fallback. Pair with per-token usage metering to attribute cost when a fallback occurs.
Engineering checklist for production
- Measure TTFB per prompt size bucket, not global averages.
- Use prefix caching for static context; send cache_control hints.
- Stream reasoning tokens; render them as progress.
- Cap thinking indirectly via prompt engineering (“think briefly”) if the provider lacks a param.
- Set a TTFB timeout that triggers gateway fallback, not raw retry.
- Monitor prefill-bound vs decode-bound latency separately.
- Load-test at expected concurrency before trusting a provider’s p50.
Decisive takeaway
DeepSeek R1 time to first token is a prefill-and-reasoning metric, not a decode metric. Treat it by shrinking prefill via caching and prompt trimming, streaming thinking for perceived responsiveness, and routing around degraded providers with fallback. Teams that do this ship snappy reasoning features; teams that stare at p50 decode latency ship timeouts.