Topic
Long-Context Latency Benchmarks
13 posts on long-context latency benchmarks — part of benchmarks & performance on the n4n AI blog.
Why long-context requests slow down time to first token
Analyzes why long context time to first token slowdown occurs, breaking down prefill cost, attention scaling, and scheduling, with practical mitigations.
Measuring latency cost of large system prompts
Step-by-step method to measure system prompt latency cost on LLM endpoints, isolating prefill overhead with streaming and controlled prompt sizes across models.
Long-context latency: Claude Opus 4.5 at 200k tokens
Analyzing Claude Opus 4.5 200k context latency: prefill dominates TTFT, prefix caching helps, and full context isn't always worth the cost.
Llama 4 Scout long-context latency at 10M tokens
Analysis of Llama 4 Scout 10M context latency: why KV-cache footprint and prefill scheduling dominate, and how to serve open-weight long-context models in production.
How much does a 100k-token prompt add to response time
Analyzing how a 100k token prompt affects LLM response time: prefill cost, time-to-first-token, caching, and practical latency tradeoffs for engineers.
How context length affects time to first token
Analyze how prompt size drives time to first token in LLM inference, why it's sublinear in practice, and how to benchmark and reduce long-context TTFT.
Long-context latency: GPT-4.1 vs Claude Sonnet 4.5
Head-to-head comparison of GPT-4.1 vs Claude Sonnet long context latency: measuring TTFT, throughput, cost, and ergonomics for engineering teams.
Long-context latency benchmark across 8 LLM providers
A head-to-head long context latency benchmark across 8 LLM providers, comparing capabilities, cost, speed, and ergonomics to guide your architecture.
Latency at 128k tokens: GPT-4o vs Claude vs Gemini
A practical 128k token context latency benchmark comparing GPT-4o, Claude, and Gemini on cost, speed, and ergonomics for long-context LLM apps.
Does prompt caching fix long-context latency?
Prompt caching reduces long-context latency for repeated prefixes but doesn't solve decode or cache-miss costs. An engineer's breakdown of tradeoffs.
Context window latency benchmark for RAG-heavy prompts
A practical guide to benchmarking context window latency RAG prompts, with actionable steps, code, and tradeoffs for engineers shipping LLM systems.
Benchmarking Gemini 1.5 Pro's 1M token context latency
A practitioner's analysis of Gemini 1.5 Pro 1M context latency: prefill dominance, measurement methodology, tradeoffs vs RAG, and production patterns.
32k vs 128k vs 1M: context length latency scaling
Context length latency scaling from 32k to 1M tokens is non-linear. This analysis explains the mechanics and gives engineers concrete mitigation strategies.
More topics in benchmarks & performance
- Agentic Workflow Performance Benchmarks14
- Benchmark Methodology and Measurement14
- Code Generation Latency for Dev Tools14
- Flagship Model Speed Showdown14
- Llama 4 Inference Speed by Provider14
- Price-Performance Rankings14
- Provider Uptime and Reliability Benchmarks14
- Reasoning Model Latency Overhead14
- Customer Support Chatbot Latency13
- DeepSeek Performance Benchmarks13
- GPU Inference Benchmarks13
- Model Size vs Inference Speed Tradeoffs13