Context window sizes by model: a 2026 comparison

Context window size by model remains the single most consequential spec when you architect LLM workflows. It determines whether your RAG pipeline fits in one call, whether you can stuff an entire codebase into the prompt, and whether you need a summarization step before the model even sees the data. Below is the 2026 landscape across the models engineers actually ship with, organized by provider, with the numbers that matter for capacity planning and routing logic.

1. GPT-4o and GPT-4.1 (OpenAI)

OpenAI’s flagship line settles at 128k input tokens and 16k output tokens. GPT-4o (May 2024) and the newer GPT-4.1 (April 2025) share this ceiling. The 128k limit is generous for most chat and single-document tasks but starts to bite when you try to embed a full repository or a multi-hour meeting transcript. Output tokens are the tighter constraint — 16k means you cannot stream a large file rewrite or generate a comprehensive test suite in one go without hitting the ceiling.

Pricing reflects the context tier: $2.50 / $10.00 per 1M input/output tokens for GPT-4o, with GPT-4.1 at a slight discount. Both models support the max_tokens parameter and will hard-stop at 16k output regardless of how much input context remains. If you need more output, you must chain calls or use a different model. OpenAI does not expose a “context window used” field in the response; you track usage via prompt_tokens and completion_tokens in the usage object.

{
  "model": "gpt-4o",
  "max_input_tokens": 128000,
  "max_output_tokens": 16384,
  "pricing_per_1m": {"input": 2.50, "output": 10.00}
}

2. Claude 3.5 Sonnet and Claude 4 Series (Anthropic)

Anthropic’s current generation — Claude 3.5 Sonnet (June 2024), Claude 3.5 Haiku (October 2024), and the Claude 4 Opus/Sonnet/Haiku trio (May 2025) — all offer a 200k token context window with 8k max output tokens. The 200k input is a meaningful step up from OpenAI’s 128k; it lets you fit roughly 150k words or a medium-sized codebase in a single prompt. The 8k output cap is lower than GPT’s 16k, so long-form generation still requires chaining.

Anthropic’s API returns usage.input_tokens and usage.output_tokens plus a usage.cache_creation_input_tokens and usage.cache_read_input_tokens when you enable prompt caching. Caching is a first-class feature here: you can mark large prefix blocks (system prompts, few-shot examples, retrieved documents) as cacheable and pay ~10% of the input price on subsequent reads. This effectively makes the 200k window cheaper for repeated workloads.

{
  "model": "claude-3-5-sonnet-20241022",
  "max_input_tokens": 200000,
  "max_output_tokens": 8192,
  "pricing_per_1m": {"input": 3.00, "output": 15.00, "cache_write": 3.75, "cache_read": 0.30}
}

3. Gemini 1.5 Pro and 2.0 (Google)

Google’s context story is distinct: Gemini 1.5 Pro (February 2024) shipped with a 1M token context window, and Gemini 1.5 Flash offers the same 1M at lower cost. Gemini 2.0 (December 2024) pushes to 2M tokens on Pro and 1M on Flash. These are not theoretical limits — the models genuinely attend across the full window, though latency and cost scale superlinearly past ~500k tokens in practice.

Output tokens are capped at 8k for 1.5 Pro/Flash and 8k for 2.0 Flash, with 2.0 Pro at 8k as well. The pricing model charges per 1M tokens with a tiered structure: inputs up to 128k are cheaper, then a higher rate applies for tokens 128k–1M, and again for 1M–2M. This means a 1.5M token prompt costs significantly more than a 128k prompt even on the same model. Google also provides usageMetadata with promptTokenCount, candidatesTokenCount, and cachedContentTokenCount when using context caching.

{
  "model": "gemini-1.5-pro",
  "max_input_tokens": 1000000,
  "max_output_tokens": 8192,
  "pricing_per_1m": {"input_lt_128k": 1.25, "input_128k_1m": 2.50, "output": 5.00}
}

4. Llama 3.1 / 3.2 / 4 (Meta)

Meta’s open-weight releases standardize on 128k context across the 8B, 70B, and 405B parameter variants of Llama 3.1 (July 2024) and the 1B/3B/11B/90B variants of Llama 3.2 (September 2024). Llama 4 (expected early 2026) is rumored to extend this but unconfirmed as of publication. The 128k window is implemented via RoPE scaling (YaRN) on the base 4k/8k pre-training length, and quality degrades noticeably past ~64k in long-context benchmarks like RULER and Needle-in-a-Haystack.

Because these are open weights, your actual context window depends on the inference engine. vLLM, TGI, and llama.cpp all support 128k with proper KV cache configuration, but you must allocate GPU memory accordingly: a 70B model at 128k context in FP8 needs ~80 GB VRAM for the KV cache alone. Output tokens are limited only by the remaining context budget — there is no separate max_output_tokens parameter in the model architecture, though serving stacks typically enforce a default (often 2k–4k) unless overridden.

# vLLM config example for Llama-3.1-70B at 128k context
model: meta-llama/Llama-3.1-70B-Instruct
max-model-len: 131072
gpu-memory-utilization: 0.9
dtype: fp8
kv-cache-dtype: fp8

5. Mistral Large 2 and Mistral Nemo (Mistral AI)

Mistral Large 2 (July 2024) and Mistral Nemo 12B (July 2024) both offer 128k context windows. Large 2 is a 123B parameter model competitive with GPT-4o on benchmarks; Nemo is a 12B model co-trained with NVIDIA, optimized for on-prem deployment. Both use a sliding window attention pattern for the first 8k tokens and full attention beyond that, which keeps KV cache growth linear rather than quadratic.

Mistral’s API (la Plateforme) prices Large 2 at $2.00 / $6.00 per 1M input/output and Nemo at $0.15 / $0.15. The API returns standard usage fields. For self-hosted deployments, the same vLLM/TGI considerations as Llama apply — 128k context on a 123B model requires multi-GPU tensor parallelism. Mistral also publishes a “context window utilization” metric in their managed API dashboard, showing peak KV cache usage per request.

{
  "model": "mistral-large-2407",
  "max_input_tokens": 128000,
  "max_output_tokens": 8192,
  "pricing_per_1m": {"input": 2.00, "output": 6.00}
}

6. Command R+ and Command R (Cohere)

Cohere’s Command R+ (April 2024) and Command R (March 2024) both feature 128k context windows with a 4k output token default (configurable up to the context limit). These models are explicitly optimized for RAG and tool use, with training data that includes long-context retrieval tasks. The 128k window is native — no RoPE scaling — which means more stable quality at the far end of the context.

Cohere’s API pricing is $2.50 / $10.00 per 1M for Command R+ and $0.15 / $0.60 for Command R. The platform includes a documents parameter for grounded generation that automatically handles citation extraction; these document tokens count against the 128k limit. Cohere also offers a “context window” dashboard showing token distribution across system, user, documents, and tool results — useful for debugging retrieval budgets.

{
  "model": "command-r-plus-08-2024",
  "max_input_tokens": 128000,
  "max_output_tokens": 4096,
  "pricing_per_1m": {"input": 2.50, "output": 10.00}
}

7. Qwen 2.5 and Qwen 3 (Alibaba)

Qwen 2.5 (September 2024) spans 0.5B to 72B parameters with 128k context across the board. Qwen 3 (expected late 2025/early 2026) extends the 72B and 110B variants to 1M context via continued pre-training on long sequences. The 128k variants use YaRN scaling from a 32k base; the 1M variants use a progressive training curriculum. Quality at 1M tokens is competitive with Gemini 1.5 Pro on Chinese and English long-context benchmarks.

As open weights, deployment is on you. The 72B model at 128k fits on 4×H100 (FP8) or 8×A100 (BF16). The 1M context 72B requires 8×H100 for the KV cache alone. Alibaba’s DashScope API offers the 128k models at ~$0.30 / $0.60 per 1M and the 1M context models at ~$2.00 / $6.00. The API returns usage.input_tokens, usage.output_tokens, and usage.total_tokens.

{
  "model": "qwen2.5-72b-instruct",
  "max_input_tokens": 128000,
  "max_output_tokens": 8192,
  "pricing_per_1m": {"input": 0.30, "output": 0.60}
}

8. DeepSeek V3 and R1 (DeepSeek)

DeepSeek V3 (December 2024) and R1 (January 2025) both offer 128k context windows. V3 is a 671B MoE model (37B active) trained on 14.8T tokens; R1 is a reasoning-specialized variant with chain-of-thought distillation. The 128k window is native, and the MoE architecture means KV cache scales with active parameters, not total — a meaningful memory advantage over dense 70B+ models at long context.

DeepSeek’s API prices V3 at $0.14 / $0.28 per 1M (cache hit / miss) and R1 at $0.55 / $2.19. The cache pricing is aggressive: a cache hit costs 1/20th of a miss, incentivizing prompt prefix reuse. Self-hosted, V3 at 128k context fits on 8×H100 (FP8) with tensor parallelism. The API returns standard usage plus prompt_cache_hit_tokens and prompt_cache_miss_tokens.

{
  "model": "deepseek-chat",
  "max_input_tokens": 128000,
  "max_output_tokens": 8192,
  "pricing_per_1m": {"input_cache_hit": 0.14, "input_cache_miss": 0.28, "output": 1.10}
}

Summary: choosing by context window size by model

The context window size by model you pick should match your retrieval strategy, not the other way around. If your RAG pipeline retrieves 50k tokens per query, a 128k model leaves ~78k for system prompt, few-shots, and output — tight but workable. If you regularly stuff 200k+ tokens, Gemini or Claude 4 are the only managed options that don’t require chunking. For self-hosted, the MoE advantage of DeepSeek V3 or the 1M context of Qwen 3 change the economics dramatically. Route accordingly.