Claude Sonnet 4.5's 200k token context window, explained

The claude sonnet context window size of 200,000 tokens means the model can process roughly 150,000 words or 500 pages of text in a single request — including both the input prompt and the generated output. This is not a hard limit on knowledge or reasoning; it is a hard limit on the total token budget available for the current conversation turn. Everything you send — system prompts, few-shot examples, retrieved documents, chat history, and the model’s response — must fit inside that budget.

How the context window works

At inference time, the model receives a single sequence of tokens. The attention mechanism computes relationships between every token and every other token in that sequence. With 200k tokens, the attention matrix is 200k × 200k — a computational reality that drives both the capability and the cost.

Anthropic achieves this window through a combination of architectural choices: rotary positional embeddings (RoPE) with a long base frequency, grouped-query attention to reduce KV cache memory, and optimized kernel implementations for the attention computation. The model was trained on sequences up to this length, so it learns to attend across long distances rather than degrading to local patterns.

The context window is shared bidirectionally. Input tokens consume capacity that output tokens cannot use, and vice versa. If you send 190k tokens of context, the model has at most 10k tokens for its response before hitting the limit. There is no separate “output window.”

# Token accounting is straightforward but often overlooked
def estimate_remaining_output_tokens(input_tokens: int, max_context: int = 200_000) -> int:
    """Return max tokens available for generation given input size."""
    return max(0, max_context - input_tokens)

# Example: 150k tokens of retrieved docs + 5k system prompt + 2k chat history
input_tokens = 150_000 + 5_000 + 2_000
remaining = estimate_remaining_output_tokens(input_tokens)
print(f"Available for response: {remaining:,} tokens")  # 43,000

Why the 200k window matters for engineering

The practical impact shows up in three patterns that change how you design LLM systems.

Single-pass document processing

You no longer need chunking pipelines for most documents. A 200k window swallows entire codebases, legal contracts, financial filings, or technical manuals whole. This eliminates retrieval-augmented generation (RAG) complexity for a large class of tasks — no embedding model, no vector database, no chunking strategy, no re-ranking.

# Before 200k: chunk → embed → retrieve → rerank → synthesize
# After 200k: stuff the whole thing in

def analyze_codebase(repo_path: str, question: str) -> str:
    """Single-pass analysis of an entire repository."""
    all_source = collect_source_files(repo_path)  # ~50k tokens for typical repo
    prompt = f"""<repository>
{all_source}
</repository>

Question: {question}

Answer thoroughly, citing specific files and line numbers."""
    
    return claude_complete(prompt, max_tokens=8000)

The trade-off is latency and cost. Processing 150k input tokens takes seconds and costs proportionally more than a 4k RAG query. But for accuracy-critical tasks where missing context means wrong answers, the single-pass approach often wins on total engineering effort and correctness.

Many-shot prompting at scale

Few-shot prompting becomes many-shot prompting. You can include hundreds of labeled examples directly in context, which often outperforms fine-tuning for classification, extraction, and style transfer tasks. The model learns the pattern from the examples without weight updates.

{
  "model": "claude-sonnet-4-5",
  "messages": [
    {"role": "system", "content": "Classify support tickets. Examples follow."},
    {"role": "user", "content": "Ticket: \"Login fails with 500 error\"\nLabel: authentication"},
    {"role": "user", "content": "Ticket: \"Export button downloads corrupted CSV\"\nLabel: data-export"},
    {"role": "user", "content": "Ticket: \"Mobile app crashes on startup\"\nLabel: mobile-crash"},
    {"role": "user", "content": "Ticket: \"Dark mode doesn't persist after refresh\"\nLabel: ui-preference"},
    {"role": "user", "content": "Ticket: \"API rate limit unclear in docs\"\nLabel: documentation"},
    {"role": "user", "content": "Ticket: \"SSO integration with Okta failing\"\nLabel": "authentication"}
  ]
}

With 200k tokens, you can fit 500–1,000 examples depending on length. This turns prompt engineering into dataset curation — a more familiar and auditable workflow for many teams.

Conversation persistence without summarization

Long-running conversations — coding agents, research assistants, customer support threads — can maintain full history without aggressive summarization. A 200k window holds roughly 20–30 substantial back-and-forth turns with code and analysis. This preserves context that summarization inevitably loses: variable names, specific error messages, discarded approaches that explain current decisions.

Concrete example: Processing a 10-K filing

A typical SEC Form 10-K runs 100–200 pages. At ~300 tokens per page, that’s 30k–60k tokens — well within the window. Here’s a production pattern:

import anthropic

client = anthropic.Anthropic()

def extract_financial_metrics(filing_text: str, metrics: list[str]) -> dict:
    """Extract specific metrics from a full 10-K filing."""
    prompt = f"""You are a financial analyst. Extract the following metrics from the 10-K filing below.
Return JSON with metric names as keys. If a metric is not found, use null.
For each metric, include: value, unit, fiscal_year, and the exact sentence where it appears.

Metrics to extract: {', '.join(metrics)}

<filing>
{filing_text}
</filing>"""
    
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=4000,
        temperature=0,
        messages=[{"role": "user", "content": prompt}]
    )
    
    return json.loads(response.content[0].text)

# Usage
filing = download_10k("AAPL", "2023")
metrics = ["total_revenue", "net_income", "operating_cash_flow", "free_cash_flow", 
           "total_assets", "total_debt", "shareholders_equity", "rd_expense"]
result = extract_financial_metrics(filing, metrics)

This replaces a pipeline of: OCR → section detection → table extraction → regex parsing → validation. One model call, full document context, structured output. The claude sonnet context window size makes this viable where 8k or 32k windows would require chunking and stitching — introducing errors at section boundaries.

Common misconceptions

“200K tokens means the model knows 200k tokens of facts”

The context window is working memory, not parametric knowledge. The model’s training cutoff and internal weights determine what it knows. A 200k window just means you can provide 200k tokens of relevant information in the prompt. If the answer isn’t in those tokens (or the model’s training data), the window size doesn’t help.

“I should always fill the window”

Filling the window with irrelevant context degrades performance. Attention is diluted across noise. The model spends compute attending to tokens that don’t matter. Empirically, precision drops when relevant signal is buried in irrelevant tokens — the “lost in the middle” phenomenon persists even at 200k. Curate aggressively.

# Bad: stuff everything
context = all_company_docs + all_slack_history + all_jira_tickets + wiki_dump

# Good: retrieve precisely what the question needs
context = retrieve_relevant_chunks(question, top_k=20, max_tokens=50_000)

“Output tokens don’t count toward the limit”

They do. The 200k limit is the total sequence length: input + output. If you request max_tokens=8192 but only have 5k tokens remaining in the window, the API will error or truncate. Always account for output budget when constructing prompts.

# Safe pattern: reserve output budget explicitly
MAX_CONTEXT = 200_000
DESIRED_OUTPUT = 8_000
MAX_INPUT = MAX_CONTEXT - DESIRED_OUTPUT - 1_000  # 1k safety margin

def build_prompt(context_docs: list[str], question: str) -> str:
    context = "\n\n".join(context_docs)
    # Truncate context to fit, preserving the question
    if count_tokens(context) > MAX_INPUT:
        context = truncate_to_token_limit(context, MAX_INPUT)
    return f"Context:\n{context}\n\nQuestion: {question}"

“Long context means slow everything”

Latency scales roughly linearly with input tokens for the prefill phase (prompt processing), but generation latency depends on output tokens, not input length. A 150k token prompt with a 100 token answer feels fast after the initial prefill. The prefill is parallelizable; decoding is sequential. For interactive use, stream the response — users see tokens while the next ones compute.

“RAG is obsolete”

RAG remains essential when:

• The corpus exceeds 200k tokens (most company knowledge bases)
• You need source attribution at the chunk level
• Data changes frequently and you can’t re-prompt the full corpus each time
• Cost per query must stay low (embedding + retrieval is cheaper than 100k token prefill)

The 200k window expands the direct stuffing regime. It doesn’t eliminate the retrieval regime. Most production systems use both: stuff the most relevant 50–100k tokens, retrieve the rest on demand.

Token counting in practice

Anthropic’s tokenizer (cl100k_base, same as GPT-4) has quirks worth knowing:

import tiktoken

enc = tiktoken.get_encoding("cl100k_base")

# Code tokens are dense
code = "def fibonacci(n):\n    if n <= 1:\n        return n\n    return fibonacci(n-1) + fibonacci(n-2)"
print(len(enc.encode(code)))  # ~35 tokens for 100 chars

# JSON is verbose
json_str = '{"user_id": 12345, "permissions": ["read", "write", "admin"], "active": true}'
print(len(enc.encode(json_str)))  # ~30 tokens for 80 chars

# Whitespace matters
print(len(enc.encode("hello world")))      # 2 tokens
print(len(enc.encode("hello  world")))     # 3 tokens (double space)
print(len(enc.encode("hello\tworld")))     # 3 tokens (tab)

Budget 1.3–1.5 tokens per English word, 3–4 tokens per line of code, 4–6 tokens per line of pretty-printed JSON. Measure your actual data rather than estimating.

Cost implications

At current pricing, 200k input tokens costs significantly more than 4k. But the comparison is against the alternative pipeline cost, not against a smaller prompt. If stuffing 100k tokens replaces a RAG system that requires embedding, vector search, re-ranking, and multiple LLM calls — the single 100k call often wins on total cost and latency.

# Rough cost comparison (illustrative, check current pricing)
def estimate_cost(input_tokens: int, output_tokens: int) -> float:
    # Sonnet 4.5 pricing per 1M tokens (hypothetical)
    INPUT_COST_PER_M = 3.00
    OUTPUT_COST_PER_M = 15.00
    return (input_tokens * INPUT_COST_PER_M + output_tokens * OUTPUT_COST_PER_M) / 1_000_000

# Single-pass 100k input
single_pass = estimate_cost(100_000, 2_000)  # ~$0.33

# RAG: 3 retrieval calls (4k each) + 1 synthesis call (8k)
rag = 3 * estimate_cost(4_000, 500) + estimate_cost(8_000, 2_000)  # ~$0.18

# But RAG misses cross-document synthesis that single-pass catches
# The "correctness premium" often justifies the delta

When to use the full window

Use the full 200k when:

• The task requires synthesis across many documents (legal review, codebase analysis, literature review)
• You have a fixed, bounded corpus that fits (a single 10-K, a repo, a contract bundle)
• Accuracy outweighs latency and cost
• You can’t define a reliable retrieval query in advance

Stick to RAG when:

• The corpus is unbounded or >200k tokens
• You need real-time data freshness
• Cost per query must be minimal
• Users need citations to specific source chunks

Summary

The claude sonnet context window size of 200k tokens is a working memory limit, not a knowledge limit. It enables single-pass processing of whole documents, many-shot prompting with hundreds of examples, and long conversations without summarization. It does not eliminate the need for RAG, does not mean you should stuff irrelevant context, and does not make output tokens free. Treat it as a budget you allocate deliberately — input tokens for context, output tokens for the answer — and measure whether the task actually benefits from the full window before paying for it.