Claude’s extended thinking mode is Anthropic’s implementation of test-time compute scaling: the model generates hidden reasoning tokens before producing its final answer. Unlike standard inference where the model outputs directly, extended thinking allocates a configurable budget of tokens for internal chain-of-thought that the user never sees. This guide covers how to enable it, how to budget for it, and the production tradeoffs you’ll hit when shipping it.
What extended thinking actually does
When you enable extended thinking, the model enters a two-phase generation. First, it produces reasoning tokens — essentially a private scratchpad where it decomposes the problem, checks intermediate steps, and self-corrects. These tokens are not streamed to the client and do not appear in the conversation history. Second, it emits the final answer tokens, which are what your application receives.
The reasoning phase consumes tokens from your output budget. If you set thinking: { type: "enabled", budget_tokens: 2000 }, the model can spend up to 2,000 tokens reasoning before it must begin the final answer. The total output (reasoning + answer) cannot exceed the model’s max output limit (8,192 for Claude 3.5 Sonnet, 12,288 for Opus).
{
"model": "claude-3-5-sonnet-20241022",
"max_tokens": 8192,
"thinking": {
"type": "enabled",
"budget_tokens": 2000
},
"messages": [
{"role": "user", "content": "Debug this recursive function..."}
]
}The reasoning tokens are billed at the same rate as output tokens. This matters for cost modeling — a 2,000 token budget with a 500 token answer costs 2,500 output tokens, not 500.
When to enable it
Extended thinking shines on tasks where the path to the answer is non-obvious: multi-step code debugging, mathematical derivation, complex logic puzzles, and architectural reasoning. It degrades on tasks that are essentially lookup or style transfer: summarization, translation, formatting, and simple classification.
A practical heuristic: if a competent engineer would need scratch paper to solve it reliably, extended thinking helps. If they’d solve it in their head, it adds latency and cost without benefit.
def should_use_extended_thinking(task_type: str, complexity: int) -> bool:
"""
Rough decision rule for production routing.
complexity: 1-5 scale
"""
reasoning_tasks = {
"code_debugging", "algorithm_design", "math_proof",
"architecture_review", "root_cause_analysis"
}
if task_type in reasoning_tasks:
return complexity >= 2
return FalseBudget sizing and token accounting
The budget_tokens parameter is a ceiling, not a reservation. The model may use fewer tokens if it converges early. However, you must set max_tokens high enough to accommodate budget_tokens + expected_answer_tokens. If the sum exceeds the model’s hard limit, the request fails.
# Valid: budget 4000, max 8192, leaves 4192 for answer
{"max_tokens": 8192, "thinking": {"type": "enabled", "budget_tokens": 4000}}
# Invalid: budget 6000, max 8192, but answer needs 3000 -> 9000 total
{"max_tokens": 8192, "thinking": {"type": "enabled", "budget_tokens": 6000}}Monitor actual usage. The response includes usage.output_tokens which equals reasoning_tokens + answer_tokens. Anthropic does not currently break these out separately in the API response, so you cannot programmatically distinguish how many tokens went to reasoning vs. answer. Plan your cost models accordingly.
Streaming and latency implications
Extended thinking adds latency proportional to the reasoning budget. A 2,000 token budget at ~50 tokens/second adds ~40 seconds of pure generation time before the first answer token appears. If you stream responses, the client sees nothing during the reasoning phase.
Two patterns mitigate this:
1. Background reasoning with progress polling — Fire the request asynchronously, poll for completion, then stream the answer. Your API returns a job ID immediately.
# Pseudocode for async pattern
async def start_reasoning_job(request: ThinkingRequest) -> JobId:
job_id = uuid4()
asyncio.create_task(run_thinking_job(job_id, request))
return job_id
async def run_thinking_job(job_id: str, request: ThinkingRequest):
response = await anthropic.messages.create(
model=request.model,
max_tokens=request.max_tokens,
thinking={"type": "enabled", "budget_tokens": request.budget_tokens},
messages=request.messages,
stream=False # Wait for full completion
)
# Store answer, mark job complete
await job_store.set(job_id, response.content[0].text)2. Hybrid mode — Use a small budget (500-1000 tokens) for interactive latency, fall back to larger budgets for batch/async workloads.
def select_budget(context: RequestContext) -> int:
if context.is_interactive:
return 800 # ~16s max reasoning
if context.is_batch:
return 4000 # ~80s max reasoning
return 1600Common pitfalls
Pitfall: Assuming reasoning tokens are free. They are billed as output tokens. A 4,000 token budget on Opus costs ~$60/million output tokens. At scale, this dominates your inference spend.
Pitfall: Setting budget too high for simple tasks. The model will often use the full budget even when unnecessary. Cap budgets per task type.
Pitfall: Expecting reasoning traces for debugging. You cannot retrieve the hidden reasoning. If you need auditability, ask the model to “show your work” in the final answer instead — this consumes answer tokens but gives you visibility.
Pitfall: Combining with tools incorrectly. Extended thinking works with tool use, but the reasoning phase cannot invoke tools. The model reasons, then emits tool calls in the answer phase. This means tool arguments are not “thought through” in the hidden scratchpad — only the decision to call the tool is.
{
"model": "claude-3-5-sonnet-20241022",
"max_tokens": 8192,
"thinking": {"type": "enabled", "budget_tokens": 2000},
"tools": [{"name": "query_db", "description": "...", "input_schema": {...}}],
"messages": [...]
}Cost optimization strategies
Route by task classification. Use a lightweight classifier (or heuristic) to send only reasoning-heavy tasks to extended thinking. Everything else uses standard mode.
Cap budgets per tier. Define tiers: low (500), medium (1500), high (4000). Map task types to tiers. Never expose raw budget_tokens to end users.
BUDGET_TIERS = {
"low": 500,
"medium": 1500,
"high": 4000
}
TASK_TIER_MAP = {
"code_debugging": "high",
"feature_design": "high",
"code_review": "medium",
"refactoring_plan": "medium",
"documentation": "low",
"style_fix": "low"
}Cache aggressively. Extended thinking answers are deterministic for a given prompt and budget (modulo temperature). Cache responses by (prompt_hash, budget_tier, model_version). This avoids re-reasoning identical queries.
Use prompt caching for context. If you send the same large context (codebase, docs) with varying questions, enable prompt caching on the context block. The reasoning phase benefits from cached context just like the answer phase.
{
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "<large_codebase>",
"cache_control": {"type": "ephemeral"}
},
{"type": "text", "text": "Find the memory leak in module X"}
]
}
]
}Evaluating quality gains
Run evals before rolling out. Extended thinking does not universally improve quality — it can overthink, hallucinate more confidently, or get stuck in reasoning loops.
def evaluate_thinking_quality(samples: list[EvalSample]) -> dict:
results = {"standard": [], "extended": []}
for sample in samples:
std = run_inference(sample.prompt, thinking=False)
ext = run_inference(sample.prompt, thinking=True, budget=2000)
results["standard"].with_trace(judge(std, sample.expected))
results["extended"].with_trace(judge(ext, sample.expected))
return {
"standard_pass_rate": pass_rate(results["standard"]),
"extended_pass_rate": pass_rate(results["extended"]),
"latency_p50_std": p50_latency(results["standard"]),
"latency_p50_ext": p50_latency(results["extended"]),
"cost_ratio": cost_ratio(results["extended"], results["standard"])
}Track pass rate delta, latency increase, and cost multiplier. Only enable extended thinking for task categories where the quality delta justifies the cost/latency penalty.
Integration checklist for production
- Classify your task types and assign budget tiers
- Implement async job pattern for high-budget requests
- Add request validation:
max_tokens >= budget_tokens + min_answer_tokens - Instrument usage: log
budget_tokens,actual_output_tokens,latency_ms,task_type - Set up cost alerts per budget tier
- Build eval harness comparing standard vs. extended per task type
- Document which tasks route to which tier for your team
- Configure prompt caching for shared context workloads
The bottom line
Claude extended thinking is a powerful lever for reasoning-heavy workloads, but it is not a free upgrade. It trades latency and token spend for answer quality on a specific class of problems. Treat it like any other infrastructure knob: measure, route selectively, cap budgets, and cache aggressively. The teams that ship it successfully are the ones who instrumented it from day one.