Max output tokens by model: GPT-4o, Claude, and Gemini

When you’re building with LLMs, the max output tokens by model determines whether your completion finishes or gets cut off mid-sentence. GPT-4o, Claude, and Gemini each enforce different ceilings, and those ceilings change as providers ship new model variants. Below is the current landscape as of mid-2024, plus the patterns you need to handle truncation gracefully in production.

1. GPT-4o

GPT-4o supports a maximum of 16,384 output tokens. That’s the highest ceiling among the three families covered here, and it matches the model’s total context window of 128,000 tokens — meaning you can theoretically consume the entire context in a single completion, though you’ll hit rate limits long before that in practice.

The 16k output limit applies to both the standard GPT-4o and the cheaper GPT-4o-mini variant. However, the default max_tokens parameter in the OpenAI API is still 4,096 unless you explicitly override it. If you’re migrating from GPT-4 Turbo (which capped at 4,096 output), you need to update your client configuration or you’ll silently truncate longer responses.

# Explicitly request the full output budget
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    max_tokens=16384,  # not the default 4096
)

One practical gotcha: OpenAI counts tokens differently for input vs. output. The 128k context window is shared, but the output portion is hard-capped at 16k. If your prompt consumes 120k tokens, you only have 8k left for the completion — the API will return a finish_reason: "length" error rather than truncating silently. Always check usage.completion_tokens against your requested max_tokens to detect this condition.

2. Claude

Claude 3.5 Sonnet and Claude 3 Opus both enforce an 8,192 output token limit. This is half of GPT-4o’s ceiling, but it’s still a substantial increase over Claude 2.1’s 4,096 limit. The total context window is 200,000 tokens for both models, so output represents roughly 4% of the available context.

Anthropic’s API behaves differently from OpenAI’s when you approach the limit. Instead of a hard error, the model will often self-terminate with a stop_reason: "max_tokens" and return whatever it generated up to that point. This is friendlier for streaming use cases — you get a partial response rather than a failed request — but it means your downstream parsing must handle incomplete JSON, cut-off code blocks, or mid-sentence cutoffs.

import anthropic

client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=8192,
    messages=[{"role": "user", "content": prompt}],
)

# Always inspect stop_reason
if message.stop_reason == "max_tokens":
    logger.warning("Claude hit output ceiling; response may be incomplete")

Claude also respects stop_sequences more reliably than the other two families. If you’re generating structured output (JSON, YAML, function calls), define a stop sequence like "\n}\n" or "</function_calls>" to guarantee well-formed termination before the token budget expires. This is a stronger contract than hoping the model closes its own brackets.

3. Gemini

Gemini 1.5 Pro and Gemini 1.5 Flash both support 8,192 output tokens against a 1,000,000 token context window (2M for 1.5 Pro via allowlist). The output ceiling is identical to Claude’s, but the massive context window changes the economics: you can stuff enormous prompts and still have the full 8k budget for the response.

Google’s Generative AI API surfaces the limit through generation_config.max_output_tokens. The default is 8,192, so you don’t need to override it to get the full budget — but you do need to set it lower if you want to constrain costs or latency.

import google.generativeai as genai

model = genai.GenerativeModel("gemini-1.5-pro")
response = model.generate_content(
    prompt,
    generation_config=genai.GenerationConfig(
        max_output_tokens=8192,  # explicit; also the default
        stop_sequences=["</response>"],  # useful for structured output
    ),
)

# Check for truncation
if response.candidates[0].finish_reason == 2:  # MAX_TOKENS
    logger.warning("Gemini hit output ceiling")

Gemini’s finish_reason enum uses integers: 1 = STOP, 2 = MAX_TOKENS, 3 = SAFETY, 4 = RECITATION. The numeric codes are stable but undocumented in the client libraries — check the protobuf definitions if you need exhaustive handling. Like Claude, Gemini returns partial output on MAX_TOKENS rather than raising an exception.

Summary: choosing and guarding against truncation

*Claude’s default varies by SDK; always set explicitly.

Three patterns keep your pipeline robust regardless of which model you route to:

1. Set max_tokens explicitly on every request. Relying on defaults is how you ship a regression when a provider changes their default or you swap models.
2. Inspect the finish reason on every response. Treat max_tokens / length / 2 as a warning, not an error — log it, alert if the rate spikes, and decide whether to retry with a higher budget or a summarization step.
3. Use stop sequences for structured output. A well-chosen stop sequence ("}, </tool_call>, ---END---) guarantees syntactic validity even when the model hits the ceiling. This is cheaper and more reliable than post-hoc repair.

If you’re routing across multiple providers — say, falling back from Claude to GPT-4o when Anthropic is degraded — normalize the finish-reason check in a thin adapter layer so your application logic doesn’t need to know which numeric code means “ran out of tokens.” The max output tokens by model will keep evolving, but the defensive patterns stay the same.