Chain-of-thought prompting on GPT-5 vs Claude

Chain-of-thought prompting on GPT-5 and Claude 3.5 reveals fundamentally different reasoning architectures. OpenAI’s models excel at structured decomposition and verifiable step-by-step logic, while Anthropic’s models favor naturalistic reasoning with stronger instruction following in multi-turn contexts. Understanding these differences determines whether your agent pipeline spends tokens on explicit reasoning traces or implicit model behavior.

Reasoning architecture and transparency

GPT-5 (building on o1/o3 patterns) exposes reasoning as a first-class output. The model emits a distinct reasoning phase before the final answer, separable via the reasoning parameter in the Responses API. You can request reasoning.effort: "high" for complex problems and inspect the trace for debugging. This makes chain of thought gpt-5 claude comparisons concrete — you literally see the steps.

{
  "model": "gpt-5",
  "input": "Solve: 3x^2 + 7x - 6 = 0",
  "reasoning": { "effort": "high", "summary": "detailed" }
}

Claude 3.5 Sonnet and Opus don’t expose a separate reasoning channel. Instead, they interleave reasoning naturally within the response when prompted. The canonical pattern remains Let's think step by step or structured XML blocks:

<reasoning>
First, identify the quadratic coefficients: a=3, b=7, c=-6.
Discriminant: b^2 - 4ac = 49 - 4(3)(-6) = 49 + 72 = 121.
Square root of 121 is 11.
Roots: (-7 ± 11) / 6 → x = 2/3 or x = -3.
</reasoning>
<answer>x = 2/3 or x = -3</answer>

The practical difference: GPT-5 lets you audit and control reasoning compute independently. Claude requires prompt engineering to elicit comparable depth, but the output feels more conversational — better for user-facing chat where you don’t want a “thinking” preamble.

Tool use and agentic workflows

Both families support function calling, but the interaction with chain-of-thought differs sharply.

GPT-5’s reasoning phase can include tool calls. The model plans, calls tools, incorporates results, and continues reasoning — all before emitting the final answer. This enables multi-hop tool use without explicit orchestration:

# GPT-5 Responses API - tool calls inside reasoning
response = client.responses.create(
    model="gpt-5",
    input="Find the current price of AAPL and calculate 5-day SMA",
    tools=[{"type": "function", "function": get_stock_price}],
    reasoning={"effort": "high"}
)
# response.reasoning contains tool call/result pairs

Claude 3.5 handles tools in the standard message loop. You get a tool_use block, execute, return tool_result, and the model continues. For chain-of-thought prompting, you must explicitly structure the prompt to reason before and after tool calls:

# Claude - explicit reasoning around tools
messages = [
    {"role": "user", "content": """Think step by step:
    1. What data do I need?
    2. Call tools to get it.
    3. Reason with the results.
    4. Final answer."""},
    {"role": "assistant", "content": "I need AAPL price history. Calling get_stock_price..."},
    # tool_use/tool_result exchange happens here
]

GPT-5 wins for autonomous agents where you want the model to self-correct across tool calls. Claude wins when you need deterministic control over each step — common in regulated workflows where every tool call must be audited.

Latency and throughput trade-offs

GPT-5’s reasoning tokens add latency linearly with reasoning.effort. At high, expect 3-8s for complex prompts; low drops to 1-2s but sacrifices depth. Throughput scales inversely — high-effort requests consume more compute quota.

Claude 3.5 Sonnet averages 800-1200ms for chain-of-thought prompts (including the reasoning tokens in-output). Opus runs 2-3x slower. No separate reasoning budget exists — all tokens count against the same output limit.

# Rough latency comparison (p50, 2k input / 500 output tokens)
# GPT-5 reasoning=low:    ~1.2s
# GPT-5 reasoning=high:   ~4.5s
# Claude 3.5 Sonnet:      ~0.9s
# Claude 3.5 Opus:        ~2.8s

For high-volume pipelines, Sonnet’s consistent sub-second latency often beats GPT-5’s variable reasoning time. If you route via a gateway like n4n.ai, automatic fallback to Sonnet when GPT-5 reasoning queues back up keeps p99 latency bounded.

Cost model implications

GPT-5 prices reasoning tokens at the same rate as output tokens (or a published multiplier). A high-effort request generating 2k reasoning + 500 output tokens costs 2.5x a standard call. Budget predictability requires capping reasoning.max_tokens.

Claude includes all reasoning in standard output tokens. No surprise multiplier, but long reasoning traces consume your output budget. Sonnet at $3/15 per MTok (input/output) remains cheaper than GPT-5’s projected pricing for equivalent reasoning depth.

# Cost estimation helper
def estimate_gpt5_cost(input_tokens, reasoning_tokens, output_tokens, 
                       input_price=5.00, output_price=15.00):
    # prices per 1M tokens (hypothetical)
    return (input_tokens * input_price + 
            (reasoning_tokens + output_tokens) * output_price) / 1_000_000

def estimate_claude_cost(input_tokens, output_tokens,
                         input_price=3.00, output_price=15.00):
    return (input_tokens * input_price + output_tokens * output_price) / 1_000_000

For workloads with predictable reasoning depth (code review, SQL generation), GPT-5’s explicit budgeting wins. For open-ended analysis where reasoning length varies wildly, Claude’s flat rate is safer.

Context handling and long-horizon reasoning

GPT-5 supports 256k context with reasoning tokens counting against the limit. The reasoning summary feature (reasoning.summary: "auto") compresses long traces for downstream passes, useful in multi-turn agents.

Claude 3.5 offers 200k context. Its strength is in-context reasoning — the model maintains coherent reasoning across very long conversations without degradation. GPT-5’s separate reasoning channel can feel disjointed in extended dialogues unless you explicitly feed summaries back.

# Multi-turn with GPT-5: feed reasoning summary back
prev_response = client.responses.create(...)
next_input = [
    {"role": "user", "content": "Now optimize this query"},
    {"role": "assistant", "content": prev_response.reasoning.summary}
]

Claude simply continues the conversation. For coding agents spanning 50+ turns, this often produces more coherent results with less orchestration code.

Instruction following and format adherence

Claude 3.5 Sonnet/Opus lead on strict format adherence. Prompt Output ONLY valid JSON matching this schema and you get valid JSON. GPT-5 follows instructions well but occasionally wraps output in markdown fences or adds conversational filler unless you use response_format: { "type": "json_schema" } (Responses API) or strict: true (Chat Completions).

For chain-of-thought prompting where the reasoning is the structured output (e.g., emitting a reasoning trace for audit logs), Claude’s natural XML/JSON compliance reduces post-processing.

// Claude - reliable structured reasoning output
{
  "reasoning_steps": [
    {"step": 1, "action": "identify_variables", "result": "a=3,b=7,c=-6"},
    {"step": 2, "action": "compute_discriminant", "result": "121"}
  ],
  "final_answer": "x = 2/3 or x = -3"
}

Ecosystem and tooling maturity

OpenAI’s Responses API, reasoning parameter, and built-in tool support (code interpreter, file search, web search) make GPT-5 plug-and-play for agent frameworks. The SDK handles streaming reasoning deltas natively.

Anthropic’s SDK is mature but lower-level. You build the reasoning loop yourself. Frameworks like LangGraph, CrewAI, and Instructor support both, but GPT-5’s native reasoning primitives reduce framework code.

# GPT-5 streaming reasoning (Responses API)
stream = client.responses.create(
    model="gpt-5",
    input="Complex problem...",
    reasoning={"effort": "high"},
    stream=True
)
for event in stream:
    if event.type == "response.reasoning_summary_text.delta":
        print(event.delta, end="", flush=True)

Limits and quotas

GPT-5 enforces per-model reasoning token limits (configurable via reasoning.max_tokens). Hitting the cap truncates reasoning silently — monitor response.reasoning.summary for truncation markers.

Claude enforces output token limits (4k for Sonnet, 8k for Opus by default, raisable to 128k). Long chain-of-thought traces can hit this ceiling mid-reasoning. The model usually detects this and compresses, but you lose the full trace.

Which to choose

Choose GPT-5 when:

• Building autonomous agents that self-correct across tool calls (coding agents, research assistants)
• You need auditable reasoning traces for compliance or debugging
• Workloads have predictable reasoning depth — budget reasoning.max_tokens and move on
• You want native code interpreter, file search, or web search without orchestration glue
• Latency variance is acceptable (batch, async, or human-in-the-loop)

Choose Claude 3.5 Sonnet when:

• Latency consistency matters — user-facing chat, real-time copilots
• Cost predictability for variable-depth reasoning (open-ended analysis, creative tasks)
• Strict output format adherence without response_format boilerplate
• Long multi-turn conversations where reasoning coherence degrades less
• You prefer prompt-based control over API parameters

Choose Claude 3.5 Opus when:

• Maximum reasoning depth on genuinely hard problems (novel math, complex architecture design)
• You can absorb 2-3x latency and cost for Sonnet
• The task benefits from Opus’s stronger instruction following on ambiguous prompts

Hybrid strategy (production reality): Route by task type. Classification → Sonnet. Multi-hop tool use → GPT-5. Open-ended analysis → Sonnet, fall back to Opus on low-confidence. A gateway handling this routing — with per-token metering and automatic fallback when a provider degrades — keeps the pipeline honest.