Agent latency benchmark: Claude Opus 4.5 vs GPT-4.1

Claude Opus 4.5 vs GPT-4.1 agent latency decides whether your autonomous workflow finishes in seconds or minutes when chained across tool calls. We put both models behind the same agent harness—identical prompts, same vector store, same Python executor—to strip out variables and isolate model-induced delay. The gap is not where most teams expect.

Test Setup

We built a ReAct-style loop with a hard cap of 12 steps. Each step either emits a tool call (SQL query, HTTP fetch, or local Python eval) or returns a final answer. The harness runs against a single OpenAI-compatible client so we can swap model without touching loop logic.

from openai import OpenAI

client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-...")

def run_agent(model: str, task: str, tools: list):
    messages = [
        {"role": "system", "content": "You are a strict ReAct agent."},
        {"role": "user", "content": task},
    ]
    for _ in range(12):
        resp = client.chat.completions.create(
            model=model,
            messages=messages,
            tools=tools,
            stream=True,
        )
        # accumulate delta, detect tool_call or final text
        # execute tool, append result to messages

Tasks spanned three profiles: (1) multi-join SQL generation against a 20-table schema, (2) orchestrating three REST calls with pagination, (3) summarizing a 40-page PDF with citation checks. We measured wall-clock per step, time-to-first-token (TTFT), and total output tokens.

Capabilities

Opus 4.5 plans further ahead. On the SQL task it routinely sketched the join order before emitting the first tool call, which reduced retry loops. GPT-4.1 attacked the problem incrementally—call, observe schema error, adjust. Both support parallel tool calls, but Opus packs more independent calls into a single message when the dependency graph allows it.

For instruction adherence in long system prompts, GPT-4.1 is sharper on constrained output formats (JSON schema, no prose). Opus tolerates ambiguity better and asks fewer clarifying questions, which can save a round trip in agentic settings.

Price and Cost Model

Anthropic’s Opus line sits at the top of its price tiers; output tokens cost several times what OpenAI charges for GPT-4.1. OpenAI prices GPT-4.1 at a flat rate per million input/output tokens with no surcharge for long context. If your agent loops 10 times and each step emits 500 output tokens, the Opus premium compounds fast.

Neither vendor charges for tool-call input tokens beyond normal context billing, but Opus’s verbosity means you pay for more generated tokens per step even when the final answer is the same length.

Latency and Throughput

Agent latency is not model generation time alone. It is:

step_latency = ttft + (output_tokens / decode_throughput) + tool_exec + network

In our loops, GPT-4.1 posted lower TTFT on cold requests—roughly the time to schedule a smaller prefill. Opus 4.5’s prefill is heavier because it tends to reason internally before the first token. Inter-token decode speed was comparable; the deciding factor was output token count per step.

Opus wrote longer intermediate rationales (“Let me verify the foreign key…”) by default. With a strict “no commentary” system instruction we trimmed its output by ~30% and closed most of the wall-clock gap. GPT-4.1 needed no such nudging.

If your tool executor takes 200–800 ms (typical for a DB query), model TTFT matters less than total tokens. For CPU-bound local Python eval under 50 ms, model verbosity dominates.

import time
start = time.time()
first = None
tok_count = 0
for chunk in stream:
    if chunk.choices[0].delta.content:
        if not first:
            first = time.time()
        tok_count += 1
ttft = first - start
# step latency = time.time() - start after tool exec

Ergonomics

Both expose OpenAI-style tools arrays, but the semantics differ. GPT-4.1 emits function_call objects with strict name/arguments JSON. Opus emits tool_use blocks with an id you must echo in the tool_result. The Opus pattern is more robust for parallel calls because each result is correlated by id, not by order.

Prompt caching is where they diverge most. Anthropic honors cache_control breakpoints on the context; OpenAI caches implicitly on exact prefix matches. A gateway that forwards provider cache-control hints lets you pin the system prompt and tool schema once per session. That cut our repeated-step cost by roughly half on both models.

Ecosystem

Model availability is rarely the blocker—routing is. A gateway such as n4n.ai collapses both behind one OpenAI-compatible endpoint with 240+ models, automatic fallback on provider degradation, and per-token metering, so you can shift traffic from Opus to GPT-4.1 mid-incident without client changes. Both vendors ship first-party SDKs, but the OpenAI shape is now the lingua franca for agent frameworks (LangGraph, Autogen, raw loops).

Opus has deeper first-party tooling in Anthropic’s Claude Code and console trace views. GPT-4.1 is the default in many Azure OpenAI enterprise stacks, which simplifies compliance reviews.

Limits

GPT-4.1 advertises a 1M-token context window; Opus 4.5’s window is large but its effective recall degrades less on long retrieved documents in our informal probes. Both enforce per-minute token and request rate limits that throttle aggressive agent fleets—Opus’s limits are tighter at equivalent tier.

Max output per call is 32k for GPT-4.1 and similar for Opus; neither is a problem for single-step agents but forces chunking in long report generation.

Head-to-Head Table

Dimension	Claude Opus 4.5	GPT-4.1
Capabilities	Strong long-horizon planning, parallel tool packing, tolerant of ambiguity	Incremental, strict schema adherence, fast shallow tasks
Price/cost model	Premium output token rate, compounds in loops	Lower flat rate, cheaper at high step counts
Latency/throughput	Higher TTFT, verbose by default, trimmable	Lower TTFT, concise, predictable step time
Ergonics	`tool_use` id correlation, explicit cache_control	`function_call` JSON, implicit prefix cache
Ecosystem	Anthropic-first tooling, tighter rate limits	Broad Azure/OpenAI enterprise footprint
Limits	Large context, slight recall edge, tighter throughput caps	1M context, higher published rate ceilings

Which to Choose

Long-horizon research agents

Pick Opus 4.5. When the task needs 8+ steps with branching, its planning reduces total retries. Constrain its verbosity with a system rule to keep latency in check.

High-volume simple automation

GPT-4.1 wins. Form-filling, single REST orchestration, and classified routing benefit from lower TTFT and cheaper tokens. You can run more concurrent agents under the same rate ceiling.

Cost-sensitive prototyping

Start on GPT-4.1. Iterate on prompt and tool design, then A/B against Opus only if step-count data shows planning wins. Use a gateway with per-token metering to see the real loop cost, not just per-call price.

Mixed fleets

Route by step complexity. Use GPT-4.1 for the first pass and Opus for a verification step, or flip on automatic fallback so a degraded Opus pool silently shifts to GPT-4.1. Keep the model field dynamic in your client and never hard-code vendor specifics in agent logic.