GPT-5 reasoning effort settings for agentic tasks

When building autonomous agents on GPT-5, the gpt-5 reasoning effort parameter is the single biggest lever you have for controlling latency, token spend, and task success. This guide lays out an ordered path to configure that setting per agent step instead of applying one global value, so you can ship agents that are both fast and reliable.

1. Understand What `reasoning_effort` Controls

The parameter tells the model how many internal reasoning tokens to spend before emitting a final answer. It is not a creativity knob like temperature; it directly scales compute inside the model. In the OpenAI-compatible schema it is a string field on the completion request:

from openai import OpenAI

client = OpenAI()  # or point base_url at your gateway
resp = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "Book a flight given these constraints"}],
    reasoning_effort="medium",
)

Values are typically low, medium, high. Low can return in a fraction of the time; high can multiply latency and billed tokens by an order of magnitude because the reasoning trace itself is metered. For agentic loops, that trace is invisible to your app but counts against context and wallet.

2. Profile Your Agent’s Sub-tasks

An agent is not one call. It is a graph of steps, each with different reasoning needs. Before writing any routing code, list the step types your executor runs:

2.1 Planning and branching

Deciding which tools to call, in what order, under partial state. This benefits from high effort. A wrong plan wastes more cycles than the reasoning tokens cost.

2.2 Deterministic tool selection

Given a clear schema and a known entity, picking send_email vs create_calendar_event is pattern matching. Use low.

2.3 Content generation

Drafting a reply, SQL, or a code block. medium is usually enough; bump to high only when the spec is ambiguous.

2.4 Verification

Checking whether a tool output satisfies a precondition. This is where silent failures hide. Keep high here even if generation was low.

3. Implement Per-Step Effort Routing

Hard-coding reasoning_effort="high" at the top of the loop is the most common waste I see in production agents. Map step type to effort explicitly:

EFFORT_MAP = {
    "plan": "high",
    "retrieve": "low",
    "generate": "medium",
    "verify": "high",
    "summarize": "low",
}

def call_gpt5(step: str, messages: list) -> str:
    resp = client.chat.completions.create(
        model="gpt-5",
        messages=messages,
        reasoning_effort=EFFORT_MAP.get(step, "medium"),
    )
    return resp.choices[0].message.content

This single change often cuts p95 latency by half without dropping task completion. The gpt-5 reasoning effort setting is now a per-step contract, not a global guess.

4. Adjust Effort From Runtime Feedback

Static maps are a start, but agents operate on messy state. If a verification step fails, the next generation should think harder. Keep the map mutable:

def run_step(step, messages, ctx):
    effort = ctx.effort_map.get(step, "medium")
    out = call_gpt5_with_effort(step, messages, effort)
    if step == "verify" and not out.passed:
        ctx.effort_map["generate"] = "high"  # escalate on retry
    return out

Do not escalate infinitely. Cap retries and fall back to a human-in-the-loop or a cheaper model after two high-effort failures.

5. Handle Limits and Preserve Cache Hints

High-effort requests are the first to get rate-limited when a provider is degraded. If you front your agents with n4n.ai, its automatic fallback will reroute to a sibling model when a provider throttles, while still forwarding your reasoning_effort directive and any provider cache-control hints you set. That matters because reasoning tokens are not cached the same way as prompt prefixes—misconfigured cache headers quietly inflate cost.

At the raw API level, forward cache control explicitly:

{
  "model": "gpt-5",
  "messages": [{"role": "system", "content": "You are a travel agent."}],
  "reasoning_effort": "low",
  "extra_headers": {"x-cache-control": "ttl=300"}
}

6. Measure Overhead, Not Just Answers

Instrument three numbers per step: wall-clock latency, final token count, and reasoning token count from the usage object.

usage = resp.usage
print(f"step={step} reason={usage.reasoning_tokens} total={usage.total_tokens}")

A high plan step that spends 4k reasoning tokens but prevents two failed retries is cheap. A high summarize step that spends 3k reasoning tokens to compress a paragraph is pure waste. Review these logs weekly; the right gpt-5 reasoning effort distribution drifts as your prompts and tools change.

7. Common Pitfalls

Global high effort. Engineers set it once to “be safe.” Safe for accuracy, lethal for latency and COGS at scale.
Ignoring reasoning token billing. The completion looks short; the invoice is not. Meter per token.
Low effort on multi-hop planning. Two low plan steps in sequence produce incoherent tool chains. Use high at branch points.
Forgetting cache forwarding. If your gateway or client strips cache-control, you re-pay for system prompts every call.
No fallback path. When the provider returns 429 on high, the agent dies. Route or degrade.

8. Recommended Starting Configuration

Follow this ordered path on your next agent build:

Enumerate step types from your executor logs.
Apply the EFFORT_MAP above as a baseline.
Run a small eval set; record reasoning tokens and success rate per step.
Escalate effort only where failures cluster, not uniformly.
Add retry-time escalation with a hard cap.
Forward cache headers and use a gateway with fallback if you operate at volume.
Re-profile every two weeks.

Treat gpt-5 reasoning effort as a tunable resource like thread pool size: allocate where contention is real, starve the rest. Agents that do this stay under latency budgets and still reason when the task actually demands it.

1. Understand What reasoning_effort Controls