Agent latency benchmark: ReAct vs plan-and-execute

The core tradeoff in agentic design shows up the moment you measure ReAct vs plan-and-execute latency under real task loads. ReAct interleaves reasoning and tool calls in a tight loop; plan-and-execute separates a single planning pass from a batch of execution steps. Both patterns solve tasks, but they tax your inference budget and tail latency very differently.

How the patterns work

ReAct loop

ReAct (Reason + Act) prompts the model to emit a thought, then an action, then observes the result, repeating until a final answer. Each iteration is a full chat completion call with the entire conversation history appended.

from openai import OpenAI
client = OpenAI()  # or any OpenAI-compatible endpoint

messages = [{"role": "system", "content": "You are a ReAct agent. Think, then call tools."}]
task = "Book a flight under $300 and email the itinerary"
messages.append({"role": "user", "content": task})

while True:
    resp = client.chat.completions.create(model="gpt-4o", messages=messages)
    text = resp.choices[0].message.content
    messages.append({"role": "assistant", "content": text})
    if "Final Answer:" in text:
        break
    action, arg = parse_action(text)
    obs = tool_run(action, arg)  # blocks
    messages.append({"role": "user", "content": f"Observation: {obs}"})

Plan-and-execute

Plan-and-execute asks the model to emit a structured plan (often JSON), then runs each step, optionally in parallel. The planner can be a stronger model; executors can be smaller and cheaper.

plan_resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "system", "content": "Output a JSON list of step objects."},
              {"role": "user", "content": task}]
)
steps = json.loads(plan_resp.choices[0].message.content)

# Sequential or parallel execution
for step in steps:
    exec_resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": step["instruction"]}]
    )
    step["result"] = exec_resp.choices[0].message.content

Capabilities

ReAct adapts mid-task. If a tool returns an unexpected error, the next thought can pivot. That makes it strong for open-ended retrieval or multi-hop questions where the path is unknown.

Plan-and-execute commits to a DAG up front. It excels when the task is decomposable and the steps are independent: data pipeline orchestration, batch document processing, or any workflow with a known shape. It cannot naturally recover from a bad plan without an explicit re-plan branch.

Cost model

ReAct resends the growing transcript on every call. A 10-step task with 2k-token thoughts and 1k-token observations per step means the final call sends ~30k tokens of context even if the step itself needs little reasoning. You pay premium pricing for the strong model on every hop.

Plan-and-execute pays once for the plan, then each step is a narrow context. You can route execution to a cheaper model (e.g., gpt-4o-mini or a local 7B) because the planner already did the hard reasoning. The tradeoff: if the plan is wrong, you may waste all executor calls.

Latency and throughput

ReAct vs plan-and-execute latency is dominated by serialization. ReAct is strictly sequential: step N cannot start until step N-1’s observation returns. A 5-step task on a 800ms p50 provider means ~4s of pure inference latency plus tool time.

Plan-and-execute adds a head-of-line planning call (same ~800ms), but independent steps can be fanned out. With 4 parallel executors, a 5-step plan finishes in roughly plan + max(step) instead of plan + sum(step). Throughput per task drops sharply for ReAct under concurrency limits because each agent holds a session open.

When running these loops against a gateway such as n4n.ai, the OpenAI-compatible endpoint forwards provider cache-control hints and automatically fails over if a provider is rate-limited, which matters because ReAct’s tight loop amplifies transient 429s into user-visible stalls.

Ergonomics

ReAct is forgiving to write. One prompt, a parser, and a tool dispatch. Debugging is painful: the trace is a linear wall of text, and a single bad thought cascades.

Plan-and-execute forces you to define a schema. That schema is a contract you can validate, unit test, and render as a DAG in a UI. Refactors are easier because the executor is decoupled from the planner. The downside is upfront design: you must anticipate step boundaries.

Ecosystem

LangChain ships both AgentExecutor (ReAct-style) and PlanAndExecute agent. LangGraph lets you model plan-and-execute as a graph with a planner node and a map-reduce executor. LlamaIndex has SubQuestionQueryEngine which is effectively plan-and-execute for RAG. AutoGen defaults to conversational ReAct but supports user-defined control flows.

None of these frameworks solve the latency equation for you; they just package the loops above.

Limits

ReAct blows up context windows on long tasks. It also tends to over-call tools (“loop of confusion”) when the model fixates. Plan-and-execute suffers from plan drift: if step 3 depends on step 2’s output but the planner assumed otherwise, you need a re-plan trigger or the whole run fails. Both patterns are vulnerable to provider degradation, but ReAct’s per-step dependency makes it more fragile.

Head-to-head table

Which to choose

Interactive assistants and ambiguous queries. Use ReAct when the user is in the loop and the problem shape is unknown. A support bot that decides whether to refund, escalate, or lookup order history benefits from mid-course correction. Keep the tool set small and cap steps at 8 to avoid context blowup.

Batch jobs and known workflows. Use plan-and-execute for ETL-like agentic tasks: “summarize these 500 tickets and file bugs.” The planner builds the list; executors run in parallel on a queue. Route execution to a smaller model and reserve the large model for planning.

Hybrid. For production systems, start with plan-and-execute and add a ReAct “repair” node that only triggers when a step fails validation. This captures most of the latency win while keeping recovery cheap.

Latency-sensitive UX. If p95 under 2s is a requirement, ReAct will fail on anything beyond 2 steps on commodity models. Plan-and-execute with parallel executors is the only pattern that fits, provided the planner can return in one shot.

Pick the pattern by measuring your own task distribution, not by benchmark blog posts. Instrument token counts per step and watch the tail.