Chain-of-thought vs ReAct: which prompting style wins

The debate over chain-of-thought vs ReAct prompting isn’t academic—it changes how many tokens you burn and whether your agent can actually call tools. Both patterns shape the control flow between a model and your code, but they optimize for different failure modes. This article compares them on the dimensions that matter when you ship.

What each pattern actually does

Chain-of-thought

Chain-of-thought (CoT) is a prompting technique that forces the model to emit intermediate reasoning steps before the final answer. You either append “Let’s think step by step” or supply few-shot examples with explicit reasoning traces. The model still produces a single completion; there is no loop, no tool call, no external state. It is pure inference over the context you provided.

ReAct

ReAct (Reason + Act) interleaves natural-language reasoning with structured actions. The model outputs a thought, then an action (e.g., search("weather London")), your code executes it, returns an observation, and the model continues. This creates a tight read-eval-print loop owned by your application. ReAct is the conceptual ancestor of today’s native function-calling APIs, but it is traditionally implemented with prompt parsing rather than schema-enforced tool calls.

Head-to-head dimensions

Capabilities

CoT excels at problems solvable entirely inside the model’s parameters and the given context: arithmetic, symbolic logic, multi-hop reading comprehension on provided text. It cannot retrieve missing facts.

ReAct handles tasks that require external state: querying a database, hitting a REST API, reading a file. Because each observation re-grounds the model, it tolerates partial information and can backtrack when a tool returns garbage.

Cost model

CoT adds tokens linearly: every hidden reasoning step is billed output. A 10x longer trace means ~10x output cost on that request, with no extra input beyond the trigger phrase.

ReAct multiplies cost per turn. Each action/observation cycle is a new completion (or a continuation with added context), so you pay for thoughts plus tool I/O plus the growing conversation history. If you route both patterns through a gateway such as n4n.ai, per-token metering makes the CoT tax visible and automatic fallback keeps ReAct loops running when a provider is degraded.

Latency and throughput

CoT is one round trip. Latency scales with output length; a 2,000-token trace blocks the user for the full generation. There is no tail latency from tool calls.

ReAct suffers compound latency. A four-step agent with 800ms median model latency and 200ms tool calls lands at ~4s minimum, before retries. Throughput drops because each step holds a connection open.

Ergonomics

CoT is a one-line prompt change. You can A/B it in minutes.

ReAct demands a parser, a tool registry, observation formatting, and loop termination logic. You own the state machine. Modern function-calling models remove some pain, but the orchestration code is non-trivial.

Ecosystem and tooling

CoT works on every text model back to GPT-3. It needs no libraries.

ReAct patterns are baked into LangChain, LlamaIndex, and most agent frameworks. However, native tool-calling (OpenAI tools, Anthropic tool use) now delivers ReAct’s benefits with structured args and less prompt fragility. Manual ReAct is mostly legacy for models without function support.

Limits

CoT hallucinates when the premise is incomplete; it cannot say “I need to look this up.” It also fails silently on long traces that exceed context.

ReAct loops can spin: the model repeats the same action, or misinterprets an observation and poisons subsequent steps. Error handling in the tool layer becomes a first-class reliability concern.

Comparison table

Implementation sketches

A minimal CoT nudge in an OpenAI-compatible call:

from openai import OpenAI

client = OpenAI()  # base_url can point to any OpenAI-compatible gateway

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a careful tutor."},
        {"role": "user", "content": "A train leaves at 2pm, travels 60mph for 2.5h, then 40mph for 1h. Total distance?"},
        {"role": "user", "content": "Think step by step before answering."}
    ]
)
print(resp.choices[0].message.content)

A bare-bones ReAct loop with manual parsing (illustrative, not production):

import re, requests

tools = {
    "get_price": lambda sym: requests.get(f"https://api.example.com/price/{sym}").json()["price"]
}

prompt = "Thought: {th}\nAction: {act}\nObservation: {obs}\n"

def parse_action(text):
    m = re.search(r"Action:\s*(\w+)\((.*?)\)", text)
    return (m.group(1), m.group(2).strip('"')) if m else (None, None)

history = "Thought: I need the price of AAPL.\n"
for _ in range(5):
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt.format(th=history, act="", obs="")}]
    )
    out = r.choices[0].message.content
    name, arg = parse_action(out)
    if name is None:
        break  # final answer
    obs = tools[name](arg)
    history += out + f"\nObservation: {obs}\n"

The second snippet shows why ReAct is heavier: you must define tools, parse free text, and manage history.

Which to choose

Use chain-of-thought when

• The task is self-contained in the prompt (classification, math on given numbers, code review of pasted snippets).
• You need lowest possible latency and simplest code.
• The model you call lacks reliable function calling.
• Cost per request must stay predictable; you can cap reasoning with max_tokens.

Use ReAct when

• The answer depends on live data, private docs, or computation outside the model.
• You already have a tool layer (APIs, SQL, vector search) and can enforce schemas.
• The task benefits from mid-course correction—e.g., a research agent that refines queries.
• You can absorb the latency and have retry/timeout guards on tools.

Hybrid notes

In practice, many shipped agents use CoT inside a ReAct step: the model reasons briefly before choosing a tool, then reasons again on the observation. If your provider supports native function calling, prefer that over hand-rolled ReAct prompts—it is the same control flow with less string parsing. Reserve explicit chain-of-thought vs ReAct prompting debates for environments where you cannot change models or libraries; otherwise, the gateway and the model’s tool API quietly decide for you.