ReAct vs Chain-of-Thought: what's the difference

ReAct vs chain-of-thought is a tradeoff every engineer hits when moving from a model that answers to a model that acts. Chain-of-thought (CoT) asks the model to think step by step before producing a final answer; ReAct interleaves reasoning traces with concrete tool calls in a loop. The gap shows up in your token bills, your latency budgets, and how gracefully your system degrades when a tool returns garbage.

What each pattern actually does

Chain-of-thought is a prompt structure, not a runtime. You ask the model to emit intermediate reasoning, then the answer. No external state changes, no side effects. The model’s parametric knowledge does all the work.

# Minimal CoT request via OpenAI-compatible chat API
import openai

resp = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Reason step by step, then give final answer."},
        {"role": "user", "content": "If a train leaves at 2pm and travels 60mph, how far in 3h?"}
    ],
    temperature=0
)
print(resp.choices[0].message["content"])

ReAct is a control loop you own. The model outputs a thought, then an action (tool call), you execute it, feed the observation back, and repeat until a final answer. The model sees the world through your tools.

# ReAct loop skeleton
def react_loop(query, max_steps=5):
    messages = [{"role": "user", "content": query}]
    for _ in range(max_steps):
        out = model_generate(messages)
        if out.get("action"):
            obs = execute_tool(out["action"], out["args"])
            messages.append({"role": "assistant", "content": out["thought"] + out["action"]})
            messages.append({"role": "user", "content": f"Observation: {obs}"})
        else:
            return out["answer"]

Reasoning trace format

CoT traces are free text. ReAct formalizes thought/action/observation triples, often with delimiters like Thought:, Action:, Observation:. Modern models with native function calling collapse the delimiters into structured JSON, but the loop semantics stay identical.

Capabilities

CoT excels at arithmetic, logic puzzles, and any task where the bottleneck is the model’s parametric reasoning. On datasets like GSM8K, explicit step-by-step prompting reliably improves accuracy over direct answers. It cannot fetch live data or mutate state.

ReAct extends the model with tools: search, SQL, API calls, code execution. It can answer “what’s the weather in Berlin right now?” because it calls a weather API. The cost is non-determinism from tool outputs—the same prompt can branch differently based on what a search returns.

Price and cost model

Both patterns bill per token. CoT adds reasoning tokens to the prompt and completion. A 100-token question might become 400 tokens with a trace. You pay once.

ReAct multiplies that across steps: each observation is re-fed into context, so context grows linearly with steps. A three-step loop with 300-token observations means the fourth model call carries ~900 extra tokens of history.

If you run on a gateway with per-token usage metering, you can attribute ReAct loop overhead to specific tool calls. That visibility matters when a misconfigured search tool triggers ten retries.

{
  "usage": {
    "prompt_tokens": 1820,
    "completion_tokens": 240,
    "total_tokens": 2060
  }
}

Token accounting example

Assume a base prompt of 200 tokens, each reasoning step emits 80 tokens, each observation adds 120 tokens. CoT: 200 + 320 + answer 50 = 570 tokens. ReAct with 3 steps: step1 (200+80+120), step2 (400+80+120), step3 (600+80+answer 50) = 1550 tokens. The loop costs ~3x for the same underlying task that didn’t need tools.

Latency and throughput

CoT is a single forward pass. Latency equals time-to-first-token plus generation of the trace and answer. ReAct is sequential: step N cannot start until step N-1’s tool returns. A three-step ReAct query has at least three model calls and three tool round-trips.

Throughput for CoT scales with batch size. ReAct loops are inherently serial per query, though you can parallelize independent agents. An inference gateway such as n4n.ai forwards provider cache-control hints, so repeated ReAct context prefixes hit prompt caches and trim step latency. Automatic fallback also prevents a rate-limited provider from stalling the loop.

Ergonomics and implementation

CoT needs a system prompt and maybe a parser to split reasoning from answer. ReAct needs a parser for actions, a tool registry, and a state machine.

// Tool registry snippet
const tools = {
  search: async (q: string) => fetch(`/api/search?q=${q}`).then(r => r.json()),
  calc: (expr: string) => eval(expr) // don't actually do this
};

Debugging ReAct means inspecting each thought/action pair. CoT debugging is just reading the trace. ReAct also forces you to handle tool timeouts, schema drift, and partial observations.

Parsing strategies

For CoT, a regex split on Final answer: works. For ReAct, you either parse delimited text or use native tool schemas. Native schemas reduce parsing errors but couple you to models that support them.

Ecosystem and tooling

CoT is supported everywhere because it’s just text. ReAct has first-class support in LangChain, LlamaIndex, and model-native function calling (OpenAI tools, Anthropic tool use). Many models now emit JSON actions directly, blurring ReAct with native tool calls.

If you use an OpenAI-compatible endpoint addressing 240+ models, you can swap the underlying model without rewriting your ReAct loop, as long as you honor the tool schema.

Limits and failure modes

CoT fails silently on tasks needing external facts. It also over-thinks simple tasks, wasting tokens. ReAct fails loudly: a tool timeout breaks the loop; a malformed action halts execution. Loops can spin if the model repeats the same action. You need max-step guards and observation truncation.

Head-to-head summary

The table compresses the reality: ReAct is strictly more capable but strictly more expensive and fragile. CoT is a subset of behavior you can often embed inside a ReAct step.

Which to choose

Use chain-of-thought when your task is self-contained: math word problems, classification, summarization with provided text, or any batch job where the model’s weights hold the answer. It’s the cheaper, faster, simpler default. You can ship it in an afternoon and monitor token spend with a single meter.

Use ReAct when the answer requires external state: current prices, user-specific DB rows, sending an email. Any agent that must act in the world needs the action interleave. If a human would need to open a browser or query a system to answer, ReAct (or native tool calling) is mandatory.

Hybrid pattern: Many production systems run CoT inside a ReAct step—reason about which tool to call, then call it. That keeps the loop purposeful and avoids reflexive tool use.

If you’re building a customer support bot that queries a ticket API, ReAct is non-negotiable. If you’re building a document classifier, CoT or zero-shot is enough. Engineers often start with ReAct because it feels agentic, then strip back to CoT for 80% of queries that don’t need tools. That’s the right instinct: reach for the loop only when the action is required, and keep the reasoning trace minimal to control cost.