6 state management mistakes that break production agents

Most production incidents with LLMs trace back to agent state management mistakes: teams bolt a chatbot onto a stateless API and assume the conversation is the only state that matters. In reality, an agent’s world includes tool outputs, intermediate plans, retry counters, and partial failures. Get the state model wrong and you get non-deterministic behavior, corrupted memory, and agents that cannot resume after a crash.

1. Keeping agent state only in process memory

The fastest way to lose an agent’s mind is to store its working set in a Python dictionary on a single worker. When the process restarts, the autoscaler kills the pod, or you deploy a new version, every in-flight conversation vanishes. Users see the agent “forget” what it was doing mid-task.

This breaks horizontally scaled deployments. If request A hits worker 1 and request B hits worker 2, they cannot share context unless state lives in a shared store. Even a single-instance setup is fragile: an OOM kill discards everything.

# Anti-pattern
agent_state = {}

@app.post("/chat")
def chat(req: dict):
    agent_state[req["id"]] = req["msg"]  # gone on restart
    return {"echo": agent_state[req["id"]]}

Use Redis, Postgres, or DynamoDB with a stable key per agent run. Persist after every meaningful transition, not just at the end.

import redis
r = redis.Redis()

def save_state(run_id: str, state: dict):
    r.set(f"agent:{run_id}", json.dumps(state), ex=3600)

2. Mutating state in place without immutable checkpoints

A second class of agent state management mistakes is treating state as a mutable blob that you overwrite on each step. When something goes wrong, you have no history to replay or debug. You cannot answer “what did the agent know before it called the refund tool?”

Append-only event logs fix this. Each step emits an event; the current state is a reduction over the log. You get auditability and the ability to fork an agent’s trajectory for testing.

# Event-sourced state
events = []

def apply_event(state, event):
    events.append(event)
    if event["type"] == "tool_result":
        state["tools"][event["id"]] = event["data"]
    return state

# state is derived, never mutated in place without recording

Version the schema. If you change how plans are stored, old events must still reduce correctly. A simple schema_version field on each event prevents silent corruption.

3. Assuming tool calls are idempotent and not recording side effects

Agents invoke APIs: charge a card, send an email, provision a VM. Networks fail. The LLM loop retries. If you do not record that a tool already executed, you double-send the email. State must store the tool’s deterministic key and result before the agent proceeds.

Compute a hash of the tool name plus normalized arguments. Check the state store before calling.

import hashlib

def tool_key(name, args):
    payload = f"{name}:{json.dumps(args, sort_keys=True)}"
    return hashlib.sha256(payload.encode()).hexdigest()

def execute_tool(state, name, args):
    key = tool_key(name, args)
    if key in state["executed_tools"]:
        return state["executed_tools"][key]  # cached result
    result = real_api_call(name, args)
    state["executed_tools"][key] = result
    return result

This also makes replays safe. If the agent crashes after the call but before persisting, the retry will see the key missing and re-execute—so persist the result to durable storage immediately after the call, not later.

4. Tightly coupling state shape to a single LLM provider’s API

Storing raw provider responses in your state is a latent outage. When you switch from one model vendor to another, the JSON shape changes: choices[0].message vs content[0].text, different finish-reason enums, different token usage fields. Your replay and evaluation code breaks.

Normalize at the boundary. Keep a provider-agnostic message list in state and store the raw response separately if needed for debugging.

def normalize_completion(raw, provider):
    if provider == "openai":
        return {"role": "assistant", "content": raw["choices"][0]["message"]["content"]}
    elif provider == "anthropic":
        return {"role": "assistant", "content": raw["content"][0]["text"]}
    # n4n.ai exposes an OpenAI-compatible endpoint across 240+ models with
    # automatic fallback, but your state layer must still normalize regardless
    # of which backend served the token.

If you use a gateway that honors client routing directives and forwards provider cache-control hints, treat those hints as ephemeral metadata. Do not let them leak into the core state schema.

5. Skipping checkpointing for long-running agent loops

An agent that runs for 50 steps to book a trip will eventually hit a timeout, a rate limit, or a worker eviction. Without checkpoints, you restart from zero and burn tokens re-deriving the same plan. Worse, the user perceives a hang.

Persist a checkpoint after each model round-trip. Include the step number, the accumulated state, and a resume cursor.

def checkpoint(run_id, step, state):
    db.execute(
        "INSERT INTO agent_checkpoints(run_id, step, state, ts) VALUES (?,?,?,now())",
        (run_id, step, json.dumps(state)),
    )

On boot, the agent queries the latest checkpoint and resumes. This turns a 30-minute job into a resumable workflow. It also gives you a natural place to inject human approval: stop at step N, wait for a signal, continue.

6. Writing unstructured logs into state instead of structured events

Engineers often dump print output or full exception traces into the state blob “for later debugging.” The store balloons, queries slow, and secrets from tool responses end up in plain text. State is not a log file.

Define a strict event schema. Each entry has type, ts, data. Redact fields at write time.

{
  "type": "tool_error",
  "ts": "2025-04-12T10:22:01Z",
  "data": {"tool": "stripe_charge", "error": "rate_limited", "retry_in": 30}
}

Separate the operational log (for observability) from the agent’s decision state (for resumption). The former can go to Elasticsearch; the latter stays in your fast key-value store.

Summary

Avoiding these agent state management mistakes is mostly discipline: treat state as a first-class product surface, not a debugging afterthought. The agents that survive production are the ones whose state can be inspected, replayed, and resumed by a human at 3 a.m.