Swapping the LLM behind a production agent is routine, but treating its memory as immutable leads to silent corruption. Effective versioning agent state requires decoupling your checkpoint schema from model weights and treating every upgrade as a schema migration event. This guide lays out an ordered path we use to keep agents reproducible across model swaps.
Why naive state breaks on model upgrade
Most agents store conversation history, tool results, and intermediate plans as a JSON blob. That blob implicitly assumes the model that produced it thinks the way the next model does. It doesn’t. A planner tuned on GPT-4 may emit structured dicts; the same prompt on a newer model might emit a different schema or reasoning style. If you load old state into a new model without transformation, you get misparsed actions or hallucinated continuations.
The bug is rarely loud. The agent continues, but its latent assumptions are wrong. You notice only when a tool call fails three steps later.
Define an explicit state schema with versions
Start by giving your checkpoint a mandatory schema_version field. Do not rely on inference from contents. Use a strict serializer (pydantic, dataclasses with validation) so illegal states fail at load.
from dataclasses import dataclass, field
import uuid
@dataclass
class AgentCheckpoint:
schema_version: int = 1
agent_id: str = field(default_factory=lambda: str(uuid.uuid4()))
model_id: str = "gpt-4o-2024-05-13"
messages: list[dict] = field(default_factory=list)
scratch: dict = field(default_factory=dict)
tool_states: dict = field(default_factory=dict)Bump schema_version whenever you change field semantics, not just when you add fields. Adding an optional field is still a version bump if old code can’t interpret it. Versioning agent state starts with refusing to let the shape drift silently.
Separate model-bound artifacts from durable memory
A common mistake is storing raw model outputs (logprobs, hidden reasoning traces) alongside durable facts. Those artifacts are worthless after upgrade and bloat storage. Split state into three buckets:
- Durable: user facts, confirmed tool outputs, long-term goals.
- Model-ephemeral: last raw completion, token logprobs, temperature.
- Routing: which model version produced this state.
Example checkpoint:
{
"schema_version": 2,
"model_id": "claude-3-5-sonnet-20241022",
"durable": {
"user_name": "Ana",
"approved_vendors": ["acme", "globex"]
},
"model_ephemeral": {
"last_prompt_tokens": 1820,
"last_completion_id": "cmpl-9x"
},
"routing": {
"gateway": "direct",
"fallback_used": false
}
}On load, drop model_ephemeral unless you explicitly need it for debugging. Your versioning agent state strategy should keep the durable core stable while letting ephemeral layers rotate freely.
Write migrations, not ad-hoc patches
Never write if version == 1: fix_in_place(). Centralize upgrades in a migration table keyed by (from, to). This makes upgrades testable and reversible in principle.
MIGRATIONS = {
(1, 2): lambda c: {**c, "model_ephemeral": {}, "routing": {"gateway": "direct"}},
(2, 3): lambda c: {**c, "durable": {**c.get("durable", {}), "prefs": c.pop("user_prefs", {})}},
}
def migrate(checkpoint: dict) -> dict:
v = checkpoint["schema_version"]
while v < CURRENT_VERSION:
step = MIGRATIONS.get((v, v + 1))
if not step:
raise ValueError(f"No migration from {v} to {v+1}")
checkpoint = step(checkpoint)
v += 1
checkpoint["schema_version"] = CURRENT_VERSION
return checkpointRun migrate() at agent boot, before the model sees the state. If a migration is missing, fail fast.
Pin and record model identity per checkpoint
Model upgrades are not just “better weights”; they change tokenization, tool-call syntax, and system prompt sensitivity. Your checkpoint must record the exact model id that wrote it. When you later load state, you have two choices:
- Replay on same model (safe, but you stall upgrades).
- Migrate then run on new model (required for cost/quality gains).
If you route through a gateway like n4n.ai, capture the resolved model id from the response metadata and store it in model_id. That lets a replay harness request the identical backend via a client routing directive, even if your default has since moved on.
# pseudo-client call capturing resolved model
resp = gateway.chat.completions.create(
model="auto",
messages=...,
routing={"pin": checkpoint["model_id"]}
)
checkpoint["model_id"] = resp.headers.get("x-resolved-model", checkpoint["model_id"])Without that pin, you cannot reproduce a failure reported on last week’s weights.
Replay tests against frozen state
Before promoting a new model, run a replay suite: load historical checkpoints, migrate them, and execute the next N steps on the candidate model in a sandbox. Assert no exceptions and that durable facts remain unchanged.
python -m pytest tests/replay.py --checkpoint-dir ./fixtures/v2 --model candidate-2025-01In tests/replay.py:
def test_replay_preserves_durable(checkpoint_path, candidate_model):
raw = json.load(open(checkpoint_path))
state = migrate(raw)
agent = Agent(model=candidate_model, state=state)
agent.step(n=3)
assert agent.state["durable"]["user_name"] == "Ana"
assert "approved_vendors" in agent.state["durable"]This catches the silent drift early. If the candidate model rewrites durable incorrectly, the test fails loudly. Solid versioning agent state practice means you never trust a model swap without this replay.
Tradeoffs: storage, latency, complexity
Versioning agent state is not free. You pay in three dimensions:
- Storage: keeping old schema versions and migration code indefinitely. We prune migrations older than two major versions but keep archived snapshots.
- Latency: running
migrate()on every load adds microseconds; negligible versus LLM call, but worth batching if you load thousands of checkpoints per minute. - Complexity: new engineers must understand the migration map. Document it beside the schema.
If your agent is short-lived (single session, no resume), you can skip most of this. The moment you persist state across days or allow model rotation, the overhead is justified.
Common pitfalls we’ve hit
Mixing prompt templates into state. A prompt template is code, not state. Store only a template hash; rebuild the prompt at load. Otherwise a template edit looks like state corruption.
Assuming semantic compatibility. Just because two models both accept OpenAI messages doesn’t mean a tool_call from one parses on the other. Migrate tool schemas explicitly.
Ignoring fallback artifacts. If your gateway fell back to a secondary provider mid-session, the state may contain mixed idioms. Record fallback_used and consider quarantining those checkpoints from auto-upgrade.
No down-migration plan. You may need to roll back a model. Keep the ability to serialize state back to the previous schema version, or at least to export durable facts to a neutral format.
Closing checklist
- Every checkpoint has
schema_versionandmodel_id. - Migrations are pure functions in a central registry.
- Ephemeral model artifacts are separated and droppable.
- Replay tests run on candidate models before rollout.
- Fallback flags captured for audit.
Versioning agent state is less about LLMs and more about disciplined data evolution. Treat your agent’s memory like a database with a schema, and model upgrades become a deploy step instead of an incident.