How to evaluate AI agent memory systems

To evaluate AI agent memory, you need more than a vibe check. A disciplined process to evaluate AI agent memory systems forces you to confront retrieval precision, persistence correctness, and cost tradeoffs before users hit edge cases.

Step 1: Specify memory behaviors and failure modes

Before writing a single test, pin down what the memory system must actually do in production. Separate episodic recall (facts from a specific session) from semantic distillation (long-term user preferences) and procedural memory (agent playbooks or workflows). Each type fails differently, and your evaluation must reflect that.

Write a spec table in the repo. Treat it as a contract that the implementation violates at its own peril.

Engineers often skip this step and jump to cosine similarity. That produces metrics with no bearing on whether the agent will embarrass the user. Define the contract first.

Decide your metrics vocabulary

Choose ahead of time: recall@k for retrieval, exact-match for forced facts, and leakage count for negative cases. Write these into the spec so later steps are unambiguous.

Step 2: Stand up a deterministic harness

Memory systems lean on wall-clock recency and vector randomness. If you don’t freeze both, your evaluate AI agent memory suite will flake. Use pytest with freezegun and a seeded RNG.

# conftest.py
import pytest
from datetime import datetime
from freezegun import freeze_time
import random

@pytest.fixture
def frozen_now():
    with freeze_time("2024-01-01 12:00:00"):
        random.seed(42)
        yield datetime(2024, 1, 1, 12, 0, 0)

Isolate the memory backend. If it’s a vector DB, spin up a local container or an in-memory fake. Never test against production data.

# test_memory.py
import pytest

@pytest.fixture
def mem_store(frozen_now):
    from myagent.memory import VectorMemory
    store = VectorMemory(namespace="test")
    store.clear()
    yield store

A clean fixture per test removes cross-test contamination, which is the silent killer of memory evaluations.

Step 3: Author contrastive test cases

A single happy-path prompt proves nothing. Build a JSONL set with input, expected_recall, and must_not_recall fields. The second field checks positive memory; the third catches leaks.

{"id": "ep1", "input": "My name is Ada", "expected_recall": ["Ada"], "must_not_recall": ["password"]}
{"id": "ep2", "input": "I prefer dark mode", "expected_recall": ["dark mode"], "must_not_recall": ["Ada"]}
{"id": "ep3", "input": "SSN 123-45-6789", "expected_recall": [], "must_not_recall": ["123-45-6789"]}

Load it parametrized:

import json

def load_cases(path):
    with open(path) as f:
        return [json.loads(l) for l in f]

@pytest.mark.parametrize("case", load_cases("tests/memory_cases.jsonl"))
def test_recall(case, mem_store):
    mem_store.ingest(case["input"])
    results = mem_store.query(case["input"])
    for token in case["expected_recall"]:
        assert token in results
    for forbidden in case["must_not_recall"]:
        assert forbidden not in results

This gives you a repeatable way to evaluate AI agent memory against regressions. The must_not_recall field is not optional—privacy failures are the most expensive ones to miss.

Generate variations programmatically

Hand-written cases cover the obvious paths. Fuzz the inputs with simple templating to catch normalization bugs.

def gen_cases():
    bases = ["My name is {n}", "Call me {n}", "I am {n}"]
    for n in ["Ada", "Lin", "Bob"]:
        for b in bases:
            yield {"input": b.format(n=n), "expected_recall": [n], "must_not_recall": []}

Step 4: Measure retrieval quality with offline metrics

Binary pass/fail hides nuance. Compute recall@k and semantic similarity. Use an embedding model via a standard OpenAI-compatible client.

import openai
import numpy as np

client = openai.OpenAI(base_url="https://api.openai.com/v1", api_key="sk-...")

def embed(text):
    resp = client.embeddings.create(model="text-embedding-3-small", input=text)
    return np.array(resp.data[0].embedding)

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

For each case, embed the expected fact and the top retrieved memory. A score above 0.8 typically means the fact is correctly surfaced.

def test_recall_at_k(case, mem_store):
    mem_store.ingest(case["input"])
    hits = mem_store.query(case["input"], k=5)
    emb_expected = embed(case["expected_recall"][0])
    scores = [cosine(emb_expected, embed(h)) for h in hits]
    assert max(scores) > 0.8

When you evaluate AI agent memory at scale, track the distribution of these scores, not just the average. A median of 0.9 with a tail of 0.2 means specific user phrasings break.

Step 5: Run end-to-end agent simulations

Unit tests on the store miss integration bugs. Run the full agent loop with a scripted user. Capture tool calls to the memory API.

from myagent.run import run_turn

def test_agent_recalls_name():
    run_turn("My name is Ada")
    out = run_turn("What's my name?")
    assert "Ada" in out["message"]
    assert out["tool_calls"][-1]["name"] == "memory_query"

If you point the OpenAI client at n4n.ai, you get automatic fallback when a provider is rate-limited and per-token usage metering, which keeps evaluation cost visible across model swaps.

Simulate session boundaries

Create two sessions with different user IDs. Assert cross-session leakage is zero.

def test_session_isolation():
    run_turn("Secret: 42", session="a")
    out = run_turn("What secret?", session="b")
    assert "42" not in out["message"]

Run this 100 times with random secrets. Any single leak fails the build.

Step 6: Profile latency and token overhead

Memory injection inflates context. Measure the delta. Wrap your agent call and log token counts.

import time

def timed_run(prompt):
    start = time.perf_counter()
    resp = run_turn(prompt)
    elapsed = time.perf_counter() - start
    return resp, elapsed, resp["usage"]["total_tokens"]

base, t1, tok1 = timed_run("Hello")
with_mem, t2, tok2 = timed_run("Recall my preferences")
print(f"overhead: {tok2 - tok1} tokens, {t2 - t1:.2f}s")

Set a budget. If memory adds >30% tokens or >200ms p95, revisit embedding size or summarization. Latency regressions are user-visible even when accuracy holds.

Step 7: Gate deploys with thresholds

Put it all in CI. Fail the build if recall@5 drops below 0.9 or any negative case leaks.

# .github/workflows/eval.yml
jobs:
  memory-eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest tests/memory_eval.py --strict-markers

Emit metrics per commit:

with open("metrics.json", "w") as f:
    json.dump({"recall@5": 0.94, "leaks": 0, "avg_overhead_tokens": 120}, f)

Verify success

You have a working evaluation when:

• pytest tests/memory_eval.py is green on a clean checkout with no fixtures shared.
• Recall@5 for curated cases exceeds threshold for three consecutive runs on different days.
• Zero must_not_recall violations across 100 session-isolation simulations.
• Token overhead stays within budget on a representative trace from production logs.

If those hold, you can evaluate AI agent memory changes with confidence and ship updates without fear of silent forgetting.