Debugging LangChain memory that forgets earlier context

LangChain memory forgetting context debug sessions usually start the same way: the first few turns work, then the model starts hallucinating or repeating itself because earlier messages silently vanished. The root cause is almost always a mismatch between your memory configuration, the model’s context window, and how you’re passing history back to the chain. This guide walks through the most common failure modes in order of likelihood, with verification steps at each stage.

Step 1: Confirm the memory type matches your use case

LangChain ships with several memory classes, and picking the wrong one is the number one reason context disappears. ConversationBufferMemory stores every message in full — fine for short chats, fatal for long ones. ConversationBufferWindowMemory keeps only the last k turns, discarding everything before that. ConversationSummaryMemory compresses history via an LLM call, which can lose nuance. ConversationSummaryBufferMemory combines both: it summarizes older turns while keeping recent ones verbatim.

Check your instantiation:

from langchain.memory import (
    ConversationBufferMemory,
    ConversationBufferWindowMemory,
    ConversationSummaryMemory,
    ConversationSummaryBufferMemory,
)
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# Problem: window size too small for your conversation length
memory = ConversationBufferWindowMemory(k=3, return_messages=True)

# Better: summary buffer with a token limit tied to your model's context
memory = ConversationSummaryBufferMemory(
    llm=llm,
    max_token_limit=2000,  # adjust for your model's context window
    return_messages=True,
)

Verify: Add a print statement after each turn to inspect memory.chat_memory.messages. Run a 10-turn conversation and confirm the message count matches your expectation. If you see fewer messages than turns, your memory class is truncating.

Step 2: Trace the actual prompt being sent to the model

Memory objects don’t automatically inject history — your chain or prompt template must reference the memory variable. The default variable name is history for string-based memories and chat_history for message-based ones. If your prompt template uses a different key, the history never reaches the model.

Inspect the formatted prompt:

from langchain.chains import ConversationChain
from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{input}"),
])

chain = ConversationChain(
    llm=llm,
    memory=memory,
    prompt=prompt,
    verbose=True,  # prints the full prompt to stdout
)

response = chain.invoke({"input": "What's my name?"})

Run this and watch the console output. You should see the full message list rendered in the prompt. If chat_history appears empty or missing, the prompt template variable name doesn’t match what the memory returns.

Verify: With verbose=True, copy the printed prompt and count the message objects. Compare against len(memory.chat_memory.messages). They must match.

Step 3: Check token counting against the model’s context window

Even with the right memory class, you can exceed the context window. ConversationSummaryBufferMemory uses the LLM’s tokenizer to estimate tokens, but the estimate can drift from the provider’s actual count. OpenAI’s tiktoken is the ground truth for GPT models.

Measure actual token usage:

import tiktoken
from langchain.schema import messages_to_dict

def count_tokens(messages, model="gpt-4o-mini"):
    encoding = tiktoken.encoding_for_model(model)
    # Approximate: each message ~4 tokens overhead + content
    total = 0
    for msg in messages:
        total += 4  # role + formatting overhead
        total += len(encoding.encode(msg.content))
    return total

current_tokens = count_tokens(memory.chat_memory.messages)
print(f"Current history tokens: {current_tokens}")
print(f"Model context window: 128000 (gpt-4o-mini)")
print(f"Available for completion: {128000 - current_tokens}")

If current_tokens approaches the window limit, the API will either truncate silently (older models) or return an error. The memory’s max_token_limit should leave headroom for the system prompt, user input, and expected completion.

Verify: Run a long conversation and log token counts per turn. Plot them — you should see a sawtooth pattern if summarization kicks in, or linear growth if it doesn’t. If tokens exceed the window, increase max_token_limit or switch to a larger-context model.

Step 4: Validate the memory key in your chain or agent

When using Runnable chains or agents, the memory variable name must be declared in the RunnableWithMessageHistory wrapper. A mismatch here is silent — the chain runs, but history is empty.

Correct wiring:

from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import InMemoryChatMessageHistory

store = {}

def get_session_history(session_id: str) -> InMemoryChatMessageHistory:
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]

runnable = prompt | llm

with_message_history = RunnableWithMessageHistory(
    runnable,
    get_session_history,
    input_messages_key="input",      # your prompt's human message variable
    history_messages_key="chat_history",  # must match MessagesPlaceholder variable_name
)

config = {"configurable": {"session_id": "user-123"}}
response = with_message_history.invoke({"input": "Hello"}, config=config)

The history_messages_key must exactly match the variable_name in your MessagesPlaceholder. The input_messages_key must match the key you pass in invoke().

Verify: Add a breakpoint or log inside get_session_history and inspect the returned InMemoryChatMessageHistory.messages after each turn. It should grow by two messages per exchange (human + AI).

Step 5: Test for async or streaming context loss

If you’re using astream or astream_log, the memory update happens after the full response completes. Interleaved streaming calls with the same session ID can race — the second call reads stale history before the first call writes its messages.

Reproduce the race:

import asyncio

async def race_condition():
    # Two concurrent requests, same session
    task1 = with_message_history.ainvoke({"input": "First"}, config=config)
    task2 = with_message_history.ainvoke({"input": "Second"}, config=config)
    await asyncio.gather(task1, task2)
    # Check history — may only have 2 messages instead of 4
    history = get_session_history("user-123")
    print(f"Messages after race: {len(history.messages)}")

asyncio.run(race_condition())

Fix: Serialize calls per session, or use a thread-safe history store (e.g., Redis-backed) with proper locking.

Verify: Run the race condition test. If message count is less than 2 * number_of_calls, you have a concurrency bug. Serialize or lock.

Step 6: Inspect provider-level truncation signals

Some model providers return metadata indicating truncation. OpenAI’s usage.prompt_tokens plus usage.completion_tokens should equal the billed tokens. If prompt_tokens exceeds the model’s context window, the request was rejected or silently truncated.

Capture raw response metadata:

from langchain.callbacks import get_openai_callback

with get_openai_callback() as cb:
    response = chain.invoke({"input": "Summarize our conversation so far"})

print(f"Prompt tokens: {cb.prompt_tokens}")
print(f"Completion tokens: {cb.completion_tokens}")
print(f"Total tokens: {cb.total_tokens}")

If prompt_tokens > context window, the provider truncated. The fix is reducing max_token_limit in your memory or summarizing more aggressively.

Verify: Compare cb.prompt_tokens against your count_tokens() function from Step 3. They should be within ~5%. Large divergence means your token counter is wrong for this model.

Step 7: Check for silent failures in custom memory classes

If you’ve subclassed BaseChatMemory or implemented a custom memory (e.g., backed by Postgres, Redis, or a vector store), the load_memory_variables and save_context methods must be symmetric. A common bug: save_context writes to one key, load_memory_variables reads from another.

Audit your custom memory:

from langchain.memory import BaseChatMemory
from typing import Dict, Any

class PostgresChatMemory(BaseChatMemory):
    def __init__(self, session_id: str, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.session_id = session_id
        # ... DB connection setup ...

    @property
    def memory_variables(self) -> list[str]:
        return ["chat_history"]  # Must match prompt variable_name

    def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, Any]:
        # BUG: returns "history" but memory_variables declares "chat_history"
        messages = self._fetch_from_db()
        return {"history": messages}  # Wrong key!

    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
        self._write_to_db(inputs, outputs)

The key in load_memory_variables return dict must match an entry in memory_variables.

Verify: Unit test your memory class in isolation:

def test_memory_roundtrip():
    mem = PostgresChatMemory(session_id="test")
    mem.save_context({"input": "Hi"}, {"output": "Hello!"})
    loaded = mem.load_memory_variables({})
    assert "chat_history" in loaded
    assert len(loaded["chat_history"]) == 2

Step 8: Verify the full chain with a deterministic test harness

Combine everything into a regression test that catches regressions. Use a fixed prompt, fixed model (or a fake LLM), and assert on message counts and content.

import pytest
from langchain_core.language_models.fake import FakeListLLM

@pytest.fixture
def test_chain():
    fake_llm = FakeListLLM(responses=[
        "Response 1",
        "Response 2",
        "Response 3",
    ])
    memory = ConversationSummaryBufferMemory(
        llm=fake_llm,
        max_token_limit=1000,
        return_messages=True,
    )
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Test bot"),
        MessagesPlaceholder(variable_name="chat_history"),
        ("human", "{input}"),
    ])
    return ConversationChain(llm=fake_llm, memory=memory, prompt=prompt)

def test_memory_persists_across_turns(test_chain):
    for i in range(3):
        test_chain.invoke({"input": f"Message {i}"})
    
    messages = test_chain.memory.chat_memory.messages
    assert len(messages) == 6  # 3 human + 3 AI
    assert messages[0].content == "Message 0"
    assert messages[-1].content == "Response 3"

Run this in CI. If it passes locally but fails in CI, check for environment differences: model version, tokenizer version, or LangChain version.

Step 9: Use a gateway that surfaces provider errors

When the underlying provider returns a 400 (context length exceeded) or 429 (rate limited), some SDKs swallow the error and return a generic failure. A gateway that forwards provider error codes and cache-control hints lets you distinguish “memory too long” from “provider degraded.”

If you’re routing through n4n.ai, the response headers include x-provider-status and x-cache-hint, and the gateway automatically falls back to another provider on 429/5xx — but it won’t silently truncate your context. You still need correct memory configuration; the gateway just makes the failure mode visible.

# Example: inspect headers from a gateway response
curl -v -H "Authorization: Bearer $N4N_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[...]}' \
  https://api.n4n.ai/v1/chat/completions

Look for x-provider-status: 400 with a body mentioning context_length_exceeded. That’s your signal to reduce max_token_limit.

Step 10: Document your memory contract

Write a one-page markdown file in your repo that states:

• Memory class and why
• max_token_limit value and how it was chosen
• Prompt template variable name
• Session store backend and TTL
• Concurrency model (serialized per session / locked / stateless)
• Token counter function and model mapping
• Test command to verify end-to-end

Example:

# Memory Contract — Chat Service

- **Class**: ConversationSummaryBufferMemory
- **Token limit**: 3000 (gpt-4o-mini: 128k context, leave 125k for prompt+completion)
- **Prompt var**: chat_history (MessagesPlaceholder)
- **Store**: Redis, TTL 24h, key prefix "chat:mem:"
- **Concurrency**: Per-session mutex via Redis SETNX
- **Tokenizer**: tiktoken encoding_for_model("gpt-4o-mini")
- **Verify**: `pytest tests/test_memory_contract.py -v`

When a new engineer asks “why did the bot forget my name?”, point them here. When you upgrade models, update the token limit and re-run the verification test.

Quick reference checklist

Run through these steps in order. Most “LangChain memory forgetting context debug” sessions resolve at Step 1, 2, or 3. The later steps catch the edge cases that survive the first pass.