LangChain memory types buffer vs summary represent two foundational strategies for managing conversational state in LLM apps: one keeps the entire raw message history, while the other maintains a continuously condensed abstract. Understanding the mechanics of each is mandatory before you ship a chat feature that survives past ten turns.
What conversational memory actually is in LangChain
In LangChain, memory is a pluggable module that reads and writes state between chain invocations. It does not magically persist across processes; you instantiate it, feed it save_context, and inject its load_memory_variables output into your prompt. The two classic implementations—ConversationBufferMemory and ConversationSummaryMemory—differ only in what they store and how they transform history before it reaches the model.
The langchain memory types buffer vs summary decision impacts your token bill, your latency profile, and the fidelity of what the model “remembers.” Everything else is configuration.
ConversationBufferMemory: the verbatim transcript
How it works
ConversationBufferMemory appends every human and AI message to an in-memory list and serializes the whole list into a string when loaded. Nothing is compressed. The following snippet builds a buffer, records two turns, and prints the injected variables:
from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory()
memory.save_context(
{"input": "What is the refund policy?"},
{"output": "Refunds are issued within 30 days."}
)
memory.save_context(
{"input": "Do you offer store credit?"},
{"output": "Yes, store credit is available on request."}
)
vars = memory.load_memory_variables({})
print(vars["history"])The printed history contains both exchanges in order. The buffer grows linearly with conversation length.
Token accounting
If your prompt template includes {history}, every token from every past message is resent on each call. A 20-turn support chat with 150 tokens per turn becomes 3,000 tokens of repetition per request. That is wasteful but lossless. There is no secondary LLM call, so the only compute cost is the enlarged context window on the main request.
ConversationSummaryMemory: the rolling abstract
How it works
ConversationSummaryMemory uses a separate LLM call to compress prior context into a prose summary, then prepends new exchanges to that summary and re-summarizes. You must pass an llm instance:
from langchain.memory import ConversationSummaryMemory
from langchain.llms import OpenAI
summary_memory = ConversationSummaryMemory(llm=OpenAI(model="gpt-3.5-turbo-instruct"))
summary_memory.save_context(
{"input": "What is the refund policy?"},
{"output": "Refunds are issued within 30 days."}
)
summary_memory.save_context(
{"input": "Do you offer store credit?"},
{"output": "Yes, store credit is available on request."}
)
print(summary_memory.load_memory_variables({})["history"])The first save_context creates an initial summary. The second triggers a summarization prompt that folds the new exchange into the existing text. The stored string stays roughly bounded by the summarizer’s output length, not the raw transcript.
Token accounting
You pay twice: once for the summarization call (input = old summary + new exchange, output = new summary) and once for the main inference that consumes the summary. Over a long session, total tokens are far lower than the buffer approach, but each summarization step adds latency and a small chance of drift.
Why the difference matters in production
The tradeoff is fidelity versus cost. Buffer memory preserves exact wording, which matters when the model must quote a user’s earlier JSON snippet or exact error message. Summary memory sacrifices that precision for a stable context size. If you route through an inference gateway such as n4n.ai, per-token usage metering makes the cost delta between replaying a full buffer and paying for periodic summarization explicit in your billing, so the choice is not abstract.
Latency is the other axis. A 10k-token buffer adds ~10k tokens of prefill to every request. Summary memory caps prefill but inserts a synchronous summarization round-trip after each turn (unless you batch it async).
Token math: a worked example
Assume each exchange is 100 tokens. After 30 turns:
- Buffer: history is 3,000 tokens. Every subsequent request sends all 3,000 tokens plus the new input. No extra calls.
- Summary: suppose the summary stabilizes at 200 tokens. Each request sends 200 + new input. After each turn, a summarization call consumes ~300 input and returns ~200 output. Total summarization tokens over 30 turns is roughly 9,000, but main-call history tokens drop from 90,000 (buffer) to 6,000.
The crossover point depends on model pricing and session length, but the curve is unambiguous: buffer cost grows unbounded; summary cost converges.
Concrete example: a support bot with swappable memory
Below is a minimal LLMChain that uses either memory type based on a flag. This pattern lets you A/B the experience without rewriting prompts.
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI
from langchain.memory import ConversationBufferMemory, ConversationSummaryMemory
template = """Support agent. Context:
{history}
Human: {input}
AI:"""
prompt = PromptTemplate(input_variables=["history", "input"], template=template)
llm = OpenAI()
def build_chain(use_summary: bool):
if use_summary:
mem = ConversationSummaryMemory(llm=llm, prompt=prompt)
else:
mem = ConversationBufferMemory()
return LLMChain(llm=llm, prompt=prompt, memory=mem)
chain = build_chain(use_summary=False)
chain.predict(input="How do I return an item?")Swap use_summary=True and the same chain now maintains a summary instead of a transcript. The prompt template does not change; only the history variable’s shape does.
Integrating with modern LCEL pipelines
The legacy LLMChain is convenient but the current LangChain core favors RunnableWithMessageHistory. You can adapt buffer or summary memory by wrapping a message history store. However, ConversationSummaryMemory does not implement the BaseChatMessageHistory interface directly; you typically use a custom BaseChatMessageHistory that calls the summarizer, or use the older ConversationChain which natively supports both memory types. This mismatch trips up engineers who assume memory classes are drop-in for get_session_history. In practice, for summary behavior in LCEL you either subclass InMemoryChatMessageHistory to periodically condense, or keep using ConversationChain.
Example of configuring a buffer-backed runnable:
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import InMemoryChatMessageHistory
store = {}
def get_history(session_id):
if session_id not in store:
store[session_id] = InMemoryChatMessageHistory()
return store[session_id]
runnable = RunnableWithMessageHistory(chain, get_history)This uses raw message lists (buffer semantics). To get summary semantics you must post-process the history before each model call.
Common misconceptions
“Summary memory deletes old messages”
False. It compresses them. The summarizer receives the prior summary plus the new turn; information can persist, but specific phrasing is lost. If you need exact recall, buffer or a retrieval layer is required.
“Buffer memory is free because there’s no extra LLM call”
It is not free. You pay for the inflated context on every subsequent request. At scale, that dominates cost compared to occasional summarization.
“You must pick one and commit”
LangChain ships ConversationSummaryBufferMemory, which keeps recent turns verbatim and summarizes older ones. That hybrid is often the right default for multi-turn agents.
“Memory persists across server restarts”
By default, both classes hold state in Python objects. You must back them with a database or use RunnableWithMessageHistory with a persistent store. The memory class alone is not a persistence layer.
“Summary memory is always shorter”
Only if the summarizer is constrained. A poorly tuned summary prompt can emit a longer summary than the original exchange. Set max_token_limit on the memory or cap the summarizer output.
When to use which
Use ConversationBufferMemory when:
- Conversations are short (<=10 turns)
- Exact wording matters (code, IDs, legal text)
- You want zero extra latency from summarization
Use ConversationSummaryMemory when:
- Sessions exceed ~20 turns routinely
- The model needs gist, not verbatim quotes
- You can tolerate periodic summarization calls
Use ConversationSummaryBufferMemory when:
- You need both recent exact context and long-term gist
- Token budget is tight but last few exchanges must be precise
Implementation gotchas
- Summarization prompt drift: The default summary prompt is generic. Override
summary_message_promptto instruct the model to retain entities like order numbers. - Concurrency:
ConversationBufferMemoryis not thread-safe. Use a per-session instance or a lock. - Token limits on summarizer: If the old summary plus new exchange exceeds the summarizer’s context window, you must truncate or use a hierarchical summary.
- Streaming: Neither classic memory class streams the summarization step; your user sees a pause after sending a message if summarization is synchronous.
- Observability: Log the
historylength on each call. A sudden jump in buffer size is the earliest signal you should switch to summary.
Bottom line
The langchain memory types buffer vs summary split is a conscious engineering decision about token spend and information fidelity. Buffer is a verbatim tape; summary is a rolling brief. Pick based on turn count, precision needs, and whether your inference stack charges per token. For most long-lived chat products, a hybrid or summary buffer wins; for scripts and short flows, the plain buffer is simpler and cheaper than it looks.