Add conversational memory to a LangChain chatbot

Most demos fall over the moment a user says “like we discussed earlier.” To add conversational memory to a LangChain chatbot, you need a state container that persists messages across turns and a way to feed them back into the prompt. This guide walks through a runnable implementation using LangChain’s message history primitives and a summary fallback for long sessions.

Step 1: Install dependencies and pin versions

LangChain’s memory APIs have churned. Use the modular packages so imports don’t break in six months.

pip install "langchain-openai>=0.1.0" "langchain-core>=0.2.0" "langchain>=0.2.0"

If you still need the legacy ConversationSummaryBufferMemory, it ships in the base langchain package. Keep it isolated; the newer RunnableWithMessageHistory pattern is cleaner for most bots.

Create a virtualenv. Don’t run this against your production API keys without a .env loader.

# config.py
import os
from dotenv import load_dotenv
load_dotenv()

Step 2: Configure the chat model

Point LangChain at any OpenAI-compatible server. If you want automatic fallback across 240+ models without writing retry logic, point ChatOpenAI at n4n.ai’s OpenAI-compatible endpoint and set your key.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0.7,
    base_url="https://api.n4n.ai/v1",  # optional: swap for your gateway
    api_key=os.environ["OPENAI_API_KEY"],
)

For local testing, gpt-3.5-turbo is fine. The memory logic doesn’t care which model answers.

Step 3: Choose a memory strategy

Two failure modes dominate:

• Buffer memory stores raw messages. Simple, perfect recall, but context window blows up after 20 turns.
• Summary memory compresses old turns into a running abstract. Bounded size, but loses exact wording.

A production bot usually runs buffer memory up to a token limit, then flushes to summary. We’ll implement both so you can see the tradeoff.

Step 4: Add conversational memory with buffer history

The modern primitive is InMemoryChatMessageHistory wrapped by RunnableWithMessageHistory. This avoids the old ConversationChain magic and gives you explicit control.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables import RunnableWithMessageHistory

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise support agent."),
    MessagesPlaceholder(variable_name="history"),
    ("human", "{input}"),
])

chain = prompt | llm

store: dict[str, InMemoryChatMessageHistory] = {}

def get_session_history(session_id: str) -> InMemoryChatMessageHistory:
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]

with_history = RunnableWithMessageHistory(
    chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="history",
)

Invoke it with a configurable session id. Each session gets its own buffer.

cfg = {"configurable": {"session_id": "user-42"}}

print(with_history.invoke({"input": "My order 881 is late."}, config=cfg).content)
# -> "I'm sorry to hear order 881 is late. Let me check..."
print(with_history.invoke({"input": "Can you refund it?"}, config=cfg).content)
# The model sees the prior message because history is injected.

That’s the minimum to add conversational memory LangChain chatbot style without dragging in a vector store.

Step 5: Bound the context with summary memory

When store["user-42"].messages exceeds your token budget, summarize. The legacy ConversationSummaryBufferMemory does this inline.

from langchain.memory import ConversationSummaryBufferMemory

summary_mem = ConversationSummaryBufferMemory(
    llm=llm,
    max_token_limit=300,
    return_messages=True,
)

summary_mem.save_context(
    {"input": "My order 881 is late."},
    {"output": "I'll check logistics for order 881."},
)
summary_mem.save_context(
    {"input": "I live in Berlin."},
    {"output": "Noted, Berlin address on file."},
)

vars = summary_mem.load_memory_variables({})
for msg in vars["history"]:
    print(msg.type, ":", msg.content)

After enough turns, the buffer drops old messages and prepends a LLM-generated summary. You can plug vars["history"] directly into the MessagesPlaceholder instead of the raw buffer.

Hybrid approach

In practice, I keep the last K messages verbatim and summarize everything older. Write a small helper:

def trimmed_history(session_id: str, k=6):
    msgs = store[session_id].messages
    if len(msgs) <= k:
        return msgs
    recent = msgs[-k:]
    older = msgs[:-k]
    # call llm to summarize older, or use summary_mem
    return recent  # stub

Wire that into get_session_history if you need tighter control than the built-in buffer.

Step 6: Build the interactive loop

A chatbot is just a REPL that persists session_id. Here’s a minimal terminal loop that uses the buffer chain from Step 4.

def chat_loop(session_id: str):
    config = {"configurable": {"session_id": session_id}}
    print("Type 'exit' to quit.")
    while True:
        user_input = input("you> ")
        if user_input.lower() == "exit":
            break
        resp = with_history.invoke({"input": user_input}, config=config)
        print("bot>", resp.content)

if __name__ == "__main__":
    chat_loop("cli-session-1")

Run it. Say “I’m allergic to peanuts” in turn one, then “What should I avoid?” in turn four. If the bot references peanuts, memory works.

Step 7: Verify success

Don’t trust a single happy path. Write a test that asserts cross-turn state.

def test_memory_recall():
    cfg = {"configurable": {"session_id": "test-1"}}
    with_history.invoke({"input": "Call me Sam"}, config=cfg)
    out = with_history.invoke({"input": "What's my name?"}, config=cfg)
    assert "Sam" in out.content

Run with pytest. If it fails, print store["test-1"].messages to inspect what reached the model.

For summary memory, assert that len(vars["history"]) stays under your limit after 50 turns. Measure tokens with tiktoken if you’re paranoid about window overflow.

Production notes

• Session store here is in-process. Swap store for Redis or Postgres in multi-worker deployments.
• RunnableWithMessageHistory is stateless; the state lives in your get_session_history. That’s a feature—it forces you to own the persistence layer.
• If you use a gateway with fallback, set model to a generic alias and let the gateway route. The memory code is model-agnostic.

Step 8: Handle multi-user concurrency

The dict in Step 4 is not thread-safe. Use a lock or a concurrent map.

import threading
lock = threading.Lock()

def get_session_history(session_id: str):
    with lock:
        if session_id not in store:
            store[session_id] = InMemoryChatMessageHistory()
        return store[session_id]

For async servers, implement aget_session_history and use RunnableWithMessageHistory’s async variant. LangChain supports both; don’t block the event loop on a slow summarizer.

Step 9: Stream responses with memory intact

Users expect tokens to appear live. The same chain works with .stream().

for chunk in with_history.stream({"input": "Summarize our chat"}, config=cfg):
    print(chunk.content, end="")

Memory is saved after the full invocation completes, so streaming doesn’t complicate state. Just don’t call save_context manually unless you bypass the Runnable.

Closing checklist

You now have a runnable pattern to add conversational memory LangChain chatbot applications can rely on: explicit buffer, optional summary compression, per-session isolation, and a test that proves recall. Swap the in-memory store for your DB of choice before shipping, and keep the summary limit tuned to your model’s context window. The rest is prompt engineering.