LangChain.js memory management for Node chat apps

LangChain.js memory management in Node chat applications determines whether your assistant remembers context across turns or hallucinates from token overflow. The library ships several memory classes, but the documentation leaves the integration details to you. This guide walks through choosing, configuring, and operating memory in production Node services — covering buffer, summary, and vector-backed approaches with working code you can drop into a running system.

Understanding the memory interface

Every LangChain.js memory class implements BaseChatMemory with two required methods: loadMemoryVariables and saveContext. The chain or agent calls loadMemoryVariables before each model invocation to inject history, then calls saveContext after the response arrives. Your job is picking a concrete implementation that fits your token budget, latency requirements, and persistence needs.

import { BaseChatMemory } from "@langchain/core/memory";

interface MyMemory extends BaseChatMemory {
  loadMemoryVariables(values: Record<string, any>): Promise<Record<string, any>>;
  saveContext(input: Record<string, any>, output: Record<string, any>): Promise<void>;
}

The input/output keys default to input and output, but you can override them per memory instance. Most chains expect a single string key called history containing the formatted conversation. If you customize the key, update your prompt template accordingly.

Buffer memory — simplest starting point

BufferMemory stores every message verbatim in an array. It works for short conversations or when you control the context window tightly. The danger is unbounded growth: a chatty user can blow past the model’s context limit in a few dozen turns.

import { BufferMemory } from "langchain/memory";
import { ChatOpenAI } from "@langchain/openai";
import { ConversationChain } from "langchain/chains";

const memory = new BufferMemory({
  returnMessages: true, // returns BaseMessage[] instead of formatted string
  inputKey: "input",
  outputKey: "response",
});

const chain = new ConversationChain({
  llm: new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0 }),
  memory,
  prompt: ChatPromptTemplate.fromMessages([
    ["system", "You are a helpful assistant."],
    new MessagesPlaceholder("history"),
    ["human", "{input}"],
  ]),
});

await chain.call({ input: "Hi, I'm Alex." });
await chain.call({ input: "What's my name?" });
// memory.chatHistory now holds [HumanMessage, AIMessage, HumanMessage, AIMessage]

Pitfall: returnMessages: true returns BaseMessage[] which works with MessagesPlaceholder. If you omit it, you get a single string — fine for legacy prompt templates but incompatible with chat model message arrays.

Tradeoff: Zero configuration, zero external dependencies. Unsuitable for long-running sessions without an eviction strategy.

Buffer window memory — fixed-turn sliding window

BufferWindowMemory keeps only the last k interactions (human + AI pairs). This bounds token usage predictably. Set k based on your model’s context window minus prompt overhead and expected response length.

import { BufferWindowMemory } from "langchain/memory";

const memory = new BufferWindowMemory({
  k: 6, // retains last 6 message pairs = 12 messages total
  returnMessages: true,
  inputKey: "input",
  outputKey: "response",
});

Choosing k: For GPT-4o-mini (128k context), a typical system prompt + few-shot examples consumes ~2k tokens. Each exchange averages 300-500 tokens. With k=6 you reserve ~3-4k tokens for history, leaving ~120k for the current turn — generous. For 8k-context models, k=3 or 4 is safer.

Pitfall: The window cuts aggressively. If a user references something from turn 7, the model has no access. No summarization happens — context simply disappears.

Conversation summary memory — compressing old turns

ConversationSummaryMemory uses an LLM to condense history into a running summary. It preserves semantic content across arbitrarily long conversations at the cost of an extra model call per turn.

import { ConversationSummaryMemory } from "langchain/memory";
import { ChatOpenAI } from "@langchain/openai";

const summaryLlm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0 });

const memory = new ConversationSummaryMemory({
  llm: summaryLlm,
  returnMessages: true,
  inputKey: "input",
  outputKey: "response",
  // Optional: provide an initial summary to prime the memory
  // summary: "User prefers TypeScript, works on backend APIs.",
});

How it works: After each saveContext, the memory invokes the summary LLM with the previous summary plus the new exchange, producing an updated summary. The summary replaces the raw history in loadMemoryVariables.

Tradeoffs:

• Extra latency: one additional LLM round-trip per turn (can run in parallel with main call if you structure it)
• Extra cost: summary tokens count against your budget
• Loss of verbatim detail: exact phrasing, code snippets, and specific numbers degrade over time

Optimization: Use a cheaper model for summarization (e.g., gpt-4o-mini summarizing gpt-4o conversations). The summary prompt is internal — you can’t easily customize it without forking the class.

Conversation summary buffer memory — hybrid approach

ConversationSummaryBufferMemory combines both: it keeps recent messages verbatim (buffer) and summarizes older ones. This is usually the right default for production chat apps.

import { ConversationSummaryBufferMemory } from "langchain/memory";
import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({ model: "gpt-4o", temperature: 0 });
const summaryLlm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0 });

const memory = new ConversationSummaryBufferMemory({
  llm: summaryLlm,
  maxTokenLimit: 3000, // target token budget for combined summary + buffer
  returnMessages: true,
  inputKey: "input",
  outputKey: "response",
});

How it works: On each save, the memory estimates tokens in the buffer. If adding the new exchange would exceed maxTokenLimit, it summarizes the oldest messages into the running summary and drops them from the buffer. The buffer always contains the most recent turns intact.

Sizing maxTokenLimit: Reserve space for your system prompt, few-shot examples, and the current user message. For a 128k model with a 4k prompt, setting 8k-16k gives ample room for the model to reason while keeping the last ~20-40 exchanges verbatim.

Pitfall: Token counting uses a rough heuristic (character-based for non-OpenAI models). For precise control, pass a custom tokenCounter function that uses your tokenizer of choice.

import { Tiktoken } from "tiktoken/lite";
import { encoding_for_model } from "tiktoken";

const enc = encoding_for_model("gpt-4o");

const memory = new ConversationSummaryBufferMemory({
  llm: summaryLlm,
  maxTokenLimit: 8000,
  tokenCounter: (text: string) => enc.encode(text).length,
  returnMessages: true,
});

Vector store memory — semantic retrieval over full history

When users reference details from hundreds of turns ago, sliding windows and summaries fail. VectorStoreRetrieverMemory embeds each exchange and retrieves the top-k most relevant passages for the current query.

import { VectorStoreRetrieverMemory } from "langchain/memory";
import { OpenAIEmbeddings } from "@langchain/openai";
import { SupabaseVectorStore } from "@langchain/community/vectorstores/supabase";
import { createClient } from "@supabase/supabase-js";

const sb = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_KEY!);
const vectorStore = new SupabaseVectorStore(new OpenAIEmbeddings(), {
  client: sb,
  tableName: "chat_history",
  queryName: "match_chat_history",
});

const memory = new VectorStoreRetrieverMemory({
  vectorStore,
  memoryKey: "history",
  inputKey: "input",
  outputKey: "response",
  k: 4, // retrieve top 4 relevant exchanges
  returnMessages: true,
});

Schema requirement: The vector store needs a table with columns for content, embedding, and metadata (session_id, timestamp, role). The Supabase integration expects a match_chat_history RPC — see LangChain.js docs for the SQL.

Tradeoffs:

• Adds infrastructure: vector DB, embedding model, retrieval latency (~50-200ms)
• Retrieval quality depends on embedding model and chunking strategy
• No guaranteed recency — a relevant but ancient exchange can crowd out yesterday’s context
• Works best combined with a small buffer for immediate history

Hybrid pattern: Use ConversationSummaryBufferMemory for recent turns + VectorStoreRetrieverMemory for long-term recall. Merge their outputs in your prompt template.

const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant."],
  new MessagesPlaceholder("recent_history"), // from buffer memory
  ["system", "Relevant past context:\n{retrieved_history}"], // from vector memory
  ["human", "{input}"],
]);

// In your route handler:
const recentVars = await bufferMemory.loadMemoryVariables({});
const retrievedVars = await vectorMemory.loadMemoryVariables({ input: userMessage });
const result = await chain.call({
  input: userMessage,
  recent_history: recentVars.history,
  retrieved_history: retrievedVars.history,
});

Persisting memory across restarts

In-memory memory classes lose state when the Node process exits. For production, you need persistence. Two patterns work well:

1. Serialize to Redis (session-scoped)

import { Redis } from "ioredis";
import { BufferMemory } from "langchain/memory";

const redis = new Redis(process.env.REDIS_URL!);

class RedisBufferMemory extends BufferMemory {
  sessionId: string;
  ttlSeconds: number;

  constructor(sessionId: string, ttlSeconds = 86400) {
    super({ returnMessages: true });
    this.sessionId = sessionId;
    this.ttlSeconds = ttlSeconds;
  }

  async loadMemoryVariables() {
    const data = await redis.get(`chat:${this.sessionId}`);
    if (data) {
      const parsed = JSON.parse(data);
      this.chatHistory = parsed.messages.map((m: any) =>
        m.type === "human" ? new HumanMessage(m.content) : new AIMessage(m.content)
      );
    }
    return super.loadMemoryVariables({});
  }

  async saveContext(input: any, output: any) {
    await super.saveContext(input, output);
    const messages = this.chatHistory.map(m => ({
      type: m._getType(),
      content: m.content,
    }));
    await redis.setex(`chat:${this.sessionId}`, this.ttlSeconds, JSON.stringify({ messages }));
  }
}

Why Redis: Sub-millisecond reads/writes, built-in TTL for session expiry, horizontal scaling. Store the serialized chatHistory array — it’s portable across memory class types.

2. Postgres with Prisma (audit + replay)

// schema.prisma
model ChatMessage {
  id        String   @id @default(cuid())
  sessionId String   @index
  role      String   // "human" | "ai"
  content   String
  tokens    Int?
  createdAt DateTime @default(now())
  metadata  Json?
}

import { PrismaClient } from "@prisma/client";

const prisma = new PrismaClient();

class PrismaBufferMemory extends BufferMemory {
  sessionId: string;

  constructor(sessionId: string) {
    super({ returnMessages: true });
    this.sessionId = sessionId;
  }

  async loadMemoryVariables() {
    const rows = await prisma.chatMessage.findMany({
      where: { sessionId: this.sessionId },
      orderBy: { createdAt: "asc" },
      take: 100, // safety cap
    });
    this.chatHistory = rows.map(r =>
      r.role === "human" ? new HumanMessage(r.content) : new AIMessage(r.content)
    );
    return super.loadMemoryVariables({});
  }

  async saveContext(input: any, output: any) {
    await super.saveContext(input, output);
    const newMessages = this.chatHistory.slice(-2); // last human + AI
    await prisma.chatMessage.createMany({
      data: newMessages.map(m => ({
        sessionId: this.sessionId,
        role: m._getType(),
        content: m.content,
      })),
    });
  }
}

Benefit: Full audit trail, SQL queries for analytics, easy replay for debugging or fine-tuning.

Token budgeting and monitoring

Memory consumes tokens. You need visibility into actual usage per request. Wrap your chain to log token counts from the model response.

import { CallbackManager } from "@langchain/core/callbacks/manager";

const chain = new ConversationChain({
  llm: new ChatOpenAI({ model: "gpt-4o", temperature: 0 }),
  memory,
  callbacks: CallbackManager.fromHandlers({
    handleLLMEnd: async (output) => {
      const usage = output.llmOutput?.tokenUsage;
      if (usage) {
        console.log(JSON.stringify({
          sessionId: memory.sessionId,
          promptTokens: usage.promptTokens,
          completionTokens: usage.completionTokens,
          totalTokens: usage.totalTokens,
          memoryType: memory.constructor.name,
        }));
      }
    },
  }),
});

Alerting: Set thresholds on promptTokens relative to your maxTokenLimit or model context. If prompt tokens exceed 80% of context, your memory configuration is too aggressive.

Common pitfalls checklist

Decision framework

Putting it together — a production-ready factory

// memory-factory.ts
import { BaseChatMemory } from "@langchain/core/memory";
import { ConversationSummaryBufferMemory } from "langchain/memory";
import { VectorStoreRetrieverMemory } from "langchain/memory";
import { ChatOpenAI } from "@langchain/openai";
import { OpenAIEmbeddings } from "@langchain/openai";
import { SupabaseVectorStore } from "@langchain/community/vectorstores/supabase";
import { createClient } from "@supabase/supabase-js";
import { Redis } from "ioredis";
import { encoding_for_model } from "tiktoken";

const redis = new Redis(process.env.REDIS_URL!);
const sb = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_KEY!);
const enc = encoding_for_model("gpt-4o");

const summaryLlm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0 });
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });

const vectorStore = new SupabaseVectorStore(embeddings, {
  client: sb,
  tableName: "chat_history",
  queryName: "match_chat_history",
});

export async function createMemory(sessionId: string): Promise<BaseChatMemory> {
  // Recent history: hybrid buffer + summary
  const bufferMemory = new ConversationSummaryBufferMemory({
    llm: summaryLlm,
    maxTokenLimit: 8000,
    tokenCounter: (text: string) => enc.encode(text).length,
    returnMessages: true,
    inputKey: "input",
    outputKey: "response",
    memoryKey: "recent_history",
  });

  // Persist buffer to Redis
  const redisKey = `chat:${sessionId}`;
  const cached = await redis.get(redisKey);
  if (cached) {
    const { messages } = JSON.parse(cached);
    bufferMemory.chatHistory = messages.map((m: any) =>
      m.type === "human" ? new HumanMessage(m.content) : new AIMessage(m.content)
    );
  }

  const originalSave = bufferMemory.saveContext.bind(bufferMemory);
  bufferMemory.saveContext = async (input, output) => {
    await originalSave(input, output);
    const messages = bufferMemory.chatHistory.map(m => ({
      type: m._getType(),
      content: m.content,
    }));
    await redis.setex(redisKey, 86400, JSON.stringify({ messages }));
  };

  // Long-term semantic recall
  const vectorMemory = new VectorStoreRetrieverMemory({
    vectorStore,
    memoryKey: "retrieved_history",
    inputKey: "input",
    outputKey: "response",
    k: 4,
    returnMessages: true,
    filter: { session_id: sessionId }, // Supabase RPC must support this
  });

  // Composite memory that merges both
  return {
    ...bufferMemory,
    ...vectorMemory,
    async loadMemoryVariables(values: Record<string, any>) {
      const [recent, retrieved] = await Promise.all([
        bufferMemory.loadMemoryVariables(values),
        vectorMemory.loadMemoryVariables(values),
      ]);
      return { ...recent, ...retrieved };
    },
    async saveContext(input: Record<string, any>, output: Record<string, any>) {
      await Promise.all([
        bufferMemory.saveContext(input, output),
        vectorMemory.saveContext(input, output),
      ]);
    },
  } as BaseChatMemory;
}

This factory gives you: bounded recent history with summarization, Redis persistence across restarts, semantic retrieval for long-term context, and a single BaseChatMemory interface your chains already expect.

What to avoid

• Don’t use ConversationBufferMemory (the string-based variant) with chat models — it forces string concatenation that breaks message roles.
• Don’t share a single memory instance across sessions — each user needs isolated state.
• Don’t skip token counting — the default character heuristic is wrong for non-English text and code.
• Don’t embed PII in vector stores without encryption — embeddings can be inverted.

Memory is the difference between a chatbot that feels smart and one that forgets the user’s name three turns in. Start with ConversationSummaryBufferMemory, add Redis persistence, then layer vector retrieval when users ask “remember that thing I said last month.” The factory above is a template — adapt the token limits, retrieval k, and persistence TTL to your actual traffic patterns.