Add long-term memory to a LlamaIndex chat engine

This llamaindex long-term memory chat engine tutorial shows how to persist conversation state beyond a single process lifetime without bolting on a separate database layer by hand. LlamaIndex already ships a ChatStore abstraction that backs its memory buffers; you just need to point it at Redis and key it per user. The build below runs against a local Redis instance and survives process restarts.

Step 1: Install dependencies and import the modules

This llamaindex long-term memory chat engine tutorial assumes Python 3.10+ and Redis 7 running locally. Install the split LlamaIndex packages so you only pull what you need:

pip install llama-index-core llama-index-llms-openai llama-index-storage-chat-store-redis redis

The core chat engine lives in llama_index.core, while the Redis store is a separate extra. Import them:

from llama_index.core.chat_engine import SimpleChatEngine, ContextChatEngine
from llama_index.core.memory import ChatMemoryBuffer
from llama_index.core.storage.chat_store import RedisChatStore, SimpleChatStore
from llama_index.llms.openai import OpenAI
from llama_index.core import VectorStoreIndex, Document

If you plan to use a RAG index, ContextChatEngine replaces SimpleChatEngine; the memory wiring is identical.

Step 2: Configure a persistent chat store

ChatMemoryBuffer is only an in-memory structure unless you give it a ChatStore. The store handles serialization and retrieval of List[ChatMessage]. RedisChatStore writes each conversation as a hash field under a key you control.

chat_store = RedisChatStore(redis_url="redis://localhost:6379/0")

# Verify connection
assert chat_store.redis.ping()

For local dev you can use SimpleChatStore with a JSON file, but Redis gives atomic updates and TTL support, which matters when serving many users.

# chat_store = SimpleChatStore.from_persist_path("memory.json")
# chat_store.persist("memory.json")

The store itself is the backend. The chat_store_key you pass later namespaces conversations within that backend. Treat it like a user ID or session ID.

Step 3: Wrap the store in a token-bounded memory buffer

Long-term does not mean infinite. You still pay per token on every request, and models cap at 8k–128k context. ChatMemoryBuffer enforces a token limit and drops oldest messages when exceeded. By default it uses tiktoken for OpenAI models; for other tokenizers pass tokenizer_fn.

USER_ID = "user_42"

memory = ChatMemoryBuffer.from_defaults(
    token_limit=4000,
    chat_store=chat_store,
    chat_store_key=USER_ID,
)

On the first run, Redis has no data for that key, so the buffer starts empty. On subsequent runs with the same key, it loads the prior message list automatically. If you need longer retention than your token budget allows, add a summarization step: periodically condense old messages with an LLM call before they get evicted. The hook is memory.get() and memory.put().

Step 4: Build the chat engine with an LLM

Wire the memory into a chat engine. Here we use SimpleChatEngine for clarity.

llm = OpenAI(model="gpt-4o-mini", api_key="sk-your-key")

chat_engine = SimpleChatEngine.from_defaults(
    llm=llm,
    memory=memory,
    system_prompt="You are a terse assistant that remembers user facts.",
)

If you route inference through n4n.ai, set api_base to its OpenAI-compatible endpoint and use any of the 240+ addressed models; the gateway forwards provider cache-control hints and falls back automatically when a provider is degraded.

llm = OpenAI(
    model="anthropic/claude-3.5-sonnet",
    api_key="sk-n4n",
    api_base="https://api.n4n.ai/v1",
)

For RAG, build an index and use ContextChatEngine:

docs = [Document(text="LlamaIndex supports persistent chat stores.")]
index = VectorStoreIndex.from_documents(docs)
chat_engine = ContextChatEngine.from_defaults(
    llm=llm, memory=memory, retriever=index.as_retriever()
)

Step 5: Drive a conversation and persist implicitly

Messages are written to Redis on every chat call. No explicit save is required for RedisChatStore.

response = chat_engine.chat("My name is Ada and I prefer Python.")
print(response.response)
# -> "Got it, Ada. I'll remember you prefer Python."

response = chat_engine.chat("What language do I like?")
print(response.response)
# -> "You mentioned you prefer Python."

Streaming works the same way with chat_engine.stream_chat(...). The memory buffer updates after the stream completes.

Step 6: Reconnect and prove memory survives

Exit the Python process. Start a new one with the same USER_ID and store connection.

from llama_index.core.chat_engine import SimpleChatEngine
from llama_index.core.memory import ChatMemoryBuffer
from llama_index.core.storage.chat_store import RedisChatStore
from llama_index.llms.openai import OpenAI

chat_store = RedisChatStore(redis_url="redis://localhost:6379/0")
memory = ChatMemoryBuffer.from_defaults(
    token_limit=4000, chat_store=chat_store, chat_store_key="user_42"
)
llm = OpenAI(model="gpt-4o-mini", api_key="sk-your-key")
chat_engine = SimpleChatEngine.from_defaults(llm=llm, memory=memory)

response = chat_engine.chat("What's my name and language?")
print(response.response)
# Expected: mentions Ada and Python.

If the response references the earlier facts, persistence works.

Step 7: Isolate users and manage lifecycle

In a real service, instantiate the engine per request using the authenticated user ID. A factory keeps boilerplate out of route handlers.

def build_engine_for_user(user_id: str, redis_url: str, llm: OpenAI) -> SimpleChatEngine:
    store = RedisChatStore(redis_url=redis_url)
    mem = ChatMemoryBuffer.from_defaults(
        token_limit=4000, chat_store=store, chat_store_key=user_id
    )
    return SimpleChatEngine.from_defaults(llm=llm, memory=mem)

RedisChatStore is safe for multiple workers, but concurrent writes to the same key use last-write-wins on put. For heavy concurrency, wrap updates in a Redis transaction or use a distributed lock.

To expire stale sessions, set a TTL on the Redis key after each interaction:

import redis
r = redis.Redis.from_url("redis://localhost:6379/0")
r.expire("user_42", 60 * 60 * 24 * 30)  # 30 days

Or delete explicitly:

chat_store.delete("user_42")

Verifying success

Write a small script that asserts cross-process memory. Using pytest:

def test_long_term_memory():
    store = RedisChatStore(redis_url="redis://localhost:6379/0")
    store.delete("test_user")
    mem = ChatMemoryBuffer.from_defaults(token_limit=2000, chat_store=store, chat_store_key="test_user")
    eng = SimpleChatEngine.from_defaults(llm=OpenAI(model="gpt-4o-mini", api_key="sk-test"), memory=mem)
    eng.chat("I live in Berlin.")
    
    # simulate restart
    mem2 = ChatMemoryBuffer.from_defaults(token_limit=2000, chat_store=store, chat_store_key="test_user")
    eng2 = SimpleChatEngine.from_defaults(llm=OpenAI(model="gpt-4o-mini", api_key="sk-test"), memory=mem2)
    out = eng2.chat("Where do I live?").response
    assert "Berlin" in out

Run it with Redis up. If it passes, your llamaindex long-term memory chat engine tutorial implementation is solid. For production, add retry on Redis connection failure, encrypt PII at rest, and monitor token usage per user to tune token_limit.