How to cache LLM responses in Express with Redis

Caching LLM responses at the boundary of your Node service cuts tail latency and avoids redundant spend on repeated prompts. This guide walks through a concrete express.js cache llm responses redis implementation that sits in front of any OpenAI-compatible inference endpoint and serves cached completions for identical requests.

Step 1: Install dependencies and run Redis

You need Node 18+ (for global fetch), Express, and a Redis client. I use the official redis package because it supports async/await cleanly and doesn’t add a cluster of opinionated wrappers.

npm init -y
npm install express redis
docker run -d --name redis-cache -p 6379:6379 redis:7

If you already run Redis in production, point the client at your existing instance. For local dev, the container above is enough.

Step 2: Stand up a minimal Express server

Create server.js. The route accepts a JSON body with model, messages, and sampling params, then forwards to the model provider. We will insert caching around this forward.

import express from 'express';
import { createClient } from 'redis';

const app = express();
app.use(express.json());

const redis = createClient({ url: 'redis://localhost:6379' });
await redis.connect();

const LLM_ENDPOINT = process.env.LLM_ENDPOINT || 'https://api.openai.com/v1/chat/completions';
const LLM_KEY = process.env.LLM_KEY;

app.post('/v1/chat', async (req, res) => {
  // TODO: caching logic goes here
  const upstream = await fetch(LLM_ENDPOINT, {
    method: 'POST',
    headers: { 'content-type': 'application/json', authorization: `Bearer ${LLM_KEY}` },
    body: JSON.stringify(req.body),
  });
  const data = await upstream.json();
  res.json(data);
});

app.listen(3000, () => console.log('listening on 3000'));

This is the uncached baseline. Every call hits the model, even if the same prompt arrived seconds ago.

Step 3: Derive a deterministic cache key

Cache correctness depends on the key. If you include a timestamp or random seed, you will never get hits. Hash the exact fields that affect the output: model, messages, temperature, top_p, max_tokens. Ignore irrelevant headers.

import { createHash } from 'crypto';

function cacheKey(reqBody) {
  const { model, messages, temperature = 1, top_p = 1, max_tokens } = reqBody;
  const normalized = JSON.stringify({ model, messages, temperature, top_p, max_tokens });
  return 'llm:' + createHash('sha256').update(normalized).digest('hex');
}

Use a prefix (llm:) so you can scan or flush the namespace without nuking other Redis data.

Step 4: Check Redis before calling the model

Insert a middleware that short-circuits on a hit. Because we are not streaming in this example, we can buffer the full JSON response.

async function cacheLookup(req, res, next) {
  const key = cacheKey(req.body);
  const hit = await redis.get(key);
  if (hit) {
    res.set('x-cache', 'HIT');
    return res.json(JSON.parse(hit));
  }
  res.locals.key = key;
  next();
}

Wire it into the route:

app.post('/v1/chat', cacheLookup, async (req, res) => {
  const upstream = await fetch(LLM_ENDPOINT, { /* same as before */ });
  const data = await upstream.json();
  await redis.set(res.locals.key, JSON.stringify(data), { EX: 3600 });
  res.set('x-cache', 'MISS');
  res.json(data);
});

The first request stores the serialized response with a one-hour TTL. Subsequent identical requests return from Redis in sub-millisecond local time. This express.js cache llm responses redis design assumes non-streaming, buffered responses.

Step 5: Route through a gateway that handles degradation

If you point the fetch at n4n.ai’s single OpenAI-compatible endpoint, you address 240+ models behind one base URL and get automatic fallback when a provider is rate-limited or degraded. The gateway forwards provider cache-control hints, so your Redis TTL can mirror the upstream semantic cache window instead of guessing.

const LLM_ENDPOINT = 'https://api.n4n.ai/v1/chat/completions';
// headers can include 'x-routing': 'anthropic:claude-3-5-sonnet' to pin a provider

That integration is transparent to the caching layer—your key is still derived from the request body, so a routed model swap only changes cache hits if you include the routing directive in the hashed payload.

Step 6: Handle non-cacheable requests

Not every LLM call should be cached. Streaming responses, requests with stream: true, or those containing volatile context (e.g., current weather) break correctness. Reject them early.

function cacheable(body) {
  if (body.stream) return false;
  if (body.messages?.some(m => m.role === 'system' && m.content.includes('{{now}}'))) return false;
  return true;
}

In the route, skip caching when !cacheable(req.body):

app.post('/v1/chat', cacheLookup, async (req, res) => {
  if (!cacheable(req.body)) {
    const upstream = await fetch(LLM_ENDPOINT, { /* ... */ });
    const data = await upstream.json();
    return res.json(data);
  }
  // existing cached path...
});

Step 7: Set TTLs based on prompt volatility

A static prompt (“Translate to French: …”) can live in cache for hours. A prompt that embeds user-specific session state should expire in seconds or not be cached at all. Store TTL alongside the key decision:

function ttlFor(body) {
  if (body.model.startsWith('gpt-4')) return 86400; // stable completions
  return 600;
}

Pass it to redis.set with { EX: ttlFor(req.body) }. Avoid infinite TTLs; they silently serve stale model versions after you change system prompts.

Step 8: Verify the cache end to end

Start the server and fire two identical requests with curl:

curl -s -X POST localhost:3000/v1/chat \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What is Redis?"}],"temperature":0}' \
  -i | grep x-cache

First call returns x-cache: MISS. The second returns x-cache: HIT. Confirm the value in Redis:

redis-cli get "llm:$(echo -n '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What is Redis?"}],"temperature":0,"top_p":1}' | sha256sum | cut -d' ' -f1)"

You should see the JSON payload. If the CLI prints (nil), your key normalization mismatches the server’s—most often because of default param omission. Log cacheKey(req.body) on both sides to diff.

Operational notes

Redis memory grows with cached payloads. Set maxmemory-policy allkeys-lru so eviction happens under pressure instead of OOM-killing the node. For multi-region deployments, run Redis per region and accept occasional cross-region misses; pushing a shared cache across continents adds latency that defeats the purpose.

Streaming remains out of scope here. If you need to cache token streams, buffer server-side, dedupe on the full completion, and only then replay—don’t try to cache partial chunks.

The express.js cache llm responses redis pattern shown above is deliberately boring. It is a synchronous request/response cache with a hashed key and a TTL. That is exactly what most LLM backends need until they have proof otherwise.