Cloudflare Durable Objects for stateful LLM chat sessions

Most LLM chat demos lose context when the tab closes or the server restarts. Cloudflare Durable Objects llm chat sessions solve this by pinning each conversation to a single edge object that owns its state with strong consistency. You get a stateful backend without standing up a database or a Redis cluster.

This guide walks through a complete implementation: a Worker that routes to a Durable Object per session, an object that persists messages, streaming from an OpenAI-compatible LLM endpoint, and a minimal chat UI. By the end you’ll have a runnable pattern you can extend.

Step 1: Scaffold the Worker and Durable Object namespace

Create a new Worker project and declare the Durable Object binding. The wrangler.toml below registers a ChatSession class and runs it on the edge environment.

name = "llm-chat-edge"
main = "src/index.js"
compatibility_date = "2024-09-23"
workers_dev = true

[[durable_objects.bindings]]
name = "CHAT"
class_name = "ChatSession"

[[migrations]]
tag = "v1"
new_classes = ["ChatSession"]

Run wrangler dev to confirm the skeleton loads. You should see a local endpoint like http://localhost:8787 with no errors about missing bindings. If you later rename the class, bump the migration tag and add a renamed_classes entry; the platform will migrate existing instances.

Step 2: Implement the Durable Object class

The core of cloudflare durable objects llm chat sessions is a class that receives all requests for a given session id. Cloudflare instantiates one object per id; all calls are serialized, so you can mutate state without locks. That serialization is the feature: two concurrent POSTs from different tabs execute one after the other, never interleaving their storage writes.

export class ChatSession {
  constructor(ctx, env) {
    this.ctx = ctx;
    this.env = env;
  }

  async fetch(request) {
    const url = new URL(request.url);
    if (request.method === "POST" && url.pathname.endsWith("/message")) {
      return this.handleMessage(request);
    }
    if (request.method === "GET" && url.pathname.endsWith("/history")) {
      return this.handleHistory();
    }
    return new Response("Not found", { status: 404 });
  }

  async handleHistory() {
    const messages = (await this.ctx.storage.get("messages")) || [];
    return Response.json(messages);
  }

  async handleMessage(request) {
    const { role, content } = await request.json();
    const messages = (await this.ctx.storage.get("messages")) || [];
    messages.push({ role, content, ts: Date.now() });
    await this.ctx.storage.put("messages", messages);
    return Response.json({ ok: true });
  }
}

This stub persists user messages and serves history. The LLM call plugs in next. Keep the fetch method thin—route to named handlers so the class stays readable as you add WebSocket support or auth.

Step 3: Persist chat history with storage APIs

Durable Object storage is transactional. Wrapping a read-modify-write in this.ctx.storage is safe because the object processes one request at a time. For longer conversations, paginate or prune:

async function appendMessage(ctx, msg) {
  const prev = (await ctx.storage.get("messages")) || [];
  prev.push(msg);
  // Keep last 100 turns to bound storage
  const trimmed = prev.slice(-100);
  await ctx.storage.put("messages", trimmed);
  return trimmed;
}

Call appendMessage from handleMessage instead of inline code. This keeps the storage logic testable and lets you swap to the SQLite API (this.ctx.storage.sql) later if you need indexed queries over messages.

Why not use KV directly

Workers KV is eventually consistent and global; writing from two edges can lose updates. Durable Objects give you a single authoritative copy per session. For chat, that authority is exactly what you need to avoid duplicate assistant replies.

Step 4: Route requests to the correct session

The Worker’s default export maps a session id from the path to a Durable Object stub. Routing requests to the correct cloudflare durable objects llm chat sessions instance is a single get call. Use idFromName so the same string always maps to the same object.

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const match = url.pathname.match(/^\/chat\/([\w-]+)/);
    if (!match) {
      return new Response("Usage: /chat/<sessionId>/...", { status: 400 });
    }
    const sessionId = match[1];
    const stub = env.CHAT.get(env.CHAT.idFromName(sessionId));
    return stub.fetch(
      new URL(url.pathname.replace(/^\/chat\/[\w-]+/, ""), url.origin) + url.search,
      request
    );
  }
};

Now /chat/room-1/message hits the ChatSession for room-1. The object sees /message. Design session ids as opaque tokens (e.g., UUIDs) if you expose them to clients; never put secrets in the path.

Step 5: Stream LLM responses and update state

A chat session is useless without a model. Call an OpenAI-compatible endpoint with the stored history. If you route through n4n.ai, its OpenAI-compatible endpoint covers 240+ models and automatically falls back when a provider is rate-limited, so you skip custom retry code. The stream from the model should be piped to the client and the full assistant reply saved after completion.

async handleMessage(request) {
  const { content } = await request.json();
  const messages = await appendMessage(this.ctx, { role: "user", content });

  const llmRes = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${this.env.OPENAI_KEY}`,
    },
    body: JSON.stringify({
      model: "gpt-4o-mini",
      messages,
      stream: true,
    }),
  });

  const reader = llmRes.body.getReader();
  const decoder = new TextDecoder();
  let assistant = "";

  const stream = new ReadableStream({
    async pull(controller) {
      const { done, value } = await reader.read();
      if (done) {
        controller.close();
        await appendMessage(this.ctx, { role: "assistant", content: assistant });
        return;
      }
      const chunk = decoder.decode(value);
      assistant += chunk;
      controller.enqueue(value);
    },
  });

  return new Response(stream, {
    headers: { "content-type": "text/plain; charset=utf-8" },
  });
}

Streaming format details

The raw body from OpenAI is Server-Sent Events with data: {json}\n\n lines. The code above forwards bytes unchanged; the browser receives a plain text stream. In production, parse data: lines, extract choices[0].delta.content, and enqueue only the token text so the UI doesn’t need to parse SSE.

Step 6: Build a minimal chat UI at the edge

A UI that talks to cloudflare durable objects llm chat sessions must send POSTs and read the streamed response. Serve a static HTML page from the Worker root:

async fetch(request, env) {
  const url = new URL(request.url);
  if (url.pathname === "/") {
    return new Response(html, { headers: { "content-type": "text/html" } });
  }
  // ... chat routing from Step 4
}

const html = `<!doctype html>
<input id="sid" placeholder="session id" value="demo">
<input id="msg" placeholder="message">
<button onclick="send()">Send</button>
<pre id="out"></pre>
<script>
async function send() {
  const sid = document.getElementById('sid').value;
  const msg = document.getElementById('msg').value;
  const res = await fetch('/chat/' + sid + '/message', {
    method: 'POST',
    headers: {'content-type':'application/json'},
    body: JSON.stringify({role:'user', content: msg})
  });
  const reader = res.body.getReader();
  const dec = new TextDecoder();
  while(true) {
    const {done, value} = await reader.read();
    if(done) break;
    document.getElementById('out').textContent += dec.decode(value);
  }
}
</script>`;

This is intentionally crude. It proves the round trip: type a message, watch tokens appear, and the session stores both sides. Swap the <pre> for a React component when you’re ready.

Step 7: Verify the full flow

Run wrangler dev. In one terminal, start a session:

curl -N -X POST http://localhost:8787/chat/demo/message \
  -H 'content-type: application/json' \
  -d '{"role":"user","content":"Explain Durable Objects in one sentence."}'

You should see streamed text. Then fetch history:

curl http://localhost:8787/chat/demo/history

The JSON must contain the user message and the assistant reply. Kill the dev server and restart, then re-run the history call. The messages persist because Durable Object storage is backed by the edge key-value store, not memory.

To test concurrency, open two browser tabs with the same session id. Messages from both tabs serialize through the same object; neither overwrites the other’s writes. If you post from tab A and tab B simultaneously, the history array contains both user messages in the order they were processed.

Operational notes

Durable Objects bill per active duration and storage. For chat, a session that idles still costs little because the object hibernates. Use this.ctx.storage.deleteAll() or a TTL sweep if you provision ephemeral rooms.

If you need cross-session analytics, meter tokens at the Worker level before calling the model. Per-token usage metering is straightforward when the LLM gateway returns usage headers, or you can count chunks.

The pattern above is the minimum viable stateful chat backend. From here, add auth by signing session ids, swap the model per user preference, or pipe the stream into a WebSocket for bidirectional UI updates. Cloudflare Durable Objects llm chat sessions give you a single authority per conversation—build on that guarantee.