How to stream LLM responses to the browser from Node.js

Building a responsive chat UI requires token-level streaming from the model to the client. To node.js stream llm responses browser efficiently, you need a server that proxies the LLM vendor’s stream without buffering, and a frontend that renders increments as they arrive. This guide implements that pipeline with Node.js, Express, and the OpenAI-compatible chat completions API.

Prerequisites

You need Node.js 18+ (for native fetch and streaming APIs) and a model endpoint that speaks the OpenAI chat completions protocol. Any compliant gateway works. If you point at n4n.ai, you get one OpenAI-compatible endpoint covering 240+ models with automatic fallback when a provider is degraded, which keeps the proxy code identical regardless of backend changes.

Install the minimal dependencies:

npm init -y
npm install express openai dotenv

Step 1: Scaffold the server and load credentials

Create server.mjs. Load environment variables and instantiate the OpenAI client with a configurable base URL. Keeping the base URL in an env var lets you swap providers without touching business logic.

import express from 'express';
import OpenAI from 'openai';
import dotenv from 'dotenv';

dotenv.config();

const client = new OpenAI({
  apiKey: process.env.LLM_API_KEY,
  baseURL: process.env.LLM_BASE_URL || 'https://api.openai.com/v1',
});

const app = express();
app.use(express.json());

Do not hardcode the model or key in route handlers. Read them from request or config so you can rotate models per request.

Step 2: Build the streaming proxy route

The browser cannot call the LLM directly without leaking your API key. The Node server authenticates, then forwards the token stream as Server-Sent Events (SSE). SSE is simpler than raw WebSocket for one-way model output and works with native EventSource or fetch.

The core technique to node.js stream llm responses browser is to set stream: true on the completion call and write each delta to the response object immediately.

app.post('/api/chat', async (req, res) => {
  const { messages, model = 'gpt-4o-mini' } = req.body;
  if (!Array.isArray(messages)) {
    res.status(400).json({ error: 'messages array required' });
    return;
  }

  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  res.flushHeaders?.();

  const abort = new AbortController();
  req.on('close', () => abort.abort());

  try {
    const stream = await client.chat.completions.create(
      { model, messages, stream: true },
      { signal: abort.signal }
    );

    for await (const chunk of stream) {
      const token = chunk.choices[0]?.delta?.content || '';
      if (token) {
        res.write(`data: ${JSON.stringify({ token })}\n\n`);
      }
    }
    res.write('data: [DONE]\n\n');
    res.end();
  } catch (err) {
    if (!abort.signal.aborted) {
      res.write(`event: error\ndata: ${JSON.stringify({ message: err.message })}\n\n`);
    }
    res.end();
  }
});

app.listen(3000, () => console.log('proxy on :3000'));

Note the req.on('close') handler. If the user navigates away, the browser drops the connection; aborting the upstream request stops you from paying for tokens you’ll discard.

Step 3: Consume the stream in the browser

Modern browsers expose ReadableStream on fetch responses. You do not need the EventSource API because we are POSTing a body; fetch + manual SSE parsing is the correct tool.

When you node.js stream llm responses browser to a React or vanilla component, append text nodes incrementally to avoid re-rendering the whole conversation.

async function sendChat(messages: { role: string; content: string }[]) {
  const res = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
  });

  if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';
  const outputEl = document.getElementById('chat')!;

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });

    const events = buffer.split('\n\n');
    buffer = events.pop() || '';

    for (const evt of events) {
      if (!evt.startsWith('data: ')) continue;
      const payload = evt.slice(6).trim();
      if (payload === '[DONE]') return;
      try {
        const { token } = JSON.parse(payload);
        if (token) outputEl.textContent += token;
      } catch {
        /* ignore malformed frame */
      }
    }
  }
}

The buffer.split('\n\n') pattern handles partial SSE frames across chunk boundaries. Never assume a single read() returns a complete event.

Step 4: Wire up an abort button

Streaming UIs must let users cancel generation. Use an AbortController in the browser and tie it to the same request close logic on the server.

let activeController: AbortController | null = null;

function stopGeneration() {
  activeController?.abort();
}

async function sendChat(messages: { role: string; content: string }[]) {
  activeController = new AbortController();
  const res = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
    signal: activeController.signal,
  });
  // ... rest of reader loop
}

The server’s req.on('close') fires when the fetch abort closes the underlying TCP connection, so no extra coordination is required.

Step 5: Forward cache and routing hints

OpenAI-compatible gateways accept headers like X-Request-Cache or provider-specific cache-control. If your frontend knows a prompt is cacheable (e.g., a long system prompt), forward that intent. A gateway that honors client routing directives and forwards provider cache-control hints will trim latency and cost without code changes.

In Express, pass headers through:

app.post('/api/chat', async (req, res) => {
  const upstreamHeaders: Record<string, string> = {};
  if (req.headers['x-cache-control']) {
    upstreamHeaders['x-cache-control'] = req.headers['x-cache-control'] as string;
  }
  // attach to client request via `defaultHeaders` or per-call header option
});

Keep this explicit. Silent header forwarding can leak browser origin data to upstreams you don’t control.

Step 6: Verify the end-to-end flow

Start the server and hit the endpoint with curl using -N (no buffering):

curl -N -X POST localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Count to five slowly."}]}'

Success looks like multiple data: {"token":"..."} lines printed one at a time, followed by data: [DONE]. If you see a single JSON blob after a long delay, streaming is broken—likely you forgot stream: true or a middleware buffered the response.

In the browser, open DevTools → Network → /api/chat → Response. You should see tokens arrive incrementally in the event stream. The chat pane should update without a full reload.

Production notes

• Run behind a reverse proxy (Nginx, Caddy) with buffering disabled (proxy_buffering off;); otherwise the proxy will hide your stream.
• Meter usage per token. Gateways that emit per-token usage metering let you bill or rate-limit accurately; capture usage from the final chunk if your provider sends it.
• Set a server-side timeout. The AbortController handles client disconnect, but add a max duration guard to kill runaway generations.
• Don’t log full message bodies in production. Streaming proxies can leak PII into stdout if you’re careless.

The pattern above is the minimum viable path to node.js stream llm responses browser with correct backpressure and cancellation. From here, add retry with exponential backoff on the proxy, or swap the SSE framing for a binary protocol if you outgrow text.