Most LLM backends buffer the full completion before sending it to the client, which wastes time and hurts perceived latency. Using express.js res.write llm streaming, you can forward tokens to the browser the moment they arrive from the model provider, keeping your Node.js service thin and responsive.
Prerequisites
You need Node.js 18 or newer because global fetch and web stream APIs are stable there. Install Express with npm i express. Set your provider API key in the environment—for example OPENAI_API_KEY. If you run behind Nginx or another proxy, confirm it is not buffering upstream responses (we will set the header to disable that).
Step 1: Disable buffering and set transport headers
Express itself does not buffer responses, but proxies and compression middleware do. If you enable compression() or sit behind Nginx with default config, chunked output will stall until the buffer fills. Set headers that tell intermediaries to pass through and declare the response as chunked.
import express from 'express';
const app = express();
app.get('/stream', (req, res) => {
res.setHeader('Content-Type', 'text/plain; charset=utf-8');
res.setHeader('Cache-Control', 'no-cache, no-transform');
res.setHeader('X-Accel-Buffering', 'no'); // Nginx: do not buffer
res.setHeader('Transfer-Encoding', 'chunked');
// streaming logic goes here
});Do not call res.json() or res.send() inside the loop. Those methods finalize the response and close the socket. res.write() pushes a raw chunk to the underlying socket and leaves the connection open for the next token.
Step 2: Request a streaming completion from the model
Any OpenAI-compatible endpoint accepts stream: true in the request body. Node 18+ ships global fetch, so you can avoid SDK bloat and retain full control of the byte stream. The snippet below posts a chat completion request and returns the raw ReadableStream.
async function streamCompletion(prompt, signal) {
const resp = await fetch('https://api.openai.com/v1/chat/completions', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: prompt }],
stream: true,
}),
signal,
});
if (!resp.ok) throw new Error(`Upstream ${resp.status}`);
return resp.body;
}If you front your model calls with a gateway such as n4n.ai, the same request shape works and you get automatic fallback when a provider is rate-limited or degraded, without changing the parsing code. The express.js res.write llm streaming pattern is agnostic to which upstream you use as long as it speaks SSE.
Step 3: Parse the SSE byte stream
The upstream sends Server-Sent Events. Each event is a line starting with data: and events are separated by a blank line (\n\n). The terminal event is data: [DONE]. We decode the byte stream, split on the separator, and parse JSON only for lines that carry payloads.
async function* parseSSE(stream) {
const reader = stream.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const parts = buffer.split('\n\n');
buffer = parts.pop() ?? '';
for (const part of parts) {
const line = part.trim();
if (!line.startsWith('data:')) continue;
const payload = line.slice(5).trim();
if (payload === '[DONE]') return;
yield JSON.parse(payload);
}
}
}Some providers also send periodic : keepalive comments. The startsWith('data:') guard skips those. Always pass { stream: true } to decode so multi-byte UTF-8 characters split across network chunks are reassembled correctly.
Step 4: Forward tokens with res.write and handle backpressure
res.write() returns false when the kernel socket buffer is full. If you ignore that return value, Node buffers the pending chunks in memory, and a slow client (or a mobile network) will silently inflate your RAM usage. Await the drain event before pulling the next token.
function writeChunk(res, text) {
return new Promise((resolve) => {
const ok = res.write(text);
if (ok) resolve();
else res.once('drain', resolve);
});
}Wire the pieces together in the route handler:
app.get('/stream', async (req, res) => {
res.setHeader('Content-Type', 'text/plain; charset=utf-8');
res.setHeader('Cache-Control', 'no-cache, no-transform');
res.setHeader('X-Accel-Buffering', 'no');
const controller = new AbortController();
req.on('close', () => controller.abort());
try {
const body = await streamCompletion(req.query.prompt ?? 'Hello', controller.signal);
for await (const event of parseSSE(body)) {
const token = event.choices?.[0]?.delta?.content;
if (token) await writeChunk(res, token);
}
res.end();
} catch (err) {
if (err.name !== 'AbortError') {
if (!res.headersSent) res.status(500);
res.write(`\n[error] ${err.message}`);
}
res.end();
}
});The await writeChunk call is what makes the express.js res.write llm streaming loop safe under load: the upstream fetch pauses while the socket catches up.
Step 5: Abort the upstream call on client disconnect
A user closing the tab should not keep your server pulling tokens from the LLM and paying for unused generation. req.on('close') fires when the HTTP socket drops. Call controller.abort() to cancel the fetch call. The fetch rejects with AbortError; catch it and exit without writing an error to a dead socket.
} catch (err) {
if (err.name === 'AbortError') return; // client gone, nothing to do
if (!res.headersSent) res.status(500);
res.write(`\n[error] ${err.message}`);
res.end();
}Step 6: Verify the stream with curl and a browser
Start the server (node server.js) and hit the route with curl using -N to disable curl’s own buffering:
curl -N "http://localhost:3000/stream?prompt=Explain%20TCP%20slow%20start"You should see words appear one or a few at a time, not all at the end. If you get the full block after a multi-second pause, a proxy is still buffering—check Nginx or remove compression() from the Express stack.
Inspect headers with -i:
curl -i -N "http://localhost:3000/stream?prompt=Hi"Look for Transfer-Encoding: chunked and X-Accel-Buffering: no. In the browser, consume the response with the Fetch API:
const resp = await fetch('/stream?prompt=Why%20is%20backpressure%20important');
const reader = resp.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
document.body.append(decoder.decode(value)); // incremental render
}Success means tokens render incrementally in the DOM and the Network panel shows a 200 with growing response size and no full-page wait.
Step 7: Production hardening
For real traffic, add three things. First, cap concurrency per IP with a simple in-memory semaphore or a Redis counter; LLM streams hold sockets open for seconds. Second, log only the completion id and token count, not the raw prompt, to avoid spilling PII into disk. Third, set a proxy_read_timeout of 0 or a few minutes on Nginx for the /stream location so the upstream proxy does not kill the connection mid-generation.
If you use a gateway that provides per-token usage metering, read the final usage object from the last SSE event (or the trailing HTTP trailer) and ship it to your billing system. The express.js res.write llm streaming code does not need to change; you just capture the last parsed event before [DONE].
Gotchas that will bite you
- Accidental
res.json(): Calling it inside the loop closes the response. Use onlyres.writeandres.end. - Multi-byte characters: Without
{ stream: true }inTextDecoder.decode, emojis or non-Latin text will corrupt. - Status after write: Once
res.writeis called,res.statusis ignored. Send errors as a trailing line of text. - Helmet/security headers: Some CSP rules block
text/plainfrom rendering in certain contexts; setContent-Security-Policyappropriately if you inject into the DOM.
Complete minimal server
import express from 'express';
const app = express();
app.get('/stream', async (req, res) => {
res.setHeader('Content-Type', 'text/plain; charset=utf-8');
res.setHeader('Cache-Control', 'no-cache, no-transform');
res.setHeader('X-Accel-Buffering', 'no');
const controller = new AbortController();
req.on('close', () => controller.abort());
try {
const resp = await fetch('https://api.openai.com/v1/chat/completions', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: req.query.prompt ?? 'Hi' }],
stream: true,
}),
signal: controller.signal,
});
if (!resp.ok) throw new Error(`Upstream ${resp.status}`);
const reader = resp.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const parts = buffer.split('\n\n');
buffer = parts.pop() ?? '';
for (const part of parts) {
const line = part.trim();
if (!line.startsWith('data:')) continue;
const payload = line.slice(5).trim();
if (payload === '[DONE]') continue;
const json = JSON.parse(payload);
const token = json.choices?.[0]?.delta?.content;
if (token) {
const ok = res.write(token);
if (!ok) await new Promise(r => res.once('drain', r));
}
}
}
res.end();
} catch (err) {
if (err.name !== 'AbortError' && !res.headersSent) res.status(500);
if (err.name !== 'AbortError') res.write(`\n[error] ${err.message}`);
res.end();
}
});
app.listen(3000);This is the full express.js res.write llm streaming path: headers, upstream call, SSE parse, backpressure, abort, and verification. Drop it behind a gateway that honors client routing directives and you get provider redundancy without touching the route.