The openai node.js sdk streaming interface is the right tool when you need tokens in the browser or terminal as they generate, not after a multi-second wait. This tutorial builds a small Node script that streams a chat completion, then layers in error handling, abort control, and HTTP delivery.
Prerequisites
- Node.js 18 or newer (global
fetchand async iterators available). - An API key from OpenAI or any OpenAI-compatible provider.
npm install openaiin a fresh project directory.
mkdir stream-demo && cd stream-demo
npm init -y
npm install openaiSet your key in the environment:
export OPENAI_API_KEY="sk-..."Set up the client
The SDK reads OPENAI_API_KEY automatically. For openai node.js sdk streaming you only need to pass stream: true in the request. Create a file basic.mjs:
import OpenAI from 'openai';
const client = new OpenAI(); // uses process.env.OPENAI_API_KEY
export { client };Stream your first completion
Request a short explanation and print each delta as it arrives.
import { client } from './basic.mjs';
const stream = await client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'Explain backpressure in 3 sentences.' }],
stream: true,
});
for await (const chunk of stream) {
const delta = chunk.choices[0]?.delta?.content || '';
process.stdout.write(delta);
}Expected output
The terminal prints a coherent three-sentence answer character-by-character, ending without a newline. Example:
Backpressure is the resistance that builds up when a consumer processes data slower than the producer emits it. In streams, it prevents memory exhaustion by signaling the producer to slow down. Without it, buffers grow until the process crashes or gets OOM-killed.Accumulate and inspect usage
Streaming does not return usage by default. Set stream_options.include_usage to get a final chunk with token counts.
let full = '';
const stream = await client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'Count to 5.' }],
stream: true,
stream_options: { include_usage: true },
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content || '';
full += content;
if (chunk.usage) {
console.log('\n---');
console.log('Full text:', full);
console.log('Usage:', chunk.usage);
}
}Expected output
1
2
3
4
5
---
Full text: 1
2
3
4
5
Usage: { prompt_tokens: 10, completion_tokens: 5, total_tokens: 15 }(Exact token counts vary by model tokenizer.)
Abort and error handling
OpenAI Node SDK accepts an AbortSignal. Wire it to a timeout or a user action. Robust openai node.js sdk streaming code always wraps the loop in try/catch.
import { client } from './basic.mjs';
const controller = new AbortController();
setTimeout(() => controller.abort(), 1500);
try {
const stream = await client.chat.completions.create(
{
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'Write a 200-word essay on rust.' }],
stream: true,
},
{ signal: controller.signal }
);
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
} catch (err) {
if (err.name === 'AbortError') {
console.error('\n[aborted] stream terminated early');
} else {
console.error('\n[error]', err.message);
}
}Expected output on abort
After ~1.5s the process prints partial text then:
[aborted] stream terminated earlyPoint the SDK at any OpenAI-compatible endpoint
The SDK is not locked to OpenAI. If you route through a gateway such as n4n.ai, which exposes one OpenAI-compatible endpoint for 240+ models with automatic fallback when a provider is degraded, change baseURL and apiKey.
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: process.env.N4N_API_KEY,
baseURL: 'https://api.n4n.ai/v1',
});
const stream = await client.chat.completions.create({
model: 'anthropic/claude-3.5-sonnet',
messages: [{ role: 'user', content: 'Hi from a gateway.' }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}The streaming contract is identical; only the base URL and model name differ.
Stream over HTTP with Express
In a web server you write deltas to the response object. Node’s res.write handles backpressure internally; for high concurrency you should check its return value.
import express from 'express';
import { client } from './basic.mjs';
const app = express();
app.get('/stream', async (req, res) => {
res.setHeader('Content-Type', 'text/plain; charset=utf-8');
res.setHeader('Cache-Control', 'no-cache');
try {
const stream = await client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'Tell me a short joke.' }],
stream: true,
});
for await (const chunk of stream) {
const ok = res.write(chunk.choices[0]?.delta?.content || '');
if (!ok) await new Promise((r) => res.once('drain', r));
}
res.end();
} catch (err) {
res.status(500).end('stream error');
}
});
app.listen(3000, () => console.log('listen on :3000'));Run node server.mjs and curl localhost:3000/stream to see tokens arrive incrementally.
Handling client disconnects
If the HTTP client hangs up, continue writing to res throws. Listen for close and abort the upstream stream.
app.get('/stream', async (req, res) => {
const controller = new AbortController();
req.on('close', () => controller.abort());
res.setHeader('Content-Type', 'text/plain');
const stream = await client.chat.completions.create(
{ model: 'gpt-4o-mini', messages: [{ role: 'user', content: 'Story.' }], stream: true },
{ signal: controller.signal }
);
try {
for await (const chunk of stream) {
if (!res.write(chunk.choices[0]?.delta?.content || '')) {
await new Promise((r) => res.once('drain', r));
}
}
res.end();
} catch (err) {
if (err.name !== 'AbortError') console.error(err);
res.end();
}
});Gotchas: partial JSON and tool calls
Streaming returns raw text deltas. If you ask for JSON, do not JSON.parse a delta. Accumulate the full string, then parse after the stream ends. Same for tool-call arguments: the tool_calls array builds across chunks, each with a function.arguments delta string. Concatenate arguments per index, then parse when chunk.choices[0].finish_reason === 'tool_calls'.
let args = '';
for await (const chunk of stream) {
const tc = chunk.choices[0]?.delta?.tool_calls?.[0];
if (tc?.function?.arguments) args += tc.function.arguments;
}
const parsed = JSON.parse(args); // safe only after stream completeWrap-up
You now have a working pattern for openai node.js sdk streaming: minimal client setup, delta iteration, usage extraction, abort signals, and HTTP piping with backpressure. Copy the Express handler as a starting point and adapt the model and prompt to your product.