When you build a client that consumes LLM completions over HTTP, you eventually need to reconnect sse stream network interruption without losing the generated text or hammering the server. Server-Sent Events are simple in the happy path, but a dropped WiFi connection or a provider failover mid-stream forces you to handle resumption explicitly. This tutorial builds a minimal TypeScript client that recovers from interruptions against an OpenAI-compatible streaming endpoint.
Prerequisites
- Node.js 18+ (global
fetchandTextDecoderavailable) tsxfor running TypeScript directly (npm i -g tsx)- An API key for an OpenAI-compatible chat completions endpoint. For the first half we run a local SSE server. For the LLM part, the same code works against n4n.ai’s single OpenAI-compatible endpoint that addresses 240+ models and provides automatic fallback when a provider is rate-limited.
The SSE wire format
A minimal SSE response is plain text. Each event is separated by a blank line. Fields are id:, event:, and data:.
curl -N -H "Authorization: Bearer $KEY" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"say hi"}],"stream":true}' \
https://api.example.com/v1/chat/completionsExpected output (truncated):
data: {"choices":[{"delta":{"content":"Hello"}}]}
data: {"choices":[{"delta":{"content":" there"}}]}
data: [DONE]There is no id: field in the OpenAI format. That matters later.
Naive streaming client (breaks on drop)
This client prints tokens but throws on any network error. It cannot reconnect sse stream network interruption.
// naive.ts
const URL = "https://api.example.com/v1/chat/completions";
const KEY = process.env.KEY!;
async function streamOnce() {
const res = await fetch(URL, {
method: "POST",
headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
body: JSON.stringify({
model: "gpt-3.5-turbo",
messages: [{ role: "user", content: "count to 3" }],
stream: true,
}),
});
const reader = res.body!.getReader();
const dec = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
process.stdout.write(dec.decode(value));
}
}
streamOnce().catch((e) => console.error("died:", e.message));Run it. If you pull your network cable, the promise rejects with fetch failed and the stream is gone. No recovery.
Build a reconnecting SSE client
First, a tiny local SSE server that respects Last-Event-ID. This proves the resume logic.
// server.ts
import http from "http";
http
.createServer((req, res) => {
res.writeHead(200, {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
Connection: "keep-alive",
});
let id = Number(req.headers["last-event-id"] || 0);
const timer = setInterval(() => {
id++;
res.write(`id: ${id}\n`);
res.write(`data: tick ${id}\n\n`);
}, 1000);
req.on("close", () => clearInterval(timer));
})
.listen(3000, () => console.log("sse on :3000"));Now the client that will reconnect sse stream network interruption using Last-Event-ID:
// reconnect.ts
function parseSSE(raw: string) {
const ev: any = {};
for (const line of raw.split("\n")) {
if (line.startsWith("id:")) ev.id = line.slice(3).trim();
if (line.startsWith("data:")) ev.data = line.slice(5).trim();
}
return ev;
}
async function run() {
let lastId = 0;
while (true) {
try {
const res = await fetch("http://localhost:3000/sse", {
headers: { "Last-Event-ID": String(lastId) },
});
const reader = res.body!.getReader();
const dec = new TextDecoder();
let buf = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buf += dec.decode(value, { stream: true });
let idx;
while ((idx = buf.indexOf("\n\n")) >= 0) {
const raw = buf.slice(0, idx);
buf = buf.slice(idx + 2);
const ev = parseSSE(raw);
if (ev.id) lastId = Number(ev.id);
console.log("got", ev.data);
}
}
} catch (e) {
console.error("stream broken, retrying in 1s");
await new Promise((r) => setTimeout(r, 1000));
}
}
}
run();Start the server, run the client. Expected output:
got tick 1
got tick 2
got tick 3Now kill and restart the server. The client prints stream broken, retrying in 1s, then resumes from tick 4 because it sent Last-Event-ID: 3. That is the core pattern to reconnect sse stream network interruption when the server supports IDs.
Adapting to LLM streams (no server-side IDs)
OpenAI-compatible endpoints do not emit id: per token. You cannot ask the server to resume from token 42. You have two options:
- Restart the whole request and dedupe overlapping text in your UI.
- Use a gateway that supports resumable streams via a client-supplied session header.
We will implement option 1: a generator that retries with exponential backoff and skips a prefix if the model repeats itself after reconnect.
// resilient-chat.ts
interface Opts {
baseUrl: string;
apiKey: string;
model: string;
messages: any[];
}
async function* resilientChat(opts: Opts) {
let attempt = 0;
let skipPrefix = "";
while (true) {
try {
const res = await fetch(`${opts.baseUrl}/v1/chat/completions`, {
method: "POST",
headers: {
Authorization: `Bearer ${opts.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: opts.model,
messages: opts.messages,
stream: true,
}),
});
if (!res.ok) throw new Error(`status ${res.status}`);
const reader = res.body!.getReader();
const dec = new TextDecoder();
let buf = "";
while (true) {
const { done, value } = await reader.read();
if (done) return;
buf += dec.decode(value, { stream: true });
let nl;
while ((nl = buf.indexOf("\n")) >= 0) {
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line.startsWith("data:")) continue;
const data = line.slice(5).trim();
if (data === "[DONE]") return;
const json = JSON.parse(data);
const token = json.choices?.[0]?.delta?.content || "";
if (skipPrefix && token.startsWith(skipPrefix)) {
skipPrefix = "";
continue;
}
yield token;
}
}
} catch (e) {
attempt++;
const backoff = Math.min(1000 * 2 ** attempt, 30000);
console.error(`Network interruption, will reconnect sse stream network interruption in ${backoff}ms`);
await new Promise((r) => setTimeout(r, backoff));
}
}
}
// usage
(async () => {
for await (const tok of resilientChat({
baseUrl: "https://api.example.com",
apiKey: process.env.KEY!,
model: "gpt-3.5-turbo",
messages: [{ role: "user", content: "write a haiku about tcp" }],
})) {
process.stdout.write(tok);
}
})();If the connection drops at “the packets”, the catch block logs the backoff, then the loop re-issues the POST. The model may regenerate from the start; skipPrefix (in a real app, set it to the last 20 chars you printed) drops the duplicate. This is crude but works for chat UIs where minor duplication is worse than a short gap.
Testing the interruption
Simulate a drop without killing the process:
# start client in one terminal
tsx resilient-chat.ts
# in another, block the port briefly (Linux)
sudo iptables -A OUTPUT -p tcp --dport 443 -j DROP
sleep 5
sudo iptables -D OUTPUT -p tcp --dport 443 -j DROPYou will see:
Network interruption, will reconnect sse stream network interruption in 2000ms
Network interruption, will reconnect sse stream network interruption in 4000msThen the stream continues. Because n4n.ai honors client routing directives and forwards provider cache-control hints, you can pin a specific provider with a header and still rely on its automatic fallback to avoid server-side drops, but the client-side retry above covers local network failures.
Checklist for production
- Use exponential backoff with jitter; cap at 30s.
- Track printed text and dedupe on restart.
- Set a total attempt ceiling; surface permanent failures to the UI.
- For browser clients,
fetch+ReadableStreamis fine; do not useEventSourcefor POST bodies. - If you control the server, emit
id:and useLast-Event-ID—it is the clean solution to reconnect sse stream network interruption.
The code here is minimal by design. Drop it into a helper module and wrap your completion calls; you will stop losing generations every time a laptop sleeps mid-stream.