n4nAI
Open console →
API integrationHow-toWebSockets vs SSE for LLM Streaming

How n4n.ai streams chat completions with SSE

Learn how to implement n4n.ai sse chat completions streaming with Server-Sent Events, including runnable Python and TS code and verification.

n4n Team3 min read645 words

Audio narration

Coming soon — every post will get a voice note here.

Implementing n4n.ai sse chat completions streaming is straightforward if you treat the HTTP response as a live event stream rather than a buffered JSON blob. The gateway exposes an OpenAI-compatible /v1/chat/completions route that emits Server-Sent Events when stream: true is set, so existing OpenAI SDKs work with a base URL swap.

Step 1: Understand the SSE wire format

Server-Sent Events over HTTP is just newline-delimited data: frames. For chat completions, each frame carries a JSON chunk mirroring the non-streaming response shape, but with choices[].delta instead of choices[].message. The stream terminates with data: [DONE].

A raw capture looks like:

curl -N "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"stream":true}'

Output:

data: {"id":"x","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}
data: {"id":"x","choices":[{"delta":{"content":"Hello"}}]}
data: {"id":"x","choices":[{"delta":{"content":" there"}}]}
data: [DONE]

No WebSocket handshake, no binary framing. Just text. That keeps client code trivial and debuggable with curl.

Why not WebSockets

WebSockets give bidirectional channels, but LLM inference is strictly server-to-client after the prompt is sent. SSE rides on plain HTTP/1.1 or HTTP/2, survives proxies, and needs no special client upgrade. For most apps, SSE is the lower-complexity choice.

Step 2: Send a correctly shaped streaming request

The request body is identical to a non-streaming call except for "stream": true. If you want final token counts, add "stream_options": {"include_usage": true}. The endpoint is OpenAI-compatible; for example, the base URL is https://api.n4n.ai/v1/chat/completions. Point any OpenAI SDK at it by changing the base URL.

{
  "model": "anthropic/claude-3.5-sonnet",
  "messages": [{"role": "user", "content": "Explain SSE in one sentence."}],
  "stream": true,
  "stream_options": {"include_usage": true}
}

Send it with curl to verify the shape before writing app code:

export API_BASE="https://api.n4n.ai"
curl -N "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $N4N_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-3.5-sonnet",
    "messages": [{"role": "user", "content": "Explain SSE."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

You should see data: lines and a closing data: [DONE]. If you get a single JSON object instead, stream was ignored—check the payload.

Step 3: Parse the stream in Python

Use requests with stream=True and iterate lines. Split on data: and skip the [DONE] sentinel. Set API_BASE in your environment to the gateway URL.

import os, json, requests

url = f"{os.environ['API_BASE']}/v1/chat/completions"
headers = {
    "Authorization": f"Bearer {os.environ['N4N_KEY']}",
    "Content-Type": "application/json",
}
body = {
    "model": "anthropic/claude-3.5-sonnet",
    "messages": [{"role": "user", "content": "Count to 3."}],
    "stream": True,
    "stream_options": {"include_usage": True},
}

with requests.post(url, headers=headers, json=body, stream=True) as r:
    r.raise_for_status()
    for line in r.iter_lines(decode_unicode=True):
        if not line or not line.startswith("data: "):
            continue
        payload = line[len("data: "):]
        if payload == "[DONE]":
            break
        chunk = json.loads(payload)
        delta = chunk["choices"][0]["delta"]
        if "content" in delta:
            print(delta["content"], end="", flush=True)
        if "usage" in chunk:
            print(f"\nTokens: {chunk['usage']}")

This prints tokens as they arrive and emits usage at the end. The with block ensures the connection closes promptly.

Step 4: Parse the stream in TypeScript

In Node 18+ or the browser, fetch returns a ReadableStream. Decode chunks and split on newlines.

const base = process.env.API_BASE!;
const res = await fetch(`${base}/v1/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.N4N_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "anthropic/claude-3.5-sonnet",
    messages: [{ role: "user", content: "Count to 3." }],
    stream: true,
    stream_options: { include_usage: true },
  }),
});

if (!res.ok) throw new Error(`Stream failed: ${res.status}`);
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("\n");
  buffer = lines.pop() ?? "";
  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;
    const payload = line.slice(6);
    if (payload === "[DONE]") return;
    const chunk = JSON.parse(payload);
    const content = chunk.choices?.[0]?.delta?.content;
    if (content) process.stdout.write(content);
    if (chunk.usage) console.log("\nUsage:", chunk.usage);
  }
}

The buffer handling is mandatory: SSE frames can arrive split across TCP segments.

Step 5: Forward cache-control and routing directives

SSE does not change request headers. You can send provider-specific hints as HTTP headers or in the JSON body. The gateway forwards cache-control hints to the upstream provider and honors client routing directives (e.g., x-routing-key) so you can pin a model version or avoid a degraded region. Set them like any other header:

curl -N "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $N4N_KEY" \
  -H "x-routing-key: provider-a" \
  -H "cache-control: max-age=3600" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"stream":true}'

Your app code should pass these through unchanged; they do not affect SSE parsing.

Step 6: Handle errors and backpressure

A 429 or 5xx arrives as a normal HTTP status before the stream opens. Check res.ok / r.raise_for_status() before reading. Mid-stream errors are rarer but possible: the connection drops, or a frame contains {"error": ...} instead of a chunk. Wrap JSON parsing in try/except and treat parse failures as retriable.

Because the gateway provides automatic fallback when a provider is rate-limited or degraded, a single request may already be resilient. Still, implement a bounded retry on the client for connection-level failures. Backpressure is implicit: if your consumer is slow, TCP buffers fill and the server throttles writes. Do not block the read loop with heavy work; ship deltas to a queue and process asynchronously.

Step 7: Verify end-to-end success

Success means three things:

  1. The client receives data: frames with choices[].delta.content.
  2. The stream ends with data: [DONE].
  3. If include_usage was set, the final frame (or a trailing frame before [DONE]) carries a usage object with prompt_tokens and completion_tokens.

Quick check with curl and grep:

curl -N "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $N4N_KEY" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}],"stream":true,"stream_options":{"include_usage":true}}' \
  | grep -c "^data: "  # should be >1

Then confirm the last line is data: [DONE]. In app logs, assert that total printed content matches usage.completion_tokens within a small margin (tokenization differs from string length). That closes the loop on n4n.ai sse chat completions streaming integration.

SSE vs WebSockets for LLM streaming: final note

If you later need to cancel a generation by sending a control message from client to server mid-stream, WebSockets or HTTP/2 trails become attractive. Until then, SSE keeps your client code to a few dozen lines and works behind any proxy. The implementation above is production-shaped: header passthrough, line buffering, sentinel handling, and usage metering.

Tagsn4n-aissechat-completionsstreaming

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationSSE reconnection with Last-Event-ID for long LLM completionsHands-on guide to SSE reconnection with Last-Event-ID for long LLM completions: resume dropped streams without token duplication, including client code.API integrationStreaming GPT-4o and Claude responses: SSE or WebSocket?A practical head-to-head comparison of SSE vs WebSocket for streaming GPT-4o and Claude responses: latency, cost, ergonomics, limits, and verdict.API integrationWebSockets for voice agents, SSE for chat streamingGuide to websocket voice agent sse chat streaming: build LLM voice and chat apps with SSE and WebSockets, including code, pitfalls, and tradeoffs.API integrationBuilding a streaming chat UI with SSE and fetch()Step-by-step streaming chat ui sse fetch tutorial: build a minimal browser chat that streams LLM responses via fetch() and SSE parsing.API integrationSSE vs WebSockets for LLM streaming: which to use in 2026Practical head-to-head comparison of SSE vs WebSockets for LLM streaming in 2026: latency, cost, ergonomics, ecosystem, limits, and which to choose for software engineers building LLM apps.API integrationWhy OpenAI and Anthropic stream over SSE, not WebSocketsExplains why LLM APIs like OpenAI and Anthropic stream tokens over Server-Sent Events instead of WebSockets, covering protocol fit, infra, and tradeoffs.

All websockets vs sse for llm streaming posts →