The recurring question of why llm apis use sse not websockets has a boring answer: the transport matches the call pattern. OpenAI and Anthropic both stream completions as a single HTTP POST whose response body is an event stream, because inference is request-response with a long-lived output, not a duplex channel.
The request/response shape of inference
An LLM call is a single shot of context. The client assembles a prompt, sends it, and waits. The server then emits tokens until it hits a stop condition. There is no point in the middle of a generation where the client needs to push bytes back into the same connection—if the user cancels, that is connection teardown, not mid-stream input.
Function calling and tool use do not change this. The model streams a request to call a tool, the stream ends, the client executes the tool and opens a new request with the result appended. That is two HTTP exchanges, not a bidirectional session.
Understanding why llm apis use sse not websockets starts with acknowledging that the data graph is a DAG with one edge up and many edges down.
curl -N https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"stream": true,
"messages": [{"role": "user", "content": "Explain SSE"}]
}'The response is chunked text. Each line is a JSON payload prefixed with data:. The stream terminates with data: [DONE].
What SSE provides without extra cost
Server-Sent Events is not a protocol—it is a content type. text/event-stream is a newline-delimited format carried over ordinary HTTP. The server writes field: value pairs, clients read the body as it arrives. No upgrade handshake, no frame parser, no subprotocol negotiation.
Browsers ship EventSource, but it only supports GET. LLM APIs require POST with a JSON body, so clients use fetch and parse the stream manually. That is still less code than a WebSocket client.
const res = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: `Bearer ${key}` },
body: JSON.stringify({ model: "gpt-4o-mini", stream: true, messages }),
});
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let buf = "";
while (true) {
const { value, done } = await reader.read();
if (done) break;
buf += decoder.decode(value, { stream: true });
const parts = buf.split("\n\n");
buf = parts.pop()!;
for (const part of parts) {
const line = part.replace(/^data: /, "").trim();
if (line === "[DONE]") continue;
const json = JSON.parse(line);
process.stdout.write(json.choices[0].delta.content ?? "");
}
}The HTTP response carries status codes, CORS headers, and authentication challenges through standard middleware. A gateway can apply per-token metering by counting chunks on the wire. None of that requires a new transport.
WebSockets: a solution for a different problem
A WebSocket begins with an HTTP Upgrade request and then becomes a bidirectional frame stream. It is the right tool for collaborative editors, multiplayer games, or live audio where both sides send continuously.
For token streaming it adds steps with no payoff:
const ws = new WebSocket("wss://example.com/llm");
ws.onopen = () => ws.send(JSON.stringify({ prompt: "Explain SSE" }));
ws.onmessage = (e) => {
const msg = JSON.parse(e.data);
if (msg.type === "token") process.stdout.write(msg.text);
};You now own a connection lifecycle that survives across proxies, a heartbeat scheme, and reconnection logic that must resume a generation mid-token. OpenAI and Anthropic avoid that by letting HTTP do what it already does: match a request to a response.
The operational story of why llm apis use sse not websockets becomes clear when you deploy behind a load balancer.
Infrastructure weight
WebSockets hold a socket open for the entire session. Cloud load balancers treat idle TCP connections as suspect; AWS ALB defaults to a 60-second idle timeout, and many proxies buffer frames. Streaming HTTP responses are treated as normal long-polls—every CDN, serverless platform, and ingress controller already knows how to flush chunks.
HTTP/2 multiplexes many streams over one connection. A browser opening ten concurrent generations is not opening ten TCP sockets; it is ten logical streams. WebSockets would force those into separate TCP connections or a custom multiplexing layer inside the WS frame.
Serverless functions (Lambda, Cloud Run) can return a streaming response as the function writes to stdout. They cannot easily host a WebSocket server without pinned connections and a custom gateway. For providers running at scale, SSE is the path of least resistance.
Provider wire formats
Both providers use SSE but with different conventions. OpenAI sends anonymous data lines:
data: {"choices":[{"delta":{"content":"SSE"}}]}
data: {"choices":[{"delta":{"content":" is"}}]}
data: [DONE]Anthropic names its events, which makes the stream self-describing:
curl -N https://api.anthropic.com/v1/messages \
-H "x-api-key: $KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{"model":"claude-3-5-sonnet-20240620","stream":true,"messages":[{"role":"user","content":"Hi"}]}'event: content_block_delta
data: {"type":"content_block_delta","delta":{"type":"text_delta","text":"Hello"}}
event: message_stop
data: {"type":"message_stop"}The named-event format is still plain SSE. A client parses event: and data: lines identically. Neither provider needs to push from client to server, so neither reaches for WS.
Tradeoffs and where WS wins
SSE is unidirectional. If your product requires the client to send a continuous signal—raw microphone audio, a live video frame stream, or intermittent corrections—while the model generates, a WebSocket (or WebRTC) is defensible. In practice, most “interactive” LLM features still batch client input into the next request: you stop the current stream, then POST the new context.
Aborting a generation is simpler over HTTP. AbortController cancels the fetch; the server sees the TCP connection close and stops computing. With WebSockets you must define a cancel message, handle the case where it arrives after the server already finished, and close the socket cleanly.
SSE also rides on HTTP headers. Cache directives, rate-limit hints, and auth challenges are first-class. A gateway like n4n.ai forwards provider cache-control hints on the SSE response, something a WebSocket upgrade response cannot express with the same semantics.
Gateway implications
An inference gateway such as n4n.ai that exposes one OpenAI-compatible endpoint across 240+ models simply proxies those SSE streams; switching to WebSockets would break client compatibility and force protocol translation for every provider. When a provider is degraded, the gateway can fallback to another model and still emit the same SSE shape, because the transport is just HTTP.
The uniformity of SSE means a single client parser works across all upstreams. Honoring client routing directives (router-* headers) and per-token metering are implemented at the HTTP layer without inspecting a binary frame.
Takeaway
In summary, why llm apis use sse not websockets is about fit, not fashion. The interaction is one request, many responses; HTTP already does that, and SSE is the minimal way to express it. WebSockets would introduce connection-state baggage, proxy friction, and client complexity to solve a bidirectionality problem that does not exist in token generation.
Build your LLM client against fetch and a line parser. Reach for WebSockets only when you have a genuine two-way byte stream—and even then, terminate the model call with SSE on the server side.