Latency compared: SSE vs WebSocket for first-token time

When you measure sse vs websocket first token latency for LLM streaming, the gap is rarely about raw byte transfer. It is about connection setup, protocol negotiation, and how quickly the server can flush the first chunk. For unidirectional token delivery, SSE over HTTP/1.1 or HTTP/2 reaches first token with less overhead because there is no WebSocket upgrade handshake.

The core misconception about transport latency

Engineers often assume WebSocket must be faster because it is a “persistent bidirectional socket.” That intuition comes from chat apps where WebSocket avoids re-establishing TCP/TLS per message. But LLM streaming is fundamentally a request-response flow: the client sends a prompt, the server streams back tokens. The request payload is sent once. There is no continuous client→server traffic during generation.

The first token appears only after the server has received the full prompt, run prefill, and sampled the first logits. No transport makes prefill faster. The transport’s job is to avoid adding avoidable delay before that first byte hits the client.

What actually determines first-token latency

Three components dominate:

1. Network round trips before server processing – TCP + TLS + any application-level handshake.
2. Server time-to-first-token (TTFT) – model load, prefill, scheduling.
3. Protocol framing and flush behavior – how the server emits the first chunk and how the client parses it.

For a cold connection, TLS 1.3 adds one round trip (or zero with 0-RTT, which most APIs disable for POST). TCP adds one. HTTP/1.1 request adds one. SSE rides on that same HTTP request: the response headers are the first bytes, then the data: frame. WebSocket requires an additional HTTP Upgrade request/response exchange before any application data flows, costing at least one extra RTT unless the upgrade is pipelined on an already-established HTTP/2 connection (rare in practice).

Connection reuse changes the math

If you keep connections alive, the gap shrinks. An HTTP/1.1 keep-alive connection or HTTP/2 multiplexed stream removes TCP/TLS cost. A WebSocket that is already open removes the upgrade cost. In a warm scenario, sse vs websocket first token latency becomes nearly identical at the transport layer. The remaining difference is framing: SSE is text lines with data: prefixes; WebSocket frames are binary or text with a 2–14 byte mask. Negligible for token-sized chunks.

SSE: one request, one stream

SSE is just HTTP with Content-Type: text/event-stream and a long-lived response. The client opens a POST, sends the prompt, and reads the stream. The server can send the first token as soon as it is ready, with no further client round trips.

import httpx

with httpx.stream(
    "POST",
    "https://api.example.com/v1/chat/completions",
    json={"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}], "stream": True},
    headers={"Authorization": "Bearer KEY"},
) as r:
    for line in r.iter_lines():
        if line.startswith("data:"):
            print(line[5:])  # first non-[DONE] line is first token

The curl equivalent makes the streaming obvious:

curl -N -X POST https://api.example.com/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}],"stream":true}'

SSE integrates with CDN and load-balancer timeouts naturally because it is HTTP. You get per-request headers, standard status codes, and cache-control. Browsers have native EventSource for GET, but for POST-based LLM calls you use fetch with a reader.

Avoiding proxy buffering

Many reverse proxies buffer responses by default. For SSE you must set X-Accel-Buffering: no (nginx) or disable buffering at the edge. Failure to do so adds hundreds of milliseconds before the first token reaches the client—a far larger penalty than any WebSocket upgrade. This operational trap is the real cause of “slow SSE” complaints in postmortems, not the protocol itself.

WebSocket: upgrade then stream

WebSocket starts as an HTTP request with Upgrade: websocket. The server replies 101 Switching Protocols. Only then can the client send the prompt and the server stream tokens. That upgrade is an extra hop.

const ws = new WebSocket("wss://api.example.com/v1/stream");
ws.onopen = () => {
  ws.send(JSON.stringify({ model: "gpt-4o", messages: [{ role: "user", content: "hi" }] }));
};
ws.onmessage = (ev) => {
  const data = JSON.parse(ev.data);
  if (data.type === "token") process.stdout.write(data.token);
};

The upside: after the upgrade, the same socket can send control messages (cancel, temperature tweak) and receive multiple interleaved generations if the server supports multiplexing. For a single generation, that capability is unused dead weight.

WebSocket frames from server to client are unmasked, so downstream overhead is small. But the client→server mask requirement adds a few bytes per send; irrelevant for a single prompt but worth noting in high-frequency duplex use.

Measuring sse vs websocket first token latency in practice

Set up a trivial proxy that adds artificial prefill delay of 200 ms, then compare cold and warm connections from a client in the same region.

Cold SSE: TCP+TLS (2 RTT) + POST (1 RTT) + server 200 ms. Cold WebSocket: TCP+TLS (2 RTT) + GET upgrade (1 RTT) + send prompt (1 RTT) + server 200 ms.

On a 30 ms RTT link, SSE first token ≈ 200 + 90 = 290 ms; WebSocket ≈ 200 + 120 = 320 ms. The 30 ms difference is the upgrade and prompt send. Not huge, but real, and it compounds when the client is mobile or cross-continent.

A minimal timing harness in Python:

import time, httpx

start = time.perf_counter()
with httpx.stream("POST", "https://api.example.com/v1/chat/completions",
                  json={"model":"gpt-4o","messages":[{"role":"user","content":"hi"}],"stream":True},
                  headers={"Authorization":"Bearer KEY"}) as r:
    for line in r.iter_lines():
        if line.startswith("data:") and "[DONE]" not in line:
            print("TTFT:", time.perf_counter() - start)
            break

Run the same against a WebSocket endpoint and compare. You will find warm connections converge; cold ones show the upgrade penalty.

HTTP/2 and head-of-line blocking

A frequent worry is that HTTP/1.1 SSE suffers from browser per-domain connection limits (six). For a chat UI with one active stream, that is irrelevant. HTTP/2 multiplexing allows many SSE streams on one connection without head-of-line blocking, matching WebSocket’s concurrency story for server→client data. WebSocket over HTTP/2 exists but most edge proxies terminate it into a plain TCP WebSocket, losing the multiplexing benefit. If you need many concurrent streams from a browser, HTTP/2 SSE is often simpler than managing a WebSocket multiplexing layer yourself.

Where WebSocket earns its keep

If your product does any of the following, WebSocket is justified:

• Interactive cancellation with low overhead – user hits stop; you send a 5-byte close frame instead of aborting an HTTP request (which may still wait for server to notice TCP reset).
• Server-initiated control – model asks for clarification, server pushes a UI hint mid-stream.
• Multiplexing many generations on one connection – a coding agent streaming 10 file edits concurrently without opening 10 HTTP connections.
• Bidirectional fine-grained protocol – client streams partial audio while server streams tokens (duplex speech).

For a typical chat completion UI, none of these apply. The user sends one prompt, gets a stream, and closes. SSE matches that shape exactly.

Gateway considerations

A gateway that fronts multiple providers must handle provider degradation without inflating TTFT. An OpenAI-compatible endpoint such as n4n.ai serves 240+ models behind one SSE route, relying on HTTP semantics for caching and fallback. Because the client contract is SSE, first-token latency stays consistent whether the request lands on a cached provider or triggers automatic fallback after a provider degradation. The gateway honors client routing directives via headers and forwards provider cache-control hints; none of that requires a WebSocket control channel.

If you build your own gateway, note that SSE simplifies metering: each chunk is a byte range on an HTTP response, and per-token usage can be sent as a final data: event with usage JSON. WebSocket forces you to define your own message types for usage, which is fine but more code.

Decision matrix

Takeaway

For the common case of streaming LLM completions from a client to a server that speaks HTTP, SSE wins on sse vs websocket first token latency because it removes a protocol handshake and maps cleanly to request-response semantics. Use WebSocket only when you need persistent bidirectional control or heavy multiplexing that justifies its connection model. Start with SSE, measure TTFT from your users’ networks, and switch transports only if the application’s interaction pattern demands it—not because WebSocket sounds faster.