Why LLM APIs use SSE instead of chunked plain text is a question of protocol design, not transport mechanics. Underneath, both rely on HTTP/1.1 chunked transfer encoding to push bytes to the client, but SSE layers a minimal event framing that matches how language models emit tokens, metadata, and control signals. If you strip the framing and stream raw text, you lose the contract that makes client code predictable across providers.
The transport layer already streams
HTTP chunked transfer encoding splits a response body into sized blocks terminated by a zero-length chunk. A server can start sending before it knows the total length. That is exactly what you want for a long generation.
A raw chunked plain-text response might look like this at the socket level:
HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked
7
Hello,
6
world.
0The client reads chunks and concatenates them. For a chatbot that only ever returns prose, this works. But LLM APIs rarely return only prose.
What SSE actually adds
Server-Sent Events is a W3C standard that defines a text stream format. Each event is a set of fields terminated by a blank line. The only field most LLM servers use is data:.
data: {"choices":[{"delta":{"content":"Hello"}}]}
data: {"choices":[{"delta":{"content":", world"}}]}
data: {"choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"total_tokens":13}}The double newline is the delimiter. The data: prefix tells the parser this is the payload. The format is agnostic to HTTP chunk boundaries: one SSE event can span multiple chunks, or one chunk can hold several events.
That framing is the answer to why LLM APIs use SSE: it gives you a message boundary without inventing a custom parser. You can split('\n\n') and JSON.parse each data: line. No length counting, no ambiguity about whether the stream is done.
Structured output needs framing
Modern LLM responses are not plain strings. Even in streaming mode, each delta is a JSON object carrying role, content, tool call fragments, finish reason, and usage. Consider the OpenAI-compatible shape:
{
"id": "chatcmpl-123",
"object": "chat.completion.chunk",
"model": "gpt-4o-mini",
"choices": [
{
"index": 0,
"delta": {"content": "The"},
"finish_reason": null
}
]
}If you streamed this as raw chunked text, you would receive a concatenation of JSON fragments with no reliable way to tell where one chunk ends and the next begins, especially because network chunks may split a JSON object mid-string. SSE solves this: each JSON object is one data: event.
A gateway that normalizes many backends benefits disproportionately. n4n.ai fronts 240+ models through one OpenAI-compatible endpoint and uses SSE to convert provider-specific token streams into a single predictable event sequence, while forwarding provider cache-control hints inside the same stream.
Client ergonomics and ecosystem
SSE is natively supported in browsers via EventSource, though EventSource only supports GET. LLM calls are POST, so most web clients use fetch and read the body stream manually. The parsing logic is still trivial:
const res = await fetch("/v1/chat/completions", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ messages, stream: true })
});
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const events = buffer.split("\n\n");
buffer = events.pop()!;
for (const evt of events) {
const line = evt.split("\n").find(l => l.startsWith("data: "));
if (!line) continue;
const json = JSON.parse(line.slice(6));
processDelta(json);
}
}In Python, the same pattern with httpx:
import httpx, json
with httpx.stream("POST", "https://api.example.com/v1/chat/completions",
json={"messages": msgs, "stream": True}) as r:
for line in r.iter_lines():
if line.startswith("data: "):
payload = line[len("data: "):]
if payload == "[DONE]":
break
delta = json.loads(payload)
print(delta["choices"][0]["delta"].get("content", ""), end="")The why llm apis use sse question becomes obvious when you write the client: the format is self-describing and works in every language with a line reader.
Error handling and control events
Raw chunked text has no in-band error channel. If the model crashes mid-generation, the server either closes the connection (client sees truncated text) or appends a custom sentinel. SSE permits structured error events before closure:
data: {"choices":[{"delta":{"content":"Partial"}}]}
event: error
data: {"message":"provider timeout","type":"upstream_unavailable"}A client can surface the error instead of silently returning half a sentence. SSE also supports event: ping and id: fields for keep-alive and resume, though LLM streams rarely use id because generations are not idempotent.
When a routing layer needs to switch providers mid-request because the primary is rate-limited, the SSE stream can emit a final error event and close cleanly. That is harder to bolt onto plain text without breaking parsers.
Tradeoffs: overhead vs standardization
SSE is not free. Each event carries the literal data: prefix and at least two newline characters. For a token stream emitting one JSON object per token, that is roughly 10–20 bytes of framing per event plus the JSON key overhead. Over a 1,000-token response, you might spend a few kilobytes on framing. Chunked plain text could send raw tokens with zero framing and save that bandwidth.
But consider the costs:
- You still need to encode metadata (finish reason, usage). That means a trailing JSON blob anyway.
- You must define your own delimiter or length prefix, which breaks if a token contains the delimiter.
- Browser and server libraries for SSE exist; custom chunked parsers are reinvented per team.
The CPU cost of splitting on \n\n is negligible compared to token generation latency. The bandwidth cost is minor against prompt and response sizes measured in kilobytes to megabytes.
There is also a subtle point: HTTP chunked encoding is hop-by-hop. Proxies may buffer or coalesce chunks. SSE’s line-based framing survives such buffering because it does not depend on chunk boundaries. A gateway that honors client routing directives and automatic fallback needs a format that survives intermediate proxies without losing message boundaries.
Takeaway
Use SSE for LLM streaming. The reason why LLM APIs use SSE instead of chunked plain text is that the latter gives you a pipe, while the former gives you a protocol: discrete, typed, JSON-friendly events that map cleanly to token deltas, usage reports, and mid-stream errors. The framing overhead is trivial, the ecosystem support is mature, and the alternative forces you to design a bespoke parser that every client must reimplement. If you are building an integration, parse data: lines and treat the stream as a sequence of completion chunks—do not try to stream raw text and hope the JSON survives.