A cloudflare worker timeout llm streaming failure usually surprises engineers because the code looks correct. You open a fetch to an OpenAI-compatible endpoint, pipe the stream to the client, and watch it die after 30 seconds with a 524 or truncated SSE. The root cause is that Cloudflare’s execution model treats your function as a short-lived compute sandbox, not a persistent proxy.
The execution model that bites you
Cloudflare Workers run on the V8 isolate platform with two distinct limits: CPU time and wall-clock time. CPU time is the actual milliseconds spent executing JavaScript. On the free plan that budget is 50ms; on paid it is 500ms by default. Wall-clock time is the total duration of the request, including time spent awaiting network I/O. Standard Worker requests are capped at 30 seconds regardless of plan.
LLM inference violates both assumptions. A single chat completion can take 10–60 seconds of generation, and streaming means the connection stays open while tokens dribble out. Even if your Worker spends near-zero CPU while piping bytes, the wall-clock timer keeps running. When it hits the cap, Cloudflare resets the connection.
// What you think you wrote
export default {
async fetch(req: Request) {
const r = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: { "content-type": "application/json", authorization: `Bearer ${KEY}` },
body: JSON.stringify({ model: "gpt-4o", stream: true, messages: [{ role: "user", content: "write 2000 words" }] })
});
return new Response(r.body, { headers: { "content-type": "text/event-stream" } });
}
}That handler returns immediately after initiating the upstream fetch, but the runtime must keep the Worker instance alive to pump r.body to the browser. The 30-second wall clock starts at the first byte and ends when the stream closes or the limit hits.
Why streaming makes it worse
Non-streaming LLM calls fail fast: you send a prompt, wait a few seconds, get JSON, return it. Streaming explicitly extends the response phase to the full generation length. A 1,500-token reply at 30 tokens/sec is 50 seconds of open socket. That alone exceeds the cloudflare worker timeout llm streaming budget by 20 seconds.
There is a second trap: subrequest limits. A Worker may issue at most 50 subrequests per invocation. If you implement token buffering, retries, or chunk reassembly inside the Worker, you can burn those quickly. Streaming a single upstream response is one subrequest, but any fallback logic multiplies it.
# Typical error from the edge
$ curl -i https://my-worker.workers.dev/stream
HTTP/2 524
date: Wed, 01 Jan 2025 00:00:30 GMT
# connection closed, client received 412 tokens then nothingThe 524 is Cloudflare’s “timeout” status, not an application error. Your logs show success; the client sees a truncated stream.
Naive proxy and its failure
The minimal proxy above works for short prompts. For a “summarize this sentence” call it may finish in 2 seconds. The moment you ask for a long blog post, the cloudflare worker timeout llm streaming boundary is reached.
Engineers often try to “fix” this by adding event.waitUntil or returning a Promise early. That does not extend the response stream lifetime. waitUntil only keeps the isolate alive for background tasks after a response is sent; it cannot keep the response socket open past the request timeout.
// This does NOT help streaming
export default {
async fetch(req: Request) {
const ctx = req as any;
const r = await fetch(UPSTREAM, { body: req.body, method: "POST", headers: { "content-type": "application/json" } });
ctx.waitUntil(Promise.resolve()); // no-op for stream duration
return new Response(r.body);
}
}Option 1: Durable Objects for long-lived streams
Durable Objects (DO) are the correct Cloudflare primitive for stateful, longer-lived connections. A DO instance is not bound by the same 30-second wall clock as a Worker front door; it can hold a connection open far longer (up to the platform’s extended duration for DO requests, which is minutes, not seconds) as long as it is actively transferring data or awaiting I/O.
You put the streaming logic inside the DO and route the client to it via a lightweight Worker that just forwards to the object.
// worker entry
export default {
async fetch(req: Request) {
const id = env.STREAMER.idFromName("global");
const stub = env.STREAMER.get(id);
return stub.fetch(req);
}
}
// durable object
export class Streamer {
async fetch(req: Request) {
const upstream = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: { "content-type": "application/json", authorization: `Bearer ${KEY}` },
body: JSON.stringify({ model: "gpt-4o", stream: true, messages: [{ role: "user", content: "long story" }] })
});
return new Response(upstream.body, { headers: { "content-type": "text/event-stream" } });
}
}Tradeoffs: DOs introduce a new class, a binding, and a migration path. They are billed per duration and GB-second, so cost rises with stream length. They also do not solve provider rate limits—if OpenAI 429s, your DO still needs fallback logic.
Option 2: Client-side direct to inference gateway
If your architecture permits, remove the Worker from the hot path entirely. Modern LLM gateways expose an OpenAI-compatible endpoint with CORS enabled. The browser opens the stream directly to the gateway, which forwards to the provider and streams back. The cloudflare worker timeout llm streaming problem disappears because there is no Cloudflare function in the loop.
An inference gateway like n4n.ai provides a single OpenAI-compatible endpoint covering 240+ models, with automatic fallback when a provider is rate-limited or degraded, and per-token usage metering. You point the client straight at it:
const res = await fetch("https://api.n4n.ai/v1/chat/completions", {
method: "POST",
headers: { "content-type": "application/json", authorization: `Bearer ${USER_KEY}` },
body: JSON.stringify({ model: "auto", stream: true, messages })
});
const reader = res.body.getReader();The gateway honors client routing directives and forwards provider cache-control hints, so you keep control over model selection. The tradeoff is key exposure: you must issue short-lived user tokens or restrict origins via the gateway’s auth layer. For internal tools or authenticated SPAs this is acceptable.
Option 3: Buffer and poll (anti-pattern)
A tempting workaround is to call the LLM from a Worker, buffer the full response in KV or R2, then have the client poll. This avoids the streaming timeout but destroys the UX benefit of tokens arriving live. It also doubles storage cost and adds polling latency. Do not do this for chat interfaces.
{ "anti-pattern": "buffer-then-poll", "latency_added_ms": 8000, "user_satisfaction": "low" }Tradeoffs and what we ship
We weighed three paths for our own deployment:
- Pure Worker proxy: zero infra overhead, fails on any prompt >30s. Rejected.
- Durable Objects: robust, keeps tokens streaming, adds ~15% cost and meaningful code complexity. Chosen for server-to-server integrations where we must mask provider keys.
- Direct gateway streaming: simplest for browser apps, requires token scoping. Chosen for our frontend playground.
If you must keep a Worker in front for auth or logging, use the DO pattern and keep the Worker itself a thin router. If you can trust the client with a scoped token, skip the edge function and stream from the gateway.
Takeaway
The cloudflare worker timeout llm streaming issue is not a misconfiguration; it is the platform enforcing its stateless compute contract. Standard Workers cannot hold a stream open for the full length of LLM generation. Use Durable Objects when you need a server-side proxy, or stream directly from an inference gateway that supports OpenAI-compatible endpoints with fallback. Pick one based on whether you need to hide provider credentials—and stop piping multi-minute SSE through a vanilla Worker.