Edge streaming with the Web Streams API on Vercel

Vercel web streams api edge streaming lets you push LLM tokens to the browser the moment they’re generated, instead of buffering a full response on a serverless function. This guide walks through wiring an Edge Function that proxies a streaming chat completion to the client using the standard ReadableStream interface, with no custom WebSocket server required.

Step 1: Create an Edge API route

Start with a Next.js App Router project (or a standalone Vercel Edge Function). The key directive is export const runtime = 'edge', which forces the function to execute in Vercel’s edge runtime where the Web Streams API is the only stream primitive available.

Create app/api/chat/route.ts:

export const runtime = 'edge';

export async function POST(req: Request): Promise<Response> {
  const { messages } = await req.json();

  // We’ll fill the streaming logic in the next steps.
  return new Response('not implemented', { status: 501 });
}

Deploy this to Vercel or run vercel dev locally. The route already runs on the edge, which means it can respond from a datacenter close to the user and start streaming with low time-to-first-byte.

Step 2: Request a streaming completion from the model

You need a model provider that returns a chunked HTTP response. OpenAI’s chat completions endpoint supports stream: true and emits Server-Sent Events (SSE). If you route through n4n.ai, you get a single OpenAI-compatible endpoint that addresses 240+ models and automatically falls back when a provider is degraded, so the edge function doesn’t need provider-specific error handling.

Replace the placeholder with a fetch call:

const upstream = await fetch('https://api.openai.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages,
    stream: true,
  }),
});

if (!upstream.ok || !upstream.body) {
  return new Response('upstream error', { status: 502 });
}

The upstream.body is a ReadableStream<Uint8Array>. Do not attempt to use Node’s Buffer or stream modules here—they are unavailable in the edge runtime.

Step 3: Parse the SSE stream with a TransformStream

The upstream emits text like:

data: {"choices":[{"delta":{"content":"Hello"}}]}

data: {"choices":[{"delta":{"content":" world"}}]}

data: [DONE]

We need to decode bytes, split on double newlines, parse the JSON, and forward only the content deltas. A TransformStream is the correct tool.

const decoder = new TextDecoderStream();
const sseToText = new TransformStream<string, string>({
  start(controller) {},
  transform(chunk, controller) {
    // Simple line-buffered SSE parser
    const lines = chunk.split('\n');
    for (const line of lines) {
      const trimmed = line.trim();
      if (!trimmed.startsWith('data:')) continue;
      const data = trimmed.slice(5).trim();
      if (data === '[DONE]') return;
      try {
        const json = JSON.parse(data);
        const content = json.choices?.[0]?.delta?.content;
        if (content) controller.enqueue(content);
      } catch {
        // ignore malformed keep-alive lines
      }
    }
  },
});

const textStream = upstream.body.pipeThrough(decoder).pipeThrough(sseToText);

This pipeline is fully composable: pipeThrough returns a new stream, so you can keep chaining transforms (e.g., to add ANSI stripping or rate limiting) without buffering.

Step 4: Return the stream from the Edge Function

Wrap the transformed stream in a Response with appropriate headers. If you are sending plain text tokens to the browser, set Content-Type: text/plain; charset=utf-8. If you prefer SSE on the client side, you can re-wrap as text/event-stream, but plain text is simpler for a basic chat UI.

return new Response(textStream, {
  headers: {
    'Content-Type': 'text/plain; charset=utf-8',
    'Cache-Control': 'no-cache, no-transform',
    'X-Accel-Buffering': 'no', // disable proxy buffering on Vercel
  },
});

The X-Accel-Buffering: no header is a pragmatic safeguard: some intermediate proxies try to buffer responses, which defeats edge streaming. Vercel’s edge network respects it.

Step 5: Read the stream in the browser

On the client, use the Fetch API and a reader. No event-source library is required for plain text.

async function send(messages: {role: string; content: string}[]) {
  const res = await fetch('/api/chat', {
    method: 'POST',
    body: JSON.stringify({ messages }),
  });
  const reader = res.body!.getReader();
  const decoder = new TextDecoder();
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    const token = decoder.decode(value, { stream: true });
    document.getElementById('output')!.textContent += token;
  }
}

For React, push tokens into state inside the loop. The UI updates incrementally because each read() resolves as soon as a chunk arrives.

Step 6: Verify the end-to-end pipeline

Run the function locally with vercel dev or deploy to a preview environment. Then test with curl using the -N flag to disable curl’s own buffering:

curl -N -X POST http://localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hi in 5 words"}]}'

You should see words appear one or two at a time, not all at the end. In the browser, open the Network tab, select the request, and watch the Response pane populate live. If you see the full text only after a delay, a proxy is buffering—check the X-Accel-Buffering header and confirm the runtime is edge, not nodejs.

Gotchas when using vercel web streams api edge streaming

Edge runtime has no Node streams

The moment you import stream from node:stream, the build fails or the function silently falls back to the Node runtime. Stick to ReadableStream, TransformStream, and TextDecoderStream. If you need JSON parsing helpers, write small pure functions.

Abort signals must be forwarded

If the user closes the tab, the browser aborts the fetch. Pass req.signal to your upstream fetch so the provider stops generating and you don’t waste tokens:

const upstream = await fetch(url, { method: 'POST', body, signal: req.signal });

CORS and content type

When calling the edge route from a different origin (e.g., a static frontend on a CDN), add Access-Control-Allow-Origin headers. For same-origin Next.js apps this is not needed, but it’s a common breakage point in micro-frontends.

Token metering and observability

Because edge functions are billed per invocation and duration, long-lived streams can cost more than you expect. If you use a gateway that provides per-token usage metering, log the final usage field from the upstream [DONE] message to reconcile cost. With vercel web streams api edge streaming, the function stays open until the model finishes, so monitor wall-clock time as well as invocation count.

SSE framing vs plain text

If you decide to emit text/event-stream to the client, you must format each chunk as data: <token>\n\n. Browsers can then use EventSource, but EventSource only supports GET. Most LLM proxies use POST, so the reader approach in Step 5 is more flexible.

Closing notes on performance

Edge streaming moves the time-to-first-token bottleneck from your origin server to the model provider. Keep the edge function thin: parse, pipe, and return. Any await that isn’t stream-related (e.g., a database write per token) will block the pipeline. If you need to persist the conversation, do it after the stream completes or via a fire-and-forget queue.

The pattern above is production-shaped: it handles backpressure automatically because ReadableStream pipes pause when the client is slow, and it cleans up when the request aborts. Once you internalize that vercel web streams api edge streaming is just Unix pipes for the web, the rest is plumbing.