Edge vs Node.js runtime on Vercel for streaming LLM responses

Choosing between vercel edge vs nodejs runtime streaming for LLM responses changes how you architect proxy logic, handle backpressure, and pay for compute. Both run on Vercel’s platform, but they differ sharply in streaming primitives, cold starts, and npm compatibility.

Runtime model

Vercel Edge Functions execute in V8 isolates distributed across the CDN. They expose Web Standard APIs (fetch, ReadableStream, crypto.subtle) and deliberately omit the Node.js standard library. Node.js functions are standard AWS Lambda-style serverless containers with the full Node runtime (currently Node 18/20 on Vercel), including fs, Buffer, stream, and the entire npm ecosystem.

For LLM streaming, the core primitive is the same Response object returned from a route handler, but the way you build the stream body is different.

Edge runtime:

// app/api/edge/route.ts
export const runtime = 'edge';

export async function POST(req: Request) {
  const upstream = await fetch('https://api.example.com/stream', {
    method: 'POST',
    body: await req.text(),
  });
  // upstream.body is a Web ReadableStream
  return new Response(upstream.body, {
    headers: { 'content-type': 'text/event-stream' },
  });
}

Node.js runtime:

// app/api/node/route.ts
export const runtime = 'nodejs';
import { Readable } from 'node:stream';

export async function POST(req: Request) {
  const upstream = await fetch('https://api.example.com/stream', {
    method: 'POST',
    body: await req.text(),
  });
  // Node 18+ exposes Web streams on fetch; return directly
  return new Response(upstream.body, {
    headers: { 'content-type': 'text/event-stream' },
  });
}

If you need to transform tokens (e.g., redact PII, parse JSON lines), Edge forces you to work with ReadableStream controllers; Node lets you use node:stream pipelines or async generators with for await.

Capabilities

Edge is intentionally minimal. You get fetch, ReadableStream, TransformStream, TextEncoder/Decoder, and crypto. You cannot require('child_process'), read files, or use native addons. For a pure pass-through proxy to an LLM provider, that is enough.

Node.js gives you the full toolkit. You can use the official OpenAI Node SDK, langchain, pdf-parse, or anything that touches Buffer. If your streaming route also writes to a Postgres pool, signs JWTs with jsonwebtoken, or uses zlib to compress, Node is the only option.

One concrete gap: Edge cannot use Node’s crypto.createHmac synchronous API—you must use crypto.subtle.sign, which is async and less ergonomic for many auth schemes.

Price and cost model

Vercel bills Edge Functions by invocation count plus compute time measured in CPU milliseconds (GB-ms equivalents). Because isolates are lightweight, a short streaming proxy costs fractions of a cent per million requests. Node.js functions are billed by execution duration rounded to the nearest 100ms and allocated memory (128MB–3008MB). A function that stays open for 30 seconds streaming a long completion will incur 300× 100ms billing increments on Node, while Edge bills only active CPU time—but Vercel caps Edge CPU per request, so you cannot busy-loop.

Qualitatively: Edge wins for high-volume, low-compute streaming. Node wins when you need longer wall-clock time and can amortize cost against heavy server-side work per request.

Latency and throughput

Edge Functions cold-start in single-digit milliseconds and execute in the region closest to the user. Time-to-first-byte for a streaming proxy is often under 30ms globally. Node functions cold-start in 100–400ms depending on bundle size and region; they run in a single region (unless you use Edge Middleware to route).

Throughput is bounded by the runtime’s CPU limit. Edge restricts total CPU time per request (tens of milliseconds on Hobby, higher on Pro), which is fine for I/O-bound streaming but lethal if you parse large responses. Node allows sustained CPU for the full function timeout (60s Hobby, 300s Pro). For LLM chat where tokens drip over 20 seconds, Edge’s low cold start gets tokens to the client faster; Node’s regional placement may add 50–100ms of cross-continent latency if your user is far from the deployment.

Ergonomics

Writing streaming code in Edge means embracing Web Streams:

const transformer = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk.toUpperCase());
  },
});
return new Response(upstream.body.pipeThrough(transformer));

That is clean, but many LLM SDKs assume Node. The OpenAI Node SDK works in Edge if you use stream: true and consume response.data as a web stream, but older versions broke. In Node, you can for await (const token of completion) with zero friction.

Error handling also differs: Edge will silently truncate streams if you exceed CPU limits; Node will throw a catchable exception and can retry.

Ecosystem

Edge supports a subset of npm packages that are marked edge-compatible (no native bindings, no Node built-ins). hono, zod, std-env work. sharp, pg, redis do not. Node supports everything deployable to Lambda.

If you rely on a vendor’s typed SDK (e.g., Anthropic, Cohere, or an OpenRouter-class gateway), check its engine field. Most publish ESM that runs on Edge, but many use Buffer internally and fail at runtime.

Limits

Vercel enforces a maximum request/response size on Edge (typically 4MB body limit) and restricts outgoing fetch to standard HTTPS. Node functions allow larger payloads and can use http agents with keep-alive to reuse connections to your LLM provider—critical for high throughput.

Which to choose

Use Edge when:

• You are building a pure pass-through proxy that adds minimal headers or routing.
• Your users are globally distributed and you need lowest time-to-first-token.
• You do not need Node-only SDKs, file access, or heavy server-side parsing.
• Volume is high and individual requests do little compute.

Example: a route that forwards a browser’s stream to a provider and back, maybe injecting a system prompt via TransformStream.

Use Node.js when:

• You need the official LLM SDK, database clients, or Buffer-based parsing.
• Your stream runs longer than Edge’s CPU budget allows (long agent loops, summarization).
• You want connection pooling to reduce TLS handshake overhead on every streaming call.
• You proxy to an inference gateway like n4n.ai, which provides one OpenAI-compatible endpoint across 240+ models with automatic fallback when a provider is degraded; the Node runtime lets you forward client routing directives and provider cache-control hints without rewriting them into edge-safe primitives.

Hybrid pattern: Put auth and regional routing in Edge Middleware, then rewrite to a Node function for the actual streaming completion. This gets you Edge’s low-latency entry and Node’s execution headroom.

For most production LLM apps that do any post-processing, logging, or multi-model fallback, Node.js is the safer default. Edge is a sharp tool for the narrow case of a globally distributed, stateless token firehose.