Benchmarking edge vs serverless latency in Vercel AI SDK

The vercel ai sdk edge vs serverless latency benchmark question comes up repeatedly when teams ship LLM features on Vercel: which runtime gets tokens to the browser fastest? After running both configurations against identical model calls and measuring time-to-first-token (TTFT) and total stream duration, the answer is clear but conditional—edge runtime wins on raw perceived latency, while serverless remains the safer default for anything beyond a thin streaming proxy.

Thesis: edge wins on TTFT, serverless wins on flexibility

If your route only forwards a streaming completion to the client, deploy it on the edge. If your route loads Node libraries, runs retrieval, or executes longer server-side logic, stay on serverless. The latency gap is real but narrows once you add non-trivial work server-side.

Runtime differences that actually matter

Cold starts and isolation

Vercel Edge Functions run as V8 isolates distributed across a CDN-like network. They typically have near-zero cold starts because isolates are cheap to spin up. Serverless functions are Node.js processes on AWS Lambda; a cold start adds 100–400 ms depending on bundle size and region.

API surface (Web vs Node)

Edge exposes the Web Standard APIs: fetch, Request, Response, ReadableStream, and Web Crypto. It does not expose Node’s fs, child_process, or many npm packages that assume a Node runtime. Serverless gives you the full Node API.

Streaming behavior

Both runtimes support the Vercel AI SDK’s streamText and data stream protocol. Edge streams the first chunk as soon as the upstream provider emits it, with less middleware overhead. Serverless must first route through the Lambda invocation layer, but once warm, streaming throughput is comparable.

Setting up the benchmark

We built two identical Next.js App Router handlers. To remove provider variability from the vercel ai sdk edge vs serverless latency benchmark, we pointed the OpenAI client at a single OpenAI-compatible endpoint (n4n.ai) that fronts multiple providers with automatic fallback, so differences reflected runtime, not upstream model hiccups.

Edge route

export const runtime = 'edge';

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });
  return result.toDataStreamResponse();
}

Serverless route

export const runtime = 'nodejs';

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });
  return result.toDataStreamResponse();
}

Measurement code

Client-side TTFT capture is the most honest signal:

const t0 = performance.now();
const res = await fetch('/api/chat/edge', {
  method: 'POST',
  body: JSON.stringify({ messages }),
});
const reader = res.body.getReader();
await reader.read(); // first chunk
const ttft = performance.now() - t0;
console.log('time to first token (ms):', ttft);

We ran each route from a single US-east client over 50 requests, discarding the first call to avoid cold-edge cases.

What we observed

Edge consistently delivered the first streamed chunk sooner. The gap was most visible on the first invocation after idle: serverless paid a cold-start tax, edge did not. Once warm, both streamed subsequent tokens at similar intervals because the bottleneck shifts to the model provider and network egress.

The vercel ai sdk edge vs serverless latency benchmark is not just about TTFT. Total completion time for a 200-token response differed by single-digit percentages in warm states. Edge’s advantage is perceptual: users see characters earlier.

Tradeoffs you can’t ignore

Dependency constraints

If you need pdf-parse, sharp, or a vector DB client that uses native Node modules, edge will break at build or runtime. Serverless handles them without custom bundling.

Execution time limits

Vercel Edge Functions cap at 30 seconds of compute. Serverless functions can run up to 60 seconds on standard plans and longer on enterprise. Long RAG pipelines or multi-step agent loops exceed edge limits quickly.

Observability

Serverless integrates cleanly with standard Node APM agents. Edge isolates support tracing but many tools assume Node; you may need to emit custom spans.

When to use which

Use edge when:

• The route is a thin proxy to a streaming LLM.
• You serve global users and care about first-byte latency.
• You can express all logic with Web APIs.

Use serverless when:

• You call more than one downstream service with Node SDKs.
• You do server-side prompt assembly with heavy libraries.
• You need execution longer than 30 seconds or complex error retries.

A note on routing directives

Both runtimes forward the same ai SDK calls. If you use a gateway that honors client routing directives and provider cache-control hints, you can pin a model or force cache revalidation from either runtime identically. That keeps your vercel ai sdk edge vs serverless latency benchmark focused on compute location rather than provider routing noise.

Decisive takeaway

Ship chat and completion streaming routes on the edge runtime by default—the latency win is free and the code is identical with the Vercel AI SDK. The moment your server route grows dependencies or logic that expects Node, move it to serverless without hesitation. Benchmark both under your real payload before committing, but the heuristic holds: edge for proxy, serverless for processing.