Slow initial latency kills the perceived quality of a streaming LLM UI. Optimizing vercel edge functions time to first token requires treating the edge as a thin, streaming-aware proxy rather than a full backend, because every millisecond spent in the function before the first chunk reaches the browser is pure dead air.
The latency budget of an edge-streamed LLM call
A user clicking “send” starts a clock. The interval ends when the first token paints on screen. That budget splits into client network, edge platform overhead, upstream connection, and model prefill.
Most teams measure vercel edge functions time to first token as the interval from client fetch to first text delta, but that hides the split between platform overhead and model prefill. You cannot fix what you do not attribute. Put a performance.now() marker in the browser before the fetch and another when the first TextDecoder output is non-empty.
Edge platform overhead includes cold start, region scheduling, and TLS termination. On Vercel’s edge runtime, cold starts are typically low but not zero if your function imports heavy modules. Upstream connection is the TCP/TLS and auth handshake to your LLM provider or gateway. Model prefill is the GPU time before the first generated token—this dominates for long prompts.
Architecture: a streaming proxy, not a processor
The fastest edge function is the one that does the least. Client calls your Edge Function; the function forwards the request to an OpenAI-compatible endpoint and returns the response body unchanged with streaming headers.
Routing through a gateway such as n4n.ai gives you one OpenAI-compatible endpoint that fronts 240+ models and fails over automatically when a provider is degraded, so your edge function skips provider-specific retry code and gets to the first token faster.
Do not attempt RAG, vector search, or prompt assembly inside the edge function if those steps add measurable delay. Do them before calling the edge, or in a regional serverless function co-located with your data.
Step 1: Write a minimal Edge Function that streams
Use the Web fetch API and return the upstream body directly. Avoid the OpenAI SDK—it adds weight and buffering.
// api/stream.ts (Vercel Edge Function)
export const config = { runtime: 'edge' };
export default async function handler(req: Request) {
const upstream = 'https://api.n4n.ai/v1/chat/completions';
const body = await req.json();
const streamReq = new Request(upstream, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${process.env.LLM_KEY}`,
},
body: JSON.stringify({ ...body, stream: true }),
});
const res = await fetch(streamReq);
return new Response(res.body, {
headers: {
'content-type': 'text/event-stream',
'cache-control': 'no-cache, no-transform',
},
});
}This function parses the JSON body once, then pipes bytes. It adds one network hop but no transformation of the token stream.
Step 2: Forward cache hints and routing directives
Providers increasingly honor cache-control for prefix caching. If your client knows a system prompt is static, let it send a hint and forward it.
const upstreamHeaders = new Headers();
upstreamHeaders.set('authorization', `Bearer ${process.env.LLM_KEY}`);
upstreamHeaders.set('content-type', 'application/json');
const clientCache = req.headers.get('cache-control');
if (clientCache) upstreamHeaders.set('cache-control', clientCache);Tradeoff: inspecting or modifying headers is cheap, but reading the body to inject prompts delays the upstream fetch. Clone the request or parse minimally.
Step 3: Avoid cold-start and runtime traps
Reducing vercel edge functions time to first token often comes down to deleting code, not adding it.
- Set
runtime: 'edge'explicitly. Node runtime on Vercel is not edge and has higher cold start. - Never import
openai,axios, orlodashin the edge path. Use globalfetch. - Pin a region only if your upstream is single-region. Otherwise let Vercel route to nearest edge; cross-region to a gateway in the same continent is usually <30ms.
- Do not
awaitlogging or analytics before the upstream call. Fire and forget.
Edge functions have execution duration limits and may be terminated if they appear idle. Streaming keeps the connection active, but a slow provider that sends no bytes for 10s can trip platform timeouts. Choose a gateway that sends periodic SSE comments.
Step 4: Transform the stream only if necessary
If you must reshape the SSE (e.g., strip data: prefixes, merge deltas), use a TransformStream. But every byte you touch is a few microseconds of extra latency and a risk of backpressure.
const { readable, writable } = new TransformStream();
const writer = writable.getWriter();
const reader = res.body!.getReader();
const decoder = new TextDecoder();
// Example: pass-through with no buffering
(async () => {
while (true) {
const { done, value } = await reader.read();
if (done) break;
await writer.write(value);
}
writer.close();
})();
return new Response(readable, { headers: { 'content-type': 'text/event-stream' } });If you do not need to modify shape, return res.body directly as in Step 1. Pass-through is fastest.
Step 5: Client consumption
The browser should read the stream incrementally and render immediately.
const res = await fetch('/api/stream', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ model: 'gpt-4o-mini', messages, stream: true }),
});
const reader = res.body!.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value, { stream: true });
// append chunk to UI state
}Measure performance.now() delta from before fetch to the first non-empty chunk. That is your real vercel edge functions time to first token from the user’s perspective.
Common pitfalls and tradeoffs
Double compression. Vercel edge may apply gzip. If your upstream already compresses and you set no-transform, you avoid re-compression cycles. Test with curl -I to see content-encoding.
Region mismatch. An edge function in Sydney calling a US-east gateway adds ~120–200ms round trip before prefill. Use a gateway with multi-region anycast or deploy your own proxy in the same region as the model.
Idle timeout. Some platforms kill functions that do not send bytes within a fixed window. Ensure the upstream sends keep-alive SSE comments or set a shorter client timeout and show a retry.
Auth leakage. The edge function holds the gateway key. Never echo authorization headers to the client or log full request bodies.
Streaming partial JSON. If you later parse the aggregated stream into JSON, that happens after TTFN, so it does not affect first token, but your client state machine must handle incomplete parses.
Measuring and monitoring
Client-side timing is the only source of truth for user-perceived latency. Send a beacon with p50, p95, and region to your analytics. Server-side logs show total function duration, not TTFN, because the function stays open until the stream ends.
Collect metrics per model. A 70B model with long prefill will dominate TTFN regardless of edge optimizations. The edge work is about removing the avoidable milliseconds, not the physics of inference.
When not to use edge
If your flow requires a 200ms vector search or a database join before the first token, an edge function in a far region will only add a hop. Put that logic in a regional function close to the data, then stream from there or hand off to the edge only for last-mile delivery.
For pure proxying of an already-assembled prompt, vercel edge functions time to first token is minimized by a lean pass-through function, correct headers, and a gateway that handles provider failover upstream.