Streaming tokens from a language model inside a Vercel Edge Function cuts time-to-first-byte and keeps interactive UIs responsive. The Vercel AI SDK gives you a clean streamText API that runs on the edge runtime, but wiring it to a route handler and a client takes a few non-obvious steps. This guide walks through a working setup for vercel edge functions ai sdk streaming using TypeScript and the Web Streams API.
Step 1: Scaffold an Edge API route
Create a Next.js (App Router) project if you don’t have one. The edge runtime is selected per-route via an exported constant, not a global config file. This matters because Vercel isolates edge deployments from the Node.js runtime, giving you different APIs and limits.
npx create-next-app@latest edge-stream-demo --ts --app
cd edge-stream-demoAdd a route handler at app/api/chat/route.ts. The critical line is export const runtime = 'edge'. Without it, Next.js defaults to Node.js and you lose the cold-start and streaming characteristics of the edge. You also cannot use Node built-ins like fs or process in this file.
// app/api/chat/route.ts
export const runtime = 'edge';
export async function POST(req: Request) {
return new Response('ok');
}Run next dev or vercel dev and hit the endpoint with curl to confirm it returns 200. At this stage you have a bare edge function; the model integration comes next.
Step 2: Install and configure the AI SDK
Install the core package and an OpenAI-compatible provider. The AI SDK abstracts model vendors behind a uniform interface, so swapping providers later is a one-line change.
npm i ai @ai-sdk/openai zodIf you point at a gateway instead of OpenAI directly, set baseURL and apiKey. For example, n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and automatically falls back when a provider is rate-limited or degraded, which works without changing the AI SDK calls below.
import { createOpenAI } from '@ai-sdk/openai';
const gateway = createOpenAI({
baseURL: 'https://api.n4n.ai/v1',
apiKey: process.env.N4N_API_KEY,
});Keep keys in environment variables, never inline. Vercel injects env vars at build time for edge functions; prefix server-only secrets with NEXT_PUBLIC_ only if the client needs them (it shouldn’t here). The zod dependency is optional but useful when you later add structured output or validate incoming message shapes.
Step 3: Implement streaming with streamText
The streamText function returns a result object with helper methods to convert to a streaming Response. On the edge, use toDataStreamResponse() if you plan to consume with the AI SDK’s React hooks, or toTextStreamResponse() for a raw text stream. When building vercel edge functions ai sdk streaming, the data stream variant is the default because it carries metadata like usage and tool calls.
import { streamText } from 'ai';
export const runtime = 'edge';
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: gateway('gpt-4o-mini'),
messages,
temperature: 0.7,
});
return result.toDataStreamResponse();
}streamText is non-blocking; it starts producing chunks immediately. The edge runtime supports the underlying ReadableStream natively, so no polyfills are required. Avoid Node-specific APIs like Buffer in this file—they will throw at runtime. If you need per-token usage metering, the AI SDK exposes result.usage after the stream closes, but on the edge you should send it via a trailing data stream event or log it server-side.
The data stream protocol prefixes each chunk with a type tag (e.g., 0: for text). This lets the client distinguish text from tool invocations without custom parsing.
Step 4: Consume the stream on the client
Use the useChat hook from @ai-sdk/react for a full chat UI, or write a minimal fetch reader. The hook handles parsing the data stream protocol automatically and updates state as tokens arrive.
'use client';
import { useChat } from '@ai-sdk/react';
export function Chat() {
const { messages, input, handleInputChange, handleSubmit } = useChat();
return (
<div>
{messages.map((m) => (
<div key={m.id}>
{m.role}: {m.content}
</div>
))}
<form onSubmit={handleSubmit}>
<input value={input} onChange={handleInputChange} />
<button type="submit">Send</button>
</form>
</div>
);
}For a lower-level check, call the route directly:
const res = await fetch('/api/chat', {
method: 'POST',
body: JSON.stringify({ messages: [{ role: 'user', content: 'Hello' }] }),
});
const reader = res.body!.getReader();
while (true) {
const { done, value } = await reader.read();
if (done) break;
console.log(new TextDecoder().decode(value));
}The data stream protocol prefixes each chunk with 0: for text parts; the React hook strips that. Raw text mode (toTextStreamResponse) emits plain strings. Either way, the browser receives bytes incrementally, which is the whole point of vercel edge functions ai sdk streaming.
Step 5: Handle abort and errors
Edge functions terminate quickly on client disconnect, but you should forward abort signals to the model call to avoid wasted tokens. streamText accepts an abortSignal from the request.
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: gateway('gpt-4o-mini'),
messages,
abortSignal: req.signal,
});
return result.toDataStreamResponse({
onError: (e) => {
console.error(e);
return 'Stream failed';
},
});
}Vercel Edge Functions have a 25-second execution limit on the hobby plan and a hard memory cap. Streaming keeps the connection open but does not extend the limit; design prompts to complete within it. If you exceed, the stream closes with an incomplete response. Always test with realistic prompt lengths before shipping.
Step 6: Deploy and verify success
Commit and push to Vercel. Set env vars in the dashboard or via CLI:
vercel env add N4N_API_KEY
vercel deploy --prodVerification steps:
- Open the deployed
/api/chatroute via your UI. - Open browser DevTools → Network → fetch entry.
- Confirm the response headers include
content-type: text/plain; charset=utf-8(for text stream) orapplication/octet-streamfor data stream. - Watch the Response tab populate token-by-token rather than after a delay.
- Run a curl streaming test locally against the dev server:
curl -N -X POST http://localhost:3000/api/chat \
-H 'content-type: application/json' \
-d '{"messages":[{"role":"user","content":"Say hi slowly"}]}'You should see incremental output, not a single blob. If you see the full response at once, you likely called await result.text() instead of the stream response helper.
Step 7: Edge-specific optimizations
The edge runtime is not Node. Replace any JSON.parse of large bodies with streaming JSON parsers if payloads are big. Use crypto.subtle instead of crypto from Node. When using vercel edge functions ai sdk streaming in production, cache model responses at the CDN level only when content is static—set Cache-Control via the response headers and let the provider forward cache hints if supported.
If you batch multiple tool calls, prefer streamObject for structured output; it streams partial JSON and parses incrementally, which pairs well with edge timeouts. Finally, monitor token usage. Per-token metering helps track cost across fallback models. The AI SDK’s onFinish callback reports final usage; log it to your analytics pipeline.
Troubleshooting
Blank responses: Ensure you call toDataStreamResponse() and not await result.text(). The latter consumes the stream.
Runtime error about Buffer: You imported a Node-only module. Remove it or move logic to a Node route.
CORS issues: Edge functions need explicit CORS headers if called from another origin. Add Access-Control-Allow-Origin in the route.
Timeouts: Reduce maxTokens or simplify the prompt. Edge limits are strict.
Following these steps gives you a production-shaped pipeline for vercel edge functions ai sdk streaming, from scaffold to verified token stream. The pattern extends to serverless Node functions with minor runtime constant changes, but the edge keeps latency low for global users.