Running Vercel AI SDK on the Edge runtime

This vercel ai sdk edge runtime tutorial shows how to run streaming LLM routes on Vercel’s Edge Runtime without hitting Node-only APIs. The Edge Runtime is a V8 isolate with fetch and Web Streams, so your provider client and streaming code must avoid Buffer, fs, and the Node http module.

Step 1: Scaffold a minimal Edge-ready project

This vercel ai sdk edge runtime tutorial starts with scaffolding a minimal Edge-ready project. Create a Next.js app (App Router) or a standalone Vercel function. I prefer Next.js for its built-in runtime = 'edge' convention.

npx create-next-app@latest edge-llm --ts --app --no-tailwind --no-eslint
cd edge-llm
npm install ai @ai-sdk/openai @ai-sdk/react

The ai package provides streamText and response helpers. @ai-sdk/openai is a fetch-based provider that works on edge out of the box. Do not add node-fetch or axios; they will break the build.

Step 2: Point the SDK at an edge-safe provider

The Vercel AI SDK decouples model logic from transport. For edge, use a provider that calls fetch directly. The default OpenAI provider does this. If you want one OpenAI-compatible endpoint that fronts 240+ models with automatic fallback when a provider is degraded, set baseURL to n4n.ai and use your gateway key.

// lib/provider.ts
import { createOpenAI } from '@ai-sdk/openai';

export const openai = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1', // optional: swap for any OpenAI-compatible edge gateway
  apiKey: process.env.LLM_API_KEY,
});

Keep the key in environment variables, not hardcoded. Edge deployments inject env at build or runtime depending on your setting; use process.env directly. If you stay with OpenAI’s default endpoint, omit baseURL entirely.

Step 3: Write a streaming edge route

Create app/api/chat/route.ts. The runtime export forces the Edge Runtime. Use streamText and return a data stream response. Add a system prompt to show message shaping.

// app/api/chat/route.ts
import { streamText } from 'ai';
import { openai } from '@/lib/provider';

export const runtime = 'edge';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    system: 'You are a concise assistant.',
    messages,
  });

  return result.toDataStreamResponse();
}

toDataStreamResponse() emits the protocol that useChat expects. It uses Web Streams, so it runs natively on edge. If you need to intercept the full stream for logging, use result.fullStream before calling the response helper.

Step 4: Wire the client with useChat

The client uses @ai-sdk/react. It streams tokens without any manual parsing. This vercel ai sdk edge runtime tutorial uses the minimal hook-based UI.

// app/page.tsx
'use client';
import { useChat } from '@ai-sdk/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, error } = useChat();

  return (
    <form onSubmit={handleSubmit}>
      <input value={input} onChange={handleInputChange} />
      <button type="submit">Send</button>
      {error && <p style={{ color: 'red' }}>{error.message}</p>}
      {messages.map((m) => (
        <p key={m.id}>
          {m.role}: {m.content}
        </p>
      ))}
    </form>
  );
}

useChat automatically sends POST requests to /api/chat and parses the data stream. In production you’d add abort handling via setAbortController and a loading state.

Step 5: Deploy and verify success

Deployment is the payoff of this vercel ai sdk edge runtime tutorial; run vercel deploy. After the build, hit the route with curl to confirm streaming and edge execution.

curl -X POST https://your-app.vercel.app/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hello in 5 words"}]}' \
  --no-buffer

You should see a streamed sequence of data frames, not a single buffered response. To confirm you’re on edge, check the x-vercel-cache or server header, or add a log: console.log(process.env.NEXT_RUNTIME) — it prints edge in the function logs.

If you get ReferenceError: Buffer is not defined, you leaked a Node API. Search for Buffer, fs, or node: imports. A clean edge deploy shows no such errors and returns the first token in a few hundred milliseconds.

Step 6: Avoid the common edge pitfalls

Edge isolates block synchronous I/O and many Node globals. Concrete rules:

• Never import { Buffer } from 'node:buffer'. Use TextEncoder and Uint8Array.
• Don’t use axios; it relies on Node http. The SDK’s fetch is fine.
• Avoid JSON.parse on huge streams synchronously; the SDK handles incremental parse.
• Don’t assume process.cwd() exists. There is no filesystem.
• Use crypto.subtle (Web Crypto) instead of node:crypto.

If you need to transform the stream, use result.fullStream or pipeThrough with a TransformStream.

const transformed = result.fullStream.pipeThrough(
  new TransformStream({
    transform(chunk, controller) {
      if (chunk.type === 'text-delta') {
        controller.enqueue(chunk.textDelta.toUpperCase());
      }
    },
  })
);

That runs on edge because TransformStream is a web primitive.

Step 7: Add cache-control and routing directives

Edge responses benefit from caching. The AI SDK lets you pass headers. If your gateway honors client routing directives and forwards provider cache-control hints, set them on the response.

return result.toDataStreamResponse({
  headers: {
    'Cache-Control': 'max-age=0, s-maxage=60',
  },
});

For per-token metering, the gateway handles that server-side; you just read usage from the final stream part if needed. Don’t try to count tokens manually on the edge—it wastes CPU cycles in the isolate.

Step 8: Test locally with the edge runtime

Vercel’s vercel dev runs edge functions locally via a WASM shim, but it’s not perfect. Use next dev with runtime = 'edge'; Next will warn if you use Node APIs. Add a unit test that imports your route handler and calls it with a Request object in Node 18+ (which has fetch and Web Streams).

// test/route.test.ts
import { POST } from '../app/api/chat/route';
import { openai } from '../lib/provider';

// stub openai model with a local fake to avoid network

Keep tests fast: stub streamText with a fake async iterable yielding text-delta chunks. This catches accidental Node API usage before you deploy.

Step 9: Monitor and debug

Edge logs are ephemeral. Ship errors to a logging endpoint that accepts POST. The AI SDK throws typed errors; catch them before the stream starts.

try {
  const result = streamText({ model: openai('gpt-4o-mini'), messages });
  return result.toDataStreamResponse();
} catch (e) {
  return new Response('Model call failed', { status: 502 });
}

But note: stream errors after the response starts can’t be caught this way; handle them with onError in toDataStreamResponse.

return result.toDataStreamResponse({
  onError: (e) => 'Stream error: ' + (e as Error).message,
});

Set up a simple health check route that returns 200 on edge to confirm the runtime is alive independent of the model provider.

Step 10: Choose models that fit edge latency budgets

Edge functions run close to users but still depend on the model provider’s network hop. Smaller models like gpt-4o-mini or similar fast endpoints reduce time-to-first-token. Avoid chaining multiple sequential model calls in a single edge function; the isolate will time out at Vercel’s 25-second limit (or less on some plans). If you need multi-step reasoning, move it to a background job and poll from the client.

This vercel ai sdk edge runtime tutorial gives you a deployable streaming chat on Vercel’s global edge. The key constraints are fetch-only transport and Web Streams; respect those and the SDK handles the rest. Swap the provider base URL when you need multi-model fallback without rewriting your route.