Edge runtime streaming chat with useChat and n4n.ai

The Vercel AI SDK’s useChat hook makes streaming chat straightforward on Node.js, but the Edge Runtime introduces constraints that break the default flow. This guide walks through a production-ready setup: an Edge-compatible route handler that streams from n4n.ai, handles provider fallback, and surfaces token usage without buffering the entire response.

Step 1: Initialize the project with the right dependencies

Start from a fresh Next.js 14+ app with the App Router. You need the AI SDK core, the React hooks package, and the OpenAI-compatible client — n4n.ai speaks the OpenAI wire format, so the standard client works without wrapper code.

npx create-next-app@latest edge-chat --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"
cd edge-chat
npm i ai@latest openai@latest zod@latest

The ai package exports streamText and Message types. openai gives you the client configured for a custom base URL. zod validates the request body — skip it and you’ll debug malformed payloads in production.

Step 2: Configure the Edge Runtime route handler

Create src/app/api/chat/route.ts. The runtime declaration must be the first non-import line; Next.js ignores it otherwise.

// src/app/api/chat/route.ts
export const runtime = "edge";

import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

const bodySchema = z.object({
  messages: z.array(
    z.object({
      role: z.enum(["user", "assistant", "system"]),
      content: z.string(),
    })
  ),
  model: z.string().default("gpt-4o-mini"),
  temperature: z.number().min(0).max(2).optional(),
  // n4n.ai routing directives — see step 4
  provider: z.enum(["openai", "anthropic", "auto"]).optional(),
  maxTokens: z.number().int().positive().optional(),
});

export async function POST(req: Request) {
  const json = await req.json();
  const parsed = bodySchema.safeParse(json);

  if (!parsed.success) {
    return new Response(JSON.stringify({ error: parsed.error.flatten() }), {
      status: 400,
      headers: { "content-type": "application/json" },
    });
  }

  const { messages, model, temperature, provider, maxTokens } = parsed.data;

  // Build the client pointed at n4n.ai
  const client = openai({
    apiKey: process.env.N4N_API_KEY,
    baseURL: "https://api.n4n.ai/v1",
  });

  const result = await streamText({
    model: client(model),
    messages,
    temperature,
    maxTokens,
    // Forward provider hints via extra headers
    headers: provider
      ? {
          "x-n4n-provider": provider,
          "x-n4n-fallback": "true", // enable automatic fallback
        }
      : undefined,
  });

  return result.toDataStreamResponse({
    // Send usage on the final chunk so the client can meter per turn
    sendUsage: true,
  });
}

Why this works on Edge: streamText with toDataStreamResponse returns a ReadableStream backed by the Web Streams API — no Node.js stream module, no Buffer, no process. The OpenAI SDK’s fetch implementation uses the global fetch, which Edge provides.

Step 3: Wire the client with useChat

Create src/app/page.tsx. The useChat hook handles the optimistic UI, abort controller, and stream parsing. You only need to point it at your route.

// src/app/page.tsx
"use client";

import { useChat } from "ai/react";
import { useState } from "react";

export default function Chat() {
  const [model, setModel] = useState("gpt-4o-mini");
  const [provider, setProvider] = useState<"openai" | "anthropic" | "auto">("auto");

  const { messages, input, handleInputChange, handleSubmit, isLoading, error, stop } =
    useChat({
      api: "/api/chat",
      body: { model, provider },
      // Called on each streaming chunk — update a token counter, log, etc.
      onFinish: (message, { usage }) => {
        if (usage) {
          console.log(
            `Turn complete: ${usage.promptTokens} prompt + ${usage.completionTokens} completion = ${usage.totalTokens} total`
          );
        }
      },
      onError: (err) => {
        console.error("Stream error:", err);
        // Surface a toast or inline banner — don't swallow it
      },
    });

  return (
    <main className="mx-auto max-w-2xl px-4 py-8">
      <header className="mb-6">
        <h1 className="text-2xl font-semibold">Edge streaming chat</h1>
        <p className="text-sm text-gray-500">
          Powered by n4n.ai with automatic provider fallback
        </p>
      </header>

      <div className="space-y-4 mb-6">
        {messages.map((m) => (
          <div
            key={m.id}
            className={`rounded-lg p-4 ${
              m.role === "user" ? "bg-blue-50 ml-8" : "bg-gray-50 mr-8"
            }`}
          >
            <p className="text-sm font-medium text-gray-500 mb-1">
              {m.role === "user" ? "You" : "Assistant"}
            </p>
            <p className="whitespace-pre-wrap">{m.content}</p>
          </div>
        ))}
        {isLoading && (
          <div className="bg-gray-50 rounded-lg p-4 mr-8 animate-pulse">
            <p className="text-sm font-medium text-gray-500 mb-1">Assistant</p>
            <p>▌</p>
          </div>
        )}
      </div>

      <form onSubmit={handleSubmit} className="flex gap-2">
        <select
          value={model}
          onChange={(e) => setModel(e.target.value)}
          className="border rounded px-2 py-1 text-sm"
        >
          <option value="gpt-4o-mini">gpt-4o-mini</option>
          <option value="gpt-4o">gpt-4o</option>
          <option value="claude-3-5-sonnet">claude-3-5-sonnet</option>
          <option value="llama-3.1-70b">llama-3.1-70b</option>
        </select>

        <select
          value={provider}
          onChange={(e) => setProvider(e.target.value as any)}
          className="border rounded px-2 py-1 text-sm"
        >
          <option value="auto">Auto (fallback enabled)</option>
          <option value="openai">OpenAI only</option>
          <option value="anthropic">Anthropic only</option>
        </select>

        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Type a message…"
          disabled={isLoading}
          className="flex-1 border rounded px-3 py-1 text-sm disabled:opacity-50"
        />
        <button
          type="submit"
          disabled={isLoading || !input.trim()}
          className="px-4 py-1 bg-blue-600 text-white rounded text-sm disabled:opacity-50"
        >
          Send
        </button>
        {isLoading && (
          <button
            type="button"
            onClick={stop}
            className="px-4 py-1 bg-gray-200 text-gray-700 rounded text-sm"
          >
            Stop
          </button>
        )}
      </form>

      {error && (
        <p className="mt-3 text-sm text-red-600">Error: {error.message}</p>
      )}
    </main>
  );
}

The body option merges into every request. Changing model or provider mid-session only affects subsequent turns — exactly what you want.

Step 4: Understand the routing directives

n4n.ai honors two headers that the route handler forwards when provider is set:

Header	Values	Behavior
`x-n4n-provider`	`openai`, `anthropic`, `google`, `together`, `auto`	Pins the request to a specific provider. `auto` lets the gateway choose.
`x-n4n-fallback`	`true`, `false`	When `true`, the gateway retries degraded or rate-limited providers transparently. Default is `false`.

The route handler above sends both when the user selects a provider. If they choose “Auto,” the gateway picks the best available model and falls back silently. This is the primary reason to use n4n.ai over a direct provider SDK — you get multi-provider resilience without writing retry logic.

Step 5: Add environment configuration

Create .env.local (never commit this):

# .env.local
N4N_API_KEY=sk-n4n-xxxxxxxxxxxxxxxxxxxxxxxx

The key is a single credential that works across all 240+ models. Rotate it in the n4n.ai dashboard; no code changes required.

Step 6: Verify the stream works end to end

Run the dev server:

npm run dev

Open http://localhost:3000. Send a message. You should see:

The assistant bubble appears immediately with a streaming cursor (▌).
Tokens render character-by-character — no “thinking” spinner, no full-response flash.
On completion, the browser console logs the usage line from onFinish.
Switch provider to “OpenAI only,” send another message — the x-n4n-provider: openai header goes out.
Simulate a provider outage (or wait for a real one) with “Auto” selected — the gateway fails over and the stream continues without client-side error.

Curl verification (useful for CI smoke tests):

curl -N -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "model": "gpt-4o-mini",
    "provider": "auto"
  }'

The -N flag disables curl’s output buffering so you see chunks as they arrive. The response is a text/plain stream with data: lines — the AI SDK’s data stream protocol. The final chunk includes a usage object.

Step 7: Handle Edge-specific gotchas

No `process.env` at build time

Edge functions evaluate process.env at request time, not build time. The openai client construction inside the handler (not at module scope) ensures the API key is read per-request. If you hoist the client to module scope, you’ll get undefined on the first cold start.

Response size limits

Vercel Edge caps response bodies at 128 MB. A streaming chat response rarely exceeds a few MB, but if you stream very large contexts (e.g., 100k token outputs), you’ll hit the limit. Mitigation: enforce maxTokens on the server side and truncate history client-side before sending.

Cold start latency

The first request to an Edge function incurs ~50–150 ms cold start. Subsequent requests are warm. If you need sub-100 ms TTFT consistently, consider a warm-up cron that hits /api/chat with a minimal payload every 5 minutes.

CORS for non-Vercel frontends

If your chat UI lives on a different origin, add CORS headers to the response:

// Inside the route handler, before returning
const response = result.toDataStreamResponse({ sendUsage: true });
response.headers.set("Access-Control-Allow-Origin", "https://your-app.com");
response.headers.set("Access-Control-Allow-Methods", "POST, OPTIONS");
response.headers.set("Access-Control-Allow-Headers", "Content-Type");
return response;

Add an OPTIONS handler if you need preflight support.

Step 8: Meter usage per conversation

The usage object in onFinish gives you prompt, completion, and total tokens for that turn. Aggregate it client-side or POST to your analytics endpoint:

onFinish: async (message, { usage }) => {
  if (!usage) return;
  await fetch("/api/usage", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      conversationId: "your-conversation-id",
      model,
      provider,
      promptTokens: usage.promptTokens,
      completionTokens: usage.completionTokens,
      totalTokens: usage.totalTokens,
      timestamp: Date.now(),
    }),
    keepalive: true, // survives page unload
  });
}

On the server, persist to your database. This lets you build cost dashboards, enforce per-user budgets, or implement token-based billing.

Step 9: Deploy to Vercel

Push to GitHub, import in Vercel, add N4N_API_KEY as an Environment Variable in Project Settings. The Edge Runtime is detected automatically from the export const runtime = "edge" line.

No Dockerfile, no server configuration, no WebSocket server. The streaming works over standard HTTP/1.1 chunked transfer encoding — compatible with every CDN and load balancer.

Step 10: Extend with tool calling (optional)

The AI SDK supports tool calls in the stream. Add a tools definition to streamText:

import { tool } from "ai";
import { z } from "zod";

const result = await streamText({
  model: client(model),
  messages,
  tools: {
    getWeather: tool({
      parameters: z.object({
        location: z.string(),
        unit: z.enum(["celsius", "fahrenheit"]).default("celsius"),
      }),
      execute: async ({ location, unit }) => {
        // Call your weather API here
        return { temperature: 22, unit, location };
      },
    }),
  },
  // ... rest of config
});

The client renders tool invocations and results automatically when you use the useChat hook — no extra client code needed. The stream includes tool_call and tool_result chunks that the hook parses into the message parts array.

You now have a streaming chat interface that runs on the Edge Runtime, routes through n4n.ai with automatic fallback, meters token usage per turn, and deploys with a single git push. The same pattern scales to multi-turn conversations, RAG pipelines, and agent workflows — just swap the messages array and add tools.