Building a streaming AI chat app with Next.js App Router

Building a next.js app router ai chat streaming experience requires wiring server logic to a language model and pushing tokens to the client as they generate. This tutorial walks through a minimal but production-shaped implementation using the Vercel AI SDK and Next.js 14 App Router, with runnable code at each step.

Prerequisites

Node.js 18.17 or later (App Router streams need stable ReadableStream support).
A package manager: npm, pnpm, or yarn.
An API key from an OpenAI-compatible provider. If you’d rather not juggle multiple vendor keys, an OpenAI-compatible gateway such as n4n.ai fronts 240+ models behind one endpoint and handles provider fallback automatically.
Basic comfort with React Server Components, TypeScript, and terminal commands.

Scaffold the project

npx create-next-app@latest ai-chat --ts --app --eslint --tailwind --src-dir --import-alias "@/*"
cd ai-chat

Accept the prompts for the remaining defaults. The --app flag creates src/app, where we’ll place routes and client components. --src-dir keeps application code out of the repo root.

Project structure

After the steps below, your tree should resemble:

src/
  app/
    api/chat/route.ts
    page.tsx
  components/chat.tsx
  lib/model.ts
.env.local

Keeping the model client in lib/model.ts makes it trivial to swap providers without touching route logic.

Install the AI SDK

We need the core ai package, the OpenAI adapter, and the React hooks.

npm install ai @ai-sdk/openai @ai-sdk/react

At time of writing, ai@^3.4 and @ai-sdk/openai@^1.0 are current. Pin them in package.json if you run reproducible builds.

Configure the model provider

Create .env.local for local dev:

OPENAI_API_KEY=sk-...

If you route through a gateway, set the base URL and key instead:

// src/lib/model.ts
import { createOpenAI } from '@ai-sdk/openai';

export const openai = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY!,
});

The rest of the code stays identical. The gateway forwards provider cache-control hints and honors client routing directives, so you can change models at request time.

Build the streaming route

App Router isolates server logic in route.ts files. We expose POST /api/chat and use streamText to return a data stream the client consumes incrementally.

// src/app/api/chat/route.ts
import { streamText } from 'ai';
import { openai } from '@/lib/model';

export const runtime = 'edge';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });

  return result.toDataStreamResponse();
}

runtime = 'edge' runs the handler on Vercel’s edge runtime, which keeps time-to-first-token low. If you need Node APIs (fs, native modules), switch to export const runtime = 'nodejs' and add export const dynamic = 'force-dynamic' to prevent static caching of the POST.

Checkpoint: curl the endpoint

Start the dev server and hit the route directly:

npm run dev
curl -X POST http://localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hello in 5 words"}]}'

You should see streamed chunks, not a single JSON blob:

0:"Hello! Hope you are well."

The 0: prefix is the AI SDK data stream protocol. The client hook decodes it into message content.

Create the chat UI

The SDK ships useChat, which manages message state, input, submission, and stream consumption. Build a client component:

// src/components/chat.tsx
'use client';

import { useChat } from '@ai-sdk/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, error, reload } = useChat();

  return (
    <div className="mx-auto max-w-lg p-4">
      <div className="space-y-2">
        {messages.map((m) => (
          <div key={m.id} className="whitespace-pre-wrap">
            <span className="font-semibold">{m.role}: </span>
            {m.content}
          </div>
        ))}
      </div>

      {error && (
        <div className="mt-2 text-red-600">
          Failed. <button onClick={reload} className="underline">Retry</button>
        </div>
      )}

      <form onSubmit={handleSubmit} className="mt-4 flex gap-2">
        <input
          className="flex-1 rounded border p-2"
          value={input}
          onChange={handleInputChange}
          placeholder="Type a message…"
        />
        <button
          type="submit"
          disabled={isLoading}
          className="rounded bg-black px-4 py-2 text-white"
        >
          Send
        </button>
      </form>
    </div>
  );
}

useChat posts to /api/chat by default. It appends prior messages to the request body, so multi-turn context works without extra code.

Mount it in a page

// src/app/page.tsx
import Chat from '@/components/chat';

export default function Home() {
  return (
    <main className="min-h-screen py-10">
      <h1 className="mb-6 text-center text-2xl font-bold">Streaming Chat</h1>
      <Chat />
    </main>
  );
}

Reload the page. Type “Explain recursion in one sentence”. Tokens appear left-to-right without a full refresh. That is the next.js app router ai chat streaming loop closed.

Inspect the browser stream

Open DevTools → Network → chat request → Response. You’ll see the same 0:"..." frames arriving as they’re produced. The hook buffers them into messages[i].content. This is plain fetch + ReadableStream, no WebSocket needed.

Error and abort handling

Streams fail. Add a try/catch and a clean error message in the route:

export async function POST(req: Request) {
  try {
    const { messages } = await req.json();
    const result = await streamText({
      model: openai('gpt-4o-mini'),
      messages,
    });
    return result.toDataStreamResponse({
      onError: (e) => 'Stream error: ' + (e as Error).message,
    });
  } catch {
    return new Response('Bad request', { status: 400 });
  }
}

The client error state surfaces this; reload() re-sends the last messages array.

Swap models without redeploying

Because the gateway honors client routing directives, you can pass a header from the browser and read it server-side:

const model = req.headers.get('x-model') ?? 'gpt-4o-mini';
const result = await streamText({ model: openai(model), messages });

Set the header in useChat via the headers option:

useChat({ headers: { 'x-model': 'claude-3-5-sonnet' } });

This lets you A/B models or fall back to a cheaper one when a provider is degraded, without changing route code.

Production checklist

Store keys in host env vars, not committed .env.local.
Rate-limit /api/chat—useChat will let a user burn tokens unchecked.
Use cacheControl hints if your provider supports them; the gateway forwards them to origin models.
Monitor spend with per-token metering rather than guessing from logs.
Set runtime = 'nodejs' if you import Node-only packages; otherwise edge is faster for streaming.

That is a complete path from create-next-app to a live chat. The next.js app router ai chat streaming pattern stays the same whether you target one model or a fleet behind a gateway.