Streaming LLM responses in Next.js with Server Actions

Streaming tokens from a language model into a React UI used to mean standing up a custom API route and hand-wiring a fetch reader. With the App Router, next.js server actions llm streaming removes that middle layer: the client calls a function, the server opens a stream, and React renders chunks as they arrive. This tutorial builds a minimal chat box that streams responses using the Vercel AI SDK and a server action.

Prerequisites

• Node.js 18.18+ or 20+ (App Router requires React Server Components).
• A Next.js 14.2+ project. If you don’t have one, create-next-app with the App Router.
• An API key from an OpenAI-compatible provider. You can use OpenAI directly, or point at a gateway that exposes one endpoint for many models.
• Basic familiarity with TypeScript and React state.

Scaffold the project

npx create-next-app@latest stream-demo --ts --app --no-tailwind --no-eslint
cd stream-demo

Pick defaults for the rest. The --app flag gives you app/page.tsx as a server component.

Install the AI SDK

We use ai and @ai-sdk/openai. The ai/rsc entry provides createStreamableValue and readStreamableValue, which are built for next.js server actions llm streaming.

npm install ai @ai-sdk/openai

Configure the provider

Create .env.local:

OPENAI_API_KEY=sk-...

If you want fallback across providers without code changes, set the base URL to an OpenAI-compatible gateway. For example, n4n.ai exposes one endpoint covering 240+ models and handles automatic fallback when a provider is rate-limited; you just set OPENAI_BASE_URL and keep the same openai() calls.

OPENAI_BASE_URL=https://api.n4n.ai/v1

In app/actions.ts, initialize the model:

'use server';

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { createStreamableValue } from 'ai/rsc';

export async function generate(prompt: string) {
  const stream = createStreamableValue('');
  const { textStream } = streamText({
    model: openai('gpt-4o-mini'),
    prompt,
    temperature: 0.7,
  });

  (async () => {
    for await (const delta of textStream) {
      stream.update(delta);
    }
    stream.done();
  })();

  return stream.value;
}

The createStreamableValue returns a serializable handle. The server action returns that handle; the client reads it with readStreamableValue. This is the core of next.js server actions llm streaming.

Build the client component

Server actions can be imported into client components and invoked like local async functions. Create app/chat.tsx:

'use client';

import { useState } from 'react';
import { readStreamableValue } from 'ai/rsc';
import { generate } from './actions';

export default function Chat() {
  const [output, setOutput] = useState('');
  const [loading, setLoading] = useState(false);

  async function handleSubmit(formData: FormData) {
    const prompt = formData.get('prompt') as string;
    if (!prompt) return;
    setOutput('');
    setLoading(true);

    const stream = await generate(prompt);
    for await (const chunk of readStreamableValue(stream)) {
      setOutput((prev) => prev + chunk);
    }
    setLoading(false);
  }

  return (
    <div>
      <form action={handleSubmit}>
        <input name="prompt" placeholder="Ask something..." />
        <button type="submit" disabled={loading}>
          {loading ? 'Streaming…' : 'Send'}
        </button>
      </form>
      <pre>{output}</pre>
    </div>
  );
}

Note the action={handleSubmit} prop. Next.js passes the FormData directly to the server action via the form. No onClick handler or fetch needed.

Replace app/page.tsx to render it:

import Chat from './chat';

export default function Page() {
  return (
    <main style={{ padding: '2rem', fontFamily: 'monospace' }}>
      <h1>Streaming Chat</h1>
      <Chat />
    </main>
  );
}

Run and verify

Start the dev server:

npm run dev

Open http://localhost:3000. Type Explain recursion in one sentence. and submit. You should see the answer appear token by token in the <pre> block. Expected partial output mid-stream:

A function that calls itself to solve smaller instances of the same problem

Final output might be:

A function that calls itself to solve smaller instances of the same problem until a base case stops the cycle.

If you see the full text pop in at once, confirm stream.update is called inside the for await loop and that you are not awaiting streamText result fully before returning.

Extend to multi-turn chat

A single prompt is a toy. Real chat keeps history. Change the action to accept a messages array:

'use server';

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { createStreamableValue } from 'ai/rsc';

type Msg = { role: 'user' | 'assistant' | 'system'; content: string };

export async function chat(messages: Msg[]) {
  const stream = createStreamableValue('');
  const { textStream } = streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });

  (async () => {
    for await (const delta of textStream) {
      stream.update(delta);
    }
    stream.done();
  })();

  return stream.value;
}

On the client, hold messages in state and append:

'use client';

import { useState } from 'react';
import { readStreamableValue } from 'ai/rsc';
import { chat } from './actions';

type Msg = { role: 'user' | 'assistant'; content: string };

export default function Chat() {
  const [messages, setMessages] = useState<Msg[]>([]);
  const [input, setInput] = useState('');
  const [streaming, setStreaming] = useState('');

  async function send() {
    if (!input) return;
    const next = [...messages, { role: 'user' as const, content: input }];
    setMessages(next);
    setInput('');
    setStreaming('');

    const stream = await chat(next);
    let acc = '';
    for await (const chunk of readStreamableValue(stream)) {
      acc += chunk;
      setStreaming(acc);
    }
    setMessages([...next, { role: 'assistant', content: acc }]);
    setStreaming('');
  }

  return (
    <div>
      {messages.map((m, i) => (
        <p key={i}><strong>{m.role}:</strong> {m.content}</p>
      ))}
      {streaming && <p><strong>assistant:</strong> {streaming}</p>}
      <input value={input} onChange={(e) => setInput(e.target.value)} />
      <button onClick={send}>Send</button>
    </div>
  );
}

The next.js server actions llm streaming pattern stays identical; only the payload shape changes.

Handle errors and aborts

Network hiccups happen. Wrap the stream consumption in try/catch and surface errors:

try {
  const stream = await chat(next);
  for await (const chunk of readStreamableValue(stream)) {
    acc += chunk;
    setStreaming(acc);
  }
} catch (err) {
  setStreaming('Error: ' + (err as Error).message);
} finally {
  setStreaming('');
}

On the server side, streamText throws if the provider returns a non-200. With a gateway that honors client routing directives, you can also forward cache-control hints to avoid repeated billing on identical prompts. Pass headers in streamText if your provider needs them.

To let users cancel, keep an AbortController and pass abortSignal to streamText. The SDK cancels the upstream request and ends the stream.

Why server actions beat route handlers here

You get end-to-end type safety: the action’s argument and return types are checked at compile time. No stringly-typed fetch URLs. The form integration means the browser handles submission, pending UI, and restoration on navigation for free. And because the stream travels inside the RSC payload, you avoid writing a custom ReadableStream parser on the client.

The trade-off is that server actions are not meant for long-poll external webhooks. Keep them for direct user-initiated generations.

Production considerations

Server actions are not cached by default, but they do serialize arguments. Don’t send huge conversation histories as raw strings without trimming; pass only the last N messages.

Streaming responses keep a connection open. On serverless platforms, watch function timeouts. If you expect long generations, set a higher timeout or run the action in a node runtime:

export const runtime = 'nodejs';
export const maxDuration = 30;

The ai/rsc streaming protocol uses a hidden <script> payload in the RSC response. It works under SSR, but if you wrap the action in useEffect manually, you lose form semantics. Prefer the <form action> pattern or the explicit useTransition wrapper.

If you use a gateway with per-token usage metering, read usage from the streamText response after the loop to log costs. That data is available on the server side only.

Wrapping up

You now have a working chat UI that streams from a server action with no custom API route. The same generate or chat function can be reused in a Route Handler or a background job. If you swap the base URL to a gateway that aggregates models, the code does not change—you only get broader model coverage and fallback.

Keep the client dumb: it renders chunks. Keep the server action focused: it opens the model stream and pipes deltas. That separation is what makes next.js server actions llm streaming maintainable when the UI grows.