Stream Claude 3.5 Sonnet replies with useChat and n4n.ai

The Vercel AI SDK’s useChat hook handles the client-side streaming machinery — message state, optimistic updates, abort controllers — so you don’t have to. Pair it with an OpenAI-compatible endpoint that speaks Anthropic’s format and you get a production-grade chat interface in under 100 lines. This tutorial walks through wiring useChat to Claude 3.5 Sonnet via n4n.ai, including server-side streaming, error handling, and a few sharp edges you’ll hit in practice.

Prerequisites

Node.js 20+ and pnpm (or npm/yarn)
An n4n.ai API key — sign up at n4n.ai if you don’t have one
Basic familiarity with Next.js App Router and TypeScript

Create a fresh project:

pnpm create next-app@latest claude-chat --typescript --tailwind --eslint --app --src-dir --import-alias "@/*" --use-pnpm
cd claude-chat
pnpm add ai @ai-sdk/openai
pnpm add -D @types/node

The ai package contains useChat and the streaming helpers. @ai-sdk/openai provides the OpenAI-compatible client that works with any endpoint speaking that protocol — including n4n.ai.

Environment configuration

Create .env.local in the project root:

# .env.local
N4N_API_KEY="your-n4n-api-key"
N4N_BASE_URL="https://api.n4n.ai/v1"

The base URL points to n4n.ai’s OpenAI-compatible endpoint. The gateway accepts standard OpenAI chat completion parameters and translates them to the upstream provider — in this case Anthropic — while handling retries, fallback, and usage metering transparently.

Server route: streaming chat completions

Create src/app/api/chat/route.ts. This is where the streaming response originates.

// src/app/api/chat/route.ts
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('claude-3-5-sonnet-20241022', {
      baseURL: process.env.N4N_BASE_URL,
      apiKey: process.env.N4N_API_KEY,
    }),
    messages,
    temperature: 0.3,
    maxTokens: 4096,
  });

  return result.toDataStreamResponse();
}

A few things worth noting:

streamText returns a StreamTextResult with helpers for different response formats. toDataStreamResponse() emits the Vercel AI SDK’s data stream protocol — a newline-delimited JSON format that useChat consumes natively.
The model identifier claude-3-5-sonnet-20241022 is the exact model slug n4n.ai exposes. Using the dated snapshot avoids surprise behavior changes when Anthropic rolls new versions.
maxDuration (seconds) configures the Vercel function timeout. Streaming responses can exceed the default 10s limit on Hobby plans.

Start the dev server and verify the endpoint:

pnpm dev

In another terminal:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Say hello in one sentence."}]}'

You should see a streaming response like:

data: {"type":"text-delta","text":"Hello! How can I help you today?"}
data: {"type":"finish","finishReason":"stop","usage":{"promptTokens":12,"completionTokens":9}}

If you get a 401, double-check N4N_API_KEY. If the model name is unrecognized, confirm the slug matches what n4n.ai currently exposes.

Client page: the useChat hook

Replace src/app/page.tsx with a minimal chat interface.

// src/app/page.tsx
'use client';

import { useChat } from 'ai/react';
import { useState } from 'react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, error } = useChat({
    api: '/api/chat',
    onError: (err) => {
      console.error('Chat error:', err);
      alert('Something went wrong. Check the console.');
    },
  });

  return (
    <main className="flex min-h-screen flex-col items-center p-4">
      <div className="w-full max-w-2xl">
        <header className="mb-6">
          <h1 className="text-2xl font-semibold">Claude 3.5 Sonnet via n4n.ai</h1>
          <p className="text-sm text-gray-500">Streaming with useChat</p>
        </header>

        <div className="border rounded-lg overflow-hidden bg-white shadow-sm">
          <div className="h-96 overflow-y-auto p-4 space-y-4">
            {messages.map((m) => (
              <div
                key={m.id}
                className={`flex ${m.role === 'assistant' ? 'justify-start' : 'justify-end'}`}
              >
                <div
                  className={`max-w-[80%] rounded-2xl px-4 py-2 ${
                    m.role === 'assistant'
                      ? 'bg-gray-100 text-gray-900 rounded-tl-none'
                      : 'bg-blue-600 text-white rounded-tr-none'
                  }`}
                >
                  <p className="whitespace-pre-wrap">{m.content}</p>
                </div>
              </div>
            ))}
            {isLoading && (
              <div className="flex justify-start">
                <div className="bg-gray-100 rounded-2xl rounded-tl-none px-4 py-2 animate-pulse">
                  <span className="text-gray-500">Claude is typing…</span>
                </div>
              </div>
            )}
          </div>

          <form onSubmit={handleSubmit} className="border-t p-4">
            <div className="flex gap-2">
              <input
                value={input}
                onChange={handleInputChange}
                placeholder="Message Claude…"
                className="flex-1 border rounded-lg px-4 py-2 focus:outline-none focus:ring-2 focus:ring-blue-500"
                disabled={isLoading}
              />
              <button
                type="submit"
                disabled={isLoading || input.trim() === ''}
                className="px-4 py-2 bg-blue-600 text-white rounded-lg disabled:opacity-50 disabled:cursor-not-allowed"
              >
                Send
              </button>
            </div>
            {error && (
              <p className="mt-2 text-sm text-red-600" role="alert">
                Error: {error.message}
              </p>
            )}
          </form>
        </div>
      </div>
    </main>
  );
}

useChat manages the entire client-side loop:

messages — array of { id, role, content } objects, updated optimistically
input / handleInputChange / handleSubmit — controlled input bindings
isLoading — true while a request is in flight
error — populated if the request fails

The hook posts to /api/chat by default, expecting the data stream protocol. No extra wiring needed.

Run pnpm dev and open http://localhost:3000. Type a message. You should see tokens appear character-by-character as the stream arrives.

Handling large contexts and tool calls

Claude 3.5 Sonnet supports 200k context tokens. The default maxTokens: 4096 in the route caps completions, not context. If you need longer replies, raise it:

maxTokens: 8192, // or up to the model's output limit

For tool calling, define tools in the streamText call:

import { tool } from 'ai';
import { z } from 'zod';

const result = streamText({
  model: openai('claude-3-5-sonnet-20241022', {
    baseURL: process.env.N4N_BASE_URL,
    apiKey: process.env.N4N_API_KEY,
  }),
  messages,
  tools: {
    getWeather: tool({
      parameters: z.object({
        location: z.string(),
        unit: z.enum(['celsius', 'fahrenheit']).default('celsius'),
      }),
      execute: async ({ location, unit }) => {
        // Your weather API call here
        return { temperature: 22, unit, condition: 'sunny' };
      },
    }),
  },
  maxSteps: 5, // allow multi-step tool use
});

maxSteps lets the model call tools iteratively. The data stream protocol surfaces tool calls and results as distinct message parts; useChat renders them automatically if you map over message.parts instead of message.content. For a minimal UI, the current approach works — tool results appear as assistant messages once the step completes.

Error handling and retries

Network hiccups happen. The gateway handles provider-level fallback (e.g., if Anthropic is degraded, it can route to another Claude-serving provider), but client-side resilience is still your responsibility.

Server-side: abort on disconnect

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('claude-3-5-sonnet-20241022', {
      baseURL: process.env.N4N_BASE_URL,
      apiKey: process.env.N4N_API_KEY,
    }),
    messages,
    temperature: 0.3,
    maxTokens: 4096,
    abortSignal: req.signal, // critical: stop generation if client disconnects
  });

  return result.toDataStreamResponse();
}

Passing req.signal as abortSignal ensures the upstream request cancels when the user navigates away or closes the tab. Without this, you burn tokens generating a response nobody sees.

Client-side: retry with exponential backoff

useChat doesn’t auto-retry. Wrap the fetch or use a custom fetch implementation:

// src/lib/fetch-with-retry.ts
export async function fetchWithRetry(
  url: string,
  options: RequestInit,
  retries = 3,
  backoffMs = 1000
): Promise<Response> {
  try {
    const res = await fetch(url, options);
    if (!res.ok && res.status >= 500 && retries > 0) {
      await new Promise((r) => setTimeout(r, backoffMs));
      return fetchWithRetry(url, options, retries - 1, backoffMs * 2);
    }
    return res;
  } catch (err) {
    if (retries > 0) {
      await new Promise((r) => setTimeout(r, backoffMs));
      return fetchWithRetry(url, options, retries - 1, backoffMs * 2);
    }
    throw err;
  }
}

Then pass a custom fetch to useChat:

const { messages, input, handleInputChange, handleSubmit, isLoading, error } = useChat({
  api: '/api/chat',
  fetch: fetchWithRetry,
  onError: (err) => console.error('Chat error:', err),
});

This retries on 5xx responses and network failures, backing off exponentially. Adjust retries and backoffMs to your SLA.

Streaming protocol details

The data stream protocol is line-oriented JSON. Each line is a complete object with a type field. Common types:

Type	Payload	When
`text-delta`	`{ text: string }`	Each token chunk
`tool-call`	`{ toolCallId, toolName, args }`	Model invokes a tool
`tool-result`	`{ toolCallId, result }`	Tool execution completes
`finish`	`{ finishReason, usage }`	Stream ends
`error`	`{ message }`	Server error mid-stream

useChat parses this automatically. If you ever need raw access — for debugging, logging, or a custom renderer — use the experimental useChat overload with experimental_onChunk:

const { messages, ... } = useChat({
  api: '/api/chat',
  experimental_onChunk: (chunk) => {
    console.log('Raw chunk:', chunk);
  },
});

Deployment notes

Vercel

Deploy as-is. The maxDuration = 30 export sets the function timeout. For Pro/Enterprise plans, you can raise this to 60s or 300s. Hobby plans are capped at 10s — if you need longer streams on Hobby, consider chunking responses or reducing maxTokens.

Docker / self-hosted

If you containerize, ensure the runtime supports streaming responses (Node.js does). Set N4N_API_KEY and N4N_BASE_URL as container env vars. No other changes needed.

Edge runtime

The ai SDK supports the Edge runtime, but @ai-sdk/openai uses fetch which works fine on Edge. Add export const runtime = 'edge'; to the route if you want lower cold starts. Note: Edge functions have stricter CPU limits; heavy tool execution may hit them.

Common pitfalls

Double streaming — If you wrap streamText in another ReadableStream or call toDataStreamResponse() twice, the client receives garbled data. Call it once, return the response directly.

Missing abortSignal — Without it, disconnected clients leave orphaned generations. Always pass req.signal.

CORS on the API route — Next.js App Router handles CORS for same-origin calls. If you call the route from a different origin (e.g., a separate frontend deploy), add a cors header or use Next.js middleware to allow it.

Model slug drift — n4n.ai may add new dated snapshots (claude-3-5-sonnet-20250115). Pin the exact slug in code. Don’t rely on aliases like claude-3-5-sonnet-latest unless you explicitly want auto-upgrades.

Token counting — The usage object in the finish chunk reflects the gateway’s accounting. It matches what you’ll see in the n4n.ai dashboard. Use it for cost tracking, not for truncation logic — the model already respects maxTokens.

What’s next

Persist conversations to a database (Postgres, SQLite, or a managed service) and hydrate initialMessages in useChat.
Add authentication and per-user rate limiting at the API route level.
Implement server-side message truncation or summarization to keep context within budget for multi-turn conversations.
Explore useCompletion for non-chat streaming use cases (code generation, summarization).

The combination of useChat, the OpenAI-compatible client, and a gateway that normalizes provider quirks gives you a maintainable stack. You own the UI and the routing logic; the gateway handles the provider mesh. That’s the right separation of concerns.