The Vercel AI SDK gives you a clean abstraction layer for LLM interactions, but the documentation assumes you’re calling providers directly. When you route through a gateway like n4n.ai, the setup changes slightly: you point the OpenAI-compatible client at a different base URL and pass routing headers. This tutorial walks through a complete, runnable Next.js app that streams Claude 3.5 Sonnet responses, handles tool calls, and degrades gracefully when the gateway returns errors.
Prerequisites
- Node.js 20+ and pnpm (or npm/yarn)
- An n4n.ai API key with access to Anthropic models
- Basic familiarity with Next.js App Router and TypeScript
Create a fresh project if you don’t have one:
pnpm create next-app@latest ai-sdk-demo --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"
cd ai-sdk-demoInstall the AI SDK packages and the OpenAI client (which n4n.ai’s endpoint speaks):
pnpm add ai @ai-sdk/openai zodEnvironment configuration
Never commit secrets. Create .env.local at the repo root:
# .env.local
N4N_API_KEY="n4n_sk_..."
N4N_BASE_URL="https://api.n4n.ai/v1"The base URL is the only thing that changes versus a direct Anthropic integration. The SDK’s OpenAI provider works because n4n.ai exposes an OpenAI-compatible /chat/completions endpoint that translates to upstream providers.
Gateway client wrapper
Create a small module that configures the provider once. This keeps route handlers clean and makes it trivial to swap models or add default headers later.
// src/lib/gateway.ts
import { createOpenAI } from '@ai-sdk/openai';
export const gateway = createOpenAI({
apiKey: process.env.N4N_API_KEY,
baseURL: process.env.N4N_BASE_URL,
});
export const sonnet = gateway('anthropic/claude-3.5-sonnet');The model ID anthropic/claude-3.5-sonnet follows n4n.ai’s provider/model convention. If you need a specific version pin, use anthropic/claude-3.5-sonnet-20241022.
Streaming chat route
The App Router route handler streams tokens to the client using streamText. This is the pattern you’ll use for most chat interfaces.
// src/app/api/chat/route.ts
import { streamText } from 'ai';
import { sonnet } from '@/lib/gateway';
import { z } from 'zod';
export const maxDuration = 30;
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: sonnet,
messages,
system: 'You are a concise coding assistant. Prefer code over prose.',
tools: {
getWeather: {
parameters: z.object({
location: z.string().describe('City and state, e.g. "San Francisco, CA"'),
unit: z.enum(['celsius', 'fahrenheit']).default('fahrenheit'),
}),
execute: async ({ location, unit }) => {
// In production, call a real weather API here
const temp = unit === 'celsius' ? 18 : 64;
return { location, temperature: temp, unit, condition: 'Partly cloudy' };
},
},
},
onError: (error) => {
console.error('[chat] stream error:', error);
},
});
return result.toDataStreamResponse({
// Send usage and finish reason in the final chunk
sendUsage: true,
sendFinishReason: true,
});
}Key points:
streamTextreturns aStreamTextResultthat handles backpressure and SSE formatting.- The
toolsobject defines functions the model can call.executeruns on your server — keep it fast and idempotent. toDataStreamResponseemits the AI SDK’s wire protocol: text deltas, tool calls, tool results, usage, and finish reason.
Client-side hook
The useChat hook manages message state, streaming, and tool call round-trips automatically.
// src/app/chat.tsx
'use client';
import { useChat } from 'ai/react';
import { useState } from 'react';
export default function Chat() {
const [isLoading, setIsLoading] = useState(false);
const { messages, input, handleInputChange, handleSubmit, status, error } = useChat({
api: '/api/chat',
onError: (err) => {
console.error('Chat error:', err);
setIsLoading(false);
},
onFinish: () => setIsLoading(false),
onResponse: (res) => {
if (!res.ok) {
throw new Error(`HTTP ${res.status}: ${res.statusText}`);
}
},
});
return (
<div className="max-w-2xl mx-auto p-4 space-y-4">
<div className="space-y-3">
{messages.map((m) => (
<div key={m.id} className={`p-3 rounded-lg ${m.role === 'user' ? 'bg-blue-50' : 'bg-gray-50'}`}>
<p className="font-mono text-sm text-gray-500">{m.role}</p>
<p className="whitespace-pre-wrap">{m.content}</p>
{m.toolInvocations?.length && (
<details className="mt-2 text-sm">
<summary className="cursor-pointer text-gray-600">Tool calls</summary>
<pre className="mt-1 p-2 bg-gray-100 rounded overflow-auto">
{JSON.stringify(m.toolInvocations, null, 2)}
</pre>
</details>
)}
</div>
))}
{status === 'streaming' && <div className="text-sm text-gray-500 animate-pulse">▌</div>}
</div>
{error && (
<div className="p-3 bg-red-50 text-red-700 rounded-lg text-sm">
{error.message}
</div>
)}
<form onSubmit={handleSubmit} className="flex gap-2">
<input
value={input}
onChange={handleInputChange}
placeholder="Ask about code, weather, anything..."
className="flex-1 px-3 py-2 border rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
disabled={status === 'streaming'}
/>
<button
type="submit"
disabled={status === 'streaming' || !input.trim()}
className="px-4 py-2 bg-blue-600 text-white rounded-lg disabled:opacity-50 disabled:cursor-not-allowed"
>
Send
</button>
</form>
</div>
);
}The hook handles the full lifecycle: optimistic UI updates, streaming deltas, tool call execution (the model calls getWeather, your server runs it, the result goes back to the model, and the final answer streams in), and error surfaces.
Wire it into a page
// src/app/page.tsx
import Chat from '@/app/chat';
export default function Home() {
return (
<main className="min-h-screen bg-gray-50">
<header className="border-b bg-white">
<div className="max-w-2xl mx-auto px-4 py-6">
<h1 className="text-2xl font-semibold">Vercel AI SDK + n4n.ai</h1>
<p className="text-gray-600 mt-1">Streaming Claude 3.5 Sonnet with tool calls</p>
</div>
</header>
<Chat />
</main>
);
}Run the dev server:
pnpm devOpen http://localhost:3000. Try: “What’s the weather in Denver?” — you’ll see the model call getWeather, your execute function run, and the final response stream back.
Expected output checkpoints
First token latency: ~300-800ms depending on gateway load and geographic proximity.
Streaming chunks: Text arrives in 10-50 token increments. The cursor (▌) in the UI blinks while status === 'streaming'.
Tool call flow (visible in the expanded “Tool calls” section):
[
{
"toolCallId": "call_abc123",
"toolName": "getWeather",
"args": { "location": "Denver, CO", "unit": "fahrenheit" },
"state": "call"
},
{
"toolCallId": "call_abc123",
"toolName": "getWeather",
"args": { "location": "Denver, CO", "unit": "fahrenheit" },
"state": "result",
"result": { "location": "Denver, CO", "temperature": 42, "unit": "fahrenheit", "condition": "Partly cloudy" }
}
]Final message: The assistant’s natural-language response incorporating the tool result.
Usage metadata (logged server-side if you add a onFinish callback):
{
"promptTokens": 142,
"completionTokens": 87,
"totalTokens": 229
}Error handling and fallbacks
The gateway returns standard HTTP codes. Handle them at the route level:
// src/app/api/chat/route.ts (additions)
import { GatewayTimeoutError } from 'ai';
export async function POST(req: Request) {
try {
// ... existing streamText call
} catch (err) {
if (err instanceof GatewayTimeoutError) {
return new Response('Gateway timeout', { status: 504 });
}
if (err instanceof Error && err.message.includes('rate_limit')) {
return new Response('Rate limited', { status: 429 });
}
return new Response('Internal error', { status: 500 });
}
}The AI SDK surfaces provider errors as APICallError with statusCode and responseBody. n4n.ai forwards upstream provider error shapes, so Anthropic’s overloaded_error or rate_limit_error come through intact.
For automatic fallback across providers, you’d configure that in the n4n.ai dashboard (routing rules, priority lists). The SDK side stays the same — you still call sonnet and the gateway handles the swap.
Production considerations
Caching: n4n.ai forwards Cache-Control hints from upstream. For deterministic prompts (classification, extraction), add cacheControl: { type: 'ephemeral' } to the streamText options to enable provider-side prompt caching where supported.
Observability: Wrap streamText with your tracing SDK (OpenTelemetry, Datadog, etc.). The onFinish callback receives { usage, finishReason, response } — log these per request.
Rate limiting: Implement per-user quotas at your API layer (Next.js middleware, Upstash Redis, etc.). The gateway enforces its own limits, but you want backpressure before requests leave your network.
Model pinning: In gateway.ts, pin to a dated snapshot (anthropic/claude-3.5-sonnet-20241022) to avoid silent behavior changes. Update intentionally during a deploy.
Testing the tool loop locally
You can exercise the full tool cycle without a browser:
curl -X POST http://localhost:3000/api/chat \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Weather in Tokyo?"}]}'Response is text/plain SSE. Look for data: {"type":"tool-call",...} and data: {"type":"tool-result",...} events before the final text deltas.
Summary
You now have a minimal, production-shaped stack:
- Gateway client (
src/lib/gateway.ts) — single source of truth for auth, base URL, and model IDs. - Streaming route (
src/app/api/chat/route.ts) —streamTextwith tools, system prompt, and error boundaries. - React hook (
src/app/chat.tsx) —useChathandles state, streaming, and tool round-trips. - Observability hooks —
onFinish,onError, and response metadata for logging.
Swap sonnet for gateway('openai/gpt-4o') or gateway('google/gemini-1.5-pro') and the rest of the code doesn’t change. That’s the point of the gateway pattern: your application code stays provider-agnostic while the routing layer handles fallbacks, caching, and cost optimization.