Building resilient LLM apps means never betting on a single model endpoint. This hands-on tutorial shows how to implement vercel ai sdk multi-model switching n4n.ai routing so your app picks the right model per request and survives provider outages. We’ll stand up a small Node script that swaps models based on task type, leverages automatic fallback, and reads per-token usage.
Prerequisites
- Node.js 18+ (fetch is built in, no extra HTTP client needed)
- An API key from a gateway that speaks the OpenAI protocol. n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and applies automatic fallback when a provider is rate-limited or degraded.
- TypeScript toolchain (
tsxorts-node) for running the script - Basic familiarity with the Vercel AI SDK
generateTextAPI
Install the required packages:
npm init -y
npm install ai @ai-sdk/openai zod
npm install -D tsxExport your key into the environment:
export N4N_API_KEY="sk-your-key"Configure the gateway as an OpenAI-compatible provider
The Vercel AI SDK’s OpenAI provider is just a thin wrapper around the chat completions wire format. Point its baseURL at the gateway instead of api.openai.com. You get every model behind the gateway without writing new client code.
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';
const gateway = createOpenAI({
baseURL: 'https://api.n4n.ai/v1',
apiKey: process.env.N4N_API_KEY,
});One provider instance now fronts the whole model catalog. You reference models by string ID, not by constructing separate SDK objects. This avoids the overhead of bundling multiple vendor SDKs and centralizes timeout and retry configuration. The gateway’s single endpoint also simplifies CORS and secret management because only one base URL and key touch your runtime.
Define a model registry
A registry keeps model choices explicit and reviewable. Use provider-prefixed IDs so the gateway knows where to route. Model IDs follow the provider/model convention that most aggregators use. If you later add a self-hosted model, register it with a custom prefix and the same switching code works.
const MODELS = {
classify: 'anthropic/claude-3-haiku',
draft: 'openai/gpt-4o-mini',
reason: 'anthropic/claude-3-opus',
} as const;
type Task = keyof typeof MODELS;Cheap, fast models handle classification. Mid-tier models draft text. Heavy reasoning goes to the largest model. This tiering is the core of cost control.
Implement switching logic
Prompt prefixes are the simplest way to signal intent in a demo. In production you’d inspect request metadata or a classified intent field. Keep the routing rule pure and synchronous. Side effects like logging belong in the caller. This makes the function trivially unit-testable with a table of inputs and expected task outputs.
function pickModel(prompt: string): Task {
if (prompt.startsWith('/classify')) return 'classify';
if (prompt.startsWith('/reason')) return 'reason';
return 'draft';
}Execute a switched request
Strip the prefix before sending the prompt to the model. The SDK returns usage directly from the gateway’s response.
async function run(prompt: string) {
const task = pickModel(prompt);
const model = gateway(MODELS[task]);
const { text, usage } = await generateText({
model,
prompt: prompt.replace(/^\/\w+\s*/, ''),
});
console.log(`[${task}] ${text.slice(0, 80)}`);
console.log('usage', usage);
}Run await run('/classify Is this invoice suspicious?'). Expected output:
[classify] The invoice shows no obvious fraud markers.
usage { promptTokens: 14, completionTokens: 9, totalTokens: 23 }The same run call with /reason Prove sqrt(2) is irrational hits the opus model and returns a longer proof with higher token usage.
Honor client routing directives
The gateway forwards provider cache-control hints and honors client routing directives sent through request headers. With the Vercel AI SDK you attach headers at model creation or per call. To force a specific upstream and disable fallback for a critical request:
const model = gateway(MODELS[task], {
headers: {
// gateway recognizes its routing directive headers
'x-n4n-routing': 'provider:openai',
'x-n4n-fallback': 'false',
},
});To take advantage of provider prompt caching, forward the standard cache-control header; the gateway passes it upstream to providers that support caching.
const { text } = await generateText({
model,
prompt: 'Long system context...',
headers: { 'cache-control': 'max-age=3600' },
});This keeps your caching strategy portable across providers without code changes.
Automatic fallback in action
Even when you leave fallback enabled, the gateway reroutes on 429 or 503 from the primary provider. You write no retry loops. You can verify fallback behavior by temporarily setting a header that forces a bad provider; the gateway should still return a completion. This is cheaper than chaos engineering against real vendor outages.
async function runWithFallback() {
const model = gateway('openai/gpt-4o');
const { text } = await generateText({
model,
prompt: 'Summarize: the quick brown fox jumps over the lazy dog',
});
console.log(text);
}If the default OpenAI path is degraded, the gateway serves the same capability from a secondary provider and returns a normal completion. Your application code stays identical.
Per-token usage metering
Every response includes usage from the SDK, reflecting exactly what the gateway metered. Because the gateway does per-token usage metering, you can attribute cost precisely across models and tasks.
const { usage } = await generateText({ model, prompt });
console.log(`Task ${task} consumed ${usage.totalTokens} tokens`);Persist this to a local file for quick audits:
import fs from 'node:fs';
fs.appendFileSync('usage.csv', `${task},${usage.totalTokens}\n`);In a real system, ship these metrics to Prometheus or your billing pipeline.
Full runnable script
Combine the pieces into index.ts:
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';
const gateway = createOpenAI({
baseURL: 'https://api.n4n.ai/v1',
apiKey: process.env.N4N_API_KEY,
});
const MODELS = {
classify: 'anthropic/claude-3-haiku',
draft: 'openai/gpt-4o-mini',
reason: 'anthropic/claude-3-opus',
} as const;
type Task = keyof typeof MODELS;
function pickModel(prompt: string): Task {
if (prompt.startsWith('/classify')) return 'classify';
if (prompt.startsWith('/reason')) return 'reason';
return 'draft';
}
async function run(prompt: string) {
const task = pickModel(prompt);
const model = gateway(MODELS[task]);
const { text, usage } = await generateText({
model,
prompt: prompt.replace(/^\/\w+\s*/, ''),
});
console.log(`[${task}] ${text.slice(0, 80)}`);
console.log('usage', usage);
}
await run('/classify Is this invoice suspicious?');
await run('/reason Prove sqrt(2) is irrational');
await run('Write a haiku about cold caches');Execute with npx tsx index.ts. You will see three model tiers invoked through one endpoint, with token counts per task printed inline.
Wire into a Next.js route handler
Framework tutorials should show the integration point. In a Next.js app router route:
// app/api/chat/route.ts
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';
const gateway = createOpenAI({
baseURL: 'https://api.n4n.ai/v1',
apiKey: process.env.N4N_API_KEY,
});
export async function POST(req: Request) {
const { prompt, task } = await req.json();
const modelId = task === 'reason' ? 'anthropic/claude-3-opus' : 'openai/gpt-4o-mini';
const { text } = await generateText({ model: gateway(modelId), prompt });
return Response.json({ text });
}The client sends a task field; the server switches models without extra dependencies. Streaming works by swapping to streamText and returning result.toDataStreamResponse(). Remember to set export const runtime = 'edge' only if the gateway is reachable from the edge runtime; otherwise use the node runtime to avoid fetch limitations.
Extending the pattern
Move switching logic into middleware that reads authenticated user tiers or request latency budgets. Use routing headers to pin a provider for regulatory reasons. The multi-model switching pattern stays the same: one provider instance, many model strings, explicit selection. No custom retry code, no per-vendor SDK lock-in.
If you need streaming, the usage object arrives in the final chunk; meter it the same way. That’s the entire mechanism.