Vercel AI SDK provider switching with n4n.ai's routing

Setting up vercel ai sdk provider switching routing lets you absorb provider outages without rewriting your call sites. This walkthrough shows how to wire the Vercel AI SDK to a single OpenAI-compatible gateway that performs automatic fallback, while still allowing explicit model selection in code.

Step 1: Install the Vercel AI SDK and OpenAI provider

The AI SDK splits core utilities from provider bindings. You need the ai package and @ai-sdk/openai, which speaks the OpenAI HTTP shape and works against any compliant endpoint.

npm install ai @ai-sdk/openai
# or
pnpm add ai @ai-sdk/openai

Target Node 18+ or any modern edge runtime. The code below uses ESM imports, so ensure your package.json has "type": "module" or use a .mjs file.

Step 2: Configure a single gateway client

Instead of instantiating separate providers for Anthropic, OpenAI, and Mistral, point the OpenAI provider at the gateway. n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models, so a single client covers most vendors.

import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

const gateway = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY ?? '',
});

// Model ids follow the gateway's naming convention, e.g. 'openai/gpt-4o', 'anthropic/claude-3-5-sonnet'
const model = gateway('anthropic/claude-3-5-sonnet');

const { text } = await generateText({
  model,
  prompt: 'Say hello in one sentence.',
});
console.log(text);

The SDK treats this as a standard OpenAI-compatible server. Streaming, tool calls, and usage metering work unchanged because the gateway translates the request to the target vendor and back.

Step 3: Build an explicit routing function

Gateway-level fallback handles degradation, but you still want task-based routing to control cost and quality. Write a small dispatcher that returns a LanguageModel instance based on intent.

type Task = 'classify' | 'summarize' | 'reason';

function routeModel(task: Task) {
  switch (task) {
    case 'classify':
      return gateway('mistral/mixtral-8x7b-instruct');
    case 'summarize':
      return gateway('openai/gpt-4o-mini');
    case 'reason':
      return gateway('anthropic/claude-3-opus');
    default:
      return gateway('openai/gpt-4o');
  }
}

Call sites stay clean:

const { text, usage } = await generateText({
  model: routeModel('summarize'),
  prompt: 'Summarize the following incident report: ...',
});

console.log(text, usage);

This pattern keeps vercel ai sdk provider switching routing declarative. The rest of the app imports routeModel and never references a vendor directly. If you later add a new model, you change one switch statement.

For dynamic routing based on live latency or quota, make the function async and query the gateway’s /models endpoint, but a static map is enough for most services.

Step 4: Send client routing directives and cache hints

The gateway honors client routing directives and forwards provider cache-control hints. With the Vercel AI SDK you can attach headers per request to express preferences:

const result = await generateText({
  model: gateway('anthropic/claude-3-5-sonnet'),
  prompt: 'Draft a contract clause about liability.',
  headers: {
    'x-routing-preference': 'anthropic; fallback=openai',
  },
});

When the gateway sees that header, it attempts Anthropic first and only shifts to OpenAI if Anthropic is rate-limited or degraded.

For vendors that support prompt caching, use the SDK’s provider extension fields. Anthropic’s cache_control is forwarded unchanged:

import { generateText } from 'ai';

const result = await generateText({
  model: gateway('anthropic/claude-3-5-sonnet'),
  messages: [
    {
      role: 'system',
      content: 'You are a legal assistant. Use plain language.',
      providerOptions: {
        anthropic: { cacheControl: { type: 'ephemeral' } },
      },
    },
    { role: 'user', content: 'Draft a clause for a SaaS agreement.' },
  ],
});

If the gateway routes to a vendor that ignores cache control, it strips the field. That avoids 400s and keeps your code portable. This is the kind of detail that matters when you care about per-token billing.

Step 5: Verify success and fallback behavior

Verification is twofold: confirm the happy path and confirm degradation handling.

Happy path test in a script or Vitest:

const res = await generateText({
  model: routeModel('reason'),
  prompt: 'What is 2+2?',
});

if (res.text && res.usage.totalTokens > 0) {
  console.log('OK', res.usage);
} else {
  throw new Error('Empty response');
}

Fallback test: set N4N_API_KEY to a key with near-zero quota, or temporarily point baseURL to a mock that returns 429. Because the gateway provides automatic fallback when a provider is rate-limited or degraded, the same generateText call should still resolve with a different backing model. n4n.ai provides per-token usage metering, so you can inspect the returned usage object or your billing dashboard to see which vendor actually served the token.

If you run a local proxy in front of the gateway, log the outgoing x-served-by header (if your gateway adds one) or simply assert that text arrived within a timeout. The key point: your application code does not change between the happy path and the fallback path.

Caveats when mixing SDK providers

If you also import @ai-sdk/anthropic directly for some calls, you bypass the unified fallback and cache forwarding. Pick one pattern: either route everything through the gateway, or build your own try/catch chain (which multiplies code and misses cache translation). For most teams, the gateway pattern is less surface area.

Type safety: gateway('model-id') returns a LanguageModel object. A typo in the model id surfaces at request time, not compile time. Mitigate by exporting a const object of approved ids:

export const MODELS = {
  fast: 'openai/gpt-4o-mini',
  smart: 'anthropic/claude-3-opus',
} as const;

function route(task: Task) {
  return gateway(MODELS[task === 'reason' ? 'smart' : 'fast']);
}

Why this beats manual try/catch

A naive implementation wraps each provider call in try/catch and retries on the next vendor. That works for one call but breaks down with streaming, tool calls, and cache control. The vercel ai sdk provider switching routing approach described here pushes the failover logic to a layer that understands LLM semantics. Your generateText call stays identical whether the request hits Claude, GPT-4o, or a fallback model after a 429.

You also get centralized metering. Instead of aggregating token counts from three vendor SDKs with different shape responses, you read one usage object and trust the gateway’s accounting.

Quick reference

• Install: ai + @ai-sdk/openai
• Point baseURL at the gateway, keep apiKey from env
• Write a routeModel(task) helper returning gateway('vendor/model')
• Pass routing headers or providerOptions for cache control
• Verify with a happy-path token check and a forced 429 to watch fallback

That is the entire integration. No vendor SDKs to version-lock, no custom retry queues, and task-based routing stays in one file.