Vercel AI SDK: switching providers without new API keys

The Vercel AI SDK switch providers without new keys approach hinges on a single abstraction: point the SDK’s OpenAI-compatible provider at a gateway that fronts many models. Instead of juggling Anthropic, OpenAI, and Mistral keys, you configure one base URL and one token, then select models by name at request time. This cuts onboarding friction and lets you shift providers in code without secret rotation.

Step 1: Provision a single gateway credential

Pick a gateway that exposes an OpenAI-compatible HTTP interface and aggregates multiple upstream providers. For example, n4n.ai provides one OpenAI-compatible endpoint addressing 240+ models, so a single key unlocks Claude, GPT, Llama, and others. Create an API key there and note the base URL.

Store both values in your environment. Never hardcode secrets in source.

# .env.local
GATEWAY_API_KEY=sk_your_gateway_token
GATEWAY_BASE_URL=https://api.n4n.ai/v1

If you run outside Node, export them or use your platform’s secret store. The rest of the steps assume these two variables are available to the process. The win here is operational: one rotated secret instead of N. When a provider changes its auth scheme, the gateway absorbs it; your client code does not move.

Step 2: Install the Vercel AI SDK and the OpenAI shim

The SDK decouples your app from provider SDKs via the @ai-sdk/openai package, which implements the OpenAI wire protocol. Install it alongside the core ai package.

npm install ai @ai-sdk/openai
# pin majors in production: ai@^4 @ai-sdk/openai@^1

Avoid pulling @ai-sdk/anthropic or @ai-sdk/google—those bind you to per-provider keys and separate client lifecycles. The gateway already translates the OpenAI shape to upstream APIs, so the shim is the only network adapter you need. Keep your lockfile tidy; stray provider packages invite accidental direct calls.

Step 3: Configure the provider to target the gateway

Create a configured OpenAI client that points at the gateway. The createOpenAI factory accepts baseURL and apiKey. From that point, any model string you pass is forwarded verbatim to the gateway. Note the base URL should be the prefix (no /chat/completions); the SDK appends paths.

// lib/gateway.ts
import { createOpenAI } from '@ai-sdk/openai';
import { env } from 'node:process';

export const gateway = createOpenAI({
  baseURL: env.GATEWAY_BASE_URL, // e.g. https://api.n4n.ai/v1
  apiKey: env.GATEWAY_API_KEY,
  // compatibility: 'strict' if your gateway mirrors OpenAI exactly
});

This object is compatible with all Vercel AI SDK functions: generateText, streamText, generateObject. You have not written a single provider-specific import. The Vercel AI SDK switch providers without new keys pattern lives or dies on this single configured instance.

Step 4: Write a model-agnostic generation function

Wrap the SDK call so the model identifier is a parameter. The gateway expects provider-prefixed model IDs (e.g., anthropic/claude-3.5-sonnet, openai/gpt-4o). Your code stays blind to the backend.

// lib/chat.ts
import { generateText } from 'ai';
import { gateway } from './gateway';

export async function ask(model: string, prompt: string) {
  const { text, usage, finishReason } = await generateText({
    model: gateway(model),
    prompt,
    maxTokens: 512,
  });
  // usage.totalTokens is metered by the gateway per token
  return { text, usage, finishReason };
}

Call it with any supported model. Swapping openai/gpt-4o for meta/llama-3.1-70b requires no code changes beyond the argument. The SDK serializes the request to the OpenAI format; the gateway maps it to the target provider and returns a normalized completion.

Step 5: Switch providers by changing the model string

In a real app, the model often comes from request metadata, feature flags, or cost rules. Build a small resolver that maps your internal tiers to gateway model IDs. This keeps provider names out of business logic.

// lib/model-map.ts
export const MODEL_MAP = {
  cheap: 'meta/llama-3.1-8b-instruct',
  balanced: 'openai/gpt-4o-mini',
  premium: 'anthropic/claude-3.5-sonnet',
} as const;

export type Tier = keyof typeof MODEL_MAP;

export function resolveModel(tier: Tier, override?: string): string {
  if (override && override in MODEL_MAP) return MODEL_MAP[override as Tier];
  return MODEL_MAP[tier];
}

Then route:

import { ask } from './chat';
import { resolveModel, type Tier } from './model-map';

export async function answer(tier: Tier, prompt: string, override?: string) {
  const model = resolveModel(tier, override);
  return ask(model, prompt);
}

You can A/B test providers by flipping a flag or passing an override from your API layer. No new keys, no new client instances. The Vercel AI SDK switch providers without new keys method turns a provider migration into a string change.

Step 6: Add fallback and forward cache hints

Gateways degrade. Use the SDK’s error boundary to retry on a secondary model when the primary is rate-limited. Because the gateway speaks OpenAI, the error shape is uniform across upstreams.

import { generateText } from 'ai';
import { gateway } from './gateway';

async function robustAsk(primary: string, fallback: string, prompt: string) {
  try {
    return await generateText({ model: gateway(primary), prompt });
  } catch (e) {
    const msg = e instanceof Error ? e.message : '';
    if (msg.includes('rate') || msg.includes('529')) {
      return await generateText({ model: gateway(fallback), prompt });
    }
    throw e;
  }
}

Some gateways honor client routing directives and forward provider cache-control hints. When using n4n.ai, you can pass headers to influence cache and routing without leaving the OpenAI schema:

await generateText({
  model: gateway('anthropic/claude-3.5-sonnet'),
  prompt,
  headers: { 'x-cache-ttl': '3600' },
});

That single header propagates to the upstream provider’s cache system. The Vercel AI SDK switch providers without new keys approach keeps these cross-provider concerns in one place instead of scattered across vendor SDKs.

Step 7: Verify the integration end to end

Write a throwaway script and run it with tsx or node --loader. Confirm you can hit two different providers with the same key and inspect token usage.

// verify.ts
import { answer } from './lib/chat'; // adjust path to your answer fn

async function main() {
  const r1 = await answer('balanced', 'Say hi in one word.');
  console.log('balanced:', r1.text, r1.usage);
  const r2 = await answer('premium', 'Say bye in one word.');
  console.log('premium:', r2.text, r2.usage);
}

main().catch(console.error);

Expected output shows two distinct model responses and token usage objects. If both succeed, your gateway credential works and the SDK routes correctly.

For a lower-level check, curl the gateway directly to confirm the OpenAI-compatible contract:

curl $GATEWAY_BASE_URL/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}'

A JSON completion confirms the wire format. Once this passes, the Vercel AI SDK switch providers without new keys setup is production-ready. You can also verify streaming with streamText and a simple for await loop to ensure tokens arrive incrementally.

Caveats worth knowing

Not every provider feature survives translation. Fine-grained JSON schemas and some tool-calling dialects may differ; test the specific capability before relying on it. Streaming works, but inspect streamText consumption to avoid orphaned connections and always drain the stream. Token metering is per-token at the gateway, so log usage to track spend and set budget alerts upstream.

The pattern trades direct provider optimizations for operational simplicity. For most apps, that trade is worth it. You lose vendor-specific extensions but gain the ability to shift models hourly based on price or latency without touching infrastructure. Keep your model map in one module, and treat provider names as implementation details rather than first-class dependencies.