n4nAI
Open console →
Framework tutorialsHow-toVercel AI SDK Multi-Model Switching

Dynamic model selection at request time in Vercel AI SDK

Learn to implement vercel ai sdk dynamic model selection at request time with runtime routing, provider fallbacks, and per-request model overrides.

n4n Team2 min read527 words

Audio narration

Coming soon — every post will get a voice note here.

Dynamic model selection at request time lets you route each inference call to the best model for the job — cheaper models for simple tasks, stronger models for reasoning, and automatic fallback when a provider degrades. The Vercel AI SDK supports this pattern natively through its provider abstraction and the generateText/streamText functions. This guide walks through a production-ready implementation with runtime model resolution, client-directed routing, and observable fallbacks.

Step 1: Set up the provider registry

Create a centralized registry that maps model identifiers to instantiated providers. This keeps model configuration out of your route handlers and makes it trivial to add or swap providers.

// lib/ai/providers.ts
import { openai } from '@ai-sdk/openai';
import { anthropic } from '@ai-sdk/anthropic';
import { google } from '@ai-sdk/google';
import { createOpenAI } from '@ai-sdk/openai';

export type ModelId = 
  | 'gpt-4o'
  | 'gpt-4o-mini'
  | 'claude-3-5-sonnet'
  | 'claude-3-haiku'
  | 'gemini-1.5-pro'
  | 'gemini-1.5-flash';

interface ProviderEntry {
  provider: ReturnType<typeof openai> | ReturnType<typeof anthropic> | ReturnType<typeof google>;
  modelId: string;
  maxTokens: number;
  supportsTools: boolean;
  supportsVision: boolean;
}

const openaiProvider = openai;
const anthropicProvider = anthropic;
const googleProvider = google;

export const modelRegistry: Record<ModelId, ProviderEntry> = {
  'gpt-4o': {
    provider: openaiProvider,
    modelId: 'gpt-4o',
    maxTokens: 4096,
    supportsTools: true,
    supportsVision: true,
  },
  'gpt-4o-mini': {
    provider: openaiProvider,
    modelId: 'gpt-4o-mini',
    maxTokens: 16384,
    supportsTools: true,
    supportsVision: true,
  },
  'claude-3-5-sonnet': {
    provider: anthropicProvider,
    modelId: 'claude-3-5-sonnet-20241022',
    maxTokens: 8192,
    supportsTools: true,
    supportsVision: true,
  },
  'claude-3-haiku': {
    provider: anthropicProvider,
    modelId: 'claude-3-haiku-20240307',
    maxTokens: 4096,
    supportsTools: true,
    supportsVision: true,
  },
  'gemini-1.5-pro': {
    provider: googleProvider,
    modelId: 'gemini-1.5-pro',
    maxTokens: 8192,
    supportsTools: true,
    supportsVision: true,
  },
  'gemini-1.5-flash': {
    provider: googleProvider,
    modelId: 'gemini-1.5-flash',
    maxTokens: 8192,
    supportsTools: true,
    supportsVision: true,
  },
};

export function getModelEntry(modelId: ModelId): ProviderEntry {
  const entry = modelRegistry[modelId];
  if (!entry) {
    throw new Error(`Unknown model: ${modelId}`);
  }
  return entry;
}

export function listAvailableModels(): ModelId[] {
  return Object.keys(modelRegistry) as ModelId[];
}

Step 2: Build a model resolver with fallback logic

The resolver encapsulates selection logic: default model, client override, capability filtering, and ordered fallback chains. This is where vercel ai sdk dynamic model selection becomes operational rather than theoretical.

// lib/ai/model-resolver.ts
import { getModelEntry, ModelId, ProviderEntry } from './providers';

export interface ResolutionContext {
  preferredModel?: ModelId;
  requireTools?: boolean;
  requireVision?: boolean;
  maxTokens?: number;
  fallbackChain?: ModelId[];
}

export interface ResolvedModel {
  entry: ProviderEntry;
  modelId: ModelId;
  fallbackChain: ModelId[];
  wasFallback: boolean;
}

const DEFAULT_MODEL: ModelId = 'gpt-4o-mini';
const DEFAULT_FALLBACK_CHAIN: ModelId[] = [
  'gpt-4o-mini',
  'claude-3-haiku',
  'gemini-1.5-flash',
  'gpt-4o',
  'claude-3-5-sonnet',
  'gemini-1.5-pro',
];

export function resolveModel(context: ResolutionContext = {}): ResolvedModel {
  const {
    preferredModel,
    requireTools = false,
    requireVision = false,
    maxTokens,
    fallbackChain = DEFAULT_FALLBACK_CHAIN,
  } = context;

  const candidates = preferredModel 
    ? [preferredModel, ...fallbackChain.filter(m => m !== preferredModel)]
    : fallbackChain;

  const filtered = candidates.filter(modelId => {
    const entry = getModelEntry(modelId);
    if (requireTools && !entry.supportsTools) return false;
    if (requireVision && !entry.supportsVision) return false;
    if (maxTokens && entry.maxTokens < maxTokens) return false;
    return true;
  });

  if (filtered.length === 0) {
    throw new Error('No model matches the required capabilities');
  }

  const primaryModelId = filtered[0];
  const entry = getModelEntry(primaryModelId);
  const wasFallback = preferredModel !== undefined && primaryModelId !== preferredModel;

  return {
    entry,
    modelId: primaryModelId,
    fallbackChain: filtered.slice(1),
    wasFallback,
  };
}

Step 3: Create a typed wrapper for generateText

Wrap generateText to inject the resolved model, attach metadata, and handle provider errors with automatic fallback. This wrapper is the core of request-time model switching.

// lib/ai/generate.ts
import { generateText, GenerateTextResult, CoreMessage } from 'ai';
import { resolveModel, ResolutionContext, ResolvedModel } from './model-resolver';
import { ModelId } from './providers';

export interface GenerateOptions {
  messages: CoreMessage[];
  context?: ResolutionContext;
  temperature?: number;
  maxTokens?: number;
  tools?: Record<string, any>;
  onStepFinish?: (step: { modelId: ModelId; wasFallback: boolean }) => void;
}

export async function generateWithModelSelection(
  options: GenerateOptions
): Promise<GenerateTextResult<Record<string, never>> & { resolvedModel: ResolvedModel }> {
  const { messages, context = {}, temperature = 0.7, maxTokens, tools, onStepFinish } = options;

  let resolved = resolveModel({
    ...context,
    maxTokens,
  });

  let lastError: Error | null = null;

  for (const attemptModelId of [resolved.modelId, ...resolved.fallbackChain]) {
    try {
      const entry = resolved.entry;
      const result = await generateText({
        model: entry.provider(entry.modelId),
        messages,
        temperature,
        maxTokens,
        tools,
      });

      if (onStepFinish) {
        onStepFinish({ modelId: resolved.modelId, wasFallback: resolved.wasFallback });
      }

      return {
        ...result,
        resolvedModel: resolved,
      };
    } catch (error) {
      lastError = error as Error;
      
      const isRateLimit = error instanceof Error && 
        (error.message.includes('rate limit') || 
         error.message.includes('429') ||
         error.message.includes('quota'));
      
      const isProviderError = error instanceof Error &&
        (error.message.includes('500') ||
         error.message.includes('502') ||
         error.message.includes('503') ||
         error.message.includes('overloaded'));

      if (!isRateLimit && !isProviderError) {
        throw error;
      }

      const nextIndex = resolved.fallbackChain.indexOf(attemptModelId) + 1;
      if (nextIndex < resolved.fallbackChain.length) {
        const nextModelId = resolved.fallbackChain[nextIndex];
        resolved = resolveModel({
          ...context,
          preferredModel: nextModelId,
          fallbackChain: resolved.fallbackChain.slice(nextIndex),
        });
        continue;
      }
      
      throw error;
    }
  }

  throw lastError || new Error('All fallback models exhausted');
}

Step 4: Implement a streaming variant with fallback

Streaming requires special handling — you can’t transparently switch models mid-stream. The pattern is to attempt the primary model, and on connection-level failure, restart the stream with the next fallback.

// lib/ai/stream.ts
import { streamText, StreamTextResult, CoreMessage } from 'ai';
import { resolveModel, ResolutionContext, ResolvedModel } from './model-resolver';
import { ModelId } from './providers';

export interface StreamOptions {
  messages: CoreMessage[];
  context?: ResolutionContext;
  temperature?: number;
  maxTokens?: number;
  tools?: Record<string, any>;
  onModelChange?: (modelId: ModelId, isFallback: boolean) => void;
}

export async function streamWithModelSelection(
  options: StreamOptions
): Promise<StreamTextResult<Record<string, never>> & { resolvedModel: ResolvedModel }> {
  const { messages, context = {}, temperature = 0.7, maxTokens, tools, onModelChange } = options;

  let resolved = resolveModel({
    ...context,
    maxTokens,
  });

  let lastError: Error | null = null;

  for (const attemptModelId of [resolved.modelId, ...resolved.fallbackChain]) {
    try {
      const entry = resolved.entry;
      const result = streamText({
        model: entry.provider(entry.modelId),
        messages,
        temperature,
        maxTokens,
        tools,
      });

      if (onModelChange) {
        onModelChange(resolved.modelId, resolved.wasFallback);
      }

      return {
        ...result,
        resolvedModel: resolved,
      };
    } catch (error) {
      lastError = error as Error;
      
      const isConnectionError = error instanceof Error &&
        (error.message.includes('ECONNRESET') ||
         error.message.includes('ETIMEDOUT') ||
         error.message.includes('network') ||
         error.message.includes('502') ||
         error.message.includes('503') ||
         error.message.includes('504'));

      if (!isConnectionError) {
        throw error;
      }

      const nextIndex = resolved.fallbackChain.indexOf(attemptModelId) + 1;
      if (nextIndex < resolved.fallbackChain.length) {
        const nextModelId = resolved.fallbackChain[nextIndex];
        resolved = resolveModel({
          ...context,
          preferredModel: nextModelId,
          fallbackChain: resolved.fallbackChain.slice(nextIndex),
        });
        continue;
      }
      
      throw error;
    }
  }

  throw lastError || new Error('All fallback models exhausted');
}

Step 5: Expose a Next.js App Router endpoint

Wire the wrapper into a route handler that accepts client routing directives. The client can specify a preferred model, required capabilities, and token budget — the server resolves the rest.

// app/api/ai/generate/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { generateWithModelSelection, GenerateOptions } from '@/lib/ai/generate';
import { ModelId } from '@/lib/ai/providers';

interface GenerateRequest {
  messages: GenerateOptions['messages'];
  model?: ModelId;
  requireTools?: boolean;
  requireVision?: boolean;
  maxTokens?: number;
  temperature?: number;
  tools?: GenerateOptions['tools'];
}

export async function POST(request: NextRequest) {
  try {
    const body: GenerateRequest = await request.json();
    
    const { messages, model, requireTools, requireVision, maxTokens, temperature, tools } = body;

    if (!messages || !Array.isArray(messages) || messages.length === 0) {
      return NextResponse.json(
        { error: 'messages array is required' },
        { status: 400 }
      );
    }

    const result = await generateWithModelSelection({
      messages,
      context: {
        preferredModel: model,
        requireTools,
        requireVision,
        maxTokens,
      },
      temperature,
      maxTokens,
      tools,
      onStepFinish: ({ modelId, wasFallback }) => {
        console.log(`[ai] Completed with model: ${modelId}${wasFallback ? ' (fallback)' : ''}`);
      },
    });

    return NextResponse.json({
      text: result.text,
      usage: result.usage,
      finishReason: result.finishReason,
      model: result.resolvedModel.modelId,
      wasFallback: result.resolvedModel.wasFallback,
      fallbackChain: result.resolvedModel.fallbackChain,
    });
  } catch (error) {
    console.error('[ai] Generation error:', error);
    return NextResponse.json(
      { error: error instanceof Error ? error.message : 'Generation failed' },
      { status: 500 }
    );
  }
}
// app/api/ai/stream/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { streamWithModelSelection, StreamOptions } from '@/lib/ai/stream';
import { ModelId } from '@/lib/ai/providers';

interface StreamRequest {
  messages: StreamOptions['messages'];
  model?: ModelId;
  requireTools?: boolean;
  requireVision?: boolean;
  maxTokens?: number;
  temperature?: number;
  tools?: StreamOptions['tools'];
}

export async function POST(request: NextRequest) {
  try {
    const body: StreamRequest = await request.json();
    
    const { messages, model, requireTools, requireVision, maxTokens, temperature, tools } = body;

    if (!messages || !Array.isArray(messages) || messages.length === 0) {
      return NextResponse.json(
        { error: 'messages array is required' },
        { status: 400 }
      );
    }

    const result = await streamWithModelSelection({
      messages,
      context: {
        preferredModel: model,
        requireTools,
        requireVision,
        maxTokens,
      },
      temperature,
      maxTokens,
      tools,
      onModelChange: (modelId, isFallback) => {
        console.log(`[ai] Streaming with model: ${modelId}${isFallback ? ' (fallback)' : ''}`);
      },
    });

    return result.toDataStreamResponse({
      headers: {
        'x-model-used': result.resolvedModel.modelId,
        'x-was-fallback': String(result.resolvedModel.wasFallback),
        'x-fallback-chain': result.resolvedModel.fallbackChain.join(','),
      },
    });
  } catch (error) {
    console.error('[ai] Stream error:', error);
    return NextResponse.json(
      { error: error instanceof Error ? error.message : 'Stream failed' },
      { status: 500 }
    );
  }
}

Step 6: Add client-side hooks for model directives

The client should be able to express intent — “use a cheap model,” “I need tools,” “this needs vision” — without hardcoding model names. A small hook encapsulates this.

// hooks/use-ai-generation.ts
'use client';

import { useCallback } from 'react';
import { ModelId } from '@/lib/ai/providers';

interface UseAIGenerationOptions {
  onModelChange?: (modelId: ModelId, wasFallback: boolean) => void;
}

export function useAIGeneration(options: UseAIGenerationOptions = {}) {
  const { onModelChange } = options;

  const generate = useCallback(async (params: {
    messages: { role: 'user' | 'assistant' | 'system'; content: string }[];
    model?: ModelId;
    requireTools?: boolean;
    requireVision?: boolean;
    maxTokens?: number;
    temperature?: number;
  }) => {
    const response = await fetch('/api/ai/generate', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(params),
    });

    if (!response.ok) {
      const error = await response.json();
      throw new Error(error.error || 'Generation failed');
    }

    const data = await response.json();
    
    if (onModelChange) {
      onModelChange(data.model, data.wasFallback);
    }

    return data;
  }, [onModelChange]);

  const stream = useCallback(async (params: {
    messages: { role: 'user' | 'assistant' | 'system'; content: string }[];
    model?: ModelId;
    requireTools?: boolean;
    requireVision?: boolean;
    maxTokens?: number;
    temperature?: number;
    onChunk?: (chunk: string) => void;
  }) => {
    const response = await fetch('/api/ai/stream', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(params),
    });

    if (!response.ok) {
      const error = await response.json();
      throw new Error(error.error || 'Stream failed');
    }

    const modelUsed = response.headers.get('x-model-used') as ModelId | null;
    const wasFallback = response.headers.get('x-was-fallback') === 'true';
    
    if (onModelChange && modelUsed) {
      onModelChange(modelUsed, wasFallback);
    }

    const reader = response.body?.getReader();
    const decoder = new TextDecoder();
    let fullText = '';

    if (reader && params.onChunk) {
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        const chunk = decoder.decode(value, { stream: true });
        fullText += chunk;
        params.onChunk(chunk);
      }
    }

    return { text: fullText, model: modelUsed, wasFallback };
  }, [onModelChange]);

  return { generate, stream };
}

Step 7: Verify the implementation end to end

Run the dev server and test each path. This checklist confirms the wiring works.

# 1. Start the dev server
npm run dev

# 2. Test non-streaming with explicit model
curl -X POST http://localhost:3000/api/ai/generate \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Say hello in one word"}],
    "model": "gpt-4o-mini"
  }'

# Expected: JSON with text, model: "gpt-4o-mini", wasFallback: false

# 3. Test capability-based routing (requires tools)
curl -X POST http://localhost:3000/api/ai/generate \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "requireTools": true
  }'

# Expected: model supports tools (all current entries do), wasFallback: false

# 4. Test fallback by simulating rate limit (set invalid key temporarily)
# Edit .env.local: OPENAI_API_KEY=invalid
# Then request gpt-4o — should fall back to claude-3-haiku or gemini-1.5-flash
curl -X POST http://localhost:3000/api/ai/generate \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello"}],
    "model": "gpt-4o"
  }'

# Expected: wasFallback: true, model: "claude-3-haiku" (or next in chain)

# 5. Test streaming endpoint
curl -X POST http://localhost:3000/api/ai/stream \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Count to 5"}],
    "model": "gpt-4o-mini"
  }' \
  --no-buffer

# Expected: SSE stream with x-model-used header

# 6. Restore valid API key and verify normal operation

Check the server logs for [ai] Completed with model: and [ai] Streaming with model: lines — they confirm the resolver picked the expected model and whether a fallback occurred.

Step 8: Add observability for production

Production systems need visibility into which models serve which requests, latency per model, and fallback rates. Add a lightweight emitter.

// lib/ai/telemetry.ts
import { ModelId } from './providers';

export interface GenerationTelemetry {
  modelId: ModelId;
  wasFallback: boolean;
  latencyMs: number;
  tokensPrompt: number;
  tokensCompletion: number;
  finishReason: string;
  error?: string;
}

const listeners: Array<(event: GenerationTelemetry) => void> = [];

export function onGenerationComplete(listener: (event: GenerationTelemetry) => void) {
  listeners.push(listener);
}

export function emitGenerationTelemetry(event: GenerationTelemetry) {
  for (const listener of listeners) {
    try {
      listener(event);
    } catch {
      // swallow listener errors
    }
  }
}

// Example: ship to your observability backend
if (process.env.NODE_ENV === 'production') {
  onGenerationComplete(async (event) => {
    await fetch('https://your-metrics-endpoint/ingest', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(event),
      keepalive: true,
    }).catch(() => {});
  });
}

Wire it into the generate wrapper:

// lib/ai/generate.ts (add to imports)
import { emitGenerationTelemetry, GenerationTelemetry } from './telemetry';

// Inside generateWithModelSelection, after successful result:
const latencyMs = Date.now() - startTime;
emitGenerationTelemetry({
  modelId: resolved.modelId,
  wasFallback: resolved.wasFallback,
  latencyMs,
  tokensPrompt: result.usage?.promptTokens ?? 0,
  tokensCompletion: result.usage?.completionTokens ?? 0,
  finishReason: result.finishReason,
});

Step 9: Extend with routing rules for cost optimization

Beyond fallbacks, you can encode business logic: route simple classifications to the cheapest model, reserve reasoning models for complex prompts, and enforce budgets per tenant.

// lib/ai/routing-rules.ts
import { ModelId, getModelEntry } from './providers';
import { ResolutionContext } from './model-resolver';

export interface RoutingRule {
  name: string;
  matches: (messages: Array<{ role: string; content: string }>, context?: ResolutionContext) => boolean;
  resolve: (context?: ResolutionContext) => ResolutionContext;
}

export const routingRules: RoutingRule[] = [
  {
    name: 'simple-classification',
    matches: (messages) => {
      const lastUser = messages.filter(m => m.role === 'user').pop();
      if (!lastUser) return false;
      const content = lastUser.content.toLowerCase();
      return content.length < 200 && 
        (content.includes('classify') || 
         content.includes('categorize') || 
         content.includes('sentiment') ||
         content.includes('yes or no'));
    },
    resolve: () => ({
      preferredModel: 'gpt-4o-mini' as ModelId,
      maxTokens: 100,
    }),
  },
  {
    name: 'code-generation',
    matches: (messages) => {
      const lastUser = messages.filter(m => m.role === 'user').pop();
      if (!lastUser) return false;
      const content = lastUser.content.toLowerCase();
      return content.includes('write code') || 
        content.includes('implement') || 
        content.includes('function') ||
        content.includes('debug');
    },
    resolve: () => ({
      preferredModel: 'claude-3-5-sonnet' as ModelId,
      requireTools: true,
      maxTokens: 4096,
    }),
  },
  {
    name: 'vision-tasks',
    matches: (messages) => {
      return messages.some(m => 
        m.role === 'user' && 
        typeof m.content === 'object' && 
        m.content !== null &&
        'image' in m.content
      );
    },
    resolve: () => ({
      requireVision: true,
      preferredModel: 'gpt-4o' as ModelId,
    }),
  },
];

export function applyRoutingRules(
  messages: Array<{ role: string; content: string }>,
  context: ResolutionContext = {}
): ResolutionContext {
  for (const rule of routingRules) {
    if (rule.matches(messages, context)) {
      return { ...context, ...rule.resolve(context) };
    }
  }
  return context;
}

Update the route handler to apply rules before resolution:

// app/api/ai/generate/route.ts (modify the POST handler)
import { applyRoutingRules } from '@/lib/ai/routing-rules';

// Inside POST, before generateWithModelSelection:
const enhancedContext = applyRoutingRules(messages, {
  preferredModel: model,
  requireTools,
  requireVision,
  maxTokens,
});

const result = await generateWithModelSelection({
  messages,
  context: enhancedContext,
  // ...
});

Verification checklist

  • Non-streaming endpoint returns model and wasFallback fields
  • Streaming endpoint emits x-model-used and x-was-fallback headers
  • Invalid provider key triggers fallback to next model in chain
  • requireTools: true excludes non-tool models (if any added)
  • requireVision: true excludes non-vision models
  • Routing rules fire for classification, code, and image prompts
  • Telemetry events fire with correct model, latency, and token counts
  • Client hook receives model info and surfaces it to UI

This implementation gives you vercel ai sdk dynamic model selection that is observable, testable, and extensible. The resolver is pure and unit-testable, the fallbacks are explicit and logged, and the client expresses intent without coupling to model names.

Tagsvercel-ai-sdkdynamic-model-selectionmulti-modeln4n-ai

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Framework tutorialsComparing model outputs side by side with Vercel AI SDKLearn how to compare model outputs side by side using Vercel AI SDK with practical patterns for multi-model evaluation, streaming, and production routing.Framework tutorialsOne provider, 240+ models: multi-model AI SDK setupA practical guide to configuring Vercel AI SDK for multi-model routing across 240+ models using a single OpenAI-compatible endpoint, with code examples and production pitfalls.Framework tutorialsPer-request model overrides in the Vercel AI SDKLearn how to implement per-request model overrides in the Vercel AI SDK with step-by-step code examples for dynamic model switching at runtime.Framework tutorialsMulti-model switching in Vercel AI SDK using n4n.ai routingLearn how to implement vercel ai sdk multi-model switching n4n.ai routing with fallback, per-token metering, and client routing directives in a runnable tutorial.Framework tutorialsSwitching models mid-conversation with Vercel AI SDKLearn how to switch models mid-conversation using Vercel AI SDK with practical code examples for streaming, tool calling, and state management.Framework tutorialsHandling rate limit fallback across models in AI SDKA step-by-step guide to implementing automatic model fallback on rate limits using the Vercel AI SDK, with runnable code and verification strategies.

All vercel ai sdk multi-model switching posts →