Switching models mid-conversation with Vercel AI SDK is a pattern that comes up when you need different capabilities at different stages — maybe a cheap model for classification, a reasoning model for complex analysis, and a fast model for final formatting. The SDK’s streamText and generateText functions make this straightforward once you understand how to thread conversation history through model changes. This guide walks through a complete implementation with streaming, tool support, and proper state handling.
Step 1: Set up the project dependencies
Start with a Next.js App Router project. The Vercel AI SDK works with any framework, but the patterns here assume Next.js 14+ with the App Router.
npm create next-app@latest model-switcher --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"
cd model-switcher
npm install ai @ai-sdk/openai @ai-sdk/anthropic zodThe ai package contains the core primitives. Provider packages like @ai-sdk/openai and @ai-sdk/anthropic give you model constructors. Zod handles schema validation for tool parameters.
Step 2: Define the conversation state type
You need a single source of truth for the conversation that survives model switches. Create a shared type that captures everything the UI and API need.
// lib/types.ts
import { Message, ToolInvocation } from 'ai'
export interface ChatMessage extends Message {
toolInvocations?: ToolInvocation[]
}
export interface ModelConfig {
id: string
name: string
provider: 'openai' | 'anthropic'
supportsTools: boolean
supportsReasoning?: boolean
}
export const AVAILABLE_MODELS: ModelConfig[] = [
{ id: 'gpt-4o-mini', name: 'GPT-4o Mini', provider: 'openai', supportsTools: true },
{ id: 'gpt-4o', name: 'GPT-4o', provider: 'openai', supportsTools: true, supportsReasoning: true },
{ id: 'claude-3-5-sonnet-20241022', name: 'Claude 3.5 Sonnet', provider: 'anthropic', supportsTools: true, supportsReasoning: true },
{ id: 'claude-3-haiku-20240307', name: 'Claude 3 Haiku', provider: 'anthropic', supportsTools: true },
]The ChatMessage type extends the SDK’s Message with optional tool invocations so the UI can render tool calls and results. ModelConfig lets you declaratively describe what each model supports — this drives the model selector in the UI.
Step 3: Create the model factory
Centralize model instantiation so the rest of your code doesn’t scatter provider imports everywhere.
// lib/models.ts
import { openai } from '@ai-sdk/openai'
import { anthropic } from '@ai-sdk/anthropic'
import { LanguageModelV1 } from 'ai'
import { ModelConfig, AVAILABLE_MODELS } from './types'
export function getModel(config: ModelConfig): LanguageModelV1 {
switch (config.provider) {
case 'openai':
return openai(config.id)
case 'anthropic':
return anthropic(config.id)
default:
throw new Error(`Unknown provider: ${config.provider}`)
}
}
export function getModelById(id: string): LanguageModelV1 {
const config = AVAILABLE_MODELS.find(m => m.id === id)
if (!config) throw new Error(`Model not found: ${id}`)
return getModel(config)
}This factory pattern keeps your route handlers clean. When you add a new provider later, you only touch this file.
Step 4: Build the streaming route handler
The route handler receives the conversation history and the selected model ID, then streams the response. This is where the vercel ai sdk switch models mid-conversation logic lives — you simply pass a different model to streamText based on the request.
// app/api/chat/route.ts
import { streamText, convertToCoreMessages, tool } from 'ai'
import { getModelById } from '@/lib/models'
import { z } from 'zod'
export const maxDuration = 30
export async function POST(req: Request) {
const { messages, modelId, systemPrompt } = await req.json()
if (!modelId) {
return new Response('modelId is required', { status: 400 })
}
const model = getModelById(modelId)
const result = streamText({
model,
system: systemPrompt ?? 'You are a helpful assistant.',
messages: convertToCoreMessages(messages),
tools: {
getWeather: tool({
parameters: z.object({
location: z.string().describe('City name'),
unit: z.enum(['celsius', 'fahrenheit']).default('fahrenheit'),
}),
execute: async ({ location, unit }) => {
// Mock implementation — replace with real API
const temp = unit === 'celsius' ? 22 : 72
return { location, temperature: temp, unit, condition: 'sunny' }
},
}),
},
maxSteps: 5,
})
return result.toDataStreamResponse()
}Key points: convertToCoreMessages normalizes the message format across providers. The tools object is optional — if the selected model doesn’t support tools, the SDK ignores it gracefully. maxSteps: 5 allows multi-step tool use (model calls tool, gets result, calls another tool, etc.).
Step 5: Build the chat UI with model selector
The client needs to maintain conversation state, render streaming tokens, and let the user pick a model for the next turn. Use the useChat hook from ai/react.
// components/ChatInterface.tsx
'use client'
import { useChat } from 'ai/react'
import { useState } from 'react'
import { AVAILABLE_MODELS, ModelConfig } from '@/lib/types'
import { ChatMessage } from '@/lib/types'
export function ChatInterface() {
const [selectedModelId, setSelectedModelId] = useState<string>(AVAILABLE_MODELS[0].id)
const [systemPrompt, setSystemPrompt] = useState<string>('')
const { messages, input, handleInputChange, handleSubmit, isLoading, stop, error } = useChat({
api: '/api/chat',
body: { modelId: selectedModelId, systemPrompt },
onError: (err) => console.error('Chat error:', err),
})
const currentModel = AVAILABLE_MODELS.find(m => m.id === selectedModelId)
return (
<div className="flex flex-col h-[calc(100vh-4rem)] p-4 gap-4">
<div className="flex gap-2 items-center">
<select
value={selectedModelId}
onChange={e => setSelectedModelId(e.target.value)}
disabled={isLoading}
className="px-3 py-2 border rounded-md text-sm"
>
{AVAILABLE_MODELS.map(model => (
<option key={model.id} value={model.id}>
{model.name} {model.supportsReasoning && '(reasoning)'}
</option>
))}
</select>
<textarea
value={systemPrompt}
onChange={e => setSystemPrompt(e.target.value)}
placeholder="System prompt (optional)"
className="flex-1 px-3 py-2 border rounded-md text-sm resize-none h-16"
/>
</div>
<div className="flex-1 overflow-y-auto space-y-4">
{messages.map((message: ChatMessage, idx) => (
<MessageBubble key={idx} message={message} />
))}
{isLoading && <StreamingIndicator />}
</div>
<form onSubmit={handleSubmit} className="flex gap-2">
<input
value={input}
onChange={handleInputChange}
placeholder={isLoading ? 'Generating...' : 'Type a message...'}
disabled={isLoading}
className="flex-1 px-4 py-2 border rounded-lg"
/>
{isLoading ? (
<button type="button" onClick={stop} className="px-4 py-2 bg-red-600 text-white rounded-lg">
Stop
</button>
) : (
<button type="submit" className="px-4 py-2 bg-blue-600 text-white rounded-lg">
Send
</button>
)}
</form>
{error && <div className="text-red-600 text-sm">Error: {error.message}</div>}
</div>
)
}
function MessageBubble({ message }: { message: ChatMessage }) {
const isUser = message.role === 'user'
return (
<div className={`flex ${isUser ? 'justify-end' : 'justify-start'}`}>
<div className={`max-w-[70%] p-3 rounded-2xl ${isUser ? 'bg-blue-600 text-white rounded-br-none' : 'bg-gray-100 rounded-bl-none'}`}>
<div className="text-sm font-medium mb-1">{message.role}</div>
<div className="whitespace-pre-wrap">{message.content}</div>
{message.toolInvocations?.map((tool, i) => (
<ToolInvocationDisplay key={i} tool={tool} />
))}
</div>
</div>
)
}
function ToolInvocationDisplay({ tool }: { tool: any }) {
return (
<details className="mt-2 text-xs border rounded p-2 bg-white/50">
<summary className="cursor-pointer font-mono">
🔧 {tool.toolName}({JSON.stringify(tool.args)})
</summary>
<pre className="mt-1 whitespace-pre-wrap">{JSON.stringify(tool.result, null, 2)}</pre>
</details>
)
}
function StreamingIndicator() {
return (
<div className="flex items-center gap-2 text-gray-500 text-sm">
<span className="animate-pulse">●</span>
<span>Generating...</span>
</div>
)
}The useChat hook handles the streaming protocol automatically. Passing body: { modelId, systemPrompt } sends those values with every request — this is how the server knows which model to use for the current turn. The selectedModelId state changes only affect the next message, which is exactly what you want for mid-conversation switching.
Step 6: Wire it into a page
// app/page.tsx
import { ChatInterface } from '@/components/ChatInterface'
export default function Home() {
return (
<main className="min-h-screen bg-gray-50">
<header className="bg-white border-b px-4 py-3">
<h1 className="text-xl font-semibold">Model Switcher Demo</h1>
<p className="text-sm text-gray-500">Switch models per message — history preserved</p>
</header>
<ChatInterface />
</main>
)
}Run npm run dev and open http://localhost:3000. Send a few messages with one model, switch the dropdown, send another — the conversation history flows through unchanged.
Step 7: Handle model capability mismatches
Not all models support tools or reasoning. If a user switches from a tool-capable model to one that isn’t, the next turn will fail silently on tool calls. Handle this in the route handler by validating capabilities.
// app/api/chat/route.ts (updated)
import { streamText, convertToCoreMessages, tool, UnsupportedFunctionalityError } from 'ai'
import { getModelById, AVAILABLE_MODELS } from '@/lib/models'
import { z } from 'zod'
const TOOLS = {
getWeather: tool({
parameters: z.object({
location: z.string().describe('City name'),
unit: z.enum(['celsius', 'fahrenheit']).default('fahrenheit'),
}),
execute: async ({ location, unit }) => {
const temp = unit === 'celsius' ? 22 : 72
return { location, temperature: temp, unit, condition: 'sunny' }
},
}),
}
export async function POST(req: Request) {
const { messages, modelId, systemPrompt } = await req.json()
if (!modelId) {
return new Response('modelId is required', { status: 400 })
}
const modelConfig = AVAILABLE_MODELS.find(m => m.id === modelId)
if (!modelConfig) {
return new Response(`Unknown model: ${modelId}`, { status: 400 })
}
const model = getModelById(modelId)
try {
const result = streamText({
model,
system: systemPrompt ?? 'You are a helpful assistant.',
messages: convertToCoreMessages(messages),
tools: modelConfig.supportsTools ? TOOLS : undefined,
maxSteps: modelConfig.supportsTools ? 5 : 1,
})
return result.toDataStreamResponse()
} catch (err) {
if (err instanceof UnsupportedFunctionalityError) {
return new Response(
`Model ${modelId} does not support required functionality: ${err.message}`,
{ status: 400 }
)
}
throw err
}
}Now the route checks supportsTools before passing tools to streamText. If a model lacks tool support, maxSteps stays at 1 (single turn, no tool loop). The UnsupportedFunctionalityError catch handles edge cases where the provider throws instead of ignoring unsupported features.
Step 8: Preserve reasoning output when switching
Some models (like OpenAI’s o1 series or Claude with thinking) emit reasoning tokens separately from the final answer. The SDK surfaces these via reasoning parts in the stream. Capture and store them so they survive model switches.
// app/api/chat/route.ts (add to streamText options)
import { streamText, convertToCoreMessages, tool, UnsupportedFunctionalityError, Message } from 'ai'
// ... inside POST handler
const result = streamText({
model,
system: systemPrompt ?? 'You are a helpful assistant.',
messages: convertToCoreMessages(messages),
tools: modelConfig.supportsTools ? TOOLS : undefined,
maxSteps: modelConfig.supportsTools ? 5 : 1,
onFinish: async ({ response }) => {
// response.messages contains the full assistant message with reasoning parts
// Persist to your database here if needed
console.log('Finished message:', JSON.stringify(response.messages, null, 2))
},
})On the client, the useChat hook already includes reasoning parts in message.parts (type 'reasoning'). Render them differently if you want visibility:
// components/ChatInterface.tsx (add to MessageBubble)
{message.parts?.map((part, i) => (
part.type === 'reasoning' && (
<details key={i} className="mt-1 text-xs text-gray-500 border-t pt-1">
<summary className="cursor-pointer">Reasoning</summary>
<pre className="whitespace-pre-wrap mt-1">{part.text}</pre>
</details>
)
))}This preserves the full chain of thought across model boundaries — useful for debugging or audit trails.
Step 9: Add server-side conversation persistence (optional)
For production, you’ll want to persist conversations. The SDK’s onFinish callback gives you the complete message array after each turn. Here’s a minimal example using a JSON file (replace with a real database):
// lib/storage.ts
import { promises as fs } from 'fs'
import { join } from 'path'
import { ChatMessage } from './types'
const DATA_FILE = join(process.cwd(), 'data', 'conversations.json')
export async function loadConversation(id: string): Promise<ChatMessage[]> {
try {
const data = await fs.readFile(DATA_FILE, 'utf-8')
const all = JSON.parse(data)
return all[id] ?? []
} catch {
return []
}
}
export async function saveConversation(id: string, messages: ChatMessage[]) {
let all: Record<string, ChatMessage[]> = {}
try {
const data = await fs.readFile(DATA_FILE, 'utf-8')
all = JSON.parse(data)
} catch {}
all[id] = messages
await fs.writeFile(DATA_FILE, JSON.stringify(all, null, 2))
}Then update the route handler to accept a conversation ID and persist on finish:
// app/api/chat/route.ts (updated imports and handler)
import { loadConversation, saveConversation } from '@/lib/storage'
export async function POST(req: Request) {
const { messages, modelId, systemPrompt, conversationId } = await req.json()
// ... validation ...
// Merge incoming messages with persisted history (incoming takes precedence for the current turn)
const persisted = conversationId ? await loadConversation(conversationId) : []
const mergedMessages = [...persisted, ...messages.slice(persisted.length)]
const result = streamText({
// ... options ...
onFinish: async ({ response }) => {
if (conversationId) {
await saveConversation(conversationId, response.messages)
}
},
})
return result.toDataStreamResponse()
}The client sends conversationId in the body option of useChat. This pattern scales to any backend — Postgres, Redis, n4n.ai’s usage metering, etc.
Step 10: Verify the implementation works
Test these scenarios manually or with Playwright:
-
Basic switch: Send 3 messages with GPT-4o Mini, switch to Claude 3.5 Sonnet, send a 4th. Verify the 4th response uses Claude’s tone and the history is intact.
-
Tool call across switch: With a tool-capable model, ask “What’s the weather in Tokyo?” — verify tool invocation. Switch to a non-tool model, ask “What was the temperature?” — verify it answers from history without calling tools.
-
Reasoning preservation: Use a reasoning model (o1, Claude with thinking), ask a complex question. Switch models. Verify the reasoning summary from the first model appears in the UI.
-
Streaming integrity: Send a long prompt, switch models mid-stream (by stopping and resending with a different model). Verify no duplicate or missing tokens.
-
Error handling: Select an invalid model ID (manually via dev tools). Verify a 400 response with a clear message.
Run the dev server and exercise each case. The console will show Finished message logs with full message structures — inspect those to confirm reasoning parts and tool results are preserved.
Common pitfalls
Passing the wrong message format to convertToCoreMessages: The hook sends messages in the correct shape, but if you manually construct messages elsewhere, ensure each has role, content, and optional toolInvocations. Missing toolInvocations on assistant messages that called tools breaks multi-step tool loops.
Forgetting maxSteps: Without it, the model can call a tool but won’t get the result back in the same turn. Set maxSteps: 5 (or higher) for tool-capable models.
Not handling UnsupportedFunctionalityError: Some providers throw instead of ignoring unsupported features. Catch it and return a 400 with a helpful message.
Losing system prompt on switch: The systemPrompt in the useChat body goes with every request. If you change it mid-conversation, the new prompt applies only to subsequent turns — previous turns keep the old prompt. This is usually the desired behavior.
When to use this pattern
Switch models mid-conversation when:
- You need cost optimization (cheap model for simple turns, expensive for complex ones)
- Different models excel at different tasks (classification vs. reasoning vs. creative writing)
- You want to let users choose per-message without losing context
Avoid it when:
- Consistency of voice matters (e.g., a single persona throughout)
- You’re building a simple chatbot where one model suffices
- Latency budget is tight — each model switch adds a round trip to a potentially different provider
The Vercel AI SDK makes this pattern viable because streamText is stateless — it only sees the messages you pass it. Your application owns the conversation history, so you decide when and how to switch. That control is the key insight: the SDK doesn’t manage conversation state, you do.