The vercel ai sdk usechat openai-compatible pattern lets you keep a single React chat UI while swapping the backend model freely. This tutorial builds a minimal Next.js app that streams responses from any endpoint speaking the OpenAI chat protocol, using the official AI SDK and nothing else.
Prerequisites
- Node.js 18.18 or later (20+ recommended).
- An existing Next.js project using the App Router and TypeScript.
- An API key and base URL for an OpenAI-compatible inference endpoint.
- Basic comfort with React client components and server routes.
If you need a fresh project, scaffold one:
npx create-next-app@latest chat-app --typescript --app
cd chat-appYou should be able to run npm run dev and see the default Next.js page before continuing.
Install the AI SDK
Install the core packages and the OpenAI provider adapter. The adapter works against any compliant base URL, not just OpenAI.
npm install ai @ai-sdk/react @ai-sdk/openai@ai-sdk/openai exports createOpenAI (imported as openai), which accepts a baseURL and apiKey. That is the entire integration surface for an OpenAI-compatible backend.
Configure environment variables
Create .env.local at the project root. Never expose the server-side key to the browser bundle.
# .env.local
OPENAI_API_KEY=sk-your-key-here
OPENAI_BASE_URL=https://api.your-provider.com/v1If you want a single OpenAI-compatible endpoint that fronts 240+ models with automatic fallback when a provider is rate-limited, point OPENAI_BASE_URL at n4n.ai’s gateway and use its issued key. The rest of the code stays identical.
Build the server route
The route handles POST requests from useChat and streams back text. Create app/api/chat/route.ts.
// app/api/chat/route.ts
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
export const runtime = 'edge';
const provider = openai({
baseURL: process.env.OPENAI_BASE_URL!,
apiKey: process.env.OPENAI_API_KEY!,
});
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: provider('gpt-4o-mini'),
messages,
});
return result.toDataStreamResponse();
}streamText returns a streaming result. toDataStreamResponse() emits the exact protocol useChat expects: a text/event-stream of data: frames. The model id string is passed straight to the backend, so use whatever your provider documents.
Expected checkpoint: start the dev server and curl the route.
npm run dev
curl -X POST http://localhost:3000/api/chat \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Say hi in 5 words"}]}'You should see a streamed sequence of data chunks, not a single JSON blob. If you get a normal JSON response, the route is not returning the stream helper.
Wire up the client
Create a client component for the chat UI. The useChat hook from @ai-sdk/react manages message state, input, and streaming.
// app/page.tsx
'use client';
import { useChat } from '@ai-sdk/react';
export default function Chat() {
const { messages, input, handleInputChange, handleSubmit } = useChat();
return (
<div style={{ maxWidth: 600, margin: '2rem auto' }}>
<div>
{messages.map((m) => (
<div key={m.id} style={{ margin: '1rem 0' }}>
<strong>{m.role}:</strong> {m.content}
</div>
))}
</div>
<form onSubmit={handleSubmit}>
<input
value={input}
onChange={handleInputChange}
placeholder="Type a message"
style={{ width: '100%', padding: 8 }}
/>
</form>
</div>
);
}The hook posts to /api/chat by default. No extra configuration is needed for the base vercel ai sdk usechat openai-compatible flow.
Verify the browser flow
Run npm run dev, open http://localhost:3000. Type a message and submit. Tokens appear incrementally in the assistant bubble. The network tab shows a text/event-stream response with data: lines.
If you see the full message pop in at once, confirm runtime = 'edge' is set and the route returns toDataStreamResponse(). The Node.js runtime also streams, but edge avoids buffering issues in local dev.
Customize the endpoint path
The default /api/chat path is convenient, but real apps often namespace routes. Override the api prop on the hook:
const { messages, input, handleInputChange, handleSubmit } = useChat({
api: '/api/llm',
});The vercel ai sdk usechat openai-compatible setup remains unchanged when you rename the route; only the client prop and the file location move.
Switch models without touching the UI
Because the provider speaks OpenAI, you change only the server route. For example, to use a different model id:
// app/api/chat/route.ts (excerpt)
const result = streamText({
model: provider('mistral-7b-instruct'),
messages,
});The client code stays identical. That decoupling is the core benefit: frontend owns rendering, server owns model selection.
Handle errors and retries
Production needs error handling. Wrap stream creation in try/catch and return a clean response.
export async function POST(req: Request) {
try {
const { messages } = await req.json();
const result = streamText({
model: provider('gpt-4o-mini'),
messages,
});
return result.toDataStreamResponse();
} catch (err) {
return new Response(JSON.stringify({ error: 'stream failed' }), {
status: 500,
headers: { 'content-type': 'application/json' },
});
}
}On the client, useChat exposes error and reload. Surface it:
const { messages, input, handleInputChange, handleSubmit, error, reload } = useChat();
if (error) {
return <div>Error: {error.message} <button onClick={() => reload()}>Retry</button></div>;
}Pass provider-specific options
Some OpenAI-compatible gateways honor cache-control or routing hints. The AI SDK forwards custom headers via the provider instance.
const provider = openai({
baseURL: process.env.OPENAI_BASE_URL!,
apiKey: process.env.OPENAI_API_KEY!,
headers: {
'x-routing-directive': 'prefer-low-latency',
},
});If your backend supports per-token usage metering or cache hints, those travel transparently. The vercel ai sdk usechat openai-compatible contract does not strip unknown fields from the outbound request.
Use a single gateway for many models
When you do not want to manage multiple base URLs, a gateway that aggregates providers helps. Point OPENAI_BASE_URL at one endpoint and change only the model string per request.
// dynamic model from request body
const { messages, model = 'gpt-4o-mini' } = await req.json();
const result = streamText({ model: provider(model), messages });Then extend the client to send a model selector in the request body. The hook accepts body options:
const { messages, input, handleInputChange, handleSubmit } = useChat({
body: { model: 'mistral-7b-instruct' },
});Add a system prompt
Server-side system messages keep instructions out of the client. Modify the route:
const result = streamText({
model: provider('gpt-4o-mini'),
system: 'You are a concise terminal assistant.',
messages,
});The system field is merged before the conversation history. The client does not need to know it exists.
Expected output structure
A successful useChat exchange produces messages like:
{
"id": "msg-1",
"role": "user",
"content": "Hello"
}
{
"id": "msg-2",
"role": "assistant",
"content": "Hi there, how can I help?"
}The streaming protocol splits assistant content into incremental data: events; the hook reassembles them into a single growing string.
Going further
Add maxTokens, temperature, or topP to streamText for sampling control. Use experimental_prepareRequestBody if you must reshape the outbound payload for a strict provider. The pattern holds: define the model and policies on the server, stream via the standard protocol, and let useChat own the UI state. You now have a runnable chat app backed by any OpenAI-compatible API, with streaming, error recovery, and model flexibility.