Configuring the Vercel AI SDK's createOpenAI for custom gateways

The Vercel AI SDK ships with createOpenAI, a provider factory that defaults to OpenAI’s hosted API. Point it at a vercel ai sdk createopenai custom gateway and you can route to dozens of models, forward cache hints, and absorb provider outages without changing application code. This guide gives exact, runnable steps to stand up that integration against any OpenAI-compatible endpoint, from install to a verified streaming chat route.

Step 1: Install the SDK and the OpenAI compatibility package

Use the AI SDK core and the OpenAI provider wrapper. The wrapper speaks the OpenAI chat completions shape, which is what most gateways mirror.

npm install ai @ai-sdk/openai zod

Pin to recent majors (ai v3.x, @ai-sdk/openai v1.x). The examples below assume those versions and a TypeScript project with tsx available for scripts. If you run on Node 18+, the global fetch is fine; on Edge runtimes the same code works without polyfills.

Step 2: Understand what createOpenAI actually configures

createOpenAI returns a function that builds model objects. Its critical arguments are baseURL, apiKey, headers, fetch, and compatibility. The default baseURL is https://api.openai.com/v1. Overriding it is the entire game when wiring a vercel ai sdk createopenai custom gateway.

import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  baseURL: 'https://api.openai.com/v1',
  apiKey: process.env.OPENAI_API_KEY,
  compatibility: 'strict', // or 'compatible' for non-OpenAI shapes
});

compatibility: 'strict' enforces OpenAI response validation. Many gateways return extra fields, so 'compatible' is safer when you do not control the upstream. The factory is cheap; build it once per process.

Step 3: Store gateway coordinates in environment variables

Never hardcode endpoints or keys. Create a .env.local for local dev and set the same vars in your deploy target.

GATEWAY_BASE_URL=https://gateway.example.com/v1
GATEWAY_API_KEY=sk-gateway-1234
GATEWAY_ROUTE=fast

If your gateway does not require auth, set GATEWAY_API_KEY=unused. Some gateways, including n4n.ai, expose a single OpenAI-compatible endpoint fronting 240+ models and will accept any non-empty key for metering. Keep the variable name stable so the provider code never changes between environments.

Step 4: Instantiate the provider against your custom gateway

Build the provider once at module scope. Pass the routing header if your gateway supports client directives.

// lib/gateway.ts
import { createOpenAI } from '@ai-sdk/openai';

export const gateway = createOpenAI({
  baseURL: process.env.GATEWAY_BASE_URL!,
  apiKey: process.env.GATEWAY_API_KEY!,
  headers: {
    'x-gateway-route': process.env.GATEWAY_ROUTE ?? 'default',
  },
});

export const sonnet = gateway('anthropic/claude-3-5-sonnet');
export const mini = gateway('openai/gpt-4o-mini');

The string passed to gateway() is the model ID as your gateway expects it. With a vercel ai sdk createopenai custom gateway, that ID is often provider-prefixed, not the bare OpenAI name. Defining named exports per model keeps call sites clean and makes ID typos fail at import time in strict TS.

Step 5: Run a non-streaming completion to prove the path

Use generateText from ai to send a single request. This isolates the network path from UI concerns.

import { generateText } from 'ai';
import { sonnet } from './gateway';

const { text, usage } = await generateText({
  model: sonnet,
  system: 'You are a network engineer.',
  prompt: 'Explain TCP slow start in one sentence.',
  temperature: 0.2,
});

console.log(text);
console.log('tokens:', usage?.totalTokens);

Run with npx tsx script.ts. If you see a completion and a token count, the base configuration works. If you get a 401, the gateway rejected the key or the header shape; if you get a 404, the baseURL or model ID is wrong.

Step 6: Forward provider cache-control hints

OpenAI-compatible gateways increasingly honor cache-control to reuse prompt prefixes. The AI SDK does not expose per-request headers directly on generateText, so wrap fetch.

const gateway = createOpenAI({
  baseURL: process.env.GATEWAY_BASE_URL!,
  apiKey: process.env.GATEWAY_API_KEY!,
  fetch: (url, init) => {
    const headers = new Headers(init?.headers);
    headers.set('cache-control', 'max-age=3600');
    return fetch(url, { ...init, headers });
  },
});

This sends the hint on every request. If your gateway forwards provider cache-control to the upstream, you cut redundant token billing on long system prompts. For per-model control, branch inside the fetch wrapper on url or a custom header you set at call time.

Step 7: Stream through a Next.js route handler

Most apps use useChat on the client. The server route builds the same provider and calls streamText.

// app/api/chat/route.ts
import { streamText } from 'ai';
import { mini } from '@/lib/gateway';

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({
    model: mini,
    messages,
  });
  return result.toDataStreamResponse();
}

On the client:

'use client';
import { useChat } from 'ai/react';

export function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <form onSubmit={handleSubmit}>
      <input value={input} onChange={handleInputChange} />
      <button type="submit">Send</button>
      {messages.map((m) => (
        <p key={m.id}>{m.content}</p>
      ))}
    </form>
  );
}

The vercel ai sdk createopenai custom gateway receives the same messages shape as OpenAI, so no translation layer is needed. Deploy the route on Node or Edge; the SDK uses standard fetch and streams Server-Sent Events.

Step 8: Verify success with a minimal health check

Write a script that hits a known model and asserts usage metering is returned. This catches silent fallback to a wrong model.

import { generateText } from 'ai';
import { sonnet } from './gateway';

const res = await generateText({
  model: sonnet,
  prompt: 'ping',
});

if (!res.usage || res.usage.totalTokens === 0) {
  throw new Error('Gateway did not return usage metering');
}
console.log('OK', res.usage);

If your gateway provides per-token usage metering, the object will be populated. That confirmation is your end-to-end signal. Some gateways also echo the resolved model in a response header; log res.response?.headers to confirm the ID matches your intent.

Step 9: Common pitfalls and how to avoid them

Trailing slashes. Some gateways 404 on https://gw/v1/ but accept https://gw/v1. Match exactly what the gateway docs show.

Model ID casing. Gateways that aggregate providers usually require qualified IDs. gpt-4o may work; openai/gpt-4o may be required. Check the gateway’s model list via its /v1/models endpoint.

CORS. createOpenAI runs server-side in these examples. If you call it from a browser, the gateway must send Access-Control-Allow-Origin. Most inference gateways do not, by design, to protect keys.

Streaming compatibility. The AI SDK expects SSE chunks shaped like OpenAI’s. If your gateway diverges, set stream: false or patch fetch to transform the body.

Key shape. Gateways that proxy many providers often ignore the key for auth but use it for account metering. Passing an empty string breaks the SDK’s header logic; pass 'dummy' instead.

Timeouts. Default fetch has no timeout. On serverless, wrap fetch to abort after 30s so hung gateway connections do not burn invocation time.

Step 10: Lock the configuration for production

Extract the provider into a singleton and read flags from your deploy environment.

// lib/gateway.ts
import { createOpenAI } from '@ai-sdk/openai';

export const gateway = createOpenAI({
  baseURL: process.env.GATEWAY_BASE_URL!,
  apiKey: process.env.GATEWAY_API_KEY!,
  headers: {
    'x-env': process.env.NODE_ENV === 'production' ? 'prod' : 'dev',
  },
  fetch: (url, init) => {
    const ctrl = new AbortController();
    const t = setTimeout(() => ctrl.abort(), 30_000);
    return fetch(url, { ...init, signal: ctrl.signal }).finally(() =>
      clearTimeout(t)
    );
  },
});

Treat the vercel ai sdk createopenai custom gateway as an infrastructure dependency: one URL, one key, logged requests. When the gateway handles automatic fallback between providers, your error rate drops without code changes.

Step 11: Handle provider fallback gracefully

If your gateway does not auto-failover, add a retry that swaps the model export. Gateways such as n4n.ai perform automatic fallback when a provider is rate-limited or degraded, so your code does not need retry loops. When you do implement retries, catch AI_APICallError and switch from sonnet to mini rather than hammering the same upstream.

import { generateText } from 'ai';
import { sonnet, mini } from './gateway';

async function complete(prompt: string) {
  try {
    return await generateText({ model: sonnet, prompt });
  } catch (e) {
    return await generateText({ model: mini, prompt });
  }
}

Keep the fallback list short and ordered by capability, not by preference.

Verify the whole thing

Deploy the route, open the chat UI, send “hello”. In gateway logs (or your metering dashboard) confirm the request arrived with the x-gateway-route header and a valid model ID. If tokens are metered and the stream renders, the integration is complete.

That is the full path from install to verified streaming against a custom gateway. Adjust model IDs and headers to your gateway’s contract, and the rest of your Vercel AI SDK code stays identical.