Avoiding cold starts in serverless Vercel AI SDK apps

A cold start in a Vercel function can add seconds of blank time before your first LLM token streams. The vercel ai sdk serverless cold starts tax shows up when you import the entire ai ecosystem and instantiate provider clients on every request. This guide gives you an end-to-end sequence to measure, cut, and verify that overhead in a real Next.js or SvelteKit deployment.

Step 1: Measure your baseline cold start

You cannot fix what you do not measure. Vercel separates the init phase (loading your code) from the invocation phase (handling the request). The function log shows Init Duration on cold runs only. Add a module-scope timer to see how much of that is your own imports.

// app/api/chat/route.ts
const boot = performance.now();

import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';

export async function POST(req: Request) {
  console.log(`module load took ${performance.now() - boot}ms`);
  // handler body
}

Deploy to a region you rarely hit, then curl it once after a 10-minute idle. The first log line tells you the truth. In a typical unoptimized Next.js AI route, module load took lands between 400ms and 1.2s because the ai core plus provider SDKs parse a lot of JavaScript. That is the number you will drive down.

Step 2: Minimize imports and bundle weight

The ai package is tree-shakeable, but the moment you write import { anthropic } from '@ai-sdk/anthropic' alongside OpenAI, both SDKs ship to the edge. Each brings its own fetch wrapper, retry queue, and Zod schemas.

// bad: pulls in every built-in provider helper
import { openai, anthropic, cohere } from '@ai-sdk/all';

// good: narrow, explicit imports
import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';

Run vercel build and inspect the output function size in .vercel/output/functions/api/chat.func. If it exceeds 2–3 MB uncompressed, you are shipping dead weight. Remove unused providers, and if you use only one model vendor, delete the others from package.json. Smaller bundles parse faster during a cold boot, directly attacking vercel ai sdk serverless cold starts.

Step 3: Lazy-initialize and reuse provider instances

Serverless containers keep the process alive for dozens or hundreds of requests. Building a provider client on every call wastes CPU and delays the first token. Store it on globalThis so initialization happens once per warm container.

// lib/llm.ts
import { createOpenAI } from '@ai-sdk/openai';

const store = globalThis as unknown as {
  llm?: ReturnType<typeof createOpenAI>;
};

export function getLLM() {
  if (!store.llm) {
    store.llm = createOpenAI({ apiKey: process.env.OPENAI_API_KEY! });
  }
  return store.llm;
}

In the route, call getLLM() inside the handler, not at module scope. On Edge runtime, globalThis is shared per isolate; on Node, it persists across invocations in the same instance. Either way, the second request skips client construction entirely.

Step 4: Pick the right runtime

Vercel exposes nodejs and edge runtimes per route. Edge isolates boot in tens of milliseconds but cap memory at 128 MB and forbid native modules. Node cold starts are slower but support the full ecosystem.

// app/api/chat/route.ts
export const runtime = 'edge';

For most chat use cases built with the vercel ai sdk serverless cold starts pattern, Edge is the better default. You do not need fs or child_process to stream LLM output. Switch to Node only if you must use a library that calls native bindings or needs long execution windows beyond Edge’s 25-second limit.

Step 5: Keep functions warm

A Vercel function reverts to cold after a short idle window (often 15–30 seconds under low traffic). A cron ping keeps the container alive.

// vercel.json
{
  "crons": [{ "path": "/api/chat", "schedule": "*/1 * * * *" }]
}

The ping must be a real POST with a valid body, or the framework middleware may short-circuit it. A robust warmer:

curl -X POST https://your-app.vercel.app/api/chat \
  -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"warm"}]}'

Schedule it from an external uptime service if you want sub-minute precision. Be aware that warmers consume invocations and can inflate cost on paid plans, so apply them only to latency-critical routes.

Step 6: Collapse multi-provider logic into one gateway

If your app falls back between OpenAI, Anthropic, and open-weight models, bundling three SDKs triples parse time. Point the AI SDK at a single OpenAI-compatible gateway instead. n4n.ai provides one endpoint covering 240+ models with automatic fallback when a provider is rate-limited, so your serverless bundle ships one small client instead of four.

import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';

const gateway = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY,
});

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({
    model: gateway('openai/gpt-4o-mini'),
    messages,
  });
  return result.toDataStreamResponse();
}

This deletes @ai-sdk/anthropic and others from your dependency tree. The gateway honors provider cache-control hints and meters per token, but the key win for cold starts is one tiny import path.

Step 7: Stream to mask residual latency

Even a warm container has network round-trips. Streaming makes the user perceive responsiveness while the model generates.

import { streamText } from 'ai';
import { getLLM } from '@/lib/llm';

export const runtime = 'edge';

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = streamText({
    model: getLLM()('gpt-4o-mini'),
    messages,
  });
  return result.toDataStreamResponse();
}

On the client, use @ai-sdk/react’s useChat to render tokens as they land. The vercel ai sdk serverless cold starts penalty becomes invisible because the first paint happens before the model finishes thinking.

Step 8: Verify success

Redeploy and test from a cold region using curl with timing:

curl -w "total: %{time_total}s\n" -X POST https://your-app.vercel.app/api/chat \
  -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"hi"}]}'

Compare the Vercel dashboard’s Init Duration against the Step 1 baseline. A trimmed Edge route with a single gateway client and a warmer should show a 70–90% reduction. If getLLM() logs construction on every hit, your globalThis cache is not surviving—check that the file is not re-bundled per request by a misconfigured loader.

Add OpenTelemetry traces if you need per-route breakdowns. The goal is a stable init under 100ms and time-to-first-token under 400ms on cold boots.

Discipline beats heroics

The vercel ai sdk serverless cold starts problem is not solved by a single trick. It is dependency hygiene, runtime choice, and a warm container. Measure first, trim imports, lazy-load the client, pick Edge, and collapse providers behind one endpoint. Do that and your users will think your LLM app is running on a always-hot box even though it is serverless.