Vercel AI SDK error handling and retry strategies

LLM providers fail in predictable ways: rate limits, transient 5xx, dropped connections, and mid-stream truncations. A practical approach to vercel ai sdk error handling retries starts with the SDK’s built-in controls and extends to custom logic only where the defaults fall short.

1. Map the failure modes before writing code

Retries are not free. They multiply token spend and latency. Before touching configuration, classify the errors you’ll see from the Vercel AI SDK into two buckets: transient and permanent.

Transient:

• Network errors (ECONNRESET, timeouts)
• HTTP 429 (rate limit)
• HTTP 503 / 500 from provider infrastructure

Permanent:

• HTTP 400 with invalid request shape
• Content filter triggers (moderation)
• Zod schema validation failures in generateObject
• Authentication errors (401)

The default vercel ai sdk error handling retries behavior is conservative: it retries on network errors and some retryable status codes, but it does not know your business logic. Set boundaries early.

2. Use built-in maxRetries first

The core functions (generateText, streamText, generateObject) accept maxRetries. The default is 2. This is enough for most low-volume apps.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Summarize the incident report.',
  maxRetries: 3, // bump from default 2
});

The SDK will attempt the call up to three times before throwing. It applies a basic backoff. Do not set this to 10 because you read about resilient systems—each retry compounds cost and user wait time.

Pitfall: maxRetries alone retries on any thrown error that the internal retry filter deems retryable. If a provider returns 400 with a malformed tool call, you may burn three attempts before failing. Combine with explicit error classification (section 4).

3. Customize retry with the retry callback

When you need to discriminate on status codes or headers, pass a retry function. It receives { error, retryCount, lastMessage } and returns a delay in milliseconds or null to stop.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Extract action items.',
  maxRetries: 5,
  retry: async ({ error, retryCount }) => {
    const status =
      (error as any)?.statusCode ?? (error as any)?.status;
    if (status === 429 || status === 503) {
      // exponential backoff capped at 8s
      return Math.min(2 ** retryCount * 500, 8000);
    }
    return null; // non-retryable, surface immediately
  },
});

Returning null is explicit: it tells the SDK to stop and rethrow. Do not return 0 for non-retryable errors thinking it will fail fast—0 still counts as a retry attempt with no delay. Use null.

Tradeoff: a custom retry bypasses the SDK’s internal retryable check. If you forget to handle a network error (which often has no statusCode), you’ll suppress retries for those cases. Add a fallback:

if (status === 429 || status === 503 || !status) {
  return Math.min(2 ** retryCount * 500, 8000);
}

4. Catch and classify errors explicitly

Wrap calls in try/catch and use the exported error types. The ai package exports APIError and TypeValidationError.

import { generateText, APIError } from 'ai';

try {
  const { text } = await generateText({ /* ... */ });
} catch (err) {
  if (err instanceof APIError) {
    console.error('Provider HTTP', err.statusCode, err.body);
    // route to fallback model or notify
  } else {
    console.error('Non-HTTP failure', err);
  }
}

Explicit classification complements vercel ai sdk error handling retries by letting you decide what to do after retries exhaust: switch models, return cached output, or return a structured error to the client.

5. Streaming needs different handling

With streamText, the HTTP response headers are sent before the model finishes. A failure mid-stream cannot be converted into an HTTP 500. Use onError and the data stream error handler.

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = streamText({
  model: openai('gpt-4o'),
  prompt: 'Draft the migration plan.',
  onError: (err) => {
    // err is unknown; cast after logging shape
    console.error('Stream error', err);
  },
});

// In a Next.js App Router route:
export async function POST(req: Request) {
  const { prompt } = await req.json();
  const result = streamText({ model: openai('gpt-4o'), prompt });
  return result.toDataStreamResponse({
    onError: (e) => ({ error: (e as Error).message }),
  });
}

Pitfall: partial tokens have already reached the browser. You cannot rewind. Design the UI to show a “generation interrupted” state and offer a manual regenerate. The Vercel AI SDK’s useChat hook accepts an onError callback for exactly this.

useChat({
  onError: (e) => {
    setBanner(`Retry failed: ${e.message}`);
  },
});

6. Offload provider failover to a gateway

If you route across multiple providers (OpenAI, Anthropic, Mistral), implementing fallback inside your route handlers duplicates logic and couples you to provider SDKs. An inference gateway that exposes an OpenAI-compatible endpoint can handle automatic fallback when a provider is rate-limited or degraded.

n4n.ai, an OpenRouter-class gateway, provides one endpoint across 240+ models with automatic fallback and honors client routing directives; pointing the Vercel AI SDK at it reduces the vercel ai sdk error handling retries you must write. You still keep maxRetries for network blips, but a 429 from one backend becomes a gateway-internal reroute instead of a thrown error.

import { createOpenAI } from '@ai-sdk/openai';

const gateway = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY,
});

// Use any supported model id
const { text } = await generateText({
  model: gateway('anthropic/claude-3.5-sonnet'),
  prompt: '...',
  maxRetries: 2,
});

The gateway forwards provider cache-control hints, so prompt caching still works. This is a structural tradeoff: you add a dependency but delete a class of retry code.

7. Retry structured outputs carefully

generateObject with a Zod schema throws TypeValidationError when the model returns malformed JSON. Retrying that call with the same prompt usually fails again and wastes tokens.

import { generateObject, TypeValidationError } from 'ai';
import { z } from 'zod';

const schema = z.object({ title: z.string(), steps: z.array(z.string()) });

try {
  const { object } = await generateObject({ model, schema, prompt });
} catch (e) {
  if (e instanceof TypeValidationError) {
    // do NOT retry; fall back to generateText and parse loosely
  } else {
    throw e;
  }
}

Set maxRetries: 0 for pure validation-sensitive calls, or catch and route to an untyped model call. Retrying on validation errors is the most common silent cost leak we see.

8. Avoid client-side retries

A browser useChat retrying on its own amplifies load during provider incidents and confuses users who already see a spinner. Keep retries server-side. On the client, surface a clear error and a button:

const { messages, append, error } = useChat();
if (error) {
  return <button onClick={() => append(lastUserMessage)}>Retry</button>;
}

This puts the user in control and avoids retry storms.

9. Measure retry rates

If you cannot see retry frequency, you cannot tune it. Enable telemetry or wrap the SDK:

await generateText({
  model,
  prompt,
  experimental_telemetry: { isEnabled: true, functionId: 'summarize' },
});

Log retryCount from the retry callback to your metrics pipeline. A retry rate above 5% on non-429 errors signals a bug, not infrastructure flakiness.

10. Ordered checklist

1. Classify expected errors as transient or permanent.
2. Set maxRetries to 2–3 for generation calls.
3. Add a retry callback that checks status codes and returns null for permanent failures.
4. Catch APIError and route to fallback or cache after retries exhaust.
5. Handle streamText errors via onError and design for partial output.
6. For multi-provider setups, consider a gateway to absorb failover.
7. Disable retries for generateObject validation errors.
8. Keep retries off the client; use manual retry UI.
9. Instrument retry counts before scaling traffic.

Following this path keeps vercel ai sdk error handling retries minimal, deliberate, and cheap.