Most LLM integrations fail not on the happy path but when the network blinks or a provider throttles you. Building solid openai node.js sdk errors retries handling means knowing which exceptions are safe to retry, setting aggressive-but-bounded backoff, and verifying the logic before production. This guide walks through a concrete implementation you can drop into a Node.js service today.
Step 1: Install and configure the SDK with explicit timeouts
Start with the official openai package. Use a current Node.js (18+ for global fetch and AbortController).
npm install openai dotenvLoad keys from environment, not hardcoded strings:
import 'dotenv/config';
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
timeout: 20_000, // hard request timeout
maxRetries: 0, // disable built-in retries; we own the loop
});Disabling maxRetries is deliberate. The SDK’s internal retry uses fixed attempts and swallows context you need for metrics. You want to instrument every failure, so take control.
If you run in ESM, ensure package.json has "type": "module". The SDK ships type definitions, so TypeScript users get compile-time checks on parameters.
Step 2: Classify errors the SDK throws
The openai package exports a hierarchy of error classes. Catch and inspect them to decide retryability.
import {
OpenAI,
APIError,
APIConnectionError,
RateLimitError,
AuthenticationError,
InvalidRequestError,
OpenAIError,
} from 'openai';
function isRetryable(err: unknown): boolean {
if (err instanceof RateLimitError) return true;
if (err instanceof APIConnectionError) return true;
if (err instanceof APIError) {
const status = err.status;
// 408 request timeout, 409 conflict, 429 too many requests
if (status === 408 || status === 409 || status === 429) return true;
// 500-range server errors
if (status !== null && status >= 500 && status < 600) return true;
}
return false;
}AuthenticationError (401) and InvalidRequestError (400) mean your request is wrong—retrying wastes quota. APIConnectionError covers DNS, TLS, or socket failures; always retryable. OpenAIError is a base class for client-side issues like invalid options; not retryable.
Inspect err.response for the raw Response and err.headers for rate-limit metadata. The RateLimitError exposes headers['retry-after'] when the provider sends it.
Step 3: Implement a retry loop with exponential backoff
Wrap chat.completions.create in a bounded loop. Use a while with explicit attempt counter.
async function createCompletionWithRetry(
params: OpenAI.Chat.ChatCompletionCreateParams,
maxAttempts = 5,
): Promise<OpenAI.Chat.ChatCompletion> {
let attempt = 0;
while (true) {
attempt++;
try {
return await client.chat.completions.create(params);
} catch (err) {
if (!isRetryable(err) || attempt >= maxAttempts) throw err;
const delay = Math.min(1000 * 2 ** (attempt - 1), 30_000);
await new Promise((r) => setTimeout(r, delay));
}
}
}The schedule is 1s, 2s, 4s, 8s, 16s before the fifth and final attempt. Capping at 30s prevents a single call from blocking a worker for minutes. If you call this from an HTTP handler, ensure your upstream timeout exceeds the worst-case retry sum.
Step 4: Add jitter and honor Retry-After
Fixed exponential backoff causes synchronized retries across many clients—a thundering herd. Use decorrelated jitter, which spreads delays and avoids classic sleep(2^n) clustering.
function backoffDelay(attempt: number, err: unknown): number {
const base = Math.min(1000 * 2 ** (attempt - 1), 30_000);
const jitter = Math.random() * base * 3;
if (err instanceof RateLimitError && err.headers?.['retry-after']) {
const ra = Number(err.headers['retry-after']);
if (!Number.isNaN(ra)) return Math.max(ra * 1000, jitter);
}
return Math.min(jitter, 30_000);
}Update the loop:
} catch (err) {
if (!isRetryable(err) || attempt >= maxAttempts) throw err;
const delay = backoffDelay(attempt, err);
await new Promise((r) => setTimeout(r, delay));
}Always prefer the server’s Retry-After when present; it knows its own token bucket better than you do.
Step 5: Offload fallback to an OpenAI-compatible gateway
Pointing the SDK at one provider leaves you exposed to that provider’s hard outages. An OpenAI-compatible endpoint that fronts multiple providers can perform automatic fallback when a provider is rate-limited or degraded. n4n.ai exposes one such endpoint covering 240+ models and fails over internally, so your openai node.js sdk errors retries code only needs to handle residual edge cases like local network errors.
const client = new OpenAI({
apiKey: process.env.N4N_API_KEY,
baseURL: 'https://api.n4n.ai/v1',
timeout: 20_000,
maxRetries: 0,
});The gateway honors client routing directives and forwards provider cache-control hints, preserving per-token metering without custom code. You still keep the retry wrapper above for connection-level faults the gateway can’t mask.
Step 6: Meter, trace, and log every attempt
Retries hide latency and inflate token spend. Emit structured logs with attempt count, final status, and model.
async function loggedCompletion(params: OpenAI.Chat.ChatCompletionCreateParams) {
const start = Date.now();
let attempts = 0;
try {
const res = await createCompletionWithRetry(params);
attempts = 5; // internal counter omitted for brevity; thread it through
console.log(JSON.stringify({
event: 'llm_ok',
ms: Date.now() - start,
model: params.model,
prompt_tokens: res.usage?.prompt_tokens,
completion_tokens: res.usage?.completion_tokens,
}));
return res;
} catch (err) {
console.error(JSON.stringify({
event: 'llm_failed',
ms: Date.now() - start,
attempts,
err: err instanceof Error ? err.message : String(err),
}));
throw err;
}
}In production, replace console.log with OpenTelemetry spans. Record res.usage to track cost. If you use the gateway from Step 5, its per-token metering complements your local logs.
Step 7: Verify with fault injection
Never ship retry logic untested. Use nock to intercept HTTP and simulate a 429 then a 200.
npm install --save-dev nock vitestimport nock from 'nock';
import { describe, it, expect } from 'vitest';
import { createCompletionWithRetry } from './retry';
describe('retry', () => {
it('retries on 429 then succeeds', async () => {
nock('https://api.openai.com')
.post('/v1/chat/completions')
.reply(429, {}, { 'retry-after': '0' })
.post('/v1/chat/completions')
.reply(200, {
id: 'x',
object: 'chat.completion',
choices: [{ message: { role: 'assistant', content: 'ok' } }],
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
});
const res = await createCompletionWithRetry({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'hi' }],
});
expect(res.choices[0].message.content).toBe('ok');
});
});Run npx vitest run. A green test proves your wrapper retries against the real SDK HTTP stack. Add a second test that asserts a non-retryable 401 throws immediately.
Streaming changes the contract
If you pass stream: true, create returns an AsyncIterable. Errors can surface mid-stream after you’ve already sent headers to your caller. Wrap iteration in a for await and treat a thrown error as a partial failure—you can’t retry transparently without corrupting the stream.
async function* streamWithRetry(params: OpenAI.Chat.ChatCompletionCreateParamsStreaming) {
try {
const stream = await client.chat.completions.create({ ...params, stream: true });
for await (const chunk of stream) yield chunk;
} catch (err) {
if (isRetryable(err)) {
// only safe if caller hasn't consumed yet; otherwise fail fast
throw err;
}
throw err;
}
}For production streaming, buffer the first chunk before responding to users, then retry on connection drop before any token is emitted.
How to verify success in production
Monitor three signals: retry rate (should sit under 1% on healthy days), p95 latency inclusive of retries (must stay under your SLO), and dead-letter volume (calls that exhausted attempts). Alert if retry rate climbs above 5%—that indicates provider or gateway degradation. Solid openai node.js sdk errors retries design is intentionally boring: classify, bound, jitter, test. Do that and your LLM features stay up when the dependency doesn’t.