When your NestJS service calls the OpenAI API at scale, you will hit HTTP 429 responses. Robust nestjs openai 429 error handling means more than logging the status code—you need retries with backoff, a global strategy for surfacing limits to clients, and a fallback path when OpenAI is saturated. This guide walks through a concrete implementation you can drop into an existing app.
Step 1: Install dependencies and configure the OpenAI client
Use the official OpenAI Node SDK. It throws typed APIError instances that expose status and headers, which we need for precise 429 detection.
npm install openaiCreate a configured client in a service:
import OpenAI from 'openai';
import { Injectable } from '@nestjs/common';
@Injectable()
export class OpenAiService {
private client: OpenAI;
constructor() {
this.client = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
// To route through a gateway, set baseURL instead:
// baseURL: 'https://api.n4n.ai/v1',
});
}
async chat(prompt: string): Promise<string> {
const completion = await this.client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: prompt }],
});
return completion.choices[0].message.content ?? '';
}
}Step 2: Implement retry with exponential backoff
The core of nestjs openai 429 error handling is respecting the Retry-After header and backing off without flooding the API. OpenAI’s SDK attaches rate-limit headers to the error object; use them directly.
import { Injectable, Logger } from '@nestjs/common';
import OpenAI from 'openai';
@Injectable()
export class OpenAiService {
private readonly logger = new Logger(OpenAiService.name);
private client: OpenAI;
constructor() {
this.client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
}
async chatWithRetry(prompt: string, maxAttempts = 5): Promise<string> {
let attempt = 0;
while (true) {
try {
attempt++;
const completion = await this.client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: prompt }],
});
return completion.choices[0].message.content ?? '';
} catch (err) {
if (err instanceof OpenAI.APIError && err.status === 429) {
if (attempt >= maxAttempts) throw err;
const retryAfter = Number(err.headers?.['retry-after'] ?? 1);
const delay = Math.min(retryAfter * 1000, 30_000);
this.logger.warn(`429 on attempt ${attempt}, retrying in ${delay}ms`);
await new Promise((r) => setTimeout(r, delay));
continue;
}
throw err;
}
}
}
}For bursty traffic, add jitter so multiple failing requests don’t retry in lockstep:
const base = Math.min(retryAfter * 1000, 30_000);
const jitter = Math.random() * 500;
await new Promise((r) => setTimeout(r, base + jitter));Step 3: Centralize 429 handling with an exception filter
Retries inside the service handle transient spikes. For limits that persist beyond maxAttempts, you still need a clean HTTP response. A NestJS exception filter maps the upstream 429 to a proper client-facing status and Retry-After header.
import {
ExceptionFilter,
Catch,
ArgumentsHost,
HttpStatus,
} from '@nestjs/common';
import { Response } from 'express';
import OpenAI from 'openai';
@Catch(OpenAI.APIError)
export class OpenAiExceptionFilter implements ExceptionFilter {
catch(exception: OpenAI.APIError, host: ArgumentsHost) {
const ctx = host.switchToHttp();
const response = ctx.getResponse<Response>();
if (exception.status === 429) {
const retryAfter = exception.headers?.['retry-after'] ?? '1';
response.setHeader('Retry-After', retryAfter);
response.status(HttpStatus.TOO_MANY_REQUESTS).json({
statusCode: 429,
message: 'Upstream rate limit exceeded',
retryAfter,
});
return;
}
response.status(exception.status ?? 500).json({
statusCode: exception.status ?? 500,
message: exception.message,
});
}
}Register it globally in main.ts:
app.useGlobalFilters(new OpenAiExceptionFilter());Now any uncaught OpenAI.APIError from a controller gets a consistent shape without repeating try/catch blocks.
Step 4: Use an interceptor for declarative retries
Another pillar of nestjs openai 429 error handling is interceptor-based retries at the route level. This keeps controllers clean and moves retry logic into the request pipeline.
import {
Injectable,
NestInterceptor,
ExecutionContext,
CallHandler,
UseInterceptors,
Post,
Body,
} from '@nestjs/common';
import { Observable, throwError, timer } from 'rxjs';
import { mergeMap, retryWhen } from 'rxjs/operators';
import OpenAI from 'openai';
@Injectable()
export class Retry429Interceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
return next.handle().pipe(
retryWhen((errors) =>
errors.pipe(
mergeMap((err, i) => {
if (err instanceof OpenAI.APIError && err.status === 429 && i < 4) {
const retryAfter = Number(err.headers?.['retry-after'] ?? 1);
return timer(retryAfter * 1000);
}
return throwError(() => err);
}),
),
),
);
}
}Bind it to a specific route:
@UseInterceptors(Retry429Interceptor)
@Post('summarize')
async summarize(@Body() dto: { text: string }) {
return this.openAiService.chat(dto.text);
}Pick either the service-level loop or the interceptor—not both—to avoid compounding delays.
Step 5: Add a fallback provider or gateway
When OpenAI is consistently degraded, retries alone won’t save you. A circuit breaker that switches to a secondary model or gateway prevents total outage. The opossum package gives you a battle-tested breaker.
npm install opossumimport CircuitBreaker from 'opossum';
import OpenAI from 'openai';
const breaker = new CircuitBreaker(
async (prompt: string) => {
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
return client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: prompt }],
});
},
{ timeout: 10_000, errorThresholdPercentage: 50, resetTimeout: 30_000 },
);
breaker.fallback(async (prompt: string) => {
const client = new OpenAI({
apiKey: process.env.GATEWAY_KEY,
baseURL: 'https://api.n4n.ai/v1',
});
const res = await client.chat.completions.create({
model: 'anthropic/claude-3.5-sonnet',
messages: [{ role: 'user', content: prompt }],
});
return res;
});n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and applies automatic fallback when a provider is rate-limited or degraded, so the fallback call above keeps the exact same SDK shape as the primary path.
Step 6: Verify your nestjs openai 429 error handling works
You need a test that forces a 429 and asserts both retry and filter behavior. Use nock to mock the OpenAI endpoint without burning quota.
npm install -D nockimport nock from 'nock';
import { Test } from '@nestjs/testing';
import { OpenAiService } from './openai.service';
it('retries on 429 then succeeds', async () => {
nock('https://api.openai.com')
.post('/v1/chat/completions')
.reply(429, {}, { 'retry-after': '0' })
.post('/v1/chat/completions')
.reply(200, {
choices: [{ message: { content: 'ok' } }],
});
const module = await Test.createTestingModule({
providers: [OpenAiService],
}).compile();
const svc = module.get(OpenAiService);
const result = await svc.chatWithRetry('hi');
expect(result).toBe('ok');
});Run the suite:
npm run test -- openai.service.spec.tsFor end-to-end verification, point OPENAI_API_KEY at a key with a tiny quota, loop requests in a script, and watch logs show 429 on attempt warnings followed by either success or a clean 429 response carrying Retry-After. Hit the route with curl -i to confirm the header is present:
curl -i -X POST localhost:3000/summarize -d '{"text":"test"}' -H 'Content-Type: application/json'Production checklist
- Cap total retry time under your HTTP client timeout (typically 30s).
- Surface
Retry-Afterto downstream callers; never swallow 429. - Track 429 rates in metrics (Prometheus, Datadog) to spot quota erosion.
- Use a gateway or circuit breaker for cross-provider resilience.
Good nestjs openai 429 error handling is layered: local retries, global filters, and fallback paths. Ship all three before relying on LLM calls in user-facing routes.