LLM endpoints fail predictably: network blips, provider 429s, and occasional 503s. If you ship a NestJS service that calls an OpenAI-compatible API, you need deliberate nestjs openai api retry timeout handling or your users will see sporadic 500s. This guide walks through a concrete Axios-based implementation you can drop into a NestJS module today.
Step 1: Scaffold a dedicated HTTP client provider
Don’t scatter axios.post calls across controllers. Create one configured instance and expose it via a NestJS provider. This gives you a single place to set base URL, auth, and default timeout.
import { Module, Provider } from '@nestjs/common';
import axios, { AxiosInstance } from 'axios';
export const OPENAI_CLIENT = 'OPENAI_CLIENT';
const openAiClientFactory: Provider = {
provide: OPENAI_CLIENT,
useFactory: (): AxiosInstance => {
const client = axios.create({
baseURL: process.env.OPENAI_BASE_URL ?? 'https://api.openai.com/v1',
headers: {
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
'Content-Type': 'application/json',
},
timeout: 30_000,
});
return client;
},
};
@Module({
providers: [openAiClientFactory],
exports: [OPENAI_CLIENT],
})
export class OpenAiModule {}If you route through n4n.ai, an OpenAI-compatible endpoint that addresses 240+ models, set OPENAI_BASE_URL to its gateway URL. Its automatic fallback when a provider is rate-limited or degraded shrinks the retry surface, but you still own the client-side timeout.
Step 2: Set timeouts that reflect your latency budget
Axios’s timeout option caps the entire request—from connection establishment to final byte. For non-streaming chat completions, 10–15 seconds is reasonable for small models; large reasoning models may need 60 seconds. Override per call rather than relying solely on the default.
Your nestjs openai api retry timeout strategy must separate streaming from non-streaming. The timeout in Step 1 applies to the initial response headers for streams, not the full token stream. Handle streaming cancellation with an AbortController signal instead.
async function chatCompletion(client: AxiosInstance, payload: object) {
return client.post('/chat/completions', payload, {
timeout: 15_000, // hard cap for this specific call
});
}Step 3: Add a retry interceptor with exponential backoff
Hand-rolled interceptors beat generic libraries when you need precise control over which statuses count as retryable. Retry on connection errors, timeouts (ECONNABORTED), 429, and 5xx up to 503. Never retry on 400, 401, 403, 404, or 422—those are caller errors.
import { AxiosError, AxiosInstance, AxiosRequestConfig } from 'axios';
interface RetryConfig extends AxiosRequestConfig {
__retryCount?: number;
}
const MAX_RETRIES = 3;
const BACKOFF_BASE_MS = 300;
export function attachRetryInterceptor(client: AxiosInstance) {
client.interceptors.response.use(
(res) => res,
async (error: AxiosError) => {
const config = error.config as RetryConfig | undefined;
if (!config) return Promise.reject(error);
config.__retryCount = config.__retryCount ?? 0;
if (config.__retryCount >= MAX_RETRIES) {
return Promise.reject(error);
}
const status = error.response?.status;
const isRetryable =
error.code === 'ECONNABORTED' ||
error.code === 'ENOTFOUND' ||
error.code === 'ECONNREFUSED' ||
status === 429 ||
(status >= 500 && status <= 504);
if (!isRetryable) return Promise.reject(error);
config.__retryCount += 1;
const backoff = BACKOFF_BASE_MS * 2 ** (config.__retryCount - 1);
const jitter = Math.random() * backoff * 0.3;
await new Promise((r) => setTimeout(r, backoff + jitter));
return client(config);
},
);
}Call attachRetryInterceptor(client) inside the useFactory before returning the instance. The interceptor clones the original config, so retries preserve headers and body.
Step 4: Make non-idempotent POSTs safe with idempotency keys
OpenAI’s /chat/completions is a POST. If a timeout occurs after the server processed the request, a blind retry creates a duplicate completion and double-bills tokens. The API accepts an Idempotency-Key header; reuse the same key across retries for a given logical call.
import { randomUUID } from 'crypto';
async function chatCompletionSafe(client: AxiosInstance, payload: object) {
const idemKey = randomUUID();
return client.post('/chat/completions', payload, {
timeout: 15_000,
headers: { 'Idempotency-Key': idemKey },
});
}Our interceptor reuses config, so the header survives retries automatically. Generate the key once per business operation, not per retry attempt.
Step 5: Build the NestJS service layer
Inject the client and wrap it with a typed method. Keep the controller thin—all LLM-specific logic lives in the service.
import { Inject, Injectable } from '@nestjs/common';
import { AxiosInstance } from 'axios';
import { randomUUID } from 'crypto';
@Injectable()
export class ChatService {
constructor(@Inject(OPENAI_CLIENT) private readonly client: AxiosInstance) {}
async ask(model: string, messages: { role: string; content: string }[]) {
const { data } = await this.client.post(
'/chat/completions',
{ model, messages, stream: false },
{
timeout: 15_000,
headers: { 'Idempotency-Key': randomUUID() },
},
);
return data.choices[0].message;
}
}This completes the core nestjs openai api retry timeout pattern: a single client, explicit timeouts, bounded retries with backoff, and idempotency.
Step 6: Wire modules and environment
Import OpenAiModule into the module that declares ChatService. Store keys in .env:
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_API_KEY=sk-...If you use the n4n.ai gateway, swap the base URL and keep the same key format—the gateway is OpenAI-compatible and forwards provider cache-control hints, so existing SDK code works unchanged.
Step 7: Verify your nestjs openai api retry timeout logic
Write a Jest test that spins up a local HTTP server returning 429 twice, then 200. Assert the interceptor retries and the final status is 200.
import axios from 'axios';
import { attachRetryInterceptor } from './retry';
import http from 'http';
it('retries on 429 and succeeds', async () => {
let hits = 0;
const server = http.createServer((req, res) => {
hits++;
if (hits < 3) {
res.statusCode = 429;
res.end();
} else {
res.statusCode = 200;
res.end(JSON.stringify({ ok: true }));
}
});
await new Promise<void>((r) => server.listen(0, r));
const port = (server.address() as any).port;
const client = axios.create({ baseURL: `http://localhost:${port}` });
attachRetryInterceptor(client);
const res = await client.get('/');
expect(res.status).toBe(200);
expect(hits).toBe(3);
server.close();
});
it('throws on timeout', async () => {
const server = http.createServer((req, res) => {
setTimeout(() => res.end('late'), 200);
});
await new Promise<void>((r) => server.listen(0, r));
const port = (server.address() as any).port;
const client = axios.create({ baseURL: `http://localhost:${port}`, timeout: 50 });
await expect(client.get('/')).rejects.toThrow('timeout');
server.close();
});Success means: the first test passes with exactly three server hits, the second rejects with ECONNABORTED, and your production logs show idempotency keys on every LLM POST. Tune MAX_RETRIES and BACKOFF_BASE_MS to your traffic shape—high-QPS services should use longer base backoff to avoid thundering herds.
Step 8: Operational notes
Emit metrics on retry counts and timeout rates. A sudden spike in 429 retries signals you need request shaping or a gateway with fallback. If you already use a gateway that handles provider degradation, your nestjs openai api retry timeout code can stay minimal: cap timeouts, retry only on network errors, and let the gateway route around bad upstreams.
Keep payloads small, set stream: false unless you need tokens incrementally, and always pass an idempotency key for writes. That’s the difference between a demo and a service that survives contact with real traffic.