n4nAI
Open console →

API integration

Guide

Express.js error handling for upstream LLM provider failures

Practical guide to express.js error handling llm provider failures: structured errors, retries, timeouts, circuit breakers, and gateway offload.

n4n Team4 min read874 words

Audio narration

Coming soon — every post will get a voice note here.

When your Express.js service sits between users and an external LLM API, upstream instability is not a question of if but when. Robust express.js error handling llm provider failures requires more than a try/catch around a fetch call; it demands a structured pipeline that classifies, isolates, and recovers from provider errors without taking down your own app. This guide lays out an ordered path to make that pipeline real.

1. Centralize Error Handling in Middleware

Express routes fail silently if you throw inside an async handler and never forward the error. The first step in express.js error handling llm provider failures is a single async wrapper that routes rejections to next(err).

import { Request, Response, NextFunction } from 'express';

export const asyncHandler =
  (fn: (req: Request, res: Response, next: NextFunction) => Promise<void>) =>
  (req: Request, res: Response, next: NextFunction) =>
    fn(req, res, next).catch(next);

Apply it to every route that touches a provider:

app.post('/v1/chat', asyncHandler(async (req, res) => {
  const completion = await callLLM(req.body);
  res.json(completion);
}));

Then define a terminal error middleware. Its signature must have four parameters, or Express will treat it as a regular handler and skip it.

app.use((err: Error, req: Request, res: Response, _next: NextFunction) => {
  console.error('request failed', { id: req.id, err });
  res.status(500).json({ error: 'internal_error' });
});

A common pitfall: mixing return res.json() with next(err) in the same branch. Pick one path. Another: catching an error and logging it but not calling next, which hangs the request.

2. Classify Upstream Errors Explicitly

Not every non-200 from an LLM provider deserves the same response. A 400 from malformed JSON is your bug; a 429 or 503 is the provider’s instability. Build a small error taxonomy so later stages can decide on retries.

export class UpstreamError extends Error {
  constructor(
    public readonly status: number,
    public readonly retryable: boolean,
    message: string,
    public readonly provider?: string
  ) {
    super(message);
    this.name = 'UpstreamError';
  }
}

When you call the model:

const resp = await fetch('https://api.openai.com/v1/chat/completions', {
  method: 'POST',
  headers: { 'content-type': 'application/json', authorization: `Bearer ${KEY}` },
  body: JSON.stringify(payload)
});

if (!resp.ok) {
  const retryable = resp.status === 429 || resp.status >= 500;
  throw new UpstreamError(resp.status, retryable, `provider ${resp.status}`, 'openai');
}

The tradeoff: each provider (Anthropic, Cohere, local vLLM) returns different error shapes. Invest in a per-provider normalizer early; retrofitting it later spreads special cases across your codebase.

3. Enforce Timeouts with AbortController

Node’s global fetch has no default timeout. A slow provider will hold connections until your event loop or load balancer forces a kill, exhausting concurrency. Use AbortController and treat aborts as retryable timeouts.

export async function fetchWithTimeout(url: string, opts: RequestInit, ms: number) {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), ms);
  try {
    return await fetch(url, { ...opts, signal: controller.signal });
  } finally {
    clearTimeout(timer);
  }
}

Wrap the call:

try {
  const r = await fetchWithTimeout(url, opts, 8000);
  // ...
} catch (err) {
  if (err instanceof Error && err.name === 'AbortError') {
    throw new UpstreamError(408, true, 'upstream timeout');
  }
  throw err;
}

Pitfall: forgetting the finally block leaks timers. Another: setting timeouts shorter than the provider’s typical p95 latency just generates spurious failures. Measure before tuning.

4. Retry Only What Is Safe

Retries are a core part of express.js error handling llm provider failures, but blind retries amplify load during incidents. Restrict them to errors flagged retryable, use exponential backoff with full jitter, and cap attempts.

async function withRetry<T>(fn: () => Promise<T>, max = 3): Promise<T> {
  let attempt = 0;
  while (true) {
    try {
      return await fn();
    } catch (err) {
      attempt++;
      const retryable = err instanceof UpstreamError && err.retryable;
      if (attempt >= max || !retryable) throw err;
      const base = Math.min(100 * 2 ** attempt, 2000);
      const jitter = base * (0.5 + Math.random());
      await new Promise(r => setTimeout(r, jitter));
    }
  }
}

Call it inside your route handler: const data = await withRetry(() => callLLM(body));. Tradeoff: retries add tail latency. Under a sustained provider outage, a 3-attempt retry at 2s backoff means a 6s worst-case before your client sees a 503. That may be acceptable for a chat endpoint but not for a synchronous classification call.

5. Add a Circuit Breaker

When a provider is degraded, retries hammer it further. A circuit breaker opens after a failure threshold and fails fast, giving the provider room to recover.

import CircuitBreaker from 'opossum';

const breaker = new CircuitBreaker(
  (body: unknown) => callLLM(body),
  { timeout: 10000, errorThresholdPercentage: 50, resetTimeout: 30000 }
);

breaker.fallback(() => {
  throw new UpstreamError(503, true, 'circuit_open');
});

Use breaker.fire(req.body) instead of calling callLLM directly. The breaker counts thrown UpstreamError as failures; make sure you do not count 4xx client mistakes against it, or you will false-trip. Tune errorThresholdPercentage against your traffic—a low-traffic endpoint needs a lower absolute request volume to open.

6. Map Errors to Safe Client Responses

Your users should never see a provider stack trace or an Anthropic API key error. The terminal middleware translates UpstreamError into a generic status and optional Retry-After.

app.use((err: Error, req: Request, res: Response, _next: NextFunction) => {
  if (err instanceof UpstreamError) {
    const status = err.retryable ? 503 : 502;
    res.set('Retry-After', err.retryable ? '5' : '0');
    return res.status(status).json({
      error: 'upstream_llm_error',
      message: err.retryable ? 'provider unavailable' : 'provider rejected request'
    });
  }
  res.status(500).json({ error: 'internal_error' });
});

Set Content-Type: application/json explicitly if you suspect middleware ordering issues. A frequent bug: returning a 200 with { error: ... } because the catch block called res.json instead of next. That breaks OpenAI-compatible clients expecting a completion object.

7. Observability and Metering

Log the error with a correlation ID and the provider name. If you retry three times, you burned three times the tokens of a single call; track that. When you proxy through a gateway, per-token usage metering lets you attribute those retry costs to the right tenant. Structured logs should include provider, status, retryable, and attempt.

console.error({
  ts: Date.now(),
  reqId: req.id,
  provider: err.provider,
  status: err.status,
  retryable: err.retryable
});

Without this data, you cannot tell whether your express.js error handling llm provider failures is masking a provider outage or your own payload bugs.

8. Offload Fallback to a Resilient Gateway

Building retries, circuit breakers, and per-provider normalizers inside Express is necessary if you talk to providers directly. One architectural tradeoff is moving that logic upstream. An OpenRouter-class inference gateway such as n4n.ai provides a single OpenAI-compatible endpoint across 240+ models with automatic fallback when a provider is rate-limited or degraded. That removes a class of express.js error handling llm provider failures from your app: you still enforce timeouts and map the gateway’s errors, but you stop writing custom fallback trees for Azure vs. OpenAI vs. Anthropic. It is not a silver bullet—the gateway can still return 503—but it shrinks your surface area.

Common Pitfalls

  • Swallowed async errors: forgetting .catch(next) or asyncHandler leaves requests pending until timeout.
  • Wrong status codes: returning 200 with an error body breaks SDKs that check response.ok.
  • Retrying non-idempotent calls: chat completions are safe to retry; billing or fine-tune job submission are not.
  • Breaker misconfiguration: counting 400s as failures opens the breaker on your own bugs.
  • Leaked details: forwarding err.stack from a provider into the client response.

Tradeoffs Summary

Implementing full express.js error handling llm provider failures in-house gives you fine-grained control over retry budgets and provider selection, but costs engineering time and adds latency under failure. Offloading fallback to a gateway simplifies code but introduces a new dependency and a single point of failure you must still handle. Whichever path you choose, the non-negotiables are: centralized error middleware, explicit error classification, hard timeouts, and client-safe responses. Everything else is tuning.

Tagsexpressjserror-handlingreliabilityllm-api

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →