LLM API status codes explained: they are the HTTP status codes returned by model inference endpoints to indicate whether a request was accepted, rejected, or failed mid-execution. While they inherit standard HTTP semantics, the practical triggers—token limits, content policy violations, provider rate limits—differ from classic web APIs and dictate your retry logic.
How LLM APIs signal errors
Most inference gateways expose an OpenAI-compatible REST surface. A request to /v1/chat/completions returns 200 OK with a streaming or JSON body on success. On failure, the endpoint returns a non-2xx status and a JSON body shaped like:
{
"error": {
"message": "context length exceeded",
"type": "invalid_request_error",
"code": "context_length_exceeded"
}
}The status code tells you the class of failure; the error.code field tells you the specific trigger. Treat the status as the primary branch in your error handler, and the code as the secondary detail.
4xx: client-side errors
These mean your request was rejected before or during validation. The server will not process it as-is. Fix the request or change your parameters.
400 Bad Request
The most common 4xx for LLMs. Causes include malformed JSON, an unknown parameter, temperature outside [0,2], or exceeding the model’s context window. OpenAI-compatible APIs return context_length_exceeded inside a 400.
import requests
resp = requests.post(
"https://api.example.com/v1/chat/completions",
json={"model": "gpt-4o", "messages": [{"role": "user", "content": "x" * 200000}]},
headers={"Authorization": "Bearer KEY"}
)
if resp.status_code == 400:
print(resp.json()["error"]["code"]) # context_length_exceeded401 Unauthorized
Missing or invalid API key. The gateway could not authenticate you. Rotate the key or check your header.
403 Forbidden
Authentication succeeded but the account or API key lacks permission for the model or route. This also appears when a content filter blocks the request pre-flight, though many providers return 400 for that.
404 Not Found
The model ID does not exist on the endpoint. If you hardcoded "gpt-5-turbo" and the gateway only has "gpt-4o", you get 404. Always validate model lists at startup.
422 Unprocessable Entity
Not universal in LLM APIs, but some gateways use it for semantically invalid input that passed JSON parsing—e.g., a max_tokens value larger than the model supports but syntactically valid. If you see 422, inspect schema validation logs.
429 Too Many Requests
Rate limit or quota exhaustion. The error body often includes retry_after in seconds. This is operational, not a code bug.
A gateway like n4n.ai implements automatic fallback when a provider is rate-limited or degraded, but will still return 429 when all routes are exhausted. Respect the Retry-After header.
curl -i https://api.example.com/v1/chat/completions \
-H "Authorization: Bearer $KEY" \
-d '{"model":"claude-3-5-sonnet","messages":[]}'
# HTTP/2 429
# retry-after: 25xx: server-side errors
These indicate the provider or gateway failed to produce a response. They are usually transient, but repeated 5xx suggest a downstream outage.
500 Internal Server Error
Generic upstream failure. The model worker crashed, or the gateway hit an unhandled exception. Safe to retry with backoff.
502 Bad Gateway
The gateway received an invalid response from the model backend—often a truncated stream or a malformed protobuf from a inference server. Common during provider deployments.
503 Service Unavailable
The provider is overloaded or in scheduled maintenance. Unlike 429, this is not about your quota; the whole fleet is saturated.
504 Gateway Timeout
The request exceeded the gateway’s upstream timeout. For long generations, the model may still be computing, but the proxy gave up. Reduce max_tokens or increase client timeout if the gateway allows.
Concrete handling example
A production client should distinguish retryable from non-retryable codes. Here is a minimal Python pattern:
import time
import requests
def complete(payload, api_key, max_retries=3):
for attempt in range(max_retries):
r = requests.post(
"https://api.example.com/v1/chat/completions",
json=payload,
headers={"Authorization": f"Bearer {api_key}"},
timeout=30
)
if r.status_code == 200:
return r.json()
if r.status_code in (400, 401, 403, 404, 422):
# non-retryable: fix the request
raise RuntimeError(f"Client error {r.status_code}: {r.text}")
if r.status_code in (429, 500, 502, 503, 504):
backoff = int(r.headers.get("retry-after", 2 ** attempt))
time.sleep(backoff)
continue
raise RuntimeError("Exhausted retries")The key is that 4xx except 429 are terminal; 429 and all 5xx are retryable with backoff.
Common misconceptions
“Any 4xx is a bug in my code.” Wrong. 429 is a capacity signal, not a defect. 403 may be a policy change on the provider side.
“A 200 means the model gave a useful answer.” No. You can get a 200 with an empty completion, a refusal, or a hallucinated JSON blob. Always validate the choices[0].message.content against your schema.
“5xx means the model is permanently down.” Almost never. LLM backends autoscale; a 503 at 14:00 UTC often clears in seconds. Exponential backoff with jitter handles this.
“Streaming avoids status codes.” False. The initial POST returns the status before the body streams. If you get 200, the stream may still break mid-way with a network error—handle that separately.
“I can ignore 422 because OpenAI doesn’t use it.” If you talk to multiple providers through one gateway, schema differences appear. A 422 from one backend is a 400 from another. Code defensively.
Retry and fallback patterns in TypeScript
For edge runtimes, use fetch with a bounded retry and a circuit breaker:
async function chatWithRetry(body: any, key: string): Promise<Response> {
let attempt = 0;
while (attempt < 4) {
const res = await fetch("https://api.example.com/v1/chat/completions", {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: `Bearer ${key}` },
body: JSON.stringify(body),
});
if (res.ok) return res;
if ([429, 500, 502, 503, 504].includes(res.status)) {
const retryAfter = Number(res.headers.get("retry-after")) || 2 ** attempt;
await new Promise((r) => setTimeout(r, retryAfter * 1000));
attempt++;
continue;
}
throw new Error(`Fatal ${res.status}`);
}
throw new Error("Retry budget exhausted");
}If you route across providers, wrap this in a loop that swaps the model field or base URL on persistent 503. That is where gateway-level fallback earns its keep.
Why this matters in production
LLM calls sit in request paths that touch users. A 429 that triggers a tight retry loop amplifies load and gets you blocked longer. A 500 that throws an unhandled exception returns a 502 to your own users. Map each status to a explicit action: terminal log, retry with backoff, or provider switch. The llm api status codes explained here are the contract you build reliability on—ignore them and you ship flicker.