How often do LLM providers hit 99.9% uptime?

Most inference vendors publish a LLM provider 99.9% uptime SLA, but the number on the status page rarely matches what your retry logic experiences at 3 a.m. The gap between contractual availability and practical request success stems from how providers define “uptime”, how they exclude scheduled maintenance, and how token-streaming failures are counted.

What the SLA actually covers

A 99.9% monthly uptime SLA permits about 43.8 minutes of downtime per 30-day window. That sounds small until you realize the clock often starts only after a provider confirms an incident. Many contracts measure availability via synthetic health checks hitting a trivial endpoint, not the expensive chat completion route your product uses.

If the health check passes but the model load balancer throws 503s for prompts over 4K tokens, the SLA remains unbreached. The LLM provider 99.9% uptime SLA is a narrow contractual claim, not a guarantee that your specific workload will succeed.

Control plane vs data plane

Providers operate two distinct surfaces: the control plane (authentication, model listing, quota endpoints) and the data plane (the inference path that streams tokens). Outages in the control plane are rare and quickly visible. Data plane degradation is where the real pain lives.

A data plane can be “up” in the sense that it accepts connections, but exhibit elevated latency or drop streams under load. We have seen cases where p99 latency triples while error rates stay under 0.1%—technically meeting the LLM provider 99.9% uptime SLA but unusable for interactive apps.

How downtime is counted (and excluded)

Standard SLA exclusions:

• Scheduled maintenance (often with 24h notice)
• Your client-side network failures
• Provider-side throttling due to your exceeding burst limits
• Third-party dependencies (e.g., CDN outages)

If a provider rotates GPUs weekly at 02:00 UTC for 10 minutes, and you are in Asia, that is 40 minutes of your prime-time downtime not counted against their SLA. The LLM provider 99.9% uptime SLA math ignores these windows. SLAs are computed per calendar month; a catastrophic day in week one is averaged with three quiet weeks, masking acute pain.

Real failure modes engineers see

Streaming resets

You call stream=True, receive 200 OK, get 30 tokens, then the connection dies with EOFError. The HTTP status was success; the SLA is intact. Your user sees a half-rendered answer.

Silent partial responses

Some providers return a complete HTTP 200 but truncate the finish_reason to length without warning when internal buffers overflow. Parsing code that assumes stop will hang or produce broken JSON. Example truncated payload:

{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "delta": {"content": "The query plan is"},
      "finish_reason": null
    }
  ]
}

The stream ends here with no finish_reason. Your client must detect the abrupt close.

Rate limits as downtime

A 429 is not “downtime” in SLA terms. But if you are rate-limited for 20 minutes because a neighbor tenant spiked, your effective uptime is zero for that period. The LLM provider 99.9% uptime SLA does not cover capacity starvation.

Status page lag

Providers update public status pages after internal confirmation, which can trail the actual incident by 10–30 minutes. If you rely on the page to trigger fallback, you are already late.

Measuring true reliability

You cannot improve what you do not measure. Instrument the client, not the provider status page.

Instrumenting your client

Wrap your calls and emit metrics on outcomes:

import time
import openai
from prometheus_client import Counter, Histogram

REQS = Counter('llm_requests', 'Total LLM requests', ['model', 'status'])
LAT = Histogram('llm_latency', 'LLM latency', ['model'])

def tracked_complete(client, **kwargs):
    start = time.time()
    try:
        resp = client.chat.completions.create(**kwargs)
        REQS.labels(model=kwargs['model'], status='ok').inc()
        return resp
    except openai.APIError as e:
        REQS.labels(model=kwargs['model'], status='error').inc()
        raise
    finally:
        LAT.labels(model=kwargs['model']).observe(time.time() - start)

Track not just exceptions but also finish_reason mismatches and stream byte counts. Log the raw response size; a 200 with zero tokens is a failure.

Defining success

A request is successful only if:

1. It returns a completion with finish_reason in {stop, tool_calls}.
2. For streams, total tokens > 0 and the stream closed cleanly.
3. Latency under your product SLO (e.g., < 5s to first token).

Anything else is a failure for your dashboard, regardless of provider SLA.

Designing for less than 99.9%

Assume the SLA will not save you. Build redundancy.

Fallback patterns

Use a primary and secondary provider with the same model family. An OpenAI-compatible gateway such as n4n.ai can automatically route around a degraded provider, but you should still encode explicit fallback in your client for cases where the gateway itself is unreachable.

import openai

clients = [
    openai.OpenAI(base_url="https://api.n4n.ai/v1", api_key="KEY"),
    openai.OpenAI(api_key="OPENAI_KEY"),
]

def complete_with_fallback(prompt):
    last_err = None
    for client in clients:
        try:
            return client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[{"role": "user", "content": prompt}],
                timeout=20,
            )
        except Exception as e:
            last_err = e
    raise last_err

This pattern adds latency on failure but keeps your p99 acceptable. Cancel the slower in-flight request when one returns to avoid double billing.

Cost and cache tradeoffs

Fallback doubles potential token cost if both calls fire. Provider-side prefix caches are invalidated when you switch endpoints; you lose the speedup from cached prompts. For high-volume RAG, that penalty may outweigh the rare outage.

Weigh: if your traffic is bursty, a single provider with aggressive retry and queue may be cheaper than multi-provider redundancy. If your product is user-facing and latency sensitive, the LLM provider 99.9% uptime SLA is insufficient and fallback is mandatory.

A note on gateway-level mitigation

Gateways that aggregate many models behind one endpoint can mask individual provider dips by honoring client routing directives and forwarding cache-control hints. That helps, but the fundamental constraint remains: if all providers behind a model family are degraded, no gateway conjures capacity.

Takeaway

The LLM provider 99.9% uptime SLA is a useful contractual floor, not an operational target. Measure real request success from your client, treat streaming resets and rate limits as downtime for your users, and implement explicit fallback for any path that cannot tolerate a 45-minute monthly blackout. Build for 99.0% real availability and you will survive the gap.