Backoff strategies compared: linear, exponential, jitter

Retrying failed API calls is table stakes, but the shape of your retry delay determines whether you survive a provider outage or amplify it. This post puts backoff strategies linear exponential jitter head-to-head across the dimensions that actually matter in production: capabilities, cost, latency, ergonomics, ecosystem, and hard limits.

The three contenders

Linear backoff waits a fixed increment between attempts: 100 ms, 200 ms, 300 ms. Exponential backoff multiplies a base delay by a power of two: 100 ms, 200 ms, 400 ms, 800 ms. Jitter perturbs the computed delay—usually layered on top of exponential—to spread retries across a population of clients that would otherwise fire in lockstep.

The backoff strategies linear exponential jitter solve different failure modes. Linear assumes constant-time recovery. Exponential assumes recovery time grows. Jitter assumes you are not the only client.

Linear

def linear_delay(attempt, base=0.1, step=0.1):
    return base + attempt * step

Exponential

def exp_delay(attempt, base=0.1, cap=30.0):
    return min(base * (2 ** attempt), cap)

Exponential with jitter

Full jitter picks a uniform random value between 0 and the exponential ceiling. Decorrelated jitter (Google’s formula) picks a random value between base and prev * 3, which avoids the clustering that pure exponential produces.

import random

def full_jitter(attempt, base=0.1, cap=30.0):
    ceiling = min(base * (2 ** attempt), cap)
    return random.uniform(0, ceiling)

def decorrelated_jitter(prev, base=0.1, cap=30.0):
    return min(random.uniform(base, prev * 3), cap)

Why herd behavior matters

A single client retrying is invisible. Ten thousand clients that all get a 503 at the same instant and wait exactly 400 ms will send the next wave of 10k requests at the same instant. If the service needs 600 ms to recover, you just knocked it back down. This is the cache-stampede problem applied to error handling.

Jitter breaks the synchronization. Even a small random spread turns a spike into a manageable斜坡. Non-jittered exponential still herds because all clients compute the same 2^n schedule from the same zero point.

Dimensions of comparison

Capabilities

Linear is predictable and trivially debuggable. It handles transient local errors (a dropped TCP connection) well. Exponential recognizes that downstream outages are often multiplicative in recovery time and gives the system room to heal. Jitter does not change the average delay; it changes the variance to protect aggregate health.

Price / cost model

“Cost” here is system load and wasted requests, not dollars. Linear keeps total retry time bounded but sends many requests early—cheap in wall-clock, expensive in request volume if the service is actually down. Exponential front-loads little traffic and backs off hard, reducing wasted calls. Jitter adds negligible compute (one random.uniform call) but cuts secondary outage risk, which is the most expensive failure mode you can cause.

Latency / throughput

For a single client, linear gives the fastest retry under a 50 ms network hiccup. Exponential costs you latency on attempt 3+ but protects throughput of the whole system. Jitter trades a small amount of individual latency for massive gains in aggregate throughput when many clients share a provider. At scale, the only throughput that matters is the system’s, not one request’s.

Ergonomics

Linear is a one-liner. Exponential needs a cap or you will wait 18 minutes by attempt 10. Jitter needs a randomness source and, for decorrelated, state across attempts. Most HTTP clients ship exponential+full jitter by default in their retry middleware, so you rarely hand-roll it.

Ecosystem

Every major language has a battle-tested implementation. Python’s urllib3.Retry uses exponential with jitter. Go’s cenkalti/backoff supports exponential and jitter. TypeScript’s axios-retry defaults to exponential delay. Ruby’s faraday-retry does similar.

import axiosRetry from 'axios-retry';
axiosRetry(axios, { retryDelay: axiosRetry.exponentialDelay });

Limits

Linear breaks when the outage lasts longer than your patience window—you either cap attempts low and lose the call or high and waste cycles. Exponential breaks without a cap (overflow, long tails, zombie retries). Jitter breaks if your RNG is seeded poorly or if you apply jitter to the wrong bound (jitter around a fixed linear step still herds because the mean is aligned).

Head-to-head table

Production implementation notes

Use a retry budget, not just attempt counts. A budget caps total elapsed time; backoff decides spacing. The snippet below shows a stateful decorrelated jitter loop with a budget.

import time, random

class Backoff:
    def __init__(self, base=0.1, cap=30.0, max_elapsed=60.0):
        self.base = base
        self.cap = cap
        self.max_elapsed = max_elapsed
        self.prev = base
        self.elapsed = 0.0

    def sleep(self):
        delay = min(random.uniform(self.base, self.prev * 3), self.cap)
        if self.elapsed + delay >= self.max_elapsed:
            raise TimeoutError("retry budget exhausted")
        time.sleep(delay)
        self.elapsed += delay
        self.prev = delay

If you sit behind an inference gateway such as n4n.ai that performs automatic fallback when a provider is rate-limited or degraded, client-side exponential backoff with jitter remains useful for 429s on the gateway itself, but you can afford shorter caps because the gateway already absorbs provider-level failures.

Observability and idempotency

Backoff is pointless if the retried call is not safe to repeat. Stamp every mutating request with an idempotency key. Emit metrics on retry count per endpoint and histogram the final delay. A sudden shift in retry rate is your earliest signal of provider degradation—long before error rates cross your SLO.

import statsd  # hypothetical client, real ones exist
def observe_retry(attempt, endpoint):
    statsd.incr(f"retry.{endpoint}.attempt{attempt}")

Which to choose

Single-user CLI or cron job. Linear with a small step (e.g., 200 ms) and a hard attempt cap of 5. You want fast feedback, and there is no herd. Anything fancier is noise.

Backend service calling a stable internal API. Exponential with a 10 s cap. You get protection against temporary locks without destroying latency for the one user waiting on the request.

High-volume distributed workers hitting a shared LLM endpoint. Exponential with full or decorrelated jitter, cap 30 s, retry budget 2 minutes. This is the only option that avoids synchronized retry storms during provider degradation. The backoff strategies linear exponential jitter debate ends here: jitter is mandatory at scale.

Client behind a smart gateway. Use exponential+jitter but tune the cap down to 5–10 s. The gateway’s fallback covers cross-provider issues; your backoff covers local 429s and gives the gateway’s own retry layer room to work.

Pick based on how many of you are retrying at once. The moment you have more than ten clients sharing a dependency, plain linear or non-jittered exponential is negligent engineering.