Handling rate limits in PHP LLM API clients

When you build a PHP service that calls an LLM, the fastest way to production outages is ignoring the 429 responses. Handling php rate limit llm api errors correctly means distinguishing transient provider throttling from hard quotas, and retrying with backoff while respecting token budgets. This guide walks through a concrete retry stack you can drop into a Laravel app or plain PHP script.

Step 1: Identify the rate limit responses from your LLM provider

Most OpenAI-compatible endpoints return HTTP 429 with a JSON body describing the limit. Some providers also embed error details in a 200 response (rare) or return 400 with a specific error code like rate_limit_exceeded. Your client must treat any 429 as retryable throttling and should inspect the body only for debugging.

use GuzzleHttp\Client;
use GuzzleHttp\Exception\RequestException;

$client = new Client(['base_uri' => 'https://api.example.com/v1/']);

try {
    $resp = $client->post('chat/completions', [
        'json' => ['model' => 'gpt-4o', 'messages' => [['role' => 'user', 'content' => 'hi']]]
    ]);
} catch (RequestException $e) {
    if ($e->hasResponse() && $e->getResponse()->getStatusCode() === 429) {
        $body = json_decode((string) $e->getResponse()->getBody(), true);
        // log $body['error']['message'] but do not surface to user
    }
}

If you use a gateway such as n4n.ai, the same OpenAI-compatible request shape applies, but the gateway may return 429 only when all upstream providers are saturated; otherwise it fails over automatically. That changes your retry math but not the fundamental need to handle the status code.

Step 2: Implement exponential backoff with jitter

Naive sleep(1) retries hammer the provider and extend the penalty. Use exponential backoff capped at a max, with full jitter to avoid thundering herd. The core of php rate limit llm api resilience is a retry wrapper that catches only 429 and leaves other exceptions to bubble up.

class LlmRetry {
    public function __construct(private int $maxRetries = 5, private int $capMs = 5000) {}

    public function call(callable $fn): mixed {
        $attempt = 0;
        while (true) {
            try {
                return $fn();
            } catch (RequestException $e) {
                $status = $e->hasResponse() ? $e->getResponse()->getStatusCode() : 0;
                if ($status !== 429 || $attempt >= $this->maxRetries) {
                    throw $e;
                }
                $base = 2 ** $attempt;
                $delay = min($this->capMs, $base * 250) + random_int(0, 250);
                usleep($delay * 1000);
                $attempt++;
            }
        }
    }
}

The $delay starts at 250 ms, doubles each attempt, and adds up to 250 ms random spread. For php rate limit llm api calls that are user-facing, keep $maxRetries low (3) and fail fast; for background workers, 5–8 retries over ~30 s is reasonable.

Step 3: Honor Retry-After and rate limit headers

Providers often send Retry-After (seconds or HTTP date) and X-RateLimit-Remaining. Ignoring Retry-After will get you banned or pushed into a longer cool-down. Parse it before falling back to jitter.

$response = $e->getResponse();
$retryAfter = $response->getHeaderLine('Retry-After');
if ($retryAfter !== '') {
    $secs = is_numeric($retryAfter)
        ? (int) $retryAfter
        : (new DateTime($retryAfter))->getTimestamp() - time();
    usleep(max(0, $secs) * 1000 * 1000);
} else {
    // use jitter backoff from Step 2
}

Persist X-RateLimit-Remaining in a shared cache (Redis or APCu) to preemptively throttle before the 429 arrives. This is critical for high-throughput php rate limit llm api workers that fan out across multiple PHP-FPM children.

Step 4: Add client-side throttling with Redis

In PHP-FPM or Laravel, each request is isolated. Use a shared Redis to enforce a token bucket per API key. The bucket refills at a fixed rate; if empty, reject locally instead of hitting the network.

class TokenBucket {
    public function __construct(
        private $redis,
        private string $key,
        private int $capacity,
        private int $refillPerSec
    ) {}

    public function consume(int $tokens = 1): bool {
        $now = microtime(true);
        $script = <<<'LUA'
        local data = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
        local tokens = tonumber(data[1]) or ARGV[2]
        local ts = tonumber(data[2]) or ARGV[1]
        local delta = math.min(ARGV[2], (ARGV[1]-ts)*ARGV[3] + tokens)
        if delta < tonumber(ARGV[4]) then return 0 end
        redis.call('HMSET', KEYS[1], 'tokens', delta-ARGV[4], 'ts', ARGV[1])
        return 1
        LUA;
        return (bool) $this->redis->eval($script, 1, $this->key,
            $now, $this->capacity, $this->refillPerSec, $tokens);
    }
}

Instantiate it with the per-minute quota from your provider docs. Call consume() before each LLM request. If false, either usleep briefly or push the work to a queue. This converts blind php rate limit llm api hits into predictable local rejections and keeps your error rate visible.

Step 5: Leverage a gateway with automatic fallback

If you are multi-model, you will trip provider-specific quotas constantly. An inference gateway that exposes one OpenAI-compatible endpoint and automatically falls back when a provider is rate-limited or degraded removes most retry logic from your code. You still wrap calls in the backoff from Step 2, but the 429 rate from the gateway is far lower because it shifts traffic to a healthy upstream.

The trade-off: you now depend on the gateway’s health and its routing directives. Honoring client-side cache-control hints and per-token metering becomes a billing concern, not a throttle concern. Keep your own token bucket anyway—gateways can still return 429 during global saturation.

Step 6: Integrate with Laravel’s queue system

For Laravel apps, push LLM calls to a queue so HTTP workers aren’t blocked. Use the built-in retry with backoff in config/queue.php and a custom job class.

// app/Jobs/CallLlmJob.php
class CallLlmJob implements ShouldQueue
{
    public $tries = 5;
    public $backoff = [1, 2, 4, 8, 16]; // seconds

    public function handle()
    {
        $client = app(Client::class);
        $retry = new LlmRetry(3);
        $response = $retry->call(
            fn() => $client->post('chat/completions', ['json' => $this->payload])
        );
        // store $response->getBody() wherever needed
    }

    public function failed(Throwable $e)
    {
        logger()->error('LLM call failed after retries', ['e' => $e]);
    }
}

The $backoff array gives Laravel-level delays between job releases; combine with the in-process retry for defense in depth. This pattern is the standard way to absorb php rate limit llm api spikes in a web app without dropping user requests. Make jobs idempotent—LLM calls are not free, and a duplicate job after a timeout wastes tokens.

Step 7: Verify your implementation

You must prove the retry path works. Mock the API with a local PHP server that returns 429 for the first three requests.

<?php
// mock_dir/index.php
session_start();
$_SESSION['count'] = ($_SESSION['count'] ?? 0) + 1;
if ($_SESSION['count'] <= 3) {
    header('HTTP/1.1 429 Too Many Requests');
    header('Retry-After: 0');
    echo json_encode(['error' => 'rate limit']);
    exit;
}
echo json_encode(['ok' => true]);

Run it with php -S localhost:8000 -t ./mock_dir and point your client base_uri there. Success is defined as: the client logs two 429 catches, waits the jitter interval, then returns {"ok":true} on the fourth attempt.

For automated coverage, use Guzzle’s MockHandler:

$mock = new MockHandler([
    new Response(429, ['Retry-After' => '0']),
    new Response(429, ['Retry-After' => '0']),
    new Response(200, [], '{"ok":true}')
]);
$client = new Client(['handler' => $mock]);
$result = (new LlmRetry(3))->call(fn() => $client->get('/'));

If the test passes without throwing, your php rate limit llm api handling is correct. Add an assertion on the number of attempts to catch silent bypasses.

Step 8: Monitor and alert on throttle rate

Emit a metric for 429 count per model and per endpoint. In Laravel, use Log::info or a Prometheus client. If throttle rate exceeds 5% of total calls, your client-side bucket (Step 4) is mis-sized or the provider changed quotas. Adjust capacity or move heavy jobs to batch processing.

Rate limits are not an edge case; they are the steady state for any LLM integration at scale. The steps above give you a layered defense: detect, back off, respect server hints, throttle locally, use a gateway, queue the rest, and prove it with tests. Do that and php rate limit llm api errors become a background metric instead of a pager alert.