A production-grade swift rate limit llm api client can’t just fire requests and hope. Providers return HTTP 429 with Retry-After headers, shed load during traffic spikes, and enforce per-minute token quotas that vary by model. This guide walks through building a Swift client that detects limits, backs off correctly, throttles locally, and fails over when a provider is degraded.
Step 1: Parse rate limit signals from the response
Most LLM APIs signal throttling via standard HTTP semantics. You’ll see 429 Too Many Requests, optionally with Retry-After: 2 (seconds) or a Unix timestamp. Some include x-ratelimit-remaining and x-ratelimit-reset.
Define a Swift type to capture these:
struct RateLimitInfo {
let retryAfterSeconds: TimeInterval?
let remaining: Int?
let reset: Date?
init?(response: HTTPURLResponse) {
let headers = response.allHeaderFields
guard response.statusCode == 429 else { return nil }
if let ra = headers["Retry-After"] as? String,
let secs = Double(ra) {
retryAfterSeconds = secs
} else if let ra = headers["Retry-After"] as? String,
let ts = Double(ra) {
retryAfterSeconds = ts - Date().timeIntervalSince1970
} else {
retryAfterSeconds = nil
}
remaining = (headers["x-ratelimit-remaining"] as? String).flatMap(Int.init)
if let resetStr = headers["x-ratelimit-reset"] as? String,
let resetTs = Double(resetStr) {
reset = Date(timeIntervalSince1970: resetTs)
} else {
reset = nil
}
}
}If the gateway returns Retry-After as a delay, use it directly. If it returns a timestamp, compute the delta.
Step 2: Wrap requests with exponential backoff
Never retry a 429 immediately. Use exponential backoff with jitter, capped at a max interval. The swift rate limit llm api pattern below uses structured concurrency.
func requestWithRetry(
_ req: URLRequest,
maxAttempts: Int = 5,
baseDelay: TimeInterval = 1.0,
maxDelay: TimeInterval = 30.0
) async throws -> (Data, HTTPURLResponse) {
var attempt = 0
while true {
attempt += 1
let (data, response) = try await URLSession.shared.data(for: req)
guard let http = response as? HTTPURLResponse else {
throw URLError(.badServerResponse)
}
if http.statusCode != 429 {
return (data, http)
}
if attempt >= maxAttempts {
throw RateLimitError.exhaustedRetries
}
let info = RateLimitInfo(response: http)
let delay = info?.retryAfterSeconds
?? min(baseDelay * pow(2, Double(attempt-1)), maxDelay)
let jitter = Double.random(in: 0..<0.5)
try await Task.sleep(nanoseconds: UInt64((delay + jitter) * 1_000_000_000))
}
}
enum RateLimitError: Error { case exhaustedRetries }This honors server-directed delays and falls back to exponential backoff when headers are missing.
Step 3: Proactively throttle with a client-side token bucket
Reacting to 429s wastes latency. A token bucket actor pre-empts limits by allocating request permits locally.
actor TokenBucket {
private var tokens: Double
private let capacity: Double
private let refillRate: Double // tokens per second
private var lastRefill: Date
init(capacity: Double, refillRate: Double) {
self.capacity = capacity
self.tokens = capacity
self.refillRate = refillRate
self.lastRefill = Date()
}
func consume(_ n: Double = 1) async {
refill()
while tokens < n {
let deficit = n - tokens
let wait = deficit / refillRate
try? await Task.sleep(nanoseconds: UInt64(wait * 1_000_000_000))
refill()
}
tokens -= n
}
private func refill() {
let now = Date()
let elapsed = now.timeIntervalSince(lastRefill)
tokens = min(capacity, tokens + elapsed * refillRate)
lastRefill = now
}
}Before each call, await bucket.consume() in your swift rate limit llm api service layer. Size the bucket to the provider’s published RPM minus a safety margin.
Step 4: Add fallback when a provider is degraded
Single-provider clients break under regional outages. If you route through an inference gateway, you can offload failover. For example, n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and performs automatic fallback when a provider is rate-limited or degraded, so your Swift code only needs to handle the unified error shape.
If you prefer direct provider relationships, implement a fallback list:
struct LLMEndpoint {
let baseURL: URL
let apiKey: String
}
let endpoints = [
LLMEndpoint(baseURL: URL(string: "https://api.provider-a.com/v1")!, apiKey: "a"),
LLMEndpoint(baseURL: URL(string: "https://api.provider-b.com/v1")!, apiKey: "b")
]
func complete(prompt: String) async throws -> String {
for endpoint in endpoints {
var req = URLRequest(url: endpoint.baseURL.appendingPathComponent("chat/completions"))
req.httpMethod = "POST"
req.setValue("Bearer \(endpoint.apiKey)", forHTTPHeaderField: "Authorization")
req.setValue("application/json", forHTTPHeaderField: "Content-Type")
req.httpBody = try JSONEncoder().encode([
"model": "gpt-4o-mini",
"messages": [["role": "user", "content": prompt]]
])
do {
let (data, _) = try await requestWithRetry(req)
return try JSONDecoder().decode(Completion.self, from: data).choices.first!.message.content
} catch RateLimitError.exhaustedRetries {
continue // try next endpoint
}
}
throw RateLimitError.exhaustedRetries
}This loops only on hard rate-limit exhaustion, not on every 429.
Step 5: Meter usage and log limits
Per-token metering lets you spot throttling trends. Capture usage from the response and log x-ratelimit-remaining if present.
struct Usage: Decodable { let prompt_tokens: Int; let completion_tokens: Int }
func logUsage(data: Data, response: HTTPURLResponse) {
if let usage = try? JSONDecoder().decode(Usage.self, from: data) {
print("tokens: \(usage.prompt_tokens + usage.completion_tokens)")
}
if let rem = response.allHeaderFields["x-ratelimit-remaining"] as? String {
print("remaining: \(rem)")
}
}A swift rate limit llm api integration should forward these metrics to your observability stack, not just stdout.
Step 6: Verify the client behaves under load
You can’t trust retry logic without a controlled test. Stand up a local mock that returns 429 with Retry-After: 0 for the first two hits, then 200.
# Using a tiny Python mock server
python3 - <<'PY'
from http.server import BaseHTTPRequestHandler, HTTPServer
class H(BaseHTTPRequestHandler):
count = 0
def do_POST(self):
self.count += 1
if self.count <= 2:
self.send_response(429)
self.send_header('Retry-After', '0')
self.end_headers()
else:
self.send_response(200)
self.end_headers()
self.wfile.write(b'{"choices":[{"message":{"content":"ok"}}]}')
HTTPServer(('127.0.0.1', 8080), H).serve_forever()
PYPoint your URLSession at http://127.0.0.1:8080/v1/chat/completions and assert the third attempt returns “ok”. Add a unit test with XCTest that injects RateLimitInfo and validates delay calculation.
Success criteria:
- First two calls sleep briefly and retry.
- Third call returns data without error.
- Token bucket rejects or delays when capacity is zero.
- Fallback switches endpoint after exhaustive retries.
Practical notes for shipping
Keep your retry budget small. Five attempts with 30s cap is 90s worst-case latency; tune for UX. Use Task.cancel() propagation so a user leaving the screen stops pending retries. Honor cache-control hints if your gateway forwards them—some providers cache prompt prefixes and that reduces token counts, indirectly easing limits.
A swift rate limit llm api client is finished when it degrades gracefully: it throttles locally, backs off remotely, and fails over without dropping user input. Build the tests first.