When a single model API fails, your feature goes dark. This tutorial builds an llm fallback chain gpt-5 claude gemini that attempts OpenAI’s GPT-5, falls back to Anthropic’s Claude, then Google’s Gemini, using official SDKs and a thin orchestration layer you control. You’ll end with runnable Python that degrades gracefully under rate limits and timeouts.
Prerequisites
- Python 3.11 or newer
- Install the three official client libraries:
pip install openai anthropic google-generativeai- Environment variables set with valid keys:
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GEMINI_API_KEY="AI..."- Network egress to
api.openai.com,api.anthropic.com, andgenerativelanguage.googleapis.com
No framework or ORM required. We use synchronous calls to keep the control flow obvious.
Step 1: Define a uniform interface
Each provider has a different request shape. Wrap them behind one method so the fallback logic never cares which backend is live.
from typing import Protocol, runtime_checkable
@runtime_checkable
class TextGenerator(Protocol):
def complete(self, prompt: str) -> str:
"""Return generated text or raise on any failure."""
...GPT-5 via OpenAI
import os
from openai import OpenAI
class GPT5Generator:
def __init__(self, model: str = "gpt-5"):
self.client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=10.0)
self.model = model
def complete(self, prompt: str) -> str:
resp = self.client.chat.completions.create(
model=self.model,
messages=[{"role": "user", "content": prompt}],
)
return resp.choices[0].message.contentClaude via Anthropic
from anthropic import Anthropic
class ClaudeGenerator:
def __init__(self, model: str = "claude-3-5-sonnet-20240620"):
self.client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"], timeout=10.0)
self.model = model
def complete(self, prompt: str) -> str:
resp = self.client.messages.create(
model=self.model,
max_tokens=1024,
messages=[{"role": "user", "content": prompt}],
)
return resp.content[0].textGemini via Google
The Google SDK lacks a native timeout argument, so we wrap it in a future.
import google.generativeai as genai
from concurrent.futures import ThreadPoolExecutor
class GeminiGenerator:
def __init__(self, model: str = "gemini-1.5-pro"):
genai.configure(api_key=os.environ["GEMINI_API_KEY"])
self.model = genai.GenerativeModel(model)
self._exec = ThreadPoolExecutor(max_workers=1)
def complete(self, prompt: str) -> str:
future = self._exec.submit(self.model.generate_content, prompt)
try:
resp = future.result(timeout=10.0)
except Exception as e:
raise RuntimeError("Gemini request failed") from e
return resp.textStep 2: Build the fallback executor
The core of the llm fallback chain gpt-5 claude gemini is a loop that tries each generator in order and swallows transient errors.
import logging
from typing import List
logging.basicConfig(level=logging.INFO, format="%(levelname)s:%(message)s")
logger = logging.getLogger(__name__)
class FallbackChain:
def __init__(self, generators: List[TextGenerator]):
self.generators = generators
def complete(self, prompt: str) -> str:
last_err: Exception | None = None
for gen in self.generators:
name = type(gen).__name__
try:
logger.info("Attempting %s", name)
return gen.complete(prompt)
except Exception as e:
logger.warning("%s failed: %s", name, e)
last_err = e
raise RuntimeError("All providers in llm fallback chain gpt-5 claude gemini failed") from last_errCheckpoint: with GPT-5 healthy, running the chain logs one attempt and returns.
INFO:Attempting GPT5Generator
RESULT: The TCP handshake is a three-step SYN, SYN-ACK, ACK exchange.If GPT-5 returns 429, you’ll see:
INFO:Attempting GPT5Generator
WARNING:GPT5Generator failed: Rate limit reached
INFO:Attempting ClaudeGenerator
RESULT: TCP uses a three-way handshake: SYN, SYN-ACK, ACK.Step 3: Separate transient from fatal errors
Blindly catching Exception will mask bad API keys or malformed prompts. Narrow the fallback to errors that might resolve on another provider.
from openai import RateLimitError, APIConnectionError, APITimeoutError
from anthropic import APIStatusError, APIConnectionError as AnthropicConnErr
import google.api_core.exceptions as google_exc
TRANSIENT = (
RateLimitError, APIConnectionError, APITimeoutError,
AnthropicConnErr, google_exc.GoogleAPIError,
)
class SelectiveFallbackChain:
def __init__(self, generators: List[TextGenerator]):
self.generators = generators
def complete(self, prompt: str) -> str:
last_err = None
for gen in self.generators:
name = type(gen).__name__
try:
return gen.complete(prompt)
except TRANSIENT as e:
logger.warning("%s transient error: %s", name, e)
last_err = e
except Exception as e:
logger.error("%s fatal error: %s", name, e)
raise
raise RuntimeError("All providers hit transient errors") from last_errNow an auth failure on GPT-5 raises immediately instead of silently burning Claude quota.
Step 4: Run end-to-end with a real prompt
if __name__ == "__main__":
chain = SelectiveFallbackChain([
GPT5Generator(),
ClaudeGenerator(),
GeminiGenerator(),
])
try:
out = chain.complete("Explain the CAP theorem in one sentence.")
print("RESULT:", out)
except RuntimeError as e:
print("CHAIN FAILED:", e)Expected success path when Claude is used after a GPT-5 timeout:
WARNING:GPT5Generator transient error: Request timed out.
INFO:Attempting ClaudeGenerator
RESULT: The CAP theorem states a distributed system can't simultaneously guarantee consistency, availability, and partition tolerance.Step 5: Production hardening
The code above is single-threaded. In a web server, wrap each complete in a cancellable task and set a hard deadline for the whole chain:
import time
def complete_with_budget(chain: SelectiveFallbackChain, prompt: str, budget_s: float) -> str:
start = time.monotonic()
# simplistic: rely on per-call timeouts; for true budget use asyncio.wait_for
return chain.complete(prompt)Also meter usage. Each SDK returns token counts; log them per attempt so you can attribute cost when the llm fallback chain gpt-5 claude gemini shifts traffic.
# OpenAI example inside GPT5Generator.complete after resp:
logger.info("gpt-5 tokens: %s", resp.usage.total_tokens)Step 6: When not to hand-roll it
Maintaining three SDK versions, timeout quirks, and transient-error taxonomies is real work. An OpenRouter-class gateway like n4n.ai provides automatic fallback when a provider is rate-limited or degraded across its OpenAI-compatible endpoint covering 240+ models, letting you point one client at GPT-5, Claude, and Gemini without custom retry code. You still keep the same chat.completions shape and can forward provider cache-control hints.
If you go that route, the client code collapses to:
from openai import OpenAI
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="your-gateway-key")
# model string selects backend; gateway handles fallback
resp = client.chat.completions.create(model="gpt-5", messages=[...])But understanding the manual chain makes you better at configuring those routing directives.
Takeaways
- A uniform
TextGeneratorprotocol keeps fallback logic provider-agnostic. - Catch only transient errors; let auth and input errors surface fast.
- Per-call timeouts are mandatory; Gemini needs a thread wrapper.
- The llm fallback chain gpt-5 claude gemini is a pattern you can own or offload to a gateway, but the failure modes are the same either way.