When OpenAI’s API experiences a hard outage, your users shouldn’t see 500s. This guide walks through a concrete fallback openai outage to anthropic pattern you can implement in a few dozen lines of Python or TypeScript, with explicit error classification and a verification harness.
Step 1: Install dependencies and load credentials
Start with the official SDKs. Both expose thin HTTP clients with sensible defaults.
pip install openai anthropicnpm install openai @anthropic-ai/sdkStore keys in environment variables. Never bake them into source.
import os
from openai import OpenAI
from anthropic import Anthropic
openai_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
anthropic_client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])import OpenAI from "openai";
import Anthropic from "@anthropic-ai/sdk";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });The pattern assumes you control the model alias. Map gpt-4o to claude-3-5-sonnet-20241022 (or current Anthropic equivalent) so the fallback preserves capability roughly.
Step 2: Define a provider-agnostic message shape
OpenAI and Anthropic differ in how they handle system prompts and message roles. Normalize to a list of {"role": "user"|"assistant", "content": str} plus a separate system string.
from dataclasses import dataclass
from typing import List, Literal
@dataclass
class ChatRequest:
system: str
messages: List[Literal["user", "assistant"]]
max_tokens: int = 1024
temperature: float = 0.2In TypeScript:
type Role = "user" | "assistant";
interface ChatMessage { role: Role; content: string; }
interface ChatRequest {
system: string;
messages: ChatMessage[];
maxTokens: number;
temperature: number;
}Keeping this shape lets you write one translation layer instead of scattering provider specifics across your codebase.
Step 3: Call OpenAI with tight timeouts and classified errors
A “fallback openai outage to anthropic” strategy is useless if the OpenAI client hangs for 30 seconds per retry. Set a 5-second timeout and catch only the errors that indicate provider-side failure.
from openai import APIConnectionError, APITimeoutError, InternalServerError, RateLimitError
def call_openai(req: ChatRequest, retries: int = 2):
attempt = 0
while attempt <= retries:
try:
resp = openai_client.chat.completions.create(
model="gpt-4o",
messages=[{"role": m["role"], "content": m["content"]} for m in req.messages],
max_tokens=req.max_tokens,
temperature=req.temperature,
timeout=5.0,
)
return {"provider": "openai", "text": resp.choices[0].message.content}
except (APIConnectionError, APITimeoutError, InternalServerError, RateLimitError) as e:
attempt += 1
if attempt > retries:
raiseNote: RateLimitError is included because during partial degradation OpenAI returns 429s. You may want to treat 429 differently from 500, but for outage fallback, routing to Anthropic is safe.
TypeScript equivalent uses error instanceof OpenAI.APIError and checks error.status.
Step 4: Translate and call Anthropic
Anthropic’s Messages API takes system as a top-level param and does not accept a system role message. Also, it requires max_tokens (required). Map roles directly.
def call_anthropic(req: ChatRequest):
resp = anthropic_client.messages.create(
model="claude-3-5-sonnet-20241022",
system=req.system,
messages=req.messages,
max_tokens=req.max_tokens,
temperature=req.temperature,
)
return {"provider": "anthropic", "text": resp.content[0].text}async function callAnthropic(req: ChatRequest) {
const resp = await anthropic.messages.create({
model: "claude-3-5-sonnet-20241022",
system: req.system,
messages: req.messages,
max_tokens: req.maxTokens,
temperature: req.temperature,
});
return { provider: "anthropic", text: resp.content[0].text };
}Anthropic returns content as an array of blocks; text is at content[0].text for standard completions.
Step 5: Orchestrate the fallback
Now wire a function that tries OpenAI, and on exhausted retries, calls Anthropic. Add a simple in-memory circuit breaker so a sustained outage doesn’t hammer OpenAI.
import time
_circuit_open = False
_circuit_opened_at = 0
def chat_with_fallback(req: ChatRequest):
global _circuit_open, _circuit_opened_at
if _circuit_open and time.time() - _circuit_opened_at < 60:
# skip OpenAI entirely
return call_anthropic(req)
try:
return call_openai(req)
except Exception:
_circuit_open = True
_circuit_opened_at = time.time()
return call_anthropic(req)This implements the core fallback openai outage to anthropic flow. In TS, use a module-level variable.
For production, externalize the circuit state to Redis so multiple workers share it.
Step 6: Verify the fallback end-to-end
You cannot claim resilience without a test. Use pytest and mock the OpenAI client to throw InternalServerError, then assert Anthropic is called.
def test_fallback(monkeypatch):
def fake_openai(*args, **kwargs):
raise InternalServerError("simulated outage")
monkeypatch.setattr("openai_client.chat.completions.create", fake_openai)
# also mock anthropic
def fake_anthropic(*args, **kwargs):
class FakeResp:
content = [type("B", (), {"text": "fallback ok"})()]
return FakeResp()
monkeypatch.setattr("anthropic_client.messages.create", fake_anthropic)
req = ChatRequest(system="", messages=[{"role": "user", "content": "hi"}])
result = chat_with_fallback(req)
assert result["provider"] == "anthropic"
assert result["text"] == "fallback ok"Run with pytest -q. For TypeScript, use vitest and vi.mock.
Additionally, do a live chaos test: point OPENAI_BASE_URL to http://localhost:9 (refused connection) and send a real request. You should see the Anthropic response within ~2 seconds.
Step 7: Hardening and a gateway alternative
The hand-rolled pattern works, but you now own timeout tuning, model mapping, and circuit state. If you’d rather delegate that, an inference gateway like n4n.ai exposes a single OpenAI-compatible endpoint with automatic fallback when a provider is rate-limited or degraded, and it honors client routing directives. That removes the custom branching from your service code.
Either way, the key discipline is classifying errors precisely. A 400 Bad Request from OpenAI is a bug in your prompt, not an outage—never route that to Anthropic.
Step 8: Streaming considerations
If you stream tokens, the fallback must be decided before the first byte. You cannot switch providers mid-stream. Open the OpenAI stream with a short connect timeout; if the connection fails or the first chunk errors, abort and open the Anthropic stream. Both SDKs support stream=True / stream: true. Buffer the first chunk in a try/except, then pipe.
def stream_with_fallback(req):
try:
stream = openai_client.chat.completions.create(model="gpt-4o", messages=req.messages, stream=True, timeout=5.0)
for chunk in stream:
yield chunk.choices[0].delta.content or ""
except Exception:
stream = anthropic_client.messages.create(model="claude-3-5-sonnet-20241022", system=req.system, messages=req.messages, max_tokens=req.max_tokens, stream=True)
for chunk in stream:
yield chunk.delta.text or ""This keeps the fallback openai outage to anthropic contract intact for interactive UIs.
Step 9: Observability
Log provider on every response and emit a counter llm_fallback_total. During an OpenAI incident you should see a spike in that counter. Per-token metering (if you use a gateway) or manual token counting via resp.usage helps track cost divergence—Anthropic and OpenAI price differently, so fallback may change your margin.
Set up an alert when fallback_rate > 0.05 for 5 minutes; that indicates OpenAI is unhealthy and you’re silently running on Anthropic.
Step 10: Clean shutdown
On process exit, flush logs and close clients. The SDKs are mostly stateless, but if you cache sessions, clear them. The fallback code itself needs no special teardown.