Consolidating multi-provider integrations: code changes

Maintaining separate SDK wrappers for OpenAI, Anthropic, and Cohere feels fine until you need to swap a model or survive a provider outage. Consolidating integrations code changes is the mechanical but unforgiving step where you collapse those divergent clients into one request path. Treat it as a refactor with strict interface boundaries, not a rewrite, and you’ll keep production stable.

Audit your current integration surface

Start by listing every file that imports a provider SDK. Include indirect calls through internal helpers. You want a count of distinct call sites, the parameters each passes, and the error types they catch.

grep -rn "anthropic\|openai\|cohere" ./src --include="*.py" | wc -l

Build a spreadsheet with columns: provider, model string, max_tokens, temperature, streaming, auth env var. This becomes your migration checklist. When scoping consolidating integrations code changes, the audit phase exposes how many hardcoded model names live in business logic and how inconsistent your timeout handling is.

Define a unified request shape

OpenAI’s chat completions schema has become the de facto lingua franca. Every major provider now accepts a compatible shape through a gateway or their own beta endpoint. Map your existing calls onto messages, model, temperature, max_tokens, and stop.

Provider-specific fields like Anthropic’s top_k or Cohere’s frequency_penalty don’t disappear; they move into an extra_body passthrough. Keep them isolated so the core call stays clean.

# unified call shape
def chat(model: str, prompt: str, **provider_extras):
    return client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.2,
        max_tokens=512,
        extra_body=provider_extras,
    )

Pick a consolidation target

A key decision in consolidating integrations code changes is whether to self-roll a polymorphic client or adopt an OpenAI-compatible gateway. Writing your own abstraction means maintaining mapper code for each provider’s error formats and auth schemes—permanent overhead. A gateway that speaks OpenAI format terminates that work at the network boundary.

An OpenAI-compatible endpoint such as n4n.ai addresses 240+ models behind one base URL and provides automatic fallback when a provider is rate-limited or degraded. That removes the retry mesh you were about to write. The tradeoff is an external dependency in your request critical path; mitigate with timeouts and health checks.

Rewrite client instantiation

Replace per-provider client construction with a single OpenAI client pointed at the gateway. Use one API key from your secrets manager.

from openai import OpenAI
import os

client = OpenAI(
    base_url=os.getenv("LLM_GATEWAY_URL", "https://api.n4n.ai/v1"),
    api_key=os.getenv("LLM_GATEWAY_KEY"),
    timeout=30.0,
)

All previous openai.ChatCompletion.create calls now route through this client. Anthropic and Cohere calls become model-string swaps: model="anthropic/claude-3-5-sonnet" or model="cohere/command-r-plus".

Normalize parameters and headers

Providers disagree on required fields. Anthropic mandates max_tokens; OpenAI does not. Set sane defaults in your wrapper and let callers override. Pass provider-specific knobs via extra_body.

def safe_chat(model, messages, max_tokens=1024, **extra):
    return client.chat.completions.create(
        model=model,
        messages=messages,
        max_tokens=max_tokens,
        extra_body=extra,
    )

Cache control is another divergence. Anthropic uses cache_control markers on messages; OpenAI uses system prompt caching automatically. A gateway that forwards provider cache-control hints lets you keep those optimizations without branching. n4n.ai honors client routing directives and forwards provider cache-control hints, so the same message body works across providers.

Delegate routing and fallback

Stop writing if provider == "openai" in your services. Encode routing in the model identifier or gateway headers. The gateway decides which upstream to call and flips to a backup on 429s.

# before: manual fallback
try:
    resp = openai_client.completions.create(model="gpt-4o", ...)
except RateLimitError:
    resp = anthropic_client.completions.create(model="claude-3-5", ...)

# after: gateway handles it
resp = client.chat.completions.create(model="openai/gpt-4o", ...)

The gateway’s per-token usage metering shows you which provider actually served the request, so cost attribution doesn’t break.

Migrate incrementally with feature flags

The safest pattern for consolidating integrations code changes is incremental flagging. Do not flip the switch globally. Wrap each call site with a flag:

if flags.use_gateway("summarizer"):
    resp = safe_chat("anthropic/claude-3-5-sonnet", msgs)
else:
    resp = legacy_anthropic_call(msgs)

Run both paths in shadow mode for a week. Compare response parity and latency. Only retire the legacy branch after error rates match.

Test against real traffic

Unit tests with mocked responses lie. Capture production requests and replay them against the gateway in a staging environment. Assert that choices[0].message.content is non-empty and that tool-call schemas survive translation.

Pay attention to streaming. OpenAI’s SSE format differs slightly from Anthropic’s event stream; a gateway normalizes this, but you must verify your parser handles the unified chunk shape.

Common pitfalls and tradeoffs

Hidden parameter loss. If you rely on Anthropic’s system as a top-level field, the OpenAI shape nests it inside messages. Forgetting to transform drops your system prompt. Write an explicit mapper.

Latency tax. A gateway adds a network hop. Measure p99 before and after; if you’re regional, place your service near the gateway edge.

Billing granularity. Consolidation simplifies code but obscures per-provider cost unless the gateway emits per-token metering. Verify you can export usage by model.

Feature lag. New provider capabilities (like Anthropic’s computer use) may not be exposed through the OpenAI shape immediately. Keep a thin escape hatch for raw provider SDK when you need bleeding-edge features.

Consolidating integrations code changes is not a one-week project if you have dozens of call sites, but the payoff is a single client to upgrade, secure, and observe. The code gets smaller; the failure modes get fewer.