A checklist for migrating from OpenAI SDK to a gateway

Migrating your stack off a single vendor client starts with a concrete openai sdk migration checklist. You’re not just changing a URL; you’re trading a hardcoded dependency for a routing layer that speaks the same protocol but aggregates many backends. Get the sequence wrong and you’ll leak provider-specific assumptions into production.

1. Map your model identifiers

The OpenAI SDK lets you pass model strings like gpt-4o or text-embedding-3-small. A gateway fronts models from multiple vendors, so those names often change. Pull every literal model ID from your codebase and config, then build a translation table.

# Before
completion = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hi"}]
)

# After (gateway model slug)
completion = client.chat.completions.create(
    model="openai/gpt-4o",  # or provider-agnostic alias
    messages=[{"role": "user", "content": "Hi"}]
)

Don’t assume feature parity. Some gateways expose 240+ models but not all support function calling, JSON mode, or vision. Verify each capability against the target slug before cutover.

2. Swap the base URL and credentials

This is the mechanical core of the openai sdk migration checklist. Instantiate the client with the gateway’s OpenAI-compatible endpoint and a gateway-issued key. The request shape stays identical.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.n4n.ai/v1",  # single endpoint, 240+ models
    api_key="gw_sk_xxx"
)

Rotate keys through your secret manager, not inline. If you use multiple environments, scope keys per environment and log the key ID in your gateway dashboard, not the raw secret.

3. Handle provider-specific parameters

OpenAI accepts temperature, max_tokens, response_format. Other providers behind the gateway may reject unknown fields or interpret them differently. Audit your calls for extensions like seed or logprobs and gate them behind capability checks.

{
  "model": "anthropic/claude-3-5-sonnet",
  "messages": [{"role": "user", "content": "Summarize"}],
  "temperature": 0.2,
  "max_tokens": 512
}

When a parameter is unsupported, the gateway may pass it through or strip it. Read the gateway’s passthrough rules and add client-side guards so you fail loud in tests, not silently in prod.

4. Implement fallback and retry logic

A gateway can route around a degraded provider, but your code should still handle partial failure. Define what “success” means: a completed stream, a parsed JSON object, or a non-429 status.

try:
    resp = client.chat.completions.create(model="openai/gpt-4o", ...)
except openai.APIStatusError as e:
    if e.status_code == 429:
        # gateway already tried fallback; surface to caller
        metrics.incr("llm_rate_limited")

n4n.ai provides automatic fallback when a provider is rate-limited or degraded, but you should still surface partial failures to callers. Don’t wrap every call in a blind retry loop; respect Retry-After and back off.

5. Preserve caching directives

Provider caching (e.g., OpenAI prompt caching, Anthropic cache_control) is expressed via request headers or message fields. A unified gateway must forward those hints or your cost and latency wins disappear.

completion = client.chat.completions.create(
    model="anthropic/claude-3-5-sonnet",
    messages=[
        {"role": "system", "content": "You are a helpful assistant"},
        {"role": "user", "content": long_context, "cache_control": {"type": "ephemeral"}}
    ]
)

Confirm the gateway honors client routing directives and forwards provider cache-control hints. If it doesn’t, you’ll pay full token price on every request and wonder why migration blew up your bill.

6. Validate streaming behavior

Streaming over SSE looks the same, but chunk boundaries differ across providers. Your tokenizer and UI must tolerate varying delta granularities. Run a diff test: same prompt, old SDK vs gateway, compare reconstructed text.

curl -N https://api.gateway.example/v1/chat/completions \
  -H "Authorization: Bearer $GW_KEY" \
  -d '{"model":"openai/gpt-4o","stream":true,"messages":[{"role":"user","content":"Count to 5"}]}'

Watch for trailing usage chunks. Some gateways emit usage only at stream end; your aggregator must not assume it arrives mid-stream.

7. Instrument usage metering

Per-token metering is non-negotiable for cost control. The gateway returns usage in the response; ship it to your metrics pipeline tagged by model and route.

resp = client.chat.completions.create(model="openai/gpt-4o", ...)
print(resp.usage.model_dump())
# {'prompt_tokens': 12, 'completion_tokens': 34, 'total_tokens': 46}

If the gateway supports per-token usage metering via response headers, capture them in middleware. Alert on sudden spikes in completion_tokens per request class.

8. Test routing and compliance constraints

Gateways let you pin a provider per request for data-residency or compliance. Encode those rules as explicit headers or model prefixes, not ambient config.

{
  "model": "eu/openai/gpt-4o",
  "messages": [{"role": "user", "content": "PII redacted input"}]
}

Run a compliance suite that asserts certain tenants never leave a region. The openai sdk migration checklist is incomplete if you can’t prove routing policy in CI.

9. Update client-side type definitions

TypeScript users: the OpenAI SDK types are vendor-shaped. Gateway responses may include extra fields like route or gateway_latency. Extend the response type rather than casting to any.

import { ChatCompletion } from "openai/resources";

interface GatewayCompletion extends ChatCompletion {
  route?: string;
  gateway_latency_ms?: number;
}

Keep the strictness. Loose types hide migration regressions until they hit production.

10. Run a shadow traffic phase

Before flipping DNS, mirror a copy of live traffic to the gateway with a no-op consumer. Compare outputs for equality or semantic similarity, and compare latency percentiles.

def shadow_call(req: dict):
    gateway_resp = client.chat.completions.create(**req)
    log_diff(prod_resp, gateway_resp)

Only after error rates and token counts match expectations do you switch the primary client. This final step on the openai sdk migration checklist de-risks the cutover.

Synthesis

Follow this openai sdk migration checklist sequentially. The protocol is compatible; the operational surface is not. Treat the gateway as a new dependency with its own failure modes, and you’ll ship the migration in days, not weeks.