Most Python services still invoke openai.ChatCompletion.create() at the point of need, and the thought of changing providers suggests a painful refactor. To migrate openai chatcompletion to gateway without a rewrite, you only need to redirect the SDK’s base URL and credentials—the request and response contracts are identical on any OpenAI-compatible endpoint. The following steps take an existing codebase from direct OpenAI calls to a unified gateway with zero changes to business logic.
Step 1: Audit existing OpenAI SDK usage
Before changing anything, find every call to the legacy ChatCompletion interface. In the openai Python package prior to v1.0, the global module functions are the default pattern:
import openai
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Summarize this ticket"}],
temperature=0.2,
max_tokens=256,
)Run a quick grep to locate the surface area:
grep -rn "ChatCompletion.create" ./srcNote the arguments you pass. Standard fields (model, messages, temperature, max_tokens, stop, stream) are forwarded unchanged by any compliant gateway. Provider-specific extensions (like logit_bias or user) are also passed through as arbitrary JSON. If you rely on openai.organization or openai.api_version, flag those—most gateways ignore them or map them to routing headers.
Step 2: Choose an OpenAI-compatible gateway
A unified gateway only saves work if it speaks the exact same HTTP protocol. Confirm three things:
- It exposes
POST /v1/chat/completionswith the same JSON schema. - It returns the same response envelope (
id,object,choices,usage). - It accepts your existing model names or documents a clear alias scheme.
For example, n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models, so the same ChatCompletion payload works without modification. You do not need a client library from the gateway; the openai package is sufficient.
Step 3: Repoint the API base URL and key
The legacy SDK reads configuration from module-level globals. Swap the endpoint and key at process startup—typically in your settings.py or main.py before any calls occur.
import os
import openai
openai.api_key = os.environ["GATEWAY_API_KEY"]
openai.api_base = "https://api.n4n.ai/v1" # replace with your gateway URLThat is the entire migration for a monolithic app. Every subsequent openai.ChatCompletion.create() now hits the gateway, which forwards to the upstream provider. If you use the newer OpenAI client class, the equivalent is OpenAI(base_url="https://api.n4n.ai/v1", api_key=...), but that requires changing ChatCompletion.create to client.chat.completions.create. To avoid a rewrite, stay on the global module pattern until later.
Step 4: Map model identifiers
Gateways often namespace models to avoid collisions: openai/gpt-4o vs anthropic/claude-3-opus. If your code hardcodes "gpt-4" in fifty places, add a single indirection rather than editing each call.
# model_router.py
import openai
MODEL_ALIASES = {
"gpt-3.5-turbo": "openai/gpt-3.5-turbo",
"gpt-4": "openai/gpt-4",
"claude-3-sonnet": "anthropic/claude-3-sonnet",
}
def chat_completion(**kwargs):
model = kwargs.get("model")
if model in MODEL_ALIASES:
kwargs["model"] = MODEL_ALIASES[model]
return openai.ChatCompletion.create(**kwargs)Call chat_completion() instead of the SDK directly. This is a one-line find-replace per file and preserves the original call signature. If your gateway accepts bare model names, skip this step.
Step 5: Preserve provider-specific parameters
The legacy SDK serializes any unknown keyword argument into the request body. This is exactly what you need for provider hints. Gateways such as n4n.ai honor client routing directives and forward provider cache-control hints, so if you already send vendor fields, they reach the backend untouched.
# Asking Anthropic via the gateway with a cache control hint
openai.ChatCompletion.create(
model="anthropic/claude-3-sonnet",
messages=[{"role": "user", "content": "Long document..."}],
temperature=0.0,
anthropic_version="2023-06-01",
# gateway forwards this through to the provider
)If you later adopt the v1 SDK, the same idea applies via extra_body={...}. No business logic changes—only the transport learns the new fields.
Step 6: Confirm streaming and async parity
Streaming is a common break point. The legacy SDK supports stream=True and yields delta objects. Verify the gateway returns the same SSE shape:
stream = openai.ChatCompletion.create(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Count to 5"}],
stream=True,
)
for chunk in stream:
delta = chunk["choices"][0]["delta"]
if "content" in delta:
print(delta["content"], end="")If your app uses openai.ChatCompletion.acreate (async), the same base-URL swap works; the coroutine returns an async generator under stream=True. No code inside the loop changes.
Step 7: Validate with a test harness
Write a smoke test that asserts structural parity. This catches a misconfigured gateway before users do.
import os
import openai
import pytest
openai.api_key = os.environ["GATEWAY_API_KEY"]
openai.api_base = os.environ["GATEWAY_BASE_URL"]
def test_chatcompletion_shape():
resp = openai.ChatCompletion.create(
model="openai/gpt-3.5-turbo",
messages=[{"role": "user", "content": "ping"}],
max_tokens=10,
)
assert resp["object"] == "chat.completion"
assert len(resp["choices"]) == 1
assert "message" in resp["choices"][0]
assert resp["usage"]["total_tokens"] > 0Run it against the gateway. A green test means the migrate openai chatcompletion to gateway effort is functionally complete. For regression coverage, record a real response with vcrpy and replay it in CI so you don’t burn tokens on every test run.
Step 8: Verify end-to-end from the shell
A direct curl confirms the gateway honors the same contract outside the SDK. This is useful when debugging auth or header issues.
curl -s "$GATEWAY_BASE_URL/chat/completions" \
-H "Authorization: Bearer $GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o",
"messages": [{"role": "user", "content": "Say hi"}],
"max_tokens": 5
}' | jq '.choices[0].message.content'If you see a normal completion, the request path is proven. The gateway’s per-token usage metering will appear in the usage field exactly as OpenAI returns it.
Step 9: Clean up configuration and monitor
Move the base URL and key to environment configuration if they aren’t there already. Do not leave openai.api_base hardcoded in a random module—set it once at entrypoint. If you previously caught openai.error.RateLimitError, note that the gateway may automatically fall back when a provider is rate-limited or degraded; your retry logic can stay, but you might widen timeouts to accommodate cross-provider routing.
Finally, log the response["model"] field. Gateways sometimes rewrite the model string to the backend’s canonical ID. Watching that field in production tells you which provider actually served the traffic after the migrate openai chatcompletion to gateway switch.
Step 10: Optional progressive rollout
If you are nervous about a big-bang cutover, use a wrapper that selects the base URL per call. The legacy SDK is global, so instead instantiate a tiny proxy module:
# gateway_client.py
import openai
def create(use_gateway=True, **kwargs):
old_base = openai.api_base
if use_gateway:
openai.api_base = "https://api.n4n.ai/v1"
try:
return openai.ChatCompletion.create(**kwargs)
finally:
openai.api_base = old_baseFlip use_gateway via a feature flag. This lets you shift a percentage of traffic without a second client library. Once stable, set the global base permanently and delete the shim.
The migration is done. No prompt templates changed, no response parsers rewritten, no model logic touched—just a redirected endpoint and a key. That is the whole point of an OpenAI-compatible gateway: the ChatCompletion interface is the stable contract, and the backend behind it is now a configuration detail.