Teams that start with OpenAI’s SDK often hit rate limits or want multi-provider redundancy once they scale. To migrate OpenAI direct to gateway API for GPT-5, you only need to change a few client parameters—but the details determine whether your streaming, tool calls, and usage accounting survive the cutover. This guide walks through an end-to-end swap with runnable snippets and a verification checklist.
Step 1: Audit your current OpenAI direct integration
Before touching code, enumerate every openai client instantiation and request path. Most Python services instantiate a global client and call chat.completions.create:
from openai import OpenAI
client = OpenAI(api_key="sk-...") # direct OpenAI
resp = client.chat.completions.create(
model="gpt-5",
messages=[{"role": "user", "content": "Summarize this log"}],
temperature=0.2,
max_tokens=512,
stream=False,
)Capture the exact model string, sampling params (temperature, top_p), max_tokens, tools, response_format, and any extra_headers (e.g., OpenAI-Beta: assistants=v1). Gateways pass these through, but model aliases and beta headers often differ. If you use the REST API with curl, save the full request body and headers. You will replay them verbatim against the gateway to confirm parity.
Pay special attention to streaming and async usage. If you use AsyncOpenAI or client.chat.completions.create(stream=True), note the exact chunk shape your parser expects. The migration should not alter delta formats.
Step 2: Select a gateway and provision keys
A gateway gives you one OpenAI-compatible surface instead of per-provider SDK config. For example, n4n.ai exposes a single endpoint that fronts 240+ models and performs automatic fallback when a provider is rate-limited or degraded. You still send the same chat.completions JSON shape, so application code stays stable.
Provision a gateway key with scoped permissions (never reuse your OpenAI org key). Set environment variables:
export GATEWAY_API_KEY="gw-..."
export GATEWAY_BASE_URL="https://api.n4n.ai/v1"
export OPENAI_API_KEY="sk-..." # keep until cutover verifiedConfirm the gateway’s model catalog lists gpt-5 under the expected identifier. Some gateways require a provider prefix (openai/gpt-5); others accept the bare name and route internally.
Step 3: Swap the base URL and credentials
The OpenAI Python SDK accepts a base_url argument. Point it at the gateway and the migration is functionally live:
from openai import OpenAI
import os
client = OpenAI(
api_key=os.environ["GATEWAY_API_KEY"],
base_url=os.environ["GATEWAY_BASE_URL"], # https://api.n4n.ai/v1
)
resp = client.chat.completions.create(
model="gpt-5",
messages=[{"role": "user", "content": "Summarize this log"}],
temperature=0.2,
max_tokens=512,
)That single change is the core of how you migrate OpenAI direct to gateway API. For raw HTTP, change the host only:
curl https://api.n4n.ai/v1/chat/completions \
-H "Authorization: Bearer $GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-5","messages":[{"role":"user","content":"ping"}]}'If you use the async client, the same base_url swap applies:
from openai import AsyncOpenAI
aclient = AsyncOpenAI(api_key=os.environ["GATEWAY_API_KEY"], base_url=os.environ["GATEWAY_BASE_URL"])Do not change model yet—prove the proxy works with the identical request shape first.
Step 4: Reconcile model identifiers and routing
Gateways rarely rename gpt-5, but ambiguity across providers may require qualification. Check the gateway docs; if it honors client routing directives, you can pin a provider without changing code paths later:
{
"model": "gpt-5",
"messages": [{"role": "user", "content": "hi"}],
"route": {"provider": "openai"}
}If the gateway does not accept a route field, set the model string exactly as documented (e.g., openai/gpt-5). Avoid hard-coding provider prefixes in business logic; inject them via config so you can migrate OpenAI direct to gateway API without scattering literals.
Version skew is real: GPT-5 on Azure may differ subtly from OpenAI direct. Lock the route during verification, then relax once parity is confirmed.
Step 5: Preserve advanced request semantics
Tool calls, structured outputs, and streaming must survive the cutover. Here is a tool-calling request that works against both direct and gateway endpoints:
tools = [{
"type": "function",
"function": {
"name": "get_status",
"description": "Return service health",
"parameters": {"type": "object", "properties": {}}
}
}]
resp = client.chat.completions.create(
model="gpt-5",
messages=[{"role": "user", "content": "Is the API up?"}],
tools=tools,
tool_choice="auto",
stream=True,
)
for chunk in resp:
if chunk.choices[0].delta.tool_calls:
print("tool call streamed")If you previously set extra_headers for OpenAI-specific features (like OpenAI-Beta), test whether the gateway forwards them. A gateway that forwards provider cache-control hints will pass Cache-Control or provider-specific headers through; otherwise drop them to avoid 400s. Structured outputs (response_format={"type":"json_schema","json_schema":{...}}) should be sent unchanged—gateways validate against the same schema.
Step 6: Add fallback and cache directives
When you migrate OpenAI direct to gateway API, you gain fallback but must configure intent. Some gateways automatically reroute on 429/5xx. To bias toward cached responses, send cache hints:
resp = client.chat.completions.create(
model="gpt-5",
messages=[{"role": "user", "content": "Repeat the spec"}],
extra_headers={"Cache-Control": "max-age=3600"},
)If the gateway honors client routing directives, specify primary and fallback explicitly:
{
"model": "gpt-5",
"messages": [{"role": "user", "content": "go"}],
"route": {"primary": "openai", "fallback": ["azure"]}
}Validate that a forced provider outage (simulate by revoking a sub-key or using a bad route) still returns a completion from the fallback. Without this test, fallback is marketing, not engineering.
Step 7: Verify usage metering and response shape
The response object should match OpenAI’s schema exactly. Check the usage field for per-token accounting:
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)Gateways with per-token usage metering return the same usage struct, sometimes with an extended provider field. For example, n4n.ai includes provider attribution in the standard object so you can bill per upstream. Diff the JSON against a direct OpenAI call:
curl -s https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{"model":"gpt-5","messages":[{"role":"user","content":"x"}]}' > direct.json
curl -s https://api.n4n.ai/v1/chat/completions \
-H "Authorization: Bearer $GATEWAY_API_KEY" \
-d '{"model":"gpt-5","messages":[{"role":"user","content":"x"}]}' > gw.json
python -c "import json; a=json.load(open('direct.json')); b=json.load(open('gw.json')); assert set(a.keys())==set(b.keys()), 'key mismatch'; print('schema ok')"Content may differ slightly due to model version skew, but the keys, types, and usage math must align. For streaming, confirm usage appears in the final chunk if you rely on it.
Step 8: Roll out behind a flag
Do not flip all traffic at once. Use a config toggle:
USE_GATEWAY = os.environ.get("USE_GATEWAY") == "1"
client = OpenAI(
api_key=os.environ["GATEWAY_API_KEY"] if USE_GATEWAY else os.environ["OPENAI_API_KEY"],
base_url=os.environ["GATEWAY_BASE_URL"] if USE_GATEWAY else "https://api.openai.com/v1",
)Shift 5% of requests, watch latency and error rates, then ramp. Because you already migrated OpenAI direct to gateway API in code, rollback is a single env var. Instrument both paths with the same span names so dashboards stay comparable.
Verifying success
Success means: (1) 200 responses with identical schema, (2) usage tokens logged to your metrics, (3) forced provider failure still yields a response, (4) no OpenAI-Beta 400s. Run the diff script from Step 7 in CI against a stubbed gateway and direct mock to catch regressions. If all green, delete the direct client path and keep the gateway as your only egress for GPT-5.