Swapping your LLM provider shouldn’t require rewriting your call sites. To migrate OpenAI SDK to n4n.ai, you change three lines: the base URL, the API key, and optionally the model string. The rest of your OpenAI client code keeps working because the gateway speaks the OpenAI chat completions protocol.
Prerequisites
- Python 3.10 or newer
openaiPython package v1.0+ (pip install -U openai)- A valid API key for the OpenAI-compatible endpoint (create one in your n4n.ai dashboard)
You should already have a working script that calls client.chat.completions.create. If you don’t, the snippet below is the baseline most teams start from.
Step 1: Isolate your existing OpenAI client
Here is a minimal, real OpenAI SDK call. This is what production code looks like after stripping retries, logging, and business logic:
from openai import OpenAI
client = OpenAI(api_key="sk-openai-1234")
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What is automatic fallback in LLM gateways?"}],
)
print(response.choices[0].message.content)Run it once to confirm your environment works. Expected output is a short prose answer from the model. Note the import, the constructor, and the create call—those are the only surfaces we will touch.
Step 2: Repoint the client in three lines
The migration is a constructor change. You set base_url to the gateway’s OpenAI-compatible endpoint and swap the key. Everything else stays identical.
from openai import OpenAI
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="sk-n4n-5678",
)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What is automatic fallback in LLM gateways?"}],
)
print(response.choices[0].message.content)That is the entire diff to migrate OpenAI SDK to n4n.ai. The base_url line and the api_key line are two of the three; the third is the model string if you decide to use a non-OpenAI model in the catalog.
Checkpoint: execute the script. You should see output structurally identical to the previous run, with latency typically within single-digit milliseconds of a direct OpenAI call.
Automatic fallback in LLM gateways refers to routing a request to a secondary
provider when the primary is rate-limited, degraded, or down, without the
calling application needing to implement its own retry mesh.If you get a 401, your key is wrong. If you get a 404 on the path, you missed /v1 in the base URL. Both are configuration errors, not code errors.
Step 3: Exploit the model catalog
Because the endpoint fronts 240+ models, you are not locked into OpenAI’s naming. Change the model argument to any supported identifier. The OpenAI SDK passes it through untouched.
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Compare fallback vs load balancing."}],
)The same chat.completions shape returns. Token counts, finish reasons, and streaming all behave as the OpenAI spec defines.
Expected output with a different model
Fallback activates only after a failure or saturation signal. Load balancing
distributes traffic proactively across healthy endpoints before any failure
occurs. A gateway typically combines both: balance under normal load, fall back
when a provider breaches its error budget.If you see a model_not_found error, your identifier is wrong. The gateway returns standard OpenAI error objects, so your existing error handling catches it without modification.
Advanced: Routing directives and cache control
The gateway honors client routing directives and forwards provider cache-control hints. In the OpenAI SDK, arbitrary headers go through extra_headers. Use them to force or suppress fallback:
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Draft a cache policy."}],
extra_headers={
"x-n4n-routing": "fallback=auto",
"x-n4n-cache": "ttl=3600",
},
)This is not a proprietary SDK method. It is plain OpenAI client behavior; the gateway reads the headers and acts. Provider cache-control hints (like Anthropic’s cache_control block inside the message body) are forwarded without modification, so your existing prompt caching code works.
Automatic fallback in practice
When an upstream provider returns 429 or 503, the gateway can retry against a secondary provider if you set fallback=auto. You do not write retry loops. Your code receives a normal completion or a standard error after exhaustion.
try:
resp = client.chat.completions.create(
model="meta-llama/llama-3.1-70b",
messages=[{"role": "user", "content": "Summarize fallback."}],
extra_headers={"x-n4n-routing": "fallback=auto"},
)
print(resp.choices[0].message.content)
except openai.APIStatusError as e:
print("All providers failed:", e.status_code)This beats hand-rolling ten try/except blocks across providers. The exception type is the same openai.APIStatusError you already catch for OpenAI.
Per-token usage metering
Billing transparency matters. The gateway emits per-token usage metering in the exact OpenAI shape:
print(response.usage.prompt_tokens, response.usage.completion_tokens)You get prompt_tokens, completion_tokens, and total_tokens. If the request hit a cache, the gateway reports prompt_tokens_details.cached_tokens when the provider supports it. No new fields to learn, no separate metering API to poll.
Streaming stays the same
If your app streams, nothing changes:
stream = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Stream a haiku."}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")The gateway streams SSE chunks compliant with OpenAI’s format. Your existing for chunk in stream loop prints tokens as they arrive. No async changes required either—AsyncOpenAI accepts the same base_url.
Verifying the migration with a smoke test
Drop this into test_smoke.py to lock the behavior in CI:
from openai import OpenAI
import os
def test_completion():
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key=os.environ["N4N_API_KEY"],
)
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "ping"}],
)
assert resp.choices[0].message.content
assert resp.usage.total_tokens > 0Run pytest test_smoke.py. A green check means your migration is complete. The test is identical to one you would write against OpenAI, except for the two constructor lines.
Handling provider-specific parameters
Some models accept parameters OpenAI’s schema doesn’t define. The SDK exposes extra_body for pass-through JSON:
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Be terse."}],
extra_body={"thinking_tokens": 0},
)The gateway forwards extra_body to the provider. If the field is unsupported, the provider returns a standard error—same as if you had called it directly.
What you did not have to change
- Retry logic (the SDK’s built-in retries still apply)
- Type hints (
from openai.types import ChatCompletionstill matches) - Async code (
AsyncOpenAIwith the samebase_urlworks) - Middleware or proxy settings that wrap the HTTP client
- Observability hooks that read
response.headersorresponse.usage
The OpenAI SDK is just an HTTP client with a typed surface. Point it at a compliant endpoint and the contract holds.
Caveats
A few behaviors differ from raw OpenAI:
- Model availability depends on the gateway’s catalog, not OpenAI’s.
- Some provider-specific extensions (like JSON mode variants) map to the nearest OpenAI equivalent.
- Rate limits are the gateway’s, not OpenAI’s, but they surface as standard 429s.
None of these require code changes beyond the initial three lines.
Final checklist
- Set
base_url="https://api.n4n.ai/v1". - Set your gateway API key.
- (Optional) Change
modelto any of 240+ identifiers. - Run your existing test suite.
If your tests passed against OpenAI, they pass here. That is the point of OpenAI compatibility—and the reason the migration fits in three lines.