Pointing the openai python sdk base_url n4n.ai at the gateway takes three lines of code and unlocks 240+ models behind one OpenAI-compatible interface. You keep your existing completion calls, tool usage, and streaming logic; you just change where the client sends requests.
Step 1: Install the OpenAI Python SDK
The official openai package is the only dependency. Use version 1.0 or later—the legacy 0.x line uses a different client shape and will fight you.
pip install "openai>=1.40"If you’re in a notebook or async framework, the same package ships AsyncOpenAI. No extra plugins required for what we’re doing.
Step 2: Configure the client with the n4n.ai endpoint
Create a single client instance and point it at the gateway. The openai python sdk base_url n4n.ai setting is just the base_url argument; the API key is the gateway key, not the upstream provider’s key.
from openai import OpenAI
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="sk-n4n-xxxxxxxxxxxxxxxx",
timeout=30.0,
)Keep the client as a module-level singleton. The SDK manages a connection pool internally; recreating it per request wastes sockets and adds latency. If you need to switch models at runtime, pass the model string per call—don’t build a new client.
Using environment variables
You can avoid hardcoding by exporting OPENAI_API_KEY and OPENAI_BASE_URL. The SDK picks them up automatically:
export OPENAI_API_KEY="sk-n4n-xxxxxxxxxxxxxxxx"
export OPENAI_BASE_URL="https://api.n4n.ai/v1"Then client = OpenAI() with no args works. This is the cleanest path for twelve-factor deployments.
Step 3: Send your first chat completion
A minimal call looks identical to calling OpenAI directly. The difference is the model field accepts any of the 240+ identifiers the gateway routes to.
resp = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[
{"role": "system", "content": "You are a terse senior engineer."},
{"role": "user", "content": "What is head-of-line blocking?"},
],
temperature=0.2,
)
print(resp.choices[0].message.content)Choosing a model
Model IDs follow the provider/family convention. Examples that work today:
openai/gpt-4oopenai/gpt-4o-minianthropic/claude-3.5-sonnetmeta-llama/llama-3.1-70b-instructgoogle/gemini-1.5-pro
If you pass an unknown ID, the gateway returns a 404 with a clear message. List available models programmatically:
models = client.models.list()
for m in models.data[:5]:
print(m.id)Step 4: Stream responses
For interactive apps, stream. The SDK yields chunks; you iterate and print deltas. This avoids waiting for the full generation and makes your UI feel responsive.
stream = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Write a haiku about TCP."}],
stream=True,
temperature=0.8,
)
for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
print(delta, end="", flush=True)Streaming also gives you early insight into malformed outputs. If the first chunk is garbage, you can abort the connection instead of paying for a full completion.
Async streaming
If you’re on FastAPI or any asyncio stack, use AsyncOpenAI and async for:
from openai import AsyncOpenAI
aclient = AsyncOpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-n4n-xxx")
async def main():
stream = await aclient.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role":"user","content":"Stream a list of HTTP codes."}],
stream=True,
)
async for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")Step 5: Pass provider routing and cache hints
n4n.ai honors client routing directives and forwards provider cache-control hints, so you can pin a provider or request caching without leaving the SDK. The OpenAI SDK doesn’t model these fields natively, but extra_body passes them straight through.
resp = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[{"role": "user", "content": "Summarize RFC 7231 section 4."}],
extra_body={
"route": {"prefer": ["anthropic"]},
"cache_control": {"type": "ephemeral", "ttl": 300},
},
)This tells the gateway to prefer Anthropic’s deployment and to treat the prompt as cacheable for five minutes. If Anthropic is degraded, the gateway still falls back per its policy, but your preference is respected when possible.
Why this matters
Provider-specific features like prompt caching normally require vendor SDKs. By forwarding hints through extra_body, you keep one code path and still get provider optimizations.
Step 6: Handle errors and fallback
The gateway provides automatic fallback when a provider is rate-limited or degraded, so most transient errors never reach your code. Still, write defensive code. Network blips, invalid model IDs, and exhausted quotas surface as APIError subclasses.
from openai import APIError, RateLimitError
try:
resp = client.chat.completions.create(
model="openai/gpt-4o",
messages=[{"role": "user", "content": "Test"}],
)
except RateLimitError as e:
# Gateway already tried fallbacks; upstream fully saturated
print("Rate limited:", e.response.json().get("error", {}).get("message"))
except APIError as e:
print("API error", e.status_code, e.message)For critical paths, implement a simple retry with exponential backoff on RateLimitError and APIConnectionError. Don’t retry on 4xx validation errors—they won’t fix themselves.
Step 7: Verify success
Verification is two-fold: confirm the request reached the gateway and confirm token metering.
First, list models to prove the base URL resolves:
assert any(m.id == "openai/gpt-4o" for m in client.models.list().data)Second, inspect usage on a real call:
resp = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Ping"}],
)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens, resp.usage.total_tokens)If those numbers are non-zero and the call returns in a reasonable time, your openai python sdk base_url n4n.ai integration is working. The gateway’s per-token usage metering matches what you see in resp.usage, so you can reconcile against your billing dashboard later.
Quick smoke test script
Put it together in a ten-line script:
from openai import OpenAI
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-n4n-xxx")
models = client.models.list()
print("Model count:", len(models.data))
r = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role":"user","content":"Say hello in JSON."}],
)
print(r.choices[0].message.content)
print("Tokens:", r.usage.total_tokens)Run it. If you see a JSON greeting and a token count, you’re done.