Using the OpenAI SDK as a drop-in client for n4n

The OpenAI SDK speaks a wire protocol that has become the de facto standard for LLM inference. If you treat n4n as an openai sdk drop-in replacement gateway, you keep your existing Python or TypeScript client code and immediately reach 240+ models behind one endpoint. This post walks through the exact configuration steps to make that switch with zero semantic changes to your call sites.

Step 1: Install the official SDK

If you already depend on openai in Python or Node, skip ahead. Otherwise, install the current major version. The client has been stable across v1.x for both languages and supports the full chat completion, embedding, and audio surfaces.

# Python
pip install openai>=1.40.0

# Node.js
npm install openai@^4.0.0

Pin the version in your lockfile. The SDK makes breaking changes rarely, but the gateway compatibility target is the v1 request shape, so anything reasonably recent works.

Step 2: Point the client at the gateway endpoint

The only mandatory change is the base_url (or baseURL in TS) and your credential. The OpenAI SDK sends requests to {base_url}/chat/completions and expects the standard response envelope. A unified gateway that is OpenAI-compatible exposes exactly that path.

# python_client.py
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.n4n.ai/v1",
    api_key=os.environ["N4N_API_KEY"],
)

// ts_client.ts
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.n4n.ai/v1",
  apiKey: process.env.N4N_API_KEY,
});

Store the key in environment variables. Do not hardcode it in source. The gateway issues a single token that meters per-token usage across all downstream providers, so you do not need per-vendor keys in your app config.

Step 3: Map model identifiers to the gateway catalog

OpenAI model names like gpt-4o will not automatically resolve to the right backend at a multi-provider gateway. Most unified catalogs use a qualified scheme: {provider}/{model}. Pull the live list from the gateway’s /v1/models endpoint and cache it at startup.

models = client.models.list()
for m in models.data[:5]:
    print(m.id)

Typical IDs look like openai/gpt-4o-mini, anthropic/claude-3-5-sonnet, or meta-llama/llama-3.1-70b-instruct. Update your call sites to use the qualified string. If your code abstracts the model name behind a config value, this is a one-line config change, not a code change.

REPLY_MODEL = "anthropic/claude-3-5-sonnet"  # was "gpt-4o"

The openai sdk drop-in replacement gateway approach pays off here: your retry logic, streaming parser, and type bindings stay identical.

Step 4: Send a minimal chat completion

Verify the wiring with the smallest possible request. Use a cheap model and a short prompt.

resp = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Say hello in one word."}],
    max_tokens=8,
)
print(resp.choices[0].message.content)

In TypeScript:

const resp = await client.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "Say hello in one word." }],
  max_tokens: 8,
});
console.log(resp.choices[0].message.content);

If you see a single word printed, the transport works. The request left your process, hit the gateway’s OpenAI-compatible endpoint, got routed to a provider, and the response parsed cleanly through the SDK’s pydantic or zod models.

Step 5: Pass routing directives and cache hints

A bare request lets the gateway pick a default route. Production systems usually want more control. The OpenAI SDK exposes extra_headers and extra_body precisely for provider-specific fields that the base interface does not model.

Because n4n honors client routing directives and forwards provider cache-control hints, you can annotate requests without writing a custom transport. For example, to prefer a cost-optimized route and set an Anthropic cache breakpoint:

resp = client.chat.completions.create(
    model="anthropic/claude-3-5-sonnet",
    messages=[
        {"role": "system", "content": "You are a terse assistant."},
        {"role": "user", "content": "Explain TCP handshakes."},
    ],
    max_tokens=256,
    extra_headers={"x-router-preference": "cost"},
    extra_body={"cache_control": {"type": "ephemeral", "ttl": "5m"}},
)

The gateway forwards the cache_control block to the upstream provider that supports it and ignores it for those that do not. Your application code does not branch on provider capabilities.

Step 6: Verify success and inspect usage metering

The OpenAI response includes a usage object. The gateway returns per-token counts after normalizing provider-specific billing shapes. Log it in development to confirm metering flows.

print(resp.usage.model_dump())
# {'prompt_tokens': 12, 'completion_tokens': 34, 'total_tokens': 46}

If you run the same prompt twice with a cache hint, you should observe a drop in prompt_tokens or a separate cache_read_tokens field depending on the upstream. The openai sdk drop-in replacement gateway keeps the field names consistent, so your existing logging pipeline needs no modification.

Set up a quick assertion in your test suite:

assert resp.usage.total_tokens > 0
assert resp.object == "chat.completion"

That catches auth failures (which raise AuthenticationError) and model-not-found errors (which raise NotFoundError) early.

Step 7: Handle degradation with automatic fallback

Networks and providers fail. A key reason to use a unified gateway is that it performs automatic fallback when a provider is rate-limited or degraded. You still need to configure the SDK’s own timeout and retry so a hung connection does not block your event loop.

client = OpenAI(
    base_url="https://api.n4n.ai/v1",
    api_key=os.environ["N4N_API_KEY"],
    timeout=30.0,
    max_retries=2,
)

The gateway will shift the request to a healthy provider that hosts an equivalent model if the primary is throwing 429s. Your code sees a normal completion or a single gateway-level error, not a provider-specific stack trace. Keep your application retries coarse; let the gateway handle the fine-grained provider shuffle.

Step 8: Stream and use async without changes

If your service uses stream=True or client.chat.completions.acreate, those code paths work unchanged. The SSE format is byte-compatible.

stream = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to 3."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

The same applies to function calling, response formatting, and embeddings. The openai sdk drop-in replacement gateway translates the uniform request to whatever the target model expects, so you do not maintain separate clients per vendor.

Verify the full migration

A clean verification checklist:

1. models.list() returns 240+ entries.
2. A non-streaming call with a qualified model ID returns chat.completion with usage.total_tokens > 0.
3. A streaming call yields deltas that concatenate to the same text as the non-streaming control.
4. An invalid model ID raises NotFoundError within 2 seconds.
5. A request with extra_headers routing hint succeeds and the response latency is logged.

Run these against a staging key before flipping production traffic. Because the client surface is identical, the diff in your repository should be limited to the client constructor and model strings—everything else stays exactly as it was when you targeted OpenAI directly.

That is the entire migration. You now have one client, one token, and one error model across every provider the gateway fronts.