Configuring LangChain to use n4n.ai as an OpenAI-compatible endpoint

A correct langchain openai-compatible endpoint config lets you swap a single base URL and immediately access hundreds of models without rewriting chains. This guide targets n4n.ai, a gateway that exposes 240+ models behind one OpenAI-compatible API, and walks through the exact steps to wire it into an existing LangChain project.

Step 1: Install the LangChain OpenAI Package

Start with a clean virtual environment to avoid version conflicts between the legacy langchain monolith and the newer modular packages.

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install langchain-openai langchain-core

If you are on LangChain v0.2 or later, langchain-openai is the supported integration. Do not install the old openai extra from langchain==0.1.x alongside it; the duplicate ChatOpenAI classes will cause subtle import errors. Verify the install:

python -c "from langchain_openai import ChatOpenAI; print('ok')"

Step 2: Point ChatOpenAI at the Gateway

The only mandatory change in your langchain openai-compatible endpoint config is the base_url. The model field stops being a single vendor’s fixed identifier and becomes a routing key.

from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model="gpt-4o",
    api_key="sk-your-gateway-key",
    base_url="https://api.n4n.ai/v1",
    temperature=0.2,
)

The api_key is the credential issued by the gateway, not the upstream provider’s key. The gateway handles provider authentication server-side and applies per-token usage metering on every request.

Use environment variables for the key

Hardcoding keys in source is a liability. Export them and read at runtime:

export N4N_API_KEY=sk-your-gateway-key

import os
from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model="gpt-4o",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
)

Route to non-OpenAI models

You are not limited to OpenAI models. Pass a qualified name to select a backend:

chat = ChatOpenAI(
    model="anthropic/claude-3-5-sonnet",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
)

The gateway honors client routing directives, so the anthropic/ prefix forwards the request to that backend. If the named provider is rate-limited or degraded, the gateway’s automatic fallback engages and returns a compatible response from a healthy provider unless you explicitly disable fallback in your request.

Step 3: Pass Provider Cache-Control Hints

OpenAI’s chat API has no native cache header, but the gateway forwards provider cache-control hints when you send them via extra headers. In LangChain, inject them through model_kwargs:

chat = ChatOpenAI(
    model="anthropic/claude-3-5-sonnet",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
    model_kwargs={
        "extra_headers": {
            "X-Cache-Control": "max-age=3600"
        }
    },
)

This matters for long system prompts that rarely change. The gateway attempts to use the upstream provider’s prompt caching and returns cached token counts in the usage payload. A raw request body that the gateway normalizes looks like this:

{
  "model": "anthropic/claude-3-5-sonnet",
  "messages": [{"role": "user", "content": "Summarize RFC 792"}],
  "extra_headers": {"X-Cache-Control": "max-age=3600"}
}

Step 4: Streaming and Async Calls

Production chains usually need streaming. LangChain’s ChatOpenAI supports it natively because the gateway streams SSE exactly like OpenAI.

for chunk in chat.stream("Explain TCP slow start in one paragraph."):
    print(chunk.content, end="", flush=True)

For async services:

out = await chat.ainvoke("Same prompt")

Because the gateway normalizes errors, a provider rate-limit surfaces as a standard 429 with Retry-After. LangChain’s default retry logic handles it without custom middleware. If you build your own retry layer, keep it shallow; the gateway already performs cross-provider fallback.

Step 5: Inspect Usage and Verify Metering

After a call, response metadata carries the raw provider usage. With the gateway, this includes the per-token counts it meters server-side.

resp = chat.invoke("What is MTU?")
print(resp.response_metadata["token_usage"])

Expected shape:

{
  "prompt_tokens": 12,
  "completion_tokens": 34,
  "total_tokens": 46
}

If you run the same prompt twice with cache-control set, the second call should show a non-zero cached prompt token count when the upstream supports it. To confirm your langchain openai-compatible endpoint config is actually hitting the gateway, check the resolved model:

print(resp.response_metadata["model"])
# -> anthropic/claude-3-5-sonnet

The returned model string reflects the resolved backend, not necessarily the exact string you passed, which proves routing occurred.

Step 6: Configure Timeouts and Retries

The underlying OpenAI SDK defaults to a 600s timeout. For interactive apps, lower it:

chat = ChatOpenAI(
    model="gpt-4o",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
    timeout=30,
    max_retries=2,
)

The gateway’s automatic fallback reduces the need for aggressive client retries. Keeping max_retries low prevents head-of-line blocking when a provider is hard-down and the gateway has already exhausted its own fallback paths.

Step 7: Use With LCEL and RAG Pipelines

The ChatOpenAI instance drops into any LangChain Expression Language chain without modification.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a netops assistant. Use RFC references."),
    ("user", "{q}"),
])

chain = prompt | chat
result = chain.invoke({"q": "Describe BGP graceful restart."})
print(result.content)

If you embed documents, swap the model string per request to compare providers:

chain.with_config({"model": "google/gemini-1.5-pro"}).invoke({"q": "TLS 1.3 handshake steps?"})

No code change beyond the config dict is required because the langchain openai-compatible endpoint config centralizes routing.

Verify Success

Run this minimal script end to end:

import os
from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model="openai/gpt-4o-mini",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
)
out = chat.invoke("Return the word 'pong'.")
print(out.content)
print(out.response_metadata["model"])

Expected output:

pong
openai/gpt-4o-mini

If the model string echoes back with a provider prefix and the content matches, your configuration is correct. Confirm a metered call appears in the gateway dashboard with token counts.

Common Pitfalls

Provider-specific params: Chains that pass frequency_penalty or logit_bias may hit a 400 if the target model rejects them. The gateway passes them through transparently. Test with minimal params first.
Hardcoding model names in branch logic: Because the gateway addresses 240+ models, avoid if model == "gpt-4": in your code. Let the routing string do the work.
Ignoring response metadata: LangChain hides usage behind response_metadata. If you need per-token cost attribution, extract it there rather than re-counting tokens locally with a tokenizer.
Mixing SDK versions: A stray import openai from an old notebook can shadow the langchain-openai client. Pin versions in requirements.txt.

Wrapping Up

A solid langchain openai-compatible endpoint config is mostly about setting base_url and trusting the gateway to handle routing, fallback, and metering. Once it’s in place, you can flip models per request without touching chain logic, and the same code path works for OpenAI, Anthropic, Google, and dozens of other backends.