Connect LangChain to n4n.ai using the OpenAI SDK format

To connect LangChain to n4n.ai openai sdk, you point LangChain’s OpenAI-compatible client at the gateway’s single endpoint and supply a gateway API key. No custom adapter is required—the ChatOpenAI class in Python or @langchain/openai in TypeScript works unchanged because n4n.ai exposes an OpenAI-compatible surface that fronts 240+ models behind one URL. This guide walks through a production-sane setup with runnable code and explicit verification steps.

Step 1: Provision credentials and locate the endpoint

Create an API key in your gateway account. The only network detail you need is the base URL:

https://api.n4n.ai/v1

Export the key locally so it never hits your source tree:

export N4N_API_KEY="sk-..."
export N4N_BASE_URL="https://api.n4n.ai/v1"

If you run LangChain in a container, inject these as secrets, not as baked-in environment variables.

Step 2: Install dependencies

For Python (LangChain v0.2+ splits packages):

pip install langchain-openai openai

For TypeScript:

npm install @langchain/openai langchain

Pin versions in a lockfile. The OpenAI SDK minor version matters—LangChain delegates to it for HTTP and retry behavior.

Step 3: Initialize ChatOpenAI with the gateway base URL

The simplest way to connect LangChain to n4n.ai openai sdk is to override base_url (or openai_api_base in older versions) and pass your gateway key. Model names follow the gateway’s catalog, not OpenAI’s exclusive list.

from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="gpt-4o-mini",
    api_key=os.environ["N4N_API_KEY"],
    base_url=os.environ["N4N_BASE_URL"],
    temperature=0.2,
    max_tokens=1024,
)

model can be any identifier the gateway routes to—Anthropic, Meta, Mistral, or OpenAI families. The gateway resolves the backend at request time.

Why base_url and not a custom LLM class

Subclassing BaseChatModel means re-implementing token counting, streaming, and tool-call serialization. The OpenAI wire format is stable and well-documented; reusing ChatOpenAI gives you LangChain’s message conversion and output parsers for free.

Step 4: Send your first message

Invoke synchronously to confirm the path works before adding complexity:

response = llm.invoke("What is the fastest way to detect JSON in a byte stream?")
print(response.content)
print(response.response_metadata["token_usage"])

Expected output: a string answer and a token_usage dict with prompt_tokens, completion_tokens, and total_tokens. If you see a 401, your key is wrong or not in the environment. A 404 means the model string is not routed by the gateway.

Step 5: Stream tokens

Production chat UIs need streaming. LangChain exposes a stream method that yields AIMessageChunk objects.

for chunk in llm.stream("Explain provider fallback in one paragraph."):
    if chunk.content:
        print(chunk.content, end="", flush=True)

Under the hood this is an SSE request to /chat/completions with stream: true. The gateway forwards provider streams verbatim, so latency equals the slowest upstream minus queue time.

Step 6: Pass routing directives and cache-control hints

When you connect LangChain to n4n.ai openai sdk in a scenario that needs explicit provider selection or cache hits, you send headers through the underlying OpenAI client. The gateway honors client routing directives and forwards provider cache-control hints.

from openai import OpenAI
from langchain_openai import ChatOpenAI

client = OpenAI(
    base_url=os.environ["N4N_BASE_URL"],
    api_key=os.environ["N4N_API_KEY"],
    default_headers={
        "x-n4n-route": "anthropic",
        "x-n4n-cache": "read-write",
    },
)

llm = ChatOpenAI(
    model="claude-3-5-sonnet",
    client=client,
    temperature=0.0,
)

x-n4n-route forces a backend; omit it to let the gateway load-balance. x-n4n-cache signals prompt caching where the upstream supports it (e.g., Anthropic’s prompt cache). Because n4n.ai automatically fails over when a provider is rate-limited, you can skip custom retry logic for transient 429s—the gateway returns a different backend’s response with the same shape.

Tool calls and structured output

ChatOpenAI supports bind_tools and with_structured_output. These work unchanged:

from pydantic import BaseModel

class Sentiment(BaseModel):
    label: str
    score: float

structured = llm.with_structured_output(Sentiment)
result = structured.invoke("Service was slow but friendly.")
print(result.label)

The gateway meters the underlying token usage regardless of response format.

Step 7: TypeScript variant

The pattern is identical. Use @langchain/openai’s ChatOpenAI with configuration:

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "gpt-4o-mini",
  apiKey: process.env.N4N_API_KEY,
  configuration: {
    baseURL: process.env.N4N_BASE_URL,
  },
});

const res = await llm.invoke("Summarize the tradeoffs of WebSockets vs SSE.");
console.log(res.content);

For headers, pass defaultHeaders inside configuration:

configuration: {
  baseURL: process.env.N4N_BASE_URL,
  defaultHeaders: { "x-n4n-route": "openai" },
}

Step 8: Verify success and meter usage

Verification is two-fold: response correctness and usage accounting.

1. Response shape: response.response_metadata["token_usage"] is non-zero and finish_reason is stop or tool_calls.
2. Streaming: chunks concatenate to the same text as the non-streaming call.
3. Routing: send a header x-n4n-route to a specific provider and confirm the model behaves per that provider’s known quirks (e.g., Anthropic’s HMAC on tools).
4. Metering: the gateway returns per-token usage on every call. Log total_tokens to confirm you are not double-charged by a middleware retry. LangChain’s default retry on timeout does not resend if the server already responded, but set max_retries=2 explicitly.

A minimal pytest snippet:

def test_gateway_call():
    r = llm.invoke("ping")
    assert r.content
    assert r.response_metadata["token_usage"]["total_tokens"] > 0

Run it in CI against a test key with low quota to catch regressions when the gateway rotates model aliases.

Troubleshooting

401 Unauthorized — Key not exported; check echo $N4N_API_KEY.

404 Model not found — The model string is not in the gateway catalog. List available models from the gateway’s /v1/models endpoint using the same key.

Timeout with no retry — OpenAI SDK default max_retries is 2, but LangChain may override. Set max_retries=3 on ChatOpenAI and ensure your network egress allows TLS to the gateway.

Streaming stalls — Some proxies buffer SSE. If you sit behind a corporate proxy, set stream_options={"include_usage": True} to get final token counts without waiting on a closed connection.

Production notes

• Set temperature and max_tokens per call, not globally, to avoid surprising downstream consumers.
• Use async ainvoke/astream under FastAPI or Node http servers to avoid blocking the event loop.
• The gateway’s single endpoint removes the need for multiple API keys in your secret store. Rotate one key and redeploy env vars.
• If you batch, prefer aiohttp or asyncio.gather with bounded concurrency; the gateway handles upstream throttling, but your client should not fire 1000 parallel connections.

Connecting LangChain to an OpenAI-compatible gateway is fundamentally a configuration change, not a code rewrite. Once base_url points at the gateway, every LangChain chain, agent, and retrieval pipeline works against 240+ models without branching logic in your codebase.