Point LangChain at n4n.ai instead of the OpenAI API

Swapping your LangChain calls from OpenAI’s hosted API to an OpenAI-compatible gateway is a two-line change if you use the right abstraction. This guide shows how to use langchain n4n.ai instead of openai by repointing the base URL on ChatOpenAI, so you keep the same LangChain primitives while gaining access to 240+ models behind one endpoint.

Step 1: Install the LangChain OpenAI package

LangChain split its provider integrations into thin packages. You need langchain-openai and langchain-core; the monolithic langchain package is not required for this swap.

pip install langchain-openai langchain-core

If you are on an existing project, pin versions to avoid silent breaking changes:

pip install langchain-openai==0.1.23 langchain-core==0.3.15

Create a dedicated virtual environment. Mixing these with an old langchain==0.0.x install will cause import errors that waste an afternoon.

python -m venv .venv && source .venv/bin/activate

Step 2: Configure the client with a custom base URL

ChatOpenAI accepts a base_url parameter. Point it at the gateway’s OpenAI-compatible endpoint. The API key is the one the gateway issued—not your OpenAI key. The client speaks the same /v1/chat/completions protocol, so no other changes are needed.

import os
from langchain_openai import ChatOpenAI

os.environ["GATEWAY_KEY"] = "sk-..."  # your gateway key

llm = ChatOpenAI(
    model="openai/gpt-4o-mini",
    api_key=os.environ["GATEWAY_KEY"],
    base_url="https://api.n4n.ai/v1",
    temperature=0.2,
)

That is the entire network-level swap. Every llm.invoke(), llm.stream(), and llm.bind_tools() call you already wrote keeps working.

Step 3: Choose a model from the unified catalog

The gateway exposes 240+ models under a provider/model naming scheme. You are not locked to OpenAI’s roster. Pass any supported string to model. This is the main operational win: one client, many backends.

models = [
    "openai/gpt-4o-mini",
    "anthropic/claude-3.5-sonnet",
    "meta-llama/llama-3.1-70b-instruct",
]

for m in models:
    llm = ChatOpenAI(
        model=m,
        api_key=os.environ["GATEWAY_KEY"],
        base_url="https://api.n4n.ai/v1",
    )
    print(m, "->", llm.invoke("ping").content[:20])

If your old code hardcoded model="gpt-4o", change it to openai/gpt-4o. The explicit provider prefix prevents ambiguous routing when a model name exists across vendors.

Step 4: Pass routing directives and cache hints

The gateway (n4n.ai) honors client routing directives and forwards provider cache-control hints. You can send these via extra headers on the LangChain client without writing custom retry logic.

llm = ChatOpenAI(
    model="anthropic/claude-3.5-sonnet",
    api_key=os.environ["GATEWAY_KEY"],
    base_url="https://api.n4n.ai/v1",
    default_headers={
        "X-Route-Prefer": "anthropic",
        "X-Cache-Control": "ephemeral",
    },
)

Do not build your own provider-selection state machine. The gateway performs automatic fallback when a provider is rate-limited or degraded, so a single request path is sufficient for most production apps. If you need strict provider pinning, the routing header is a hint, not a guarantee—design your error budget accordingly.

Step 5: Stream tokens without changing your logic

LangChain’s streaming interface is identical regardless of backend. Use .stream() for sync loops or astream() in async services.

from langchain_core.messages import HumanMessage

llm = ChatOpenAI(
    model="meta-llama/llama-3.1-70b-instruct",
    api_key=os.environ["GATEWAY_KEY"],
    base_url="https://api.n4n.ai/v1",
    streaming=True,
)

for chunk in llm.stream([HumanMessage(content="Explain TCP slow start in 3 lines")]):
    print(chunk.content, end="", flush=True)

The gateway meters per-token usage even on streams. Inspect chunk.response_metadata on the final chunk for usage details. Some providers omit usage on intermediate chunks; do not assert on it mid-stream.

For async FastAPI handlers:

async for chunk in llm.astream(messages):
    await websocket.send_text(chunk.content)

Step 6: Verify the integration end to end

A minimal verification script confirms the swap worked and that you are hitting the gateway, not OpenAI directly.

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="openai/gpt-4o-mini",
    api_key=os.environ["GATEWAY_KEY"],
    base_url="https://api.n4n.ai/v1",
)

resp = llm.invoke("Return the word 'ok' and nothing else.")
print("CONTENT:", resp.content)
print("USAGE:", resp.response_metadata.get("usage"))
assert resp.content.strip().lower() == "ok"

Success criteria:

• The script prints CONTENT: ok.
• USAGE shows prompt_tokens and completion_tokens greater than zero.
• No traffic leaves to api.openai.com. Confirm with mitmproxy or tcpdump if you need forensic proof.

Step 7: Migrate an existing LangChain project

If your codebase already uses ChatOpenAI, you likely set openai_api_key and model. Do a scoped refactor:

1. Replace openai_api_key= with api_key= and read from your gateway env var.
2. Add base_url="https://api.n4n.ai/v1".
3. Prefix model names with the provider where needed.

Before:

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o", openai_api_key=os.environ["OPENAI_KEY"])

After repointing the client:

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    model="openai/gpt-4o",
    api_key=os.environ["GATEWAY_KEY"],
    base_url="https://api.n4n.ai/v1",
)

That is the whole diff for client construction. Chains, memory, and tool-calling code stay put. If you used OpenAI (the legacy completion class), migrate to ChatOpenAI first—the gateway speaks chat completions, not the old /v1/completions format.

Step 8: Handle tool calls and structured output

LangChain’s function-calling layer translates to the OpenAI tools schema. The gateway passes it through to providers that support it. Test with a simple tool to ensure the route works.

from langchain_core.tools import tool
from langchain_core.messages import HumanMessage

@tool
def get_weather(city: str) -> str:
    """Get weather for a city."""
    return f"Sunny in {city}"

llm = ChatOpenAI(
    model="anthropic/claude-3.5-sonnet",
    api_key=os.environ["GATEWAY_KEY"],
    base_url="https://api.n4n.ai/v1",
).bind_tools([get_weather])

msg = llm.invoke("Weather in Berlin?")
print(msg.tool_calls)

If tool_calls is populated, the gateway correctly forwarded the schema and the provider answered in the expected format. For structured output, use with_structured_output:

from pydantic import BaseModel

class Answer(BaseModel):
    city: str
    temp_c: int

structured = llm.with_structured_output(Answer)
print(structured.invoke("Extract: 22C in Paris"))

Not every model supports JSON mode; the gateway returns the provider’s native error if you pick one that doesn’t. Keep a fallback model string in your config.

Step 9: Observability and cost control

Because the gateway meters per-token usage, log response_metadata["usage"] on every call. Pipe it into your existing metrics stack:

usage = resp.response_metadata.get("usage", {})
statsd.incr("llm.tokens", usage.get("total_tokens", 0))

Set a model allowlist in your deployment config. Just because 240+ models are reachable doesn’t mean every service should roam freely. A retrieval chain should default to a cheap instruction-tuned model; reserve large reasoning models for explicit user requests.

Gotchas

• Some providers reject temperature=0 for certain models; the gateway passes it through unchanged. Test edge cases.
• response_metadata shape varies by provider. Don’t assume usage is present on the first chunk.
• If you use AzureChatOpenAI, that’s a different class with its own URL logic. Stick with ChatOpenAI for OpenAI-compatible gateways.
• LangChain may send user field; the gateway forwards it. Scrub PII before calling if your compliance regime requires it.

Pointing LangChain at the gateway instead of OpenAI is not a rewrite. It’s a base URL and a key. The rest is configuration and model selection.