LangChain quickstart: n4n.ai as your model provider

This langchain quickstart n4n.ai provider guide gets you from zero to a working multi-model LangChain app in under ten minutes. Instead of hardcoding vendor SDKs, you point LangChain’s OpenAI-compatible client at a single gateway endpoint and gain access to hundreds of models with automatic failover.

Prerequisites

• Python 3.10 or newer
• langchain-openai (LangChain’s OpenAI integration)
• An API key for the gateway (set as N4N_API_KEY in your environment)

If you’re on Node, the same patterns apply with @langchain/openai.

Install and configure

pip install langchain-openai openai

Set the key:

export N4N_API_KEY="sk-..."

Wire up ChatOpenAI

LangChain’s ChatOpenAI wraps the OpenAI Python client. Point base_url at the gateway and use provider-prefixed model slugs.

from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="openai/gpt-4o-mini",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
    temperature=0.1,
)

The model string follows provider/model. If you omit the provider prefix, the gateway applies its default routing.

First call

resp = llm.invoke("What is the difference between a gateway and a proxy?")
print(resp.content)

That’s it. You’re now sending traffic through the gateway.

Routing across models

A key reason to use a gateway is to avoid rewriting code when you switch backends. The langchain quickstart n4n.ai provider pattern keeps model selection declarative.

claude = ChatOpenAI(
    model="anthropic/claude-3.5-sonnet",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
)

If you need to force a fallback order, pass it through model_kwargs. The gateway honors client routing directives via the request body.

llm_with_fallback = ChatOpenAI(
    model="anthropic/claude-3.5-sonnet",
    api_key=os.environ["N4N_API_KEY"],
    base_url="https://api.n4n.ai/v1",
    model_kwargs={"fallback_models": ["openai/gpt-4o-mini", "meta-llama/llama-3.1-70b-instruct"]},
)

Check your gateway’s exact field name; the above mirrors common OpenRouter-style schemas.

Automatic fallback in practice

n4n.ai exposes a single OpenAI-compatible endpoint that fronts 240+ models and applies automatic fallback when a provider is degraded or rate-limited. You don’t get a 429 from the upstream; you get a response from the next healthy provider. This is convenient, but understand the tradeoff: the response may come from a different model than you requested, so guard any code that assumes specific tokenizer behavior or JSON schema strictness.

When streaming, fallback typically triggers before the first token. If the stream has started, the gateway will not silently swap models mid-stream. Design your UI to handle a brief delay rather than partial text from model A followed by model B.

Cache control and provider hints

Some providers support prompt caching via cache-control headers or body fields. The gateway forwards provider cache-control hints unchanged. In LangChain, pass them via model_kwargs or a system message with extra metadata.

Example using Anthropic’s cache control through the gateway:

from langchain_core.messages import SystemMessage, HumanMessage

messages = [
    SystemMessage(content="You are a strict SQL reviewer." * 50,
                  additional_kwargs={"cache_control": {"type": "ephemeral"}}),
    HumanMessage(content="Review: SELECT * FROM users WHERE id = 1"),
]

llm.invoke(messages)

If the underlying provider ignores cache control, the gateway does not synthesize it. You pay full token cost.

Per-token usage metering

Every response includes usage metadata. LangChain surfaces it in response_metadata.

resp = llm.invoke("Count to five.")
print(resp.response_metadata["token_usage"])

The gateway meters per-token usage across all providers, so your billing line items stay consistent even when fallback swaps the backend. Log this field; it’s the only reliable way to attribute cost in a multi-model setup.

Error handling and retries

LangChain does not automatically retry on gateway-level 5xx. Wrap calls if you expect flaky networks:

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def ask(prompt: str):
    return llm.invoke(prompt)

But note: if the gateway already performed fallback, a retry just hits the same logic. Only retry on network errors, not on content filter rejections.

Structured output caveats

Using llm.with_structured_output(SomePydantic) relies on function calling or JSON mode. Not all 240+ models support that. Test on your specific slug before shipping. If a fallback model lacks tool support, the call will fail even though the primary model would have succeeded.

Common pitfalls

Trailing slash on base_url. OpenAI client concatenates paths. https://api.n4n.ai/v1/ often yields double slashes and 404s. Use no trailing slash.

Model slug drift. Providers rename models. gpt-4o vs openai/gpt-4o. Always prefix once you adopt a gateway.

Temperature out of range. Some open-weight models clamp or ignore temperature. Don’t assume a low temperature yields deterministic output.

Streaming + fallback assumptions. As noted, fallback is pre-stream. If you see a slow first token, it may be cross-provider negotiation, not network latency.

Hidden system messages. Gateways may inject routing or policy messages. Inspect resp.response_metadata for any gateway-added flags.

Tradeoffs and when to use this

The langchain quickstart n4n.ai provider approach trades direct provider coupling for operational simplicity. You get one key, one client, one usage schema. You lose native SDK features like provider-specific batch endpoints or fine-tune management UIs.

Use it when:

• You prototype across many models weekly.
• You need resilience to single-provider outages.
• You want unified token metering.

Skip it when:

• You depend on a provider’s proprietary tooling (e.g., OpenAI assistants).
• You need sub-millisecond cold starts (extra hop adds latency).
• Compliance requires direct contracts with each model vendor.

A minimal TS variant

For Node users, the shape is identical:

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "openai/gpt-4o-mini",
  apiKey: process.env.N4N_API_KEY,
  configuration: { baseURL: "https://api.n4n.ai/v1" },
});

Same routing directives go in modelKwargs.

Closing checklist

• Set base_url without trailing slash.
• Prefix model slugs with provider.
• Pass fallback list via model_kwargs if needed.
• Log token_usage from every response.
• Test streaming behavior under simulated provider outage.

That’s the full path. The langchain quickstart n4n.ai provider pattern is boring in the best way: it works, and it gets out of your way when a provider falls over.