Set up LangChain with n4n.ai in under 10 minutes

To set up LangChain with n4n.ai, you point LangChain’s OpenAI-compatible chat model at the n4n.ai gateway endpoint and supply an API key. This tutorial walks through a working Python configuration, from install to first streaming response, in less than ten minutes.

Step 1: Install the LangChain OpenAI package

LangChain does not ship model integrations in the core package. For any OpenAI-compatible endpoint you only need langchain-openai, which wraps the official OpenAI SDK and exposes the ChatOpenAI class.

pip install langchain-openai langchain-core python-dotenv

If you plan to use async flows or streaming in a web framework, also install httpx (already a transitive dependency, but pin it explicitly to avoid surprises):

pip install httpx

Verify the install by importing the class:

from langchain_openai import ChatOpenAI
print(ChatOpenAI.__name__)

A clean import with no ModuleNotFoundError means you are ready to configure credentials.

Step 2: Configure credentials and base URL

The gateway exposes a single OpenAI-compatible endpoint that addresses 240+ models. Put your key and the base URL in a .env file so they never touch source control.

# .env
N4N_API_KEY=sk-your-real-key-here
N4N_BASE_URL=https://api.n4n.ai/v1

Load it early in your application entrypoint:

from dotenv import load_dotenv
load_dotenv()

If you already have OPENAI_API_KEY wired into other tools, do not reuse it—the gateway expects its own key. The base URL must end with /v1 to match the OpenAI SDK’s path construction; otherwise you will get 404s on /chat/completions.

Step 3: Initialize the chat model

ChatOpenAI accepts api_key, base_url, model, and standard sampling params. The model string is forwarded verbatim to the gateway, so use any model identifier the gateway supports (e.g., gpt-4o-mini, claude-3-5-sonnet, or a provider-agnostic alias).

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    api_key=os.environ["N4N_API_KEY"],
    base_url=os.environ["N4N_BASE_URL"],
    model="gpt-4o-mini",
    temperature=0.2,
    max_tokens=512,
    timeout=30,
    max_retries=2,
)

A few notes from production use:

• max_retries inside the SDK handles transient 5xx and connection resets. The gateway itself performs automatic fallback when a upstream provider is rate-limited or degraded, so a single retry config here is enough.
• timeout should be set explicitly; the default 600s will hang a CLI tool if a provider stalls.
• model is not validated client-side. A typo returns a 400 from the gateway with a clear message.

Step 4: Send a test request

A minimal synchronous call proves the wiring:

response = llm.invoke("What is the difference between a thread and a process in Linux?")
print(response.content)

Expected output is a concise technical answer. The returned AIMessage also carries usage metadata:

print(response.usage_metadata)
# {'input_tokens': 14, 'output_tokens': 128, 'total_tokens': 142}

Because the gateway performs per-token usage metering, those numbers reflect what the upstream provider reported (or the gateway’s own count for models that omit it). If you see total_tokens: 0, your model or provider path is not returning usage; check the gateway dashboard, not LangChain.

Step 5: Verify success

Success is not just a printed string. Confirm all three:

1. HTTP 200: run with export DEBUG=1 or langchain.debug = True to see the raw request/response.
2. Correct model: the response response.response_metadata["model"] should match what you requested (some providers rewrite the id).
3. Usage present: as shown above, usage_metadata is populated.

A quick assertion-based check:

assert response.content.strip()
assert response.usage_metadata["total_tokens"] > 0
print("OK: LangChain is talking to the gateway")

If this prints OK, you have completed the core task to set up LangChain with n4n.ai.

Step 6: Pass routing directives and cache hints

The gateway honors client routing directives and forwards provider cache-control hints. LangChain forwards arbitrary headers through default_headers. Use this to pin a provider or set a cache TTL without changing the model string.

llm_with_headers = ChatOpenAI(
    api_key=os.environ["N4N_API_KEY"],
    base_url=os.environ["N4N_BASE_URL"],
    model="claude-3-5-sonnet",
    default_headers={
        "Cache-Control": "max-age=300",
    },
)

Standard HTTP Cache-Control is understood by providers that support prompt caching (e.g., Anthropic). Proprietary routing headers follow the gateway’s documented schema; pass them the same way. Do not implement your own fallback loop—the gateway already shifts traffic when a provider returns 429/5xx.

Step 7: Stream tokens

For CLI or chat UIs, streaming avoids the dead wait. Enable it with stream=True and iterate:

from langchain_core.messages import HumanMessage

stream = llm.stream([HumanMessage(content="Write a 5-line Python decorator that retries on exception.")])
for chunk in stream:
    print(chunk.content, end="", flush=True)
print()

The streaming generator yields AIMessageChunk objects. Concatenate .content to reconstruct the final text. If you need usage at the end of a stream, capture the chunk.response_metadata from the final chunk—some providers only emit usage on the last frame.

Async streaming works identically with astream:

async for chunk in llm.astream("Explain RAID 5 vs RAID 10"):
    print(chunk.content, end="", flush=True)

Step 8: Compose with LangChain primitives

Once the model is initialized, drop it into a chain. A common pattern is a prompt template plus a parser:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a senior SRE. Answer in bullet points."),
    ("user", "{question}")
])

chain = prompt | llm | StrOutputParser()

print(chain.invoke({"question": "How do I debug OOM kills in Kubernetes?"}))

This is the same llm object; the gateway sees a normal chat completion request. No changes to LCEL are required.

Troubleshooting

401 Unauthorized
Key missing or wrong env var name. Print os.environ["N4N_API_KEY"][:4] to confirm it loaded.

404 Not Found
Base URL missing /v1, or you set base_url to the dashboard host instead of the API host.

Model not found
The gateway returns 400 with a list of valid families. Model aliases are case-sensitive.

Streaming hangs
You passed stream=True but called .invoke(). Use .stream() or .astream().

Timeouts on long outputs
Raise max_tokens and timeout. The gateway will not truncate mid-stream, but the client will disconnect if it gives up first.

What you have now

You can set up LangChain with n4n.ai in a single file: load env, build ChatOpenAI with the gateway base URL, and call .invoke or .stream. The gateway handles provider selection, fallback, and token metering behind one endpoint. From here, swap model strings to compare providers, attach headers for caching, and compose chains with the rest of LangChain’s ecosystem without further gateway-specific code.