Connect LangChain.js to n4n.ai with the OpenAI package

Wiring up the langchain.js n4n.ai openai package takes about five minutes if you treat the gateway as a drop-in OpenAI replacement. You point the ChatOpenAI client at the gateway’s base URL, hand it an API key, and pick a model from the 240+ available behind that single endpoint. This guide walks through a working Node.js setup, from install to first streamed response, with the gotchas that aren’t in the README.

Step 1: Install the dependencies

LangChain.js splits model integrations into thin packages. For OpenAI-compatible backends you only need @langchain/openai and the core langchain package. The underlying openai SDK is pulled in transitively, but pin it explicitly to avoid version drift.

npm init -y
npm install @langchain/openai langchain openai dotenv

If you’re on TypeScript, add the Node types and tsx for execution:

npm install -D typescript @types/node tsx

Step 2: Set up environment variables

Never hard-code credentials. Put the gateway key and base URL in .env. The OpenAI client reads OPENAI_API_KEY and OPENAI_BASE_URL by convention, so we reuse those names to avoid custom loader code.

OPENAI_API_KEY=sk-n4n-your-real-key
OPENAI_BASE_URL=https://api.n4n.ai/v1
MODEL_NAME=anthropic/claude-3.5-sonnet

Load them before importing the LLM:

import * as dotenv from "dotenv";
dotenv.config();

Step 3: Initialize ChatOpenAI against the gateway

The langchain.js n4n.ai openai package integration is just ChatOpenAI with a different configuration.baseURL. Model names follow the gateway’s routing scheme—usually provider/model. Temperature and max tokens behave exactly as they would against OpenAI.

import { ChatOpenAI } from "@langchain/openai";

const chat = new ChatOpenAI({
  model: process.env.MODEL_NAME!,
  temperature: 0.2,
  maxTokens: 1024,
  apiKey: process.env.OPENAI_API_KEY,
  configuration: {
    baseURL: process.env.OPENAI_BASE_URL,
  },
});

One subtlety: LangChain’s ChatOpenAI wraps the openai SDK’s AzureOpenAI or OpenAI class depending on the URL. Pointing at a non-Azure URL forces the standard client, so streaming and function calls work unchanged.

Verify the client builds

console.log(chat.model);
console.log(chat.client.baseURL);

If those print your model string and https://api.n4n.ai/v1/, the wiring is correct before any network call.

Step 4: Send a blocking chat request

A minimal invoke returns an AIMessage with .content and .usage_metadata. The gateway meters per token; those numbers come straight from the provider response.

import { HumanMessage } from "@langchain/core/messages";

const res = await chat.invoke([
  new HumanMessage("Explain TCP slow start in one paragraph."),
]);

console.log(res.content);
console.log("Tokens:", res.usage_metadata);

Expected shape:

{
  "input_tokens": 11,
  "output_tokens": 84,
  "total_tokens": 95
}

If you get a 401, the key is wrong. A 404 means the model string isn’t routed—check the gateway’s model list.

Step 5: Stream tokens to the caller

Blocking calls waste round-trip time on long outputs. Use .stream() and iterate the async generator. Each chunk carries a delta; concatenate them yourself.

const stream = await chat.stream([
  new HumanMessage("List three Rust crates for async HTTP and why each exists."),
]);

let full = "";
for await (const chunk of stream) {
  const text = chunk.content as string;
  full += text;
  process.stdout.write(text);
}
console.log("\n--- stream done ---");

In a server context, pipe stream into the response body. LangChain’s LangChainAdapter (Vercel AI SDK) or a simple ReadableStream both work; the gateway sends SSE frames the OpenAI SDK already parses.

Step 6: Pin a provider and forward cache hints

The langchain.js n4n.ai openai package setup shines when you need provider-specific behavior without swapping clients. n4n.ai honors client routing directives and forwards provider cache-control hints, so you can reduce latency and cost without custom middleware. It also triggers automatic fallback when a provider is rate-limited or degraded, so the same model string keeps working across outages.

To request cached prompt handling, pass the header the gateway expects. With ChatOpenAI you inject headers via defaultHeaders in the configuration:

const cachedChat = new ChatOpenAI({
  model: "anthropic/claude-3.5-sonnet",
  apiKey: process.env.OPENAI_API_KEY,
  configuration: {
    baseURL: process.env.OPENAI_BASE_URL,
    defaultHeaders: {
      "X-N4N-Cache-Control": "ephemeral",
    },
  },
});

Routing is just the model prefix. Swap anthropic/ for openai/ or google/ and the request lands on that provider’s fleet. If that provider returns 429, the gateway retries on a healthy peer and the LangChain caller sees a single clean response.

Error handling

Wrap calls in try/catch. The OpenAI SDK throws APIError with .status. Map 429 to a user-facing “slow down” message; map 4xx validation to logs.

import { APIError } from "openai";

try {
  await chat.invoke([new HumanMessage("hi")]);
} catch (e) {
  if (e instanceof APIError && e.status === 429) {
    console.warn("Rate limited upstream; gateway already retried.");
  } else {
    throw e;
  }
}

Step 7: Verify success end to end

A complete verification script should (1) load env, (2) send a short message, (3) assert the response is non-empty, and (4) confirm token usage is present. Run it with tsx:

import * as dotenv from "dotenv";
import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage } from "@langchain/core/messages";

dotenv.config();

const chat = new ChatOpenAI({
  model: process.env.MODEL_NAME!,
  apiKey: process.env.OPENAI_API_KEY,
  configuration: { baseURL: process.env.OPENAI_BASE_URL },
});

const msg = await chat.invoke([new HumanMessage("Ping. Reply with 'pong' only.")]);
const text = msg.content as string;
if (!text.includes("pong")) throw new Error("Unexpected response");
if (!msg.usage_metadata?.total_tokens) throw new Error("No token metering");
console.log("OK:", text.trim(), msg.usage_metadata);

If that prints OK: pong {...} you have a working langchain.js n4n.ai openai package pipeline.

Smoke test from curl

Before writing TypeScript, confirm the endpoint independently:

curl $OPENAI_BASE_URL/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"anthropic/claude-3.5-sonnet","messages":[{"role":"user","content":"pong?"}]}'

A valid JSON completion means the credential and base URL are correct, isolating LangChain issues from network issues.

Notes on production shape

Keep the ChatOpenAI instance singleton. It holds a pooled HTTP client; recreating it per request leaks sockets. In a serverless function, instantiate at module scope.

For batch workloads, use chat.batch([...]) rather than a loop of invoke. The gateway accepts parallel requests; the client schedules them efficiently.

If you need structured output, prefer chat.withStructuredOutput(schema) from LangChain’s experimental parsers. It sets response_format and the gateway forwards it to providers that support JSON mode.

The langchain.js n4n.ai openai package path gives you the entire LangChain toolchain—retrievers, agents, callbacks—while the gateway absorbs provider heterogeneity. You write one client, one model string, and let the routing and fallback handle the rest.