LangChain JS streaming chat with n4n.ai's chat completions API

Wiring up langchain js streaming chat completions against an OpenAI-compatible gateway saves you from vendor lock-in and keeps your app resilient when a model provider hiccups. This tutorial builds a minimal Node.js script that streams tokens from n4n.ai’s chat completions API using LangChain’s JS bindings, then extends it into a small interactive REPL.

Prerequisites

• Node.js 18+ (native fetch and async iterators)
• npm or pnpm
• An API key from the gateway, exported as N4N_API_KEY
• Comfort with ES modules and basic TypeScript

Scaffold the project

mkdir lc-stream && cd lc-stream
npm init -y
npm pkg set type=module
npm install @langchain/openai @langchain/core dotenv

If you run TypeScript directly, add tsx as a dev dependency. The code samples below use ESM imports that work in both .js and .ts.

Point LangChain at the gateway

LangChain’s ChatOpenAI class talks to any OpenAI-compatible /chat/completions endpoint. You override the base URL and pass your gateway key. The model field accepts any of the 240+ model IDs routed by the gateway.

// chat.ts
import { ChatOpenAI } from "@langchain/openai";
import * as dotenv from "dotenv";

dotenv.config();

export const chat = new ChatOpenAI({
  apiKey: process.env.N4N_API_KEY,
  model: "anthropic/claude-3.5-sonnet", // or "openai/gpt-4o-mini"
  streaming: true,
  configuration: {
    baseURL: "https://api.n4n.ai/v1",
  },
});

Set streaming: true explicitly. Without it, .stream() may still function but LangChain can buffer internally, defeating the purpose.

Stream your first completion

The core of langchain js streaming chat completions is the .stream() method, which returns an async iterable of ChatGenerationChunk objects. Each chunk carries a content string fragment.

import { chat } from "./chat.ts";

const stream = await chat.stream([
  { role: "user", content: "Explain TCP slow start in three sentences." },
]);

for await (const chunk of stream) {
  process.stdout.write(chunk.content);
}
console.log("\n--- done ---");

Run it with node chat.ts (or tsx chat.ts). Tokens appear one by one on stdout.

Expected output

Exact text varies by model, but the terminal prints incrementally:

TCP slow start is a congestion control mechanism that gradually increases
the amount of data sent at the beginning of a connection. It starts with
a small congestion window and doubles it every round-trip time until
packet loss or a threshold is observed. This avoids overwhelming the
network path with a full burst of data immediately.
--- done ---

A 401 means the env var is missing. A 404 means the model ID is not routed; check the gateway’s /models list.

Add conversation memory

A single user message is boring. Keep a mutable array of messages:

const messages = [
  { role: "system", content: "You are a terse networking tutor." },
];

async function ask(q: string) {
  messages.push({ role: "user", content: q });
  const stream = await chat.stream(messages);
  let reply = "";
  for await (const chunk of stream) {
    process.stdout.write(chunk.content);
    reply += chunk.content;
  }
  messages.push({ role: "assistant", content: reply });
  console.log("\n---");
}

await ask("What is a SYN flood?");
await ask("How does slow start mitigate that?");

This keeps context without a vector store. For long sessions, trim older messages to stay under the model’s context window.

Build a readline REPL

Pipe stdin to the same loop for an interactive toy:

import readline from "node:readline";
import { chat } from "./chat.ts";

const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
rl.setPrompt("you> ");
rl.prompt();

for await (const line of rl) {
  if (line.trim() === "exit") break;
  const stream = await chat.stream([{ role: "user", content: line }]);
  process.stdout.write("bot> ");
  for await (const chunk of stream) {
    process.stdout.write(chunk.content);
  }
  console.log("\n");
  rl.prompt();
}

Type a question, hit enter, watch tokens stream. exit or Ctrl+C quits.

Observe token usage

LangChain chunks may include response_metadata. The gateway meters per-token usage even during streams; you can capture the final count if the provider sends it:

let completionTokens = 0;
const stream = await chat.stream([{ role: "user", content: "Count to 10." }]);
for await (const chunk of stream) {
  process.stdout.write(chunk.content);
  const usage = chunk.response_metadata?.usage;
  if (usage?.completion_tokens) completionTokens = usage.completion_tokens;
}
console.error(`\n[usage] completion tokens: ${completionTokens}`);

If usage is undefined mid-stream, don’t panic—some providers only emit it on the final chunk. The gateway’s per-token metering still records the real count for billing.

Resilience via gateway fallback

Network calls fail. A provider returns 429, or a region degrades. Because n4n.ai performs automatic fallback when a provider is rate-limited or degraded, the single ChatOpenAI config above keeps working across underlying provider outages without code changes. You should still wrap streams in try/catch to handle hard errors and abort hung connections:

import { AbortController } from "node:abort-controller";

const ac = new AbortController();
setTimeout(() => ac.abort(), 30_000);

try {
  const stream = await chat.stream(
    [{ role: "user", content: "Long story about distributed systems." }],
    { signal: ac.signal }
  );
  for await (const chunk of stream) process.stdout.write(chunk.content);
} catch (err) {
  console.error("\nStream aborted or failed:", err.message);
}

LangChain forwards the signal to the underlying fetch call. Combine this with the gateway’s fallback and you get a robust streaming client with minimal code.

Routing directives and cache hints

The gateway honors client routing directives and forwards provider cache-control hints. If you need to pin a provider or set a cache key, pass extra headers through LangChain’s configuration or clientOptions depending on your version. For most apps, default routing is fine; the point is that the OpenAI-compatible surface means you don’t rewrite anything when you switch models.

Where to go next

Swap the model string to any of the 240+ routed IDs and compare latency. Replace process.stdout.write with a WebSocket push to get browser streaming. Or plug chat into LangChain’s ConversationChain for higher-level memory. The langchain js streaming chat completions pattern stays the same: configure base URL, set streaming: true, iterate the async generator.

That’s the whole integration. No custom transport, no polling—just standard LangChain against an OpenAI-shaped endpoint.