Build a Next.js chatbot with LangChain.js

Building a langchain.js nextjs chatbot forces you to confront real architectural decisions: where to terminate streams, how to scope API keys, and when to let the model call tools. This guide walks through a working implementation using Next.js App Router, LangChain.js, and a typed React client, without hiding the sharp edges.

Step 1: Scaffold the project and install dependencies

Run the following to create a TypeScript Next.js app with the App Router and a src/ directory:

npx create-next-app@latest chatbot --ts --app --eslint --src-dir --no-tailwind
cd chatbot
npm install @langchain/core @langchain/openai

The @langchain/openai package provides ChatOpenAI, an OpenAI-compatible chat model client that supports streaming and tool binding. We deliberately avoid the umbrella langchain package to keep the dependency tree small. Node 18+ is required for the ReadableStream APIs used below.

Verify the scaffold boots before writing code:

npm run dev

Open http://localhost:3000 and confirm the default template renders. Kill the dev server; we will replace the page and add an API route.

Step 2: Configure the model and environment

Create .env.local at the project root. Never import this file into a "use client" component.

OPENAI_API_KEY=sk-...

If you’d rather not juggle multiple provider keys, an OpenAI-compatible gateway like n4n.ai exposes one endpoint for 240+ models and automatically fails over when a provider is rate-limited or degraded. Point ChatOpenAI at it by setting baseURL and apiKey in a server-only module.

Create src/lib/model.ts:

import { ChatOpenAI } from "@langchain/openai";

export function getChatModel() {
  return new ChatOpenAI({
    model: "gpt-4o-mini",
    streaming: true,
    // baseURL: "https://api.n4n.ai/v1",
    // apiKey: process.env.N4N_API_KEY,
  });
}

The model factory must stay server-side. Importing it into client code will embed your key in the browser bundle, which is a critical failure.

Step 3: Build the streaming API route

App Router route handlers run on the server and can return a ReadableStream. Create src/app/api/chat/route.ts:

import { NextRequest } from "next/server";
import { getChatModel } from "@/lib/model";

export const runtime = "nodejs";
export const dynamic = "force-dynamic";

export async function POST(req: NextRequest) {
  const { messages } = await req.json();
  // messages: Array<{ role: "user" | "assistant" | "system"; content: string }>
  const model = getChatModel();
  const stream = await model.stream(messages);

  const readable = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      for await (const chunk of stream) {
        const text = typeof chunk.content === "string" ? chunk.content : "";
        controller.enqueue(encoder.encode(text));
      }
      controller.close();
    },
  });

  return new Response(readable, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
}

This returns raw text tokens. For a production chatbot you would typically use Server-Sent Events (SSE) or the Vercel AI SDK’s LangChainAdapter, but raw streaming is the most transparent way to learn the mechanics. Note that model.stream yields AIMessageChunk objects; we extract .content and ignore non-string parts (such as tool calls) for now.

Step 4: Create the React chat client

Replace src/app/page.tsx with a client component that posts the message history and reads the stream:

"use client";

import { useState } from "react";

type Message = { role: "user" | "assistant"; content: string };

export default function Chat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");
  const [busy, setBusy] = useState(false);

  async function send() {
    if (!input.trim() || busy) return;
    const next = [...messages, { role: "user", content: input }];
    setMessages(next);
    setInput("");
    setBusy(true);

    const res = await fetch("/api/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ messages: next }),
    });
    if (!res.body) throw new Error("No response stream");

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let acc = "";
    setMessages([...next, { role: "assistant", content: "" }]);

    try {
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        acc += decoder.decode(value, { stream: true });
        setMessages([...next, { role: "assistant", content: acc }]);
      }
    } finally {
      setBusy(false);
    }
  }

  return (
    <div style={{ maxWidth: 600, margin: "2rem auto" }}>
      {messages.map((m, i) => (
        <p key={i}>
          <strong>{m.role}:</strong> {m.content}
        </p>
      ))}
      <input
        value={input}
        onChange={(e) => setInput(e.target.value)}
        onKeyDown={(e) => e.key === "Enter" && send()}
        disabled={busy}
        style={{ width: "80%" }}
      />
      <button onClick={send} disabled={busy}>
        Send
      </button>
    </div>
  );
}

The component maintains the full conversation array locally and sends it on each turn—LangChain’s ChatOpenAI expects the complete history, not just the latest message. Re-rendering on every token is acceptable for a demo; in a high-frequency UI, buffer decoder output and flush with requestAnimationFrame.

Step 5: Add a tool call

LangChain.js binds tools via the OpenAI function-calling interface. Extend src/lib/model.ts:

export function getChatModelWithTools() {
  const model = new ChatOpenAI({ model: "gpt-4o-mini", streaming: true });
  return model.bind({
    tools: [
      {
        type: "function",
        function: {
          name: "get_weather",
          parameters: {
            type: "object",
            properties: { city: { type: "string" } },
            required: ["city"],
          },
        },
      },
    ],
  });
}

In the route, tool calls arrive as a separate chunk field (chunk.tool_calls) when streaming. A complete implementation would detect that chunk, execute the function server-side, append the result as a tool message, and stream a second model pass. That loop is mechanical but outside the scope of a minimal langchain.js nextjs chatbot; the binding alone proves the wiring.

Step 6: Verify end-to-end

Start the dev server and open the page:

npm run dev

Type “Explain streaming in one sentence” and press Enter. The assistant text should appear incrementally. Open Chrome DevTools → Network → chat request → Response tab: you will see raw text chunks arriving, confirming the stream is not buffered.

Check for key leakage:

grep -r "sk-" .next 2>/dev/null && echo "LEAK" || echo "CLEAN"

The output should be CLEAN. If you wired the gateway, swap getChatModel to the commented variant and re-run; the chatbot behavior is identical but routes through the unified endpoint.

Success criteria:

• First token renders in the browser within ~1s on localhost.
• No provider key in the client bundle.
• Server logs show no unhandled rejection from the stream.

Step 7: Production hardening

Set export const dynamic = "force-dynamic" on the route (already done) to prevent Next.js from attempting static optimization. If you deploy to Vercel, runtime = "edge" works with @langchain/openai for pure streaming, but tool execution that uses Node built-ins (e.g., fs or certain fetch patterns) will break—keep Node runtime unless you’ve validated the edge build.

For per-token metering, capture chunk.usage on the final chunk if your provider sends it, or sum prompt and completion tokens from your gateway’s response headers. The langchain.js nextjs chatbot architecture above keeps model orchestration strictly server-side, so swapping providers or adding fallback logic never touches the React layer.