Deploying a LangChain.js app to Vercel

Deploying a LangChain.js app to Vercel requires understanding how serverless functions handle long-running LLM requests, streaming responses, and environment variable management. This guide walks through a production-ready setup that handles timeouts, cold starts, and provider failover without vendor lock-in.

Step 1: Initialize the project with the right dependencies

Start with a fresh Next.js project using the App Router — it gives you native streaming support via ReadableStream and edge runtime compatibility.

npx create-next-app@latest langchain-vercel --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"
cd langchain-vercel

Install the LangChain core packages and your provider of choice. For this guide we’ll use OpenAI-compatible endpoints so you can swap providers (including n4n.ai) without code changes:

npm install @langchain/core @langchain/openai langchain
npm install -D @types/node

The @langchain/openai package works with any OpenAI-compatible API — just change the baseURL and apiKey.

Step 2: Create a reusable LLM client factory

Avoid instantiating models inside route handlers. Create a singleton factory that reads configuration from environment variables and applies sensible defaults for serverless environments.

// src/lib/llm.ts
import { ChatOpenAI } from "@langchain/openai";

type ModelConfig = {
  model: string;
  temperature?: number;
  maxTokens?: number;
  streaming?: boolean;
};

const DEFAULT_CONFIG: ModelConfig = {
  model: process.env.DEFAULT_MODEL ?? "gpt-4o-mini",
  temperature: 0.7,
  maxTokens: 2000,
  streaming: true,
};

export function createChatModel(overrides: Partial<ModelConfig> = {}) {
  const config = { ...DEFAULT_CONFIG, ...overrides };

  return new ChatOpenAI({
    modelName: config.model,
    temperature: config.temperature,
    maxTokens: config.maxTokens,
    streaming: config.streaming,
    openAIApiKey: process.env.LLM_API_KEY,
    configuration: {
      baseURL: process.env.LLM_BASE_URL ?? "https://api.openai.com/v1",
      defaultHeaders: {
        "HTTP-Referer": process.env.NEXT_PUBLIC_APP_URL ?? "http://localhost:3000",
        "X-Title": "LangChain Vercel Demo",
      },
    },
  });
}

Key decisions here:

• Streaming enabled by default — Vercel’s Node.js runtime supports streaming responses up to 60s (Pro) or 300s (Enterprise). The Edge runtime has a hard 30s limit.
• Base URL configurable — Point to any OpenAI-compatible gateway. This is where you’d swap in a fallback-enabled endpoint if you need automatic provider failover.
• Headers for observability — HTTP-Referer and X-Title help providers identify your traffic for analytics and abuse prevention.

Step 3: Build a streaming chat route handler

Create a route that accepts messages, streams tokens back to the client, and handles the Vercel-specific response format.

// src/app/api/chat/route.ts
import { createChatModel } from "@/lib/llm";
import { HumanMessage, SystemMessage, AIMessage } from "@langchain/core/messages";
import { NextRequest, NextResponse } from "next/server";

export const runtime = "nodejs"; // Required for streaming >30s
export const maxDuration = 60; // Vercel Pro: 60s, Enterprise: 300s

export async function POST(req: NextRequest) {
  try {
    const { messages, systemPrompt, model } = await req.json();

    if (!messages || !Array.isArray(messages)) {
      return NextResponse.json(
        { error: "messages array is required" },
        { status: 400 }
      );
    }

    const llm = createChatModel({ model, streaming: true });

    // Convert plain objects to LangChain message classes
    const lcMessages = messages.map((m: { role: string; content: string }) => {
      switch (m.role) {
        case "system":
          return new SystemMessage(m.content);
        case "assistant":
          return new AIMessage(m.content);
        default:
          return new HumanMessage(m.content);
      }
    });

    if (systemPrompt) {
      lcMessages.unshift(new SystemMessage(systemPrompt));
    }

    const stream = await llm.stream(lcMessages);

    // Transform async iterable to ReadableStream for Vercel
    const encoder = new TextEncoder();
    const readable = new ReadableStream({
      async start(controller) {
        for await (const chunk of stream) {
          const content = chunk.content;
          if (typeof content === "string" && content.length > 0) {
            controller.enqueue(encoder.encode(`data: ${JSON.stringify({ content })}\n\n`));
          }
        }
        controller.enqueue(encoder.encode("data: [DONE]\n\n"));
        controller.close();
      },
    });

    return new NextResponse(readable, {
      headers: {
        "Content-Type": "text/event-stream",
        "Cache-Control": "no-cache, no-transform",
        Connection: "keep-alive",
      },
    });
  } catch (error) {
    console.error("Chat API error:", error);
    return NextResponse.json(
      { error: error instanceof Error ? error.message : "Internal server error" },
      { status: 500 }
    );
  }
}

Critical Vercel-specific details:

• export const runtime = "nodejs" — The Edge runtime cannot stream beyond 30 seconds. Node.js runtime gives you up to 60s on Pro plans.
• export const maxDuration = 60 — Explicitly declare the timeout so Vercel doesn’t kill the function early.
• SSE format with data: [DONE]\n\n — Matches the OpenAI streaming protocol that most frontend clients expect.

Step 4: Add a minimal frontend to test streaming

Replace the default page with a simple chat interface that consumes the SSE stream.

// src/app/page.tsx
"use client";

import { useState, FormEvent, ChangeEvent } from "react";

export default function ChatPage() {
  const [messages, setMessages] = useState<Array<{ role: string; content: string }>>([]);
  const [input, setInput] = useState("");
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);

  const handleSubmit = async (e: FormEvent) => {
    e.preventDefault();
    if (!input.trim() || loading) return;

    const userMessage = { role: "user", content: input };
    setMessages((prev) => [...prev, userMessage]);
    setInput("");
    setLoading(true);
    setError(null);

    try {
      const response = await fetch("/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ messages: [...messages, userMessage] }),
      });

      if (!response.ok) {
        throw new Error(`HTTP ${response.status}`);
      }

      const reader = response.body?.getReader();
      const decoder = new TextDecoder();
      let assistantContent = "";

      if (!reader) throw new Error("No response body");

      setMessages((prev) => [...prev, { role: "assistant", content: "" }]);

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        const chunk = decoder.decode(value);
        const lines = chunk.split("\n").filter((line) => line.startsWith("data: "));

        for (const line of lines) {
          const data = line.slice(6);
          if (data === "[DONE]") continue;

          try {
            const parsed = JSON.parse(data);
            if (parsed.content) {
              assistantContent += parsed.content;
              setMessages((prev) => {
                const next = [...prev];
                next[next.length - 1] = { role: "assistant", content: assistantContent };
                return next;
              });
            }
          } catch {
            // Ignore parse errors on partial chunks
          }
        }
      }
    } catch (err) {
      setError(err instanceof Error ? err.message : "Request failed");
      setMessages((prev) => prev.slice(0, -1)); // Remove pending assistant message
    } finally {
      setLoading(false);
    }
  };

  return (
    <main className="max-w-2xl mx-auto p-4">
      <h1 className="text-2xl font-bold mb-4">LangChain.js on Vercel</h1>

      <div className="h-96 overflow-y-auto border rounded p-4 space-y-3 mb-4">
        {messages.map((msg, i) => (
          <div key={i} className={`flex ${msg.role === "user" ? "justify-end" : "justify-start"}`}>
            <div
              className={`max-w-[80%] p-3 rounded-lg ${
                msg.role === "user" ? "bg-blue-500 text-white" : "bg-gray-100"
              }`}
            >
              {msg.content}
            </div>
          </div>
        ))}
        {loading && (
          <div className="flex justify-start">
            <div className="bg-gray-100 p-3 rounded-lg animate-pulse">...</div>
          </div>
        )}
      </div>

      {error && <div className="text-red-500 text-sm mb-2">{error}</div>}

      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          type="text"
          value={input}
          onChange={(e: ChangeEvent<HTMLInputElement>) => setInput(e.target.value)}
          placeholder="Ask something..."
          className="flex-1 border rounded px-3 py-2"
          disabled={loading}
        />
        <button type="submit" disabled={loading || !input.trim()} className="px-4 py-2 bg-blue-500 text-white rounded disabled:opacity-50">
          Send
        </button>
      </form>
    </main>
  );
}

Step 5: Configure environment variables

Create .env.local for local development and add the same variables in Vercel’s dashboard.

# .env.local
LLM_API_KEY=sk-your-key-here
LLM_BASE_URL=https://api.openai.com/v1
DEFAULT_MODEL=gpt-4o-mini
NEXT_PUBLIC_APP_URL=http://localhost:3000

In Vercel: Project Settings → Environment Variables → add each key for Production, Preview, and Development environments.

If you’re using a gateway that supports automatic fallback (like n4n.ai), set LLM_BASE_URL to that gateway’s endpoint. The same code works — the gateway handles provider selection, retries, and cache-control forwarding transparently.

Step 6: Add Vercel configuration for function tuning

Create vercel.json at the project root to control function behavior beyond what maxDuration provides.

// vercel.json
{
  "functions": {
    "src/app/api/chat/route.ts": {
      "maxDuration": 60,
      "memory": 1024
    }
  },
  "headers": [
    {
      "source": "/api/(.*)",
      "headers": [
        { "key": "Access-Control-Allow-Origin", "value": "*" },
        { "key": "Access-Control-Allow-Methods", "value": "POST, OPTIONS" },
        { "key": "Access-Control-Allow-Headers", "value": "Content-Type" }
      ]
    }
  ]
}

• Memory: 1024 MB — LangChain with large contexts can exceed the default 128 MB. 1024 MB is the minimum for comfortable streaming with tool calls.
• CORS headers — Required if you call the API from a different origin (e.g., a separate frontend deployment).

Step 7: Deploy to Vercel

Push to GitHub, then import in Vercel:

git init
git add .
git commit -m "Initial LangChain.js Vercel deployment"
git branch -M main
git remote add origin https://github.com/yourusername/langchain-vercel.git
git push -u origin main

In Vercel:

1. Import Project → Select your repo
2. Framework Preset: Next.js (auto-detected)
3. Environment Variables: Add the four from Step 5
4. Deploy

Vercel will run npm run build and deploy the Node.js functions. The first deploy takes 2-3 minutes.

Step 8: Verify the deployment

Visit your production URL (e.g., https://langchain-vercel.vercel.app). You should see the chat interface.

Test checklist:

1. Basic request — Send “Hello”. Tokens should stream in word-by-word.
2. Long response — Ask “Write a 500-word essay on serverless architecture”. Verify it completes without 504 timeout.
3. Conversation history — Send a follow-up. The model should reference prior context.
4. Error handling — Temporarily set an invalid LLM_API_KEY, send a message, confirm the UI shows a friendly error.
5. Cold start — Wait 10+ minutes, send a request. First token latency should be <3s on Node.js runtime.

Check function logs in Vercel: Functions tab → api/chat → View Function Logs. Look for:

• POST /api/chat 200 on success
• Stream chunks logged if you add console.log in the route
• No Function timeout errors

Step 9: Handle production concerns

Rate limiting

Add a simple in-memory rate limiter per IP for the chat endpoint:

// src/lib/rate-limit.ts
const requests = new Map<string, number[]>();
const WINDOW_MS = 60_000;
const MAX_REQUESTS = 20;

export function checkRateLimit(ip: string): boolean {
  const now = Date.now();
  const windowStart = now - WINDOW_MS;
  const userRequests = requests.get(ip)?.filter((t) => t > windowStart) ?? [];

  if (userRequests.length >= MAX_REQUESTS) return false;

  userRequests.push(now);
  requests.set(ip, userRequests);
  return true;
}

Use it in the route:

import { checkRateLimit } from "@/lib/rate-limit";

export async function POST(req: NextRequest) {
  const ip = req.headers.get("x-forwarded-for")?.split(",")[0]?.trim() ?? "unknown";
  if (!checkRateLimit(ip)) {
    return NextResponse.json({ error: "Rate limit exceeded" }, { status: 429 });
  }
  // ... rest of handler
}

Structured logging

Replace console.log with a structured logger that Vercel can parse:

// src/lib/logger.ts
export function log(event: string, data: Record<string, unknown> = {}) {
  console.log(JSON.stringify({ timestamp: new Date().toISOString(), event, ...data }));
}

Health check endpoint

Add a lightweight health check for load balancer probes:

// src/app/api/health/route.ts
export async function GET() {
  return NextResponse.json({ status: "ok", timestamp: new Date().toISOString() });
}

Step 10: Optional — Enable Edge caching for non-streaming requests

If you have endpoints that don’t stream (e.g., embeddings, classification), deploy them to the Edge runtime for lower latency and automatic caching:

// src/app/api/embed/route.ts
import { OpenAIEmbeddings } from "@langchain/openai";
export const runtime = "edge";

export async function POST(req: NextRequest) {
  const { text } = await req.json();
  const embeddings = new OpenAIEmbeddings({
    openAIApiKey: process.env.LLM_API_KEY,
    configuration: { baseURL: process.env.LLM_BASE_URL },
  });
  const vector = await embeddings.embedQuery(text);
  return NextResponse.json({ vector });
}

Edge functions cold-start faster (~50ms vs ~300ms) but remember: no streaming beyond 30s.

Summary of key Vercel-specific decisions

This setup has been running in production across multiple projects. The only thing that changes per deployment is the LLM_BASE_URL — swap between OpenAI, Azure, or a multi-provider gateway without touching application code.