Vercel AI SDK chatbot with Claude 3.5 Sonnet and n4n.ai

This tutorial walks through building a streaming chatbot with the Vercel AI SDK, Next.js App Router, and Anthropic’s Claude 3.5 Sonnet accessed through n4n.ai. You’ll end up with a working chat interface that handles streaming responses, tool calls, and provider fallbacks — patterns that scale to production workloads.

Prerequisites

• Node.js 20+ and pnpm (or npm/yarn)
• An n4n.ai API key (get one at n4n.ai)
• Basic familiarity with Next.js App Router and React Server Components

The stack: Next.js 14+, Vercel AI SDK 4.x, TypeScript, Tailwind CSS for styling.

Project setup

Create the project and install dependencies:

pnpm create next-app@latest ai-chatbot --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"
cd ai-chatbot
pnpm add ai @ai-sdk/anthropic zod
pnpm add -D @types/node

The @ai-sdk/anthropic package provides the Anthropic provider for the AI SDK. We’ll route requests through n4n.ai by configuring the base URL.

Environment configuration

Create .env.local in the project root:

# .env.local
N4N_API_KEY="your-n4n-api-key"
N4N_BASE_URL="https://api.n4n.ai/v1"

The AI SDK’s Anthropic provider accepts a baseURL option, so we can point it at n4n.ai without changing any other code.

The route handler

Create src/app/api/chat/route.ts. This is where the AI SDK’s streamText does the heavy lifting:

// src/app/api/chat/route.ts
import { streamText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: anthropic("claude-3-5-sonnet-20241022", {
      baseURL: process.env.N4N_BASE_URL,
      headers: {
        Authorization: `Bearer ${process.env.N4N_API_KEY}`,
      },
    }),
    messages,
    system: "You are a helpful assistant. Be concise.",
    temperature: 0.3,
    maxTokens: 4096,
  });

  return result.toDataStreamResponse();
}

A few things to note:

• streamText returns a ReadableStream that the client consumes incrementally
• The anthropic provider function accepts an options object where we inject the n4n.ai base URL and API key
• maxDuration extends the Vercel function timeout to 30 seconds for longer streams
• toDataStreamResponse() handles the proper SSE formatting and headers

The chat interface

Now build the client component. Create src/components/chat.tsx:

// src/components/chat.tsx
"use client";

import { useChat } from "ai/react";
import { useState } from "react";

export function Chat() {
  const [input, setInput] = useState("");
  const { messages, append, isLoading, error, stop } = useChat({
    api: "/api/chat",
    onError: (err) => {
      console.error("Chat error:", err);
      alert("Something went wrong. Check the console.");
    },
  });

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (!input.trim() || isLoading) return;
    append({ role: "user", content: input });
    setInput("");
  };

  return (
    <div className="flex flex-col h-[600px] w-full max-w-2xl mx-auto border rounded-lg overflow-hidden bg-white">
      <div className="flex-1 overflow-y-auto p-4 space-y-4">
        {messages.map((msg) => (
          <div
            key={msg.id}
            className={`flex ${msg.role === "user" ? "justify-end" : "justify-start"}`}
          >
            <div
              className={`max-w-[70%] p-3 rounded-lg ${
                msg.role === "user"
                  ? "bg-blue-600 text-white rounded-br-none"
                  : "bg-gray-100 text-gray-900 rounded-bl-none"
              }`}
            >
              <p className="whitespace-pre-wrap">{msg.content}</p>
            </div>
          </div>
        ))}
        {isLoading && (
          <div className="flex justify-start">
            <div className="bg-gray-100 p-3 rounded-lg rounded-bl-none animate-pulse">
              <span className="text-gray-500">Thinking...</span>
            </div>
          </div>
        )}
      </div>

      <form onSubmit={handleSubmit} className="p-4 border-t bg-gray-50">
        <div className="flex gap-2">
          <input
            value={input}
            onChange={(e) => setInput(e.target.value)}
            placeholder={isLoading ? "Waiting for response..." : "Type a message..."}
            disabled={isLoading}
            className="flex-1 px-4 py-2 border rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500 disabled:opacity-50"
          />
          <button
            type="submit"
            disabled={!input.trim() || isLoading}
            className="px-6 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 disabled:opacity-50 disabled:cursor-not-allowed"
          >
            Send
          </button>
          {isLoading && (
            <button
              type="button"
              onClick={stop}
              className="px-6 py-2 bg-red-600 text-white rounded-lg hover:bg-red-700"
            >
              Stop
            </button>
          )}
        </div>
        {error && (
          <p className="mt-2 text-sm text-red-600">Error: {error.message}</p>
        )}
      </form>
    </div>
  );
}

The useChat hook from ai/react handles the streaming connection, message state, and optimistic updates. It expects the endpoint to return a data stream — which toDataStreamResponse() provides.

Wire it into a page

Replace src/app/page.tsx:

// src/app/page.tsx
import { Chat } from "@/components/chat";

export default function Home() {
  return (
    <main className="min-h-screen bg-gray-50 py-12 px-4">
      <div className="max-w-3xl mx-auto">
        <header className="mb-8 text-center">
          <h1 className="text-3xl font-bold text-gray-900">AI Chatbot</h1>
          <p className="text-gray-600 mt-2">
            Powered by Claude 3.5 Sonnet via n4n.ai
          </p>
        </header>
        <Chat />
      </div>
    </main>
  );
}

Run and verify

Start the dev server:

pnpm dev

Open http://localhost:3000. Type a message. You should see tokens stream in real time — no waiting for the full response.

Expected behavior:

• User message appears immediately on the right (optimistic update)
• Assistant response streams token-by-token on the left
• “Thinking…” indicator shows while streaming
• Stop button aborts the in-flight request
• Errors surface in a red banner below the input

Adding tool calling

Claude 3.5 Sonnet excels at tool use. Let’s add a weather tool to demonstrate. First, define the tool schema in the route handler:

// src/app/api/chat/route.ts
import { streamText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { z } from "zod";

const weatherTool = {
  parameters: z.object({
    location: z.string().describe("City and state, e.g. San Francisco, CA"),
    unit: z.enum(["celsius", "fahrenheit"]).default("fahrenheit"),
  }),
  execute: async ({ location, unit }: { location: string; unit: "celsius" | "fahrenheit" }) => {
    // In production, call a real weather API here
    const temp = unit === "celsius" ? 22 : 72;
    return { location, temperature: temp, unit, condition: "Sunny" };
  },
};

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: anthropic("claude-3-5-sonnet-20241022", {
      baseURL: process.env.N4N_BASE_URL,
      headers: {
        Authorization: `Bearer ${process.env.N4N_API_KEY}`,
      },
    }),
    messages,
    system: "You are a helpful assistant. Use tools when users ask about weather.",
    temperature: 0.3,
    maxTokens: 4096,
    tools: { weather: weatherTool },
  });

  return result.toDataStreamResponse();
}

The tools object maps tool names to definitions with description, parameters (a Zod schema), and an execute function. The AI SDK handles the tool call loop automatically: it sends the tool schema to the model, receives tool calls, executes them, and feeds results back — all within the same stream.

Test it: ask “What’s the weather in Seattle?” The model invokes the tool, the execute function runs, and the response includes the result.

Handling provider failures with fallback

One reason to use n4n.ai is automatic fallback when a provider degrades. The AI SDK supports this via the fallback model option. Update the route:

// src/app/api/chat/route.ts
import { streamText, fallback } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

// ... weatherTool definition ...

export async function POST(req: Request) {
  const { messages } = await req.json();

  const primary = anthropic("claude-3-5-sonnet-20241022", {
    baseURL: process.env.N4N_BASE_URL,
    headers: { Authorization: `Bearer ${process.env.N4N_API_KEY}` },
  });

  // Fallback to GPT-4o via n4n.ai if Claude is unavailable
  const fallbackModel = openai("gpt-4o", {
    baseURL: process.env.N4N_BASE_URL,
    headers: { Authorization: `Bearer ${process.env.N4N_API_KEY}` },
  });

  const result = await streamText({
    model: fallback([primary, fallbackModel]),
    messages,
    system: "You are a helpful assistant. Use tools when users ask about weather.",
    temperature: 0.3,
    maxTokens: 4096,
    tools: { weather: weatherTool },
  });

  return result.toDataStreamResponse();
}

Add the OpenAI provider dependency:

pnpm add @ai-sdk/openai

The fallback function takes an array of models and tries each in order until one succeeds. n4n.ai routes both Anthropic and OpenAI requests through the same endpoint, so the same API key works for both.

Streaming tool results to the client

The client currently only renders msg.content. Tool invocations and results live in msg.parts. Update the chat component to surface them:

// src/components/chat.tsx (replace the message mapping)
{messages.map((msg) => (
  <div
    key={msg.id}
    className={`flex ${msg.role === "user" ? "justify-end" : "justify-start"}`}
  >
    <div
      className={`max-w-[70%] p-3 rounded-lg ${
        msg.role === "user"
          ? "bg-blue-600 text-white rounded-br-none"
          : "bg-gray-100 text-gray-900 rounded-bl-none"
      }`}
    >
      {msg.parts?.map((part, i) => {
        if (part.type === "text") {
          return <p key={i} className="whitespace-pre-wrap">{part.text}</p>;
        }
        if (part.type === "tool-invocation") {
          return (
            <details key={i} className="text-sm text-gray-600 mt-1">
              <summary className="font-mono cursor-pointer">
                Tool: {part.toolInvocation.toolName}
              </summary>
              <pre className="mt-1 p-2 bg-gray-200 rounded text-xs overflow-auto">
                {JSON.stringify(part.toolInvocation.args, null, 2)}
              </pre>
              {part.toolInvocation.state === "result" && (
                <pre className="mt-1 p-2 bg-green-50 rounded text-xs overflow-auto">
                  {JSON.stringify(part.toolInvocation.result, null, 2)}
                </pre>
              )}
            </details>
          );
        }
        return null;
      })}
    </div>
  </div>
))}

Now when the model calls the weather tool, the user sees an expandable block showing the tool name, arguments, and result — useful for debugging and transparency.

Production considerations

Rate limiting and quotas

The AI SDK doesn’t include built-in rate limiting. At minimum, add a middleware or use Vercel’s Edge Config for IP-based limits. For per-user quotas, integrate with your auth system (Clerk, NextAuth, etc.) and track usage in a database.

Conversation persistence

useChat keeps messages in memory. For persistence, implement onFinish on the server to save to a database, and hydrate initial messages from the server:

// Server: in streamText options
onFinish: async ({ messages, usage }) => {
  await db.chat.create({ data: { messages, usage } });
}

// Client: pass initialMessages to useChat
const { messages } = useChat({
  api: "/api/chat",
  initialMessages: await fetch("/api/chat/history").then(r => r.json()),
});

Observability

Log usage from onFinish — it includes promptTokens, completionTokens, and totalTokens. n4n.ai returns provider-level usage in response headers (x-n4n-usage-prompt, x-n4n-usage-completion), which the AI SDK surfaces automatically.

Caching

Anthropic supports prompt caching via the cache_control parameter. The AI SDK doesn’t expose this directly yet, but n4n.ai forwards provider cache-control hints. For now, structure system prompts to maximize prefix reuse.

Deployment

Deploy to Vercel:

pnpm build
vercel deploy --prod

Set N4N_API_KEY and N4N_BASE_URL in Vercel’s environment variables. The streaming response works on both Node.js and Edge runtimes — choose Edge for lower cold-start latency.

What you’ve built

A streaming chatbot with:

• Real-time token streaming via Vercel AI SDK’s streamText
• Tool calling with Zod-validated schemas and automatic loop handling
• Provider fallback (Claude → GPT-4o) via n4n.ai’s unified endpoint
• Client-side rendering of tool invocations and results
• Error boundaries, stop controls, and loading states

The patterns here — streaming responses, tool loops, fallback models — are the foundation of any production LLM application. The AI SDK abstracts the protocol complexity; n4n.ai abstracts the provider complexity. You write application logic.