Build a RAG chatbot with Vercel AI SDK and n4n.ai

This rag chatbot vercel ai sdk n4n.ai tutorial walks through building a retrieval-augmented generation chatbot in a Next.js app. You’ll embed a small knowledge base, store vectors locally, and stream grounded answers through the Vercel AI SDK’s chat hooks. The pattern scales to production with a real vector store and a gateway that routes to 240+ models.

Prerequisites

• Node.js 18.18 or later
• A Next.js 14 app using the App Router (npx create-next-app@latest)
• An API key from an OpenAI-compatible inference gateway (we point at n4n.ai’s endpoint)
• TypeScript familiarity and comfort with React Server Routes

Install the required packages:

npm install ai @ai-sdk/openai zod
npm install -D tsx

Create .env.local and add your key:

N4N_API_KEY=your_key_here

Project setup

The Vercel AI SDK talks to providers through a uniform interface. We configure a custom provider that targets the gateway’s OpenAI-compatible base URL.

lib/provider.ts:

import { createOpenAI } from '@ai-sdk/openai';

export const gateway = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY,
});

We point the Vercel AI SDK at n4n.ai’s OpenAI-compatible endpoint to access 240+ models with automatic fallback when a provider is degraded. The returned gateway object exposes .chat() and .embedding() factories identical to the standard OpenAI provider.

Embed the knowledge base

For a runnable demo we embed two documents and persist them to disk. In a real system you’d batch and store in a vector DB.

scripts/embed.ts:

import { embed } from 'ai';
import { gateway } from '../lib/provider';
import fs from 'fs';

const docs = [
  { id: 1, text: 'n4n.ai is an LLM inference gateway with 240+ models and automatic fallback.' },
  { id: 2, text: 'Vercel AI SDK provides a unified API for chat, embeddings, and streaming.' },
  { id: 3, text: 'RAG reduces hallucination by injecting retrieved context into the prompt.' },
];

async function main() {
  const out = [];
  for (const d of docs) {
    const { embedding } = await embed({
      model: gateway.embedding('text-embedding-3-small'),
      value: d.text,
    });
    out.push({ ...d, embedding });
  }
  fs.mkdirSync('data', { recursive: true });
  fs.writeFileSync('data/vectors.json', JSON.stringify(out, null, 2));
  console.log(`Embedded ${out.length} documents`);
}

main().catch(console.error);

Run it:

npx tsx scripts/embed.ts

Expected output:

Embedded 3 documents

The data/vectors.json file now contains 1536-dimensional vectors. Snippet:

[
  {
    "id": 1,
    "text": "n4n.ai is an LLM inference gateway with 240+ models and automatic fallback.",
    "embedding": [0.0123, -0.0341, 0.0089]
  }
]

Build the RAG API route

The route handler retrieves the closest documents via cosine similarity, then streams a completion.

app/api/chat/route.ts:

import { streamText, embed } from 'ai';
import { gateway } from '@/lib/provider';
import fs from 'fs';

function cosine(a: number[], b: number[]) {
  const dot = a.reduce((s, v, i) => s + v * b[i], 0);
  const magA = Math.sqrt(a.reduce((s, v) => s + v * v, 0));
  const magB = Math.sqrt(b.reduce((s, v) => s + v * v, 0));
  return dot / (magA * magB);
}

export async function POST(req: Request) {
  const { messages } = await req.json();
  const userMsg = messages.filter(m => m.role === 'user').pop()?.content ?? '';

  const { embedding } = await embed({
    model: gateway.embedding('text-embedding-3-small'),
    value: userMsg,
  });

  const vectors = JSON.parse(fs.readFileSync('data/vectors.json', 'utf8'));
  const context = vectors
    .map(v => ({ text: v.text, score: cosine(embedding, v.embedding) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, 2)
    .map(x => x.text)
    .join('\n');

  const result = await streamText({
    model: gateway('gpt-4o-mini'),
    messages: [
      { role: 'system', content: 'Answer the question using only the provided context.' },
      ...messages.slice(0, -1),
      { role: 'user', content: `Context:\n${context}\n\nQuestion: ${userMsg}` },
    ],
  });

  return result.toDataStreamResponse();
}

toDataStreamResponse() emits the Vercel AI SDK’s data stream protocol that useChat consumes natively.

Test the route with curl

Before wiring the UI, verify the backend:

curl -X POST http://localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"What is n4n.ai?"}]}'

You’ll see streamed chunks prefixed with 0: (text parts). The assembled answer reads:

n4n.ai is an LLM inference gateway with 240+ models and automatic fallback.

Wire the frontend

The client uses useChat from ai/react. It posts messages to /api/chat and renders streamed tokens.

app/page.tsx:

'use client';
import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat();
  return (
    <main style={{ maxWidth: 600, margin: '2rem auto', fontFamily: 'sans-serif' }}>
      <h1>RAG Chat</h1>
      {messages.map(m => (
        <p key={m.id}>
          <strong>{m.role}:</strong> {m.content}
        </p>
      ))}
      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask about the docs..."
          style={{ width: '80%', padding: '0.5rem' }}
        />
        <button type="submit">Send</button>
      </form>
    </main>
  );
}

Run and verify

Start the dev server:

npm run dev

Open http://localhost:3000. Type: “What does Vercel AI SDK do?” Expected streamed response:

Vercel AI SDK provides a unified API for chat, embeddings, and streaming.

The answer is grounded in the embedded context, not the model’s prior weights.

Production notes

The file-based vector store is a teaching aid. Swap data/vectors.json for pgvector, Pinecone, or DuckDB when document count grows. The gateway already forwards provider cache-control hints, so you can annotate long system prompts with cache_control blocks if the upstream model supports prompt caching.

In this rag chatbot vercel ai sdk n4n.ai tutorial we kept retrieval single-shot: we embed only the final user message. For multi-turn dialogue, concatenate the recent conversation window before embedding to improve recall.

Extending the rag chatbot vercel ai sdk n4n.ai tutorial with metadata filters is straightforward: add fields to each vector object and filter before cosine scoring. Because the gateway normalizes provider errors, a rate-limited upstream automatically falls back without code changes.

You now have a minimal but complete RAG pipeline: embed, retrieve, inject, stream. Replace the three seed documents with your own corpus and ship it.