Building a nextjs vercel edge functions streaming chatbot requires handling bidirectional state on the client and a serverless edge route that proxies LLM tokens with low latency. This guide walks through a minimal but production-shaped implementation using the Next.js App Router, Edge Runtime, and the OpenAI streaming protocol.
Prerequisites
- Node.js 18.18+ and npm
- A Vercel account (free tier works)
- An LLM provider API key (OpenAI, or any OpenAI-compatible endpoint)
- Familiarity with React state and
fetchstreams
Create a project directory and scaffold:
npx create-next-app@latest chatbot-edge --ts --app --no-tailwind --no-eslint
cd chatbot-edgeSet your API key in .env.local:
echo "OPENAI_API_KEY=sk-your-key" > .env.localIf you want automatic fallback when a provider is degraded, you can point the same OpenAI-compatible call at n4n.ai’s single endpoint, which addresses 240+ models and forwards provider cache-control hints. The rest of this tutorial uses the standard OpenAI URL; swap the base URL if you choose a gateway.
Scaffold the edge API route
Create app/api/chat/route.ts. The Edge Runtime keeps cold starts low and lets you stream the provider response straight to the browser.
export const runtime = 'edge';
export async function POST(req: Request) {
const { messages } = await req.json();
const upstream = await fetch('https://api.openai.com/v1/chat/completions', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model: 'gpt-3.5-turbo',
messages,
stream: true,
}),
});
return new Response(upstream.body, {
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache, no-transform',
},
});
}This route does not buffer. It pipes the LLM’s SSE stream directly to the client. Expected checkpoint: curl from local dev returns data: {...} lines immediately as tokens generate.
curl -N -X POST http://localhost:3000/api/chat \
-H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Say hi"}]}'You should see incremental data: {"choices":[{"delta":{"content":"Hello"}}]} lines, ending with data: [DONE].
Build the client chat component
Create app/components/Chat.tsx. Mark it 'use client' and manage three pieces of state: sent messages, current input, and the in-flight streamed text.
'use client';
import { useState } from 'react';
type Msg = { role: 'user' | 'assistant'; content: string };
export default function Chat() {
const [messages, setMessages] = useState<Msg[]>([]);
const [input, setInput] = useState('');
const [streaming, setStreaming] = useState('');
async function send() {
if (!input.trim()) return;
const next = [...messages, { role: 'user' as const, content: input }];
setMessages(next);
setInput('');
setStreaming('');
const res = await fetch('/api/chat', {
method: 'POST',
body: JSON.stringify({ messages: next }),
});
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n');
buffer = lines.pop() ?? '';
for (const line of lines) {
if (!line.startsWith('data:')) continue;
const payload = line.slice(5).trim();
if (payload === '[DONE]') continue;
try {
const json = JSON.parse(payload);
const token = json.choices?.[0]?.delta?.content ?? '';
setStreaming((prev) => prev + token);
} catch {
// ignore malformed keep-alive
}
}
}
setMessages([...next, { role: 'assistant', content: streaming }]);
setStreaming('');
}
return (
<div style={{ maxWidth: 600, margin: '2rem auto' }}>
{messages.map((m, i) => (
<p key={i}><strong>{m.role}:</strong> {m.content}</p>
))}
{streaming && <p><strong>assistant:</strong> {streaming}</p>}
<input
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={(e) => e.key === 'Enter' && send()}
placeholder="Type a message"
style={{ width: '80%' }}
/>
<button onClick={send}>Send</button>
</div>
);
}Wire the component into the page
Replace app/page.tsx with a server component that renders the client chat:
import Chat from './components/Chat';
export default function Page() {
return (
<main>
<h1>Edge Streaming Chatbot</h1>
<Chat />
</main>
);
}Run npm run dev. Open http://localhost:3000. Type “Explain edge runtime” and press Enter. The assistant text appears token-by-token in the streaming paragraph. That is the core of a nextjs vercel edge functions streaming chatbot working locally.
Hardening the stream parser
The naive line split above can break if a JSON payload spans two chunks. In production, accumulate until you see \n\n (SSE event boundary). A small helper keeps it robust:
function parseSSE(buffer: string): { events: string[]; rest: string } {
const parts = buffer.split('\n\n');
const rest = parts.pop() ?? '';
const events = parts
.map((p) => p.split('\n').filter((l) => l.startsWith('data:')))
.flat()
.map((l) => l.slice(5).trim());
return { events, rest };
}Integrate it in the while loop by feeding buffer through parseSSE after each decode. This avoids dropping partial data: lines when the network flakes.
Deploy to Vercel
Commit and push to GitHub, then import the repo in Vercel. Set OPENAI_API_KEY in the project’s Environment Variables. Vercel detects runtime = 'edge' and deploys the route to the edge network automatically.
git add -A
git commit -m "initial nextjs vercel edge functions streaming chatbot"
git pushAfter deploy, the same streaming behavior works globally. Check the Functions tab: the /api/chat route should show Edge as the runtime, not Node.js.
Checkpoint: expected production output
In the deployed chat, send “Write a haiku about latency”. You should observe:
- Immediate first token within ~300–800ms (provider dependent).
- Smooth incremental append with no full-page refetch.
- Network panel shows a single
POST /api/chatwithtext/event-streamresponse that stays open until[DONE].
If you see a buffered response instead of streaming, confirm the route file exports runtime = 'edge' and that no middleware is stripping Cache-Control. Also verify the upstream fetch uses stream: true and you return upstream.body unwrapped.
Why edge matters for this pattern
A nextjs vercel edge functions streaming chatbot moves the proxy close to the user and avoids a regional Node function buffering the LLM response. The edge function pays only for the milliseconds it spends forwarding bytes. Because the LLM call stays server-side, your provider key never reaches the browser, and you can enforce rate limits or prompt guards in the same route before calling the model.
Keep the client dumb: it sends messages, reads a stream, and renders. All retry or fallback logic (e.g., switching models on 429) belongs in the edge route, where you can inspect the upstream status before piping. That separation is what makes the architecture maintainable as you add features like tool calls or multi-turn summarization.