Chatbot UI design patterns for Next.js and Vercel AI SDK

The Vercel AI SDK has become the de facto standard for building chatbot UIs in Next.js, but most tutorials stop at “it works.” Production chatbot UI design patterns for Next.js and Vercel AI SDK require handling streaming edge cases, tool call visualization, optimistic updates that don’t flicker, and error recovery that doesn’t lose context. This guide walks through the patterns we’ve shipped repeatedly, ordered by dependency — each section builds on the last.

Project structure and streaming foundations

Start with the App Router. The AI SDK’s useChat hook expects a streaming endpoint at /api/chat by default. Keep the route handler minimal — it should only validate auth, enforce rate limits, and delegate to your model abstraction.

// app/api/chat/route.ts
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { getServerSession } from 'next-auth';
import { authOptions } from '@/lib/auth';

export const maxDuration = 30;

export async function POST(req: Request) {
  const session = await getServerSession(authOptions);
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { messages, model, system } = await req.json();

  const result = streamText({
    model: openai(model ?? 'gpt-4o-mini'),
    system,
    messages,
    maxTokens: 4096,
    temperature: 0.3,
  });

  return result.toDataStreamResponse();
}

Pitfall: Don’t put business logic in the route handler. Extract model selection, tool definitions, and system prompt composition into a separate service layer. This lets you unit test prompt construction and swap providers without touching the HTTP layer.

The client page component stays thin:

// app/chat/page.tsx
import { ChatInterface } from '@/components/chat/ChatInterface';

export default function ChatPage() {
  return <ChatInterface />;
}

Message components that don’t re-render everything

The biggest performance trap is rendering the entire message list on every token. The AI SDK’s useChat returns messages as a stable array reference — new messages are appended, existing ones are never mutated. Exploit this with React.memo and key stability.

// components/chat/Message.tsx
import { Message as AIMessage } from 'ai';
import { Markdown } from '@/components/ui/Markdown';
import { ToolCallView } from './ToolCallView';

interface MessageProps {
  message: AIMessage;
  isStreaming: boolean;
}

export const Message = React.memo(function Message({ message, isStreaming }: MessageProps) {
  if (message.role === 'tool') return null; // Handled inline with assistant message

  return (
    <div className={`message ${message.role} ${isStreaming ? 'streaming' : ''}`}>
      <div className="message-header">
        <span className="role-badge">{message.role}</span>
        {message.role === 'assistant' && isStreaming && <StreamingIndicator />}
      </div>
      <div className="message-content">
        <Markdown content={message.content} />
        {message.toolInvocations?.map((invocation) => (
          <ToolCallView key={invocation.toolCallId} invocation={invocation} />
        ))}
      </div>
    </div>
  );
});

Tradeoff: The isStreaming prop must come from the parent tracking status === 'streaming' && message.id === messages[messages.length - 1]?.id. Don’t derive it inside the message component — that couples presentation to hook internals.

For the markdown renderer, use a streaming-aware component that doesn’t re-parse on every token:

// components/ui/Markdown.tsx
import { marked } from 'marked';
import { useEffect, useRef, useState } from 'react';

interface MarkdownProps {
  content: string;
}

export function Markdown({ content }: MarkdownProps) {
  const [html, setHtml] = useState('');
  const prevContentRef = useRef(content);

  useEffect(() => {
    if (content === prevContentRef.current) return;
    prevContentRef.current = content;
    setHtml(marked.parse(content, { async: false }) as string);
  }, [content]);

  return <div className="markdown" dangerouslySetInnerHTML={{ __html: html }} />;
}

This avoids the react-markdown re-render tax on streaming tokens. For code blocks, add syntax highlighting via a useEffect that runs once per message completion, not per token.

Optimistic input and send handling

Users expect instant feedback. The pattern: clear the input immediately, show the user message optimistically, then reconcile when the server responds.

// components/chat/ChatInput.tsx
import { useChat } from 'ai/react';
import { useRef, useCallback } from 'react';

export function ChatInput() {
  const { input, setInput, handleSubmit, status, stop, isSubmitting } = useChat({
    onFinish: (message) => {
      // Analytics, logging, etc.
    },
    onError: (error) => {
      toast.error('Failed to send message. Please try again.');
    },
  });

  const textareaRef = useRef<HTMLTextAreaElement>(null);

  const handleKeyDown = useCallback((e: React.KeyboardEvent) => {
    if (e.key === 'Enter' && !e.shiftKey) {
      e.preventDefault();
      handleSubmit(e);
    }
  }, [handleSubmit]);

  const handleSubmit = useCallback((e: React.FormEvent) => {
    e.preventDefault();
    if (!input.trim() || isSubmitting) return;
    handleSubmit(e);
  }, [input, isSubmitting, handleSubmit]);

  return (
    <form onSubmit={handleSubmit} className="chat-input-form">
      <textarea
        ref={textareaRef}
        value={input}
        onChange={(e) => setInput(e.target.value)}
        onKeyDown={handleKeyDown}
        placeholder={status === 'streaming' ? 'Generating...' : 'Type a message...'}
        disabled={isSubmitting}
        rows={1}
        style={{ height: 'auto', minHeight: '44px' }}
      />
      <div className="input-actions">
        {status === 'streaming' ? (
          <button type="button" onClick={stop} className="stop-btn">
            Stop
          </button>
        ) : (
          <button type="submit" disabled={!input.trim() || isSubmitting}>
            Send
          </button>
        )}
      </div>
    </form>
  );
}

Pitfall: The AI SDK’s handleSubmit clears input after the fetch starts. If the request fails, the user’s text is gone. Wrap handleSubmit to preserve input on error:

const handleSubmit = useCallback(async (e: React.FormEvent) => {
  e.preventDefault();
  const currentInput = input;
  if (!currentInput.trim() || isSubmitting) return;

  try {
    await handleSubmit(e); // SDK's handler
  } catch {
    setInput(currentInput); // Restore on failure
  }
}, [input, isSubmitting, handleSubmit, setInput]);

Tool call visualization patterns

Tool calls are where chatbot UIs differentiate. Three patterns cover most cases:

1. Inline status chips (for fast tools)

// components/chat/ToolCallView.tsx
import { ToolInvocation } from 'ai';

interface ToolCallViewProps {
  invocation: ToolInvocation;
}

export function ToolCallView({ invocation }: ToolCallViewProps) {
  const { toolCallId, toolName, args, state, result } = invocation;

  return (
    <div className={`tool-call ${state}`} data-tool-call-id={toolCallId}>
      <div className="tool-call-header">
        <span className="tool-name">{toolName}</span>
        <span className={`tool-state ${state}`}>{state}</span>
      </div>
      <details className="tool-args">
        <summary>Arguments</summary>
        <pre>{JSON.stringify(args, null, 2)}</pre>
      </details>
      {state === 'result' && (
        <details className="tool-result">
          <summary>Result</summary>
          <pre>{JSON.stringify(result, null, 2)}</pre>
        </details>
      )}
    </div>
  );
}

2. Rich cards for structured results (search, lookup)

// components/chat/ToolCallView.tsx (extended)
import { SearchResultCard } from './SearchResultCard';

export function ToolCallView({ invocation }: ToolCallViewProps) {
  const { toolName, state, result } = invocation;

  if (toolName === 'search' && state === 'result') {
    return (
      <div className="tool-call result-card">
        <SearchResultCard results={result as SearchResult[]} />
      </div>
    );
  }
  // ... fallback to inline
}

3. Interactive confirmations (for mutating tools)

// components/chat/ToolCallView.tsx (extended)
export function ToolCallView({ invocation }: ToolCallViewProps) {
  const { toolName, state, args, toolCallId } = invocation;

  if (toolName === 'send_email' && state === 'calling') {
    return (
      <ToolConfirmation
        toolCallId={toolCallId}
        title="Send email?"
        details={args}
        onConfirm={() => confirmToolCall(toolCallId)}
        onReject={() => rejectToolCall(toolCallId)}
      />
    );
  }
  // ...
}

The confirmation component must call back into useChat’s addToolResult — this is the only way to resume the stream after human-in-the-loop:

// components/chat/ToolConfirmation.tsx
import { useChat } from 'ai/react';

interface ToolConfirmationProps {
  toolCallId: string;
  title: string;
  details: unknown;
  onConfirm: () => void;
  onReject: () => void;
}

export function ToolConfirmation({ toolCallId, title, details, onConfirm, onReject }: ToolConfirmationProps) {
  const { addToolResult } = useChat();

  const handleConfirm = () => {
    addToolResult({
      toolCallId,
      result: { confirmed: true },
    });
    onConfirm();
  };

  const handleReject = () => {
    addToolResult({
      toolCallId,
      result: { confirmed: false, reason: 'User rejected' },
    });
    onReject();
  };

  return (
    <div className="tool-confirmation">
      <h4>{title}</h4>
      <pre>{JSON.stringify(details, null, 2)}</pre>
      <div className="confirmation-actions">
        <button onClick={handleConfirm} className="btn-primary">Confirm</button>
        <button onClick={handleReject} className="btn-secondary">Cancel</button>
      </div>
    </div>
  );
}

Critical: The tool definition on the server must return a result shape that includes confirmed: boolean. The model sees this result and continues — or doesn’t — based on your prompt instructions.

Error recovery without losing context

Network failures, rate limits, and provider outages happen. The AI SDK’s onError callback fires, but the message history stays intact. Build a retry mechanism that preserves the conversation.

// components/chat/ChatInterface.tsx
import { useChat, Message } from 'ai/react';
import { useCallback, useState } from 'react';

export function ChatInterface() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [error, setError] = useState<Error | null>(null);

  const { append, status, stop, reload } = useChat({
    initialMessages: messages,
    onFinish: (message) => {
      setMessages((prev) => [...prev, message]);
    },
    onError: (err) => {
      setError(err);
      // Don't clear messages — they're already in state
    },
  });

  const handleRetry = useCallback(() => {
    setError(null);
    reload();
  }, [reload]);

  const handleSend = useCallback(async (input: string) => {
    setError(null);
    await append({ role: 'user', content: input });
  }, [append]);

  return (
    <div className="chat-interface">
      <MessageList messages={messages} status={status} />
      {error && (
        <div className="error-banner">
          <p>{error.message}</p>
          <button onClick={handleRetry}>Retry</button>
          <button onClick={() => setError(null)}>Dismiss</button>
        </div>
      )}
      <ChatInput onSend={handleSend} status={status} onStop={stop} />
    </div>
  );
}

Pitfall: reload() re-sends the last user message. If the user sent multiple messages while the first was streaming, only the last one retries. For true resilience, implement a custom sendMessage that tracks pending user messages and retries the full pending queue.

Streaming UX details that matter

Typing indicators

Don’t show a generic spinner. Show token arrival rate:

// components/chat/StreamingIndicator.tsx
import { useEffect, useState } from 'react';

export function StreamingIndicator() {
  const [tokensPerSecond, setTokensPerSecond] = useState(0);
  const tokenCountRef = useRef(0);
  const lastTimeRef = useRef(Date.now());

  useEffect(() => {
    tokenCountRef.current = 0;
    lastTimeRef.current = Date.now();
    const interval = setInterval(() => {
      const now = Date.now();
      const elapsed = (now - lastTimeRef.current) / 1000;
      if (elapsed > 0) {
        setTokensPerSecond(Math.round(tokenCountRef.current / elapsed));
      }
    }, 500);
    return () => clearInterval(interval);
  }, []);

  // Call this from Message component on each content update
  window.__registerToken?.(() => {
    tokenCountRef.current++;
  });

  return tokensPerSecond > 0 ? (
    <span className="tps-indicator">{tokensPerSecond} tok/s</span>
  ) : (
    <span className="streaming-dots"><span>.</span><span>.</span><span>.</span></span>
  );
}

Scroll management

Auto-scroll only when the user is at the bottom. Preserve scroll position when they’re reading history:

// components/chat/MessageList.tsx
import { useRef, useEffect, useImperativeHandle, forwardRef } from 'react';
import { Message } from 'ai/react';

interface MessageListProps {
  messages: Message[];
  status: 'submitted' | 'streaming' | 'ready' | 'error';
}

export const MessageList = forwardRef<HTMLDivElement, MessageListProps>(
  function MessageList({ messages, status }, ref) {
    const containerRef = useRef<HTMLDivElement>(null);
    const userScrolledRef = useRef(false);
    const mutationObserverRef = useRef<MutationObserver>();

    useImperativeHandle(ref, () => ({
      scrollToBottom: () => {
        containerRef.current?.scrollTo({ top: containerRef.current.scrollHeight, behavior: 'smooth' });
        userScrolledRef.current = false;
      },
    }));

    useEffect(() => {
      const container = containerRef.current;
      if (!container) return;

      const handleScroll = () => {
        const { scrollTop, scrollHeight, clientHeight } = container;
        userScrolledRef.current = scrollTop + clientHeight < scrollHeight - 50;
      };

      container.addEventListener('scroll', handleScroll, { passive: true });

      mutationObserverRef.current = new MutationObserver(() => {
        if (!userScrolledRef.current && status === 'streaming') {
          container.scrollTop = container.scrollHeight;
        }
      });

      mutationObserverRef.current.observe(container, { childList: true, subtree: true, characterData: true });

      return () => {
        container.removeEventListener('scroll', handleScroll);
        mutationObserverRef.current?.disconnect();
      };
    }, [status]);

    return (
      <div ref={containerRef} className="message-list" role="log" aria-live="polite">
        {messages.map((message) => (
          <Message key={message.id} message={message} isStreaming={status === 'streaming'} />
        ))}
      </div>
    );
  }
);

Provider abstraction for production

Hardcoding openai('gpt-4o-mini') in the route handler works for demos. Production needs provider fallbacks, model routing, and usage metering. This is where a gateway like n4n.ai fits — one OpenAI-compatible endpoint addressing 240+ models with automatic fallback when a provider is degraded, per-token usage metering, and client routing directives honored. Your route handler becomes:

// app/api/chat/route.ts (production)
import { streamText } from 'ai';
import { createGateway } from '@/lib/gateway';

const gateway = createGateway({
  baseUrl: process.env.GATEWAY_URL,
  apiKey: process.env.GATEWAY_KEY,
});

export async function POST(req: Request) {
  const { messages, model, system, provider } = await req.json();

  const result = streamText({
    model: gateway(model, { provider }), // Handles routing, fallback, cache hints
    system,
    messages,
    maxTokens: 4096,
  });

  return result.toDataStreamResponse({
    sendUsage: true, // Critical for metering
  });
}

The client can now pass routing hints via the model field (e.g., gpt-4o-mini@lowest-latency or claude-3.5-sonnet@cheapest) without the frontend knowing provider details.

Accessibility checklist

Chat UIs fail accessibility audits constantly. Non-negotiables:

ARIA live regions: The message list needs aria-live="polite" and role="log". New messages announce automatically.
Focus management: After sending, focus returns to the textarea. After tool confirmation, focus stays on the confirmation buttons.
Keyboard navigation: Every interactive element (tool cards, confirmation buttons, retry links) reachable and operable via keyboard.
Color contrast: Streaming indicators, tool states, and error banners meet WCAG AA.
Reduced motion: Respect prefers-reduced-motion for scrolling animations and streaming dots.

/* globals.css */
@media (prefers-reduced-motion: reduce) {
  .message-list { scroll-behavior: auto; }
  .streaming-dots span { animation: none; opacity: 1; }
}

Testing the streaming contract

Unit test the message components. Integration test the streaming flow with MSW:

// __tests__/chat/streaming.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import { http, HttpResponse } from 'msw';
import { setupServer } from 'msw/node';
import { ChatInterface } from '@/components/chat/ChatInterface';

const server = setupServer(
  http.post('/api/chat', async ({ request }) => {
    const encoder = new TextEncoder();
    const stream = new ReadableStream({
      async start(controller) {
        const chunks = ['Hello', ', ', 'world', '!'];
        for (const chunk of chunks) {
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ type: 'text-delta', textDelta: chunk })}\n\n`));
          await new Promise((r) => setTimeout(r, 10));
        }
        controller.enqueue(encoder.encode(`data: ${JSON.stringify({ type: 'finish', finishReason: 'stop' })}\n\n`));
        controller.close();
      },
    });
    return new HttpResponse(stream, { headers: { 'Content-Type': 'text/plain; charset=utf-8' } });
  })
);

beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());

test('streams tokens incrementally', async () => {
  render(<ChatInterface />);
  const input = screen.getByPlaceholderText('Type a message...');
  fireEvent.change(input, { target: { value: 'Hi' } });
  fireEvent.submit(screen.getByRole('form'));

  await waitFor(() => expect(screen.getByText('Hello')).toBeInTheDocument());
  await waitFor(() => expect(screen.getByText('Hello, world!')).toBeInTheDocument());
});

Performance budget

Metric	Target	Measurement
First token latency	< 800ms p95	`Time to First Byte` on `/api/chat`
Token render cost	< 2ms/token	React DevTools Profiler on streaming
Input latency (keystroke → paint)	< 16ms	`Interaction to Next Paint`
Bundle size (chat route)	< 50KB gzipped	`next build --profile`

Profile with next build --profile and ANALYZE=true. The AI SDK client is ~12KB gzipped. Marked is ~8KB. Your components should stay under 30KB.

What to build next

Conversation branching — Let users fork from any assistant message. Store message trees, not flat arrays.
Streaming tool results — For long-running tools, stream partial results back via the same data stream channel.
Message editing — Allow users to edit their last message and re-run from that point (replaces the tail of the conversation).
Offline queue — Persist pending messages to IndexedDB, flush on reconnect.

The patterns above handle the 90% case. The remaining 10% is where your product differentiates.