Handling loading and error states in useChat

The useChat hook from Vercel AI SDK handles the heavy lifting of streaming responses, but the documentation treats loading and error states as an afterthought. In production, these states determine whether your chat interface feels responsive or broken. This tutorial walks through the complete state machine, shows patterns that survive real traffic, and explains where the SDK falls short.

Step 1: Understand the state machine

useChat exposes three primary status values that map to distinct UI phases:

type ChatStatus = 'submitted' | 'streaming' | 'ready' | 'error'

• submitted — Request sent, waiting for first token
• streaming — Tokens arriving incrementally
• ready — Stream complete, idle
• error — Terminal failure state

The hook also provides isLoading: boolean (true during submitted + streaming) and error: Error | undefined. Most tutorials stop here. The gap: isLoading conflates “waiting for first byte” with “receiving tokens,” but these need different UX.

// app/chat/page.tsx
'use client'

import { useChat } from 'ai/react'

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, status, error, isLoading } = useChat({
    api: '/api/chat',
  })

  return (
    <div className="flex flex-col h-screen">
      <Messages messages={messages} status={status} />
      <Composer
        input={input}
        onChange={handleInputChange}
        onSubmit={handleSubmit}
        disabled={isLoading}
        status={status}
        error={error}
      />
    </div>
  )
}

Step 2: Distinguish submitted from streaming

The submitted phase is where users perceive latency. A blank composer with a disabled send button feels broken. Show a “thinking” indicator immediately after submit, before the first token arrives.

// components/Composer.tsx
'use client'

import { useState, FormEvent } from 'react'

interface ComposerProps {
  input: string
  onChange: (e: React.ChangeEvent<HTMLTextAreaElement>) => void
  onSubmit: (e: FormEvent<HTMLFormElement>) => void
  disabled: boolean
  status: 'submitted' | 'streaming' | 'ready' | 'error'
  error: Error | undefined
}

export function Composer({ input, onChange, onSubmit, disabled, status, error }: ComposerProps) {
  const [isComposing, setIsComposing] = useState(false)

  const handleSubmit = (e: FormEvent<HTMLFormElement>) => {
    e.preventDefault()
    if (!input.trim() || disabled) return
    setIsComposing(true)
    onSubmit(e)
  }

  return (
    <form onSubmit={handleSubmit} className="border-t p-4">
      <div className="flex gap-2">
        <textarea
          value={input}
          onChange={onChange}
          disabled={disabled}
          placeholder={status === 'submitted' ? 'Sending...' : status === 'streaming' ? 'Receiving...' : 'Type a message...'}
          className="flex-1 min-h-[60px] max-h-[200px] resize-none px-3 py-2 border rounded-lg"
          rows={1}
        />
        <button
          type="submit"
          disabled={disabled || !input.trim()}
          className="px-4 py-2 bg-blue-600 text-white rounded-lg disabled:opacity-50 disabled:cursor-not-allowed"
        >
          {status === 'submitted' && <Spinner className="w-4 h-4" />}
          {status !== 'submitted' && 'Send'}
        </button>
      </div>
      
      {status === 'submitted' && (
        <p className="mt-2 text-sm text-gray-500 animate-pulse">Waiting for response...</p>
      )}
      
      {error && (
        <ErrorBanner error={error} onRetry={() => { setIsComposing(false); onSubmit(new Event('submit') as any) }} />
      )}
    </form>
  )
}

function Spinner({ className }: { className?: string }) {
  return (
    <svg className={className} viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
      <circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4" />
      <path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z" />
      <animateTransform attributeName="transform" type="rotate" from="0 12 12" to="360 12 12" dur="1s" repeatCount="indefinite" />
    </svg>
  )
}

Verify: Submit a message. You should see “Waiting for response…” immediately, before any tokens appear. The send button shows a spinner during submitted, then re-enables during streaming if you allow follow-up messages.

Step 3: Render streaming tokens incrementally

The messages array contains the assistant message with content: '' during submitted, then populates incrementally during streaming. Render it as a single streaming element, not a completed message.

// components/Messages.tsx
'use client'

import { Message } from 'ai/react'

interface MessagesProps {
  messages: Message[]
  status: 'submitted' | 'streaming' | 'ready' | 'error'
}

export function Messages({ messages, status }: MessagesProps) {
  return (
    <div className="flex-1 overflow-y-auto p-4 space-y-4">
      {messages.map((message, i) => (
        <MessageBubble key={message.id} message={message} isLast={i === messages.length - 1} status={status} />
      ))}
      {status === 'streaming' && <ScrollAnchor />}
    </div>
  )
}

function MessageBubble({ message, isLast, status }: { message: Message; isLast: boolean; status: string }) {
  const isStreaming = isLast && (status === 'streaming' || status === 'submitted')
  
  return (
    <div className={`flex gap-3 ${message.role === 'assistant' ? '' : 'flex-row-reverse'}`}>
      <div className={`max-w-[70%] ${message.role === 'assistant' ? 'bg-gray-100' : 'bg-blue-600 text-white'} rounded-2xl px-4 py-2`}>
        {message.role === 'assistant' ? (
          <>
            <Prose content={message.content} isStreaming={isStreaming} />
            {isStreaming && <CursorBlink />}
          </>
        ) : (
          <p className="whitespace-pre-wrap">{message.content}</p>
        )}
      </div>
    </div>
  )
}

function Prose({ content, isStreaming }: { content: string; isStreaming: boolean }) {
  // Simple markdown-ish rendering; replace with react-markdown in production
  return (
    <div className="prose prose-sm max-w-none">
      {content.split('\n').map((line, i) => (
        <p key={i} className={isStreaming && i === content.split('\n').length - 1 ? 'min-h-[1.5em]' : ''}>
          {line || <br />}
        </p>
      ))}
    </div>
  )
}

function CursorBlink() {
  return <span className="inline-block w-1 h-4 bg-gray-400 animate-pulse ml-0.5 align-bottom" />
}

function ScrollAnchor() {
  const ref = useRef<HTMLDivElement>(null)
  useEffect(() => { ref.current?.scrollIntoView({ behavior: 'smooth' }) }, [])
  return <div ref={ref} />
}

Verify: Send a long prompt. Tokens should appear character-by-character with a blinking cursor on the last line. No layout shift when streaming completes.

Step 4: Handle errors with context

The error object contains the raw fetch error. In practice, you need to distinguish:

• Network failures (retryable)
• Rate limits (retryable with backoff)
• Auth failures (require re-login)
• Upstream model errors (may need different model)

// components/ErrorBanner.tsx
'use client'

import { useCallback } from 'react'

interface ErrorBannerProps {
  error: Error
  onRetry: () => void
  onDismiss: () => void
}

export function ErrorBanner({ error, onRetry, onDismiss }: ErrorBannerProps) {
  const errorType = classifyError(error)
  
  return (
    <div className="mt-3 p-3 bg-red-50 border border-red-200 rounded-lg flex items-start gap-3">
      <svg className="w-5 h-5 text-red-500 mt-0.5 flex-shrink-0" fill="currentColor" viewBox="0 0 20 20">
        <path fillRule="evenodd" d="M10 18a8 8 0 100-16 8 8 0 000 16zM8.707 7.293a1 1 0 00-1.414 1.414L8.586 10l-1.293 1.293a1 1 0 101.414 1.414L10 11.414l1.293 1.293a1 1 0 001.414-1.414L11.414 10l1.293-1.293a1 1 0 00-1.414-1.414L10 8.586 8.707 7.293z" clipRule="evenodd" />
      </svg>
      <div className="flex-1 min-w-0">
        <p className="text-sm font-medium text-red-800">{errorType.title}</p>
        <p className="text-sm text-red-600 mt-1">{errorType.message}</p>
      </div>
      <div className="flex gap-2 flex-shrink-0">
        {errorType.retryable && (
          <button onClick={onRetry} className="text-sm text-red-700 hover:text-red-900 underline">
            Retry
          </button>
        )}
        <button onClick={onDismiss} className="text-sm text-red-500 hover:text-red-700">
          Dismiss
        </button>
      </div>
    </div>
  )
}

function classifyError(error: Error): { title: string; message: string; retryable: boolean } {
  // Network errors
  if (error instanceof TypeError && error.message.includes('fetch')) {
    return { title: 'Connection failed', message: 'Check your network and try again.', retryable: true }
  }
  
  // HTTP errors from the API route
  if ('status' in error && typeof error.status === 'number') {
    switch (error.status) {
      case 401:
        return { title: 'Session expired', message: 'Please sign in again.', retryable: false }
      case 429:
        return { title: 'Rate limited', message: 'Too many requests. Waiting a moment may help.', retryable: true }
      case 500:
      case 502:
      case 503:
        return { title: 'Server error', message: 'Our servers are having issues. Retry in a few seconds.', retryable: true }
      default:
        return { title: `Error ${error.status}`, message: error.message, retryable: error.status >= 500 }
    }
  }
  
  // AbortError from user navigation
  if (error.name === 'AbortError') {
    return { title: 'Request cancelled', message: 'The request was cancelled.', retryable: false }
  }
  
  return { title: 'Unknown error', message: error.message, retryable: true }
}

Verify: Disconnect network and send a message. Banner shows “Connection failed” with retry button. Click retry after reconnecting — message sends again.

Step 5: Implement exponential backoff retry

Naive retry loops hammer degraded endpoints. Implement client-side backoff with jitter, and expose attempt count so the UI can show progress.

// hooks/useChatWithRetry.ts
'use client'

import { useChat, UseChatOptions } from 'ai/react'
import { useCallback, useRef, useState } from 'react'

interface UseChatWithRetryOptions extends UseChatOptions {
  maxRetries?: number
  baseDelayMs?: number
  maxDelayMs?: number
}

export function useChatWithRetry(options: UseChatWithRetryOptions = {}) {
  const { maxRetries = 3, baseDelayMs = 1000, maxDelayMs = 10000, ...chatOptions } = options
  const [retryState, setRetryState] = useState<{ attempt: number; maxAttempts: number } | null>(null)
  const abortControllerRef = useRef<AbortController | null>(null)
  const retryTimeoutRef = useRef<NodeJS.Timeout | null>(null)

  const chat = useChat({
    ...chatOptions,
    onError: handleError,
    onFinish: handleFinish,
  })

  function handleError(error: Error) {
    chatOptions.onError?.(error)
    
    if (retryState && retryState.attempt >= maxRetries) {
      setRetryState(null)
      return
    }

    if (!isRetryableError(error)) {
      return
    }

    const attempt = (retryState?.attempt ?? 0) + 1
    const delay = Math.min(baseDelayMs * Math.pow(2, attempt - 1) + Math.random() * 1000, maxDelayMs)
    
    setRetryState({ attempt, maxAttempts: maxRetries })
    
    retryTimeoutRef.current = setTimeout(() => {
      setRetryState(prev => prev ? { ...prev, attempt: prev.attempt + 1 } : null)
      chat.reload()
    }, delay)
  }

  function handleFinish() {
    if (retryTimeoutRef.current) {
      clearTimeout(retryTimeoutRef.current)
    }
    setRetryState(null)
  }

  const cancelRetry = useCallback(() => {
    if (retryTimeoutRef.current) {
      clearTimeout(retryTimeoutRef.current)
    }
    setRetryState(null)
  }, [])

  return {
    ...chat,
    retryState,
    cancelRetry,
  }
}

function isRetryableError(error: Error): boolean {
  if (error instanceof TypeError && error.message.includes('fetch')) return true
  if ('status' in error && typeof error.status === 'number') {
    return error.status === 429 || error.status >= 500
  }
  return false
}

Update the composer to show retry progress:

// components/Composer.tsx (add to existing)
{retryState && (
  <div className="mt-2 text-sm text-amber-700 bg-amber-50 p-2 rounded flex items-center gap-2">
    <Spinner className="w-4 h-4" />
    <span>Retrying... attempt {retryState.attempt} of {retryState.maxAttempts}</span>
    <button onClick={cancelRetry} className="ml-2 underline">Cancel</button>
  </div>
)}

Verify: Simulate a 500 error from your API route. Watch the retry count increment with increasing delays. After max retries, the error banner appears with a manual retry button.

Step 6: Handle abort and navigation

Users navigate away mid-stream. The SDK aborts the fetch, throwing an AbortError that lands in onError. Don’t treat this as a user-facing error.

// app/chat/page.tsx (update useChat call)
const { messages, input, handleInputChange, handleSubmit, status, error, isLoading, reload, stop } = useChatWithRetry({
  api: '/api/chat',
  onError: (error) => {
    if (error.name === 'AbortError') return // Ignore navigation aborts
    // Log to error tracking service
    console.error('Chat error:', error)
  },
  onFinish: (message) => {
    // Persist to localStorage or analytics
  },
})

Add a stop button during streaming:

// components/Composer.tsx (add to button group)
{status === 'streaming' && (
  <button
    type="button"
    onClick={stop}
    className="px-4 py-2 bg-gray-100 text-gray-700 rounded-lg hover:bg-gray-200"
  >
    Stop
  </button>
)}

Verify: Start a long response, click “Stop” — streaming halts, composer re-enables. Navigate to another page mid-stream — no error banner appears.

Step 7: Persist and restore state

Refresh mid-conversation loses everything. Persist messages to localStorage and hydrate on mount.

// hooks/usePersistedChat.ts
'use client'

import { useChatWithRetry } from './useChatWithRetry'
import { useEffect, useState } from 'react'
import { Message } from 'ai/react'

const STORAGE_KEY = 'chat:messages'

export function usePersistedChat(options: Parameters<typeof useChatWithRetry>[0] = {}) {
  const [hydrated, setHydrated] = useState(false)
  const chat = useChatWithRetry({
    ...options,
    initialMessages: [],
  })

  // Hydrate on mount
  useEffect(() => {
    try {
      const stored = localStorage.getItem(STORAGE_KEY)
      if (stored) {
        const messages = JSON.parse(stored) as Message[]
        chat.setMessages(messages)
      }
    } catch {
      // Corrupted storage — ignore
    } finally {
      setHydrated(true)
    }
  }, [chat])

  // Persist on change
  useEffect(() => {
    if (!hydrated) return
    if (chat.messages.length === 0) {
      localStorage.removeItem(STORAGE_KEY)
    } else {
      localStorage.setItem(STORAGE_KEY, JSON.stringify(chat.messages))
    }
  }, [chat.messages, hydrated])

  return { ...chat, hydrated }
}

Verify: Send a few messages, refresh the page. Conversation history restores. Clear localStorage in DevTools — chat starts empty.

Step 8: Optimize re-renders during streaming

useChat triggers a render per token. For long responses, this creates jank. Memoize message components and use a virtualized list.

// components/Messages.tsx (optimized)
'use client'

import { Message } from 'ai/react'
import { memo, useMemo, useRef, useEffect } from 'react'
import { FixedSizeList as List } from 'react-window'

interface MessagesProps {
  messages: Message[]
  status: 'submitted' | 'streaming' | 'ready' | 'error'
}

export const Messages = memo(function Messages({ messages, status }: MessagesProps) {
  const listRef = useRef<List>(null)
  const isAtBottomRef = useRef(true)

  const itemData = useMemo(() => ({ messages, status }), [messages, status])

  const handleScroll = ({ scrollTop, scrollHeight, clientHeight }: any) => {
    isAtBottomRef.current = scrollTop + clientHeight >= scrollHeight - 50
  }

  const Item = ({ index, style }: { index: number; style: React.CSSProperties }) => (
    <div style={style}>
      <MessageBubble message={messages[index]} isLast={index === messages.length - 1} status={status} />
    </div>
  )

  useEffect(() => {
    if (isAtBottomRef.current && listRef.current) {
      listRef.current.scrollToItem(messages.length - 1, 'auto')
    }
  }, [messages.length, status])

  return (
    <div className="flex-1 overflow-hidden">
      <List
        ref={listRef}
        height={600}
        itemCount={messages.length}
        itemSize={120}
        itemData={itemData}
        onScroll={handleScroll}
        width="100%"
      >
        {Item}
      </List>
      {status === 'streaming' && <div style={{ height: 20 }} />}
    </div>
  )
})

Verify: Stream a 2000-token response. CPU stays low, no frame drops. Scroll up during streaming — auto-scroll pauses. Scroll to bottom — auto-scroll resumes.

Step 9: Server-side error shaping

Client error handling only works if the API route returns structured errors. Don’t let raw provider errors leak.

// app/api/chat/route.ts
import { streamText } from 'ai'
import { openai } from '@ai-sdk/openai'

export async function POST(req: Request) {
  try {
    const { messages } = await req.json()
    
    const result = await streamText({
      model: openai('gpt-4o'),
      messages,
      maxTokens: 2000,
      temperature: 0.7,
    })
    
    return result.toDataStreamResponse()
  } catch (error: any) {
    // Normalize provider errors
    if (error.statusCode === 429) {
      return new Response(JSON.stringify({ error: 'Rate limited' }), {
        status: 429,
        headers: { 'Content-Type': 'application/json', 'Retry-After': '60' },
      })
    }
    
    if (error.statusCode >= 500) {
      return new Response(JSON.stringify({ error: 'Model unavailable' }), {
        status: 503,
        headers: { 'Content-Type': 'application/json' },
      })
    }
    
    return new Response(JSON.stringify({ error: 'Invalid request' }), {
      status: 400,
      headers: { 'Content-Type': 'application/json' },
    })
  }
}

If you’re routing through a gateway that normalizes provider errors (like n4n.ai does with its automatic fallback and unified error format), the client sees consistent status codes regardless of which upstream model serves the request.

Verify: Trigger each error path. Network tab shows structured JSON with appropriate status codes. Client classifyError maps them correctly.

Step 10: Test the complete flow

Write an integration test that exercises the full state machine.

// __tests__/chat-flow.test.tsx
import { render, screen, waitFor, act } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { Chat } from '@/app/chat/page'
import { http, HttpResponse } from 'msw'
import { setupServer } from 'msw/node'

const server = setupServer(
  http.post('/api/chat', async ({ request }) => {
    const { messages } = await request.json()
    const lastMessage = messages[messages.length - 1]?.content ?? ''
    
    if (lastMessage.includes('trigger-500')) {
      return HttpResponse.json({ error: 'Model unavailable' }, { status: 503 })
    }
    
    if (lastMessage.includes('trigger-429')) {
      return HttpResponse.json({ error: 'Rate limited' }, { status: 429, headers: { 'Retry-After': '1' } })
    }
    
    // Stream a response
    const stream = new ReadableStream({
      async start(controller) {
        const encoder = new TextEncoder()
        for (const chunk of ['Hello', ', ', 'world', '!']) {
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ content: chunk })}\n\n`))
          await new Promise(r => setTimeout(r, 50))
        }
        controller.enqueue(encoder.encode('data: [DONE]\n\n'))
        controller.close()
      }
    })
    
    return new HttpResponse(stream, {
      headers: { 'Content-Type': 'text/plain; charset=utf-8' },
    })
  })
)

beforeAll(() => server.listen())
afterEach(() => server.resetHandlers())
afterAll(() => server.close())

test('complete chat flow with retry', async () => {
  const user = userEvent.setup()
  render(<Chat />)
  
  // Initial state
  expect(screen.getByPlaceholderText('Type a message...')).toBeInTheDocument()
  
  // Send message
  await user.type(screen.getByRole('textbox'), 'Hello')
  await user.click(screen.getByRole('button', { name: /send/i }))
  
  // Submitted state
  expect(screen.getByText('Waiting for response...')).toBeInTheDocument()
  expect(screen.getByRole('button', { name: /send/i })).toBeDisabled()
  
  // Streaming state
  await waitFor(() => expect(screen.getByText('Hello, world!')).toBeInTheDocument())
  expect(screen.getByRole('button', { name: /stop/i })).toBeInTheDocument()
  
  // Ready state
  await waitFor(() => expect(screen.queryByRole('button', { name: /stop/i })).not.toBeInTheDocument())
  expect(screen.getByRole('button', { name: /send/i })).not.toBeDisabled()
  
  // Error flow
  await user.type(screen.getByRole('textbox'), 'trigger-500')
  await user.click(screen.getByRole('button', { name: /send/i }))
  
  await waitFor(() => expect(screen.getByText('Server error')).toBeInTheDocument())
  expect(screen.getByRole('button', { name: /retry/i })).toBeInTheDocument()
  
  // Retry succeeds
  await user.click(screen.getByRole('button', { name: /retry/i }))
  await waitFor(() => expect(screen.getByText('Hello, world!')).toBeInTheDocument())
})

Verify: Run npm test. All assertions pass. The test covers submitted → streaming → ready → error → retry → ready.

What the SDK doesn’t solve

useChat gives you primitives. You still need to build:

• Message persistence across sessions (localStorage, IndexedDB, or backend)
• Optimistic updates for instant perceived latency
• Token usage display during streaming
• Branch/regenerate UX for alternative completions
• Attachment handling (images, files) — the hook only manages text
• Multi-turn tool calling state visualization

The patterns above cover the 80% case. The remaining 20% is where product differentiation lives.