Building a react optimistic ui streaming chat means rendering the user’s message instantly while the assistant response streams token by token. The hard part is reconciling local state with server truth when streams interrupt, models fall back, or the user sends three messages in a row.
1. Model the conversation as a local-first log
Start with a reducer that stores messages in an array keyed by insertion order. Each message carries a client-generated id, a role, content, a status, and optional server metadata. Keeping the shape flat makes optimistic updates trivial.
interface ChatMessage {
id: string;
role: 'user' | 'assistant' | 'system';
content: string;
status: 'pending' | 'streaming' | 'complete' | 'error';
model?: string;
serverId?: string;
}
interface ChatState {
messages: ChatMessage[];
pendingTokens: Record<string, string>;
}Never store derived streaming state outside the message object. If you keep a separate streamingText variable, you will desync when the user edits or resends.
2. Optimistically append the user turn
On submit, mint a UUID and dispatch the user message with status: 'pending'. Fire the request immediately. The UI shows the bubble before the server validates the payload.
function sendMessage(state: ChatState, text: string): ChatState {
const id = crypto.randomUUID();
const msg: ChatMessage = { id, role: 'user', content: text, status: 'pending' };
return { ...state, messages: [...state.messages, msg] };
}Common pitfall: disabling the input until the previous turn completes. That destroys the chat feel. Instead, allow queuing. Track an inFlight count, not a boolean, so multiple streams can run concurrently if your backend supports it.
If the POST fails with 400, flip the message to error and keep it visible. Do not silently drop it; the user typed it.
3. Stream tokens into a placeholder assistant message
Create the assistant message with empty content and status: 'streaming' in the same tick as the fetch. Then parse the stream and append tokens.
async function streamChat(
messages: ChatMessage[],
onToken: (t: string) => void,
signal: AbortSignal
) {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ messages: messages.map(m => ({ role: m.role, content: m.content })) }),
signal,
});
if (!res.body) throw new Error('missing body');
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buf = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buf += decoder.decode(value, { stream: true });
const lines = buf.split('\n');
buf = lines.pop() ?? '';
for (const line of lines) {
if (!line.startsWith('data: ')) continue;
const data = line.slice(6).trim();
if (data === '[DONE]') return;
const json = JSON.parse(data);
const token = json.choices?.[0]?.delta?.content;
if (token) onToken(token);
}
}
}Wire onToken to a dispatch that appends to the assistant message’s content. Keep the client id stable; the server has not confirmed anything yet.
Tradeoff: parsing SSE manually is verbose but avoids adding an event-source polyfill that breaks on POST. If your gateway uses plain JSON lines instead of SSE, adjust the split logic accordingly.
4. Reconcile temporary IDs with server truth
When the stream ends, the backend should return a small footer: the canonical serverId for the assistant message and the actual model used. Patch the message.
function finalizeMessage(state: ChatState, clientId: string, serverId: string, model: string): ChatState {
return {
...state,
messages: state.messages.map(m =>
m.id === clientId ? { ...m, serverId, model, status: 'complete' } : m
),
};
}If you proxy through a gateway such as n4n.ai, which exposes an OpenAI-compatible endpoint across 240+ models with automatic fallback when a provider is rate-limited, the response may carry a different model name than you requested. Capture the x-model header or final chunk metadata and show it in a faint footer. Hiding that mismatch erodes trust when the output style changes mid-session.
5. Handle partial streams and errors honestly
A stream can die after 40 tokens. Mark the message error, but keep the partial content. Add a retry affordance that resumes from the last user turn, not from zero.
function markError(state: ChatState, id: string): ChatState {
return {
...state,
messages: state.messages.map(m =>
m.id === id ? { ...m, status: 'error' } : m
),
};
}Pitfall: auto-retrying silently. If the user already typed a follow-up, a silent retry duplicates context. Expose a single “Regenerate” button on the failed assistant bubble.
Abort signals matter. When the user hits stop, call controller.abort(). The fetch reader throws; catch it and set status to complete if content exists, else error.
6. Throttle renders to protect interaction
Token bursts at 100/sec will thrash React. Batch appends with requestAnimationFrame:
let frame: number | null = null;
let acc = '';
function scheduleToken(dispatch: React.Dispatch<any>, id: string, token: string) {
acc += token;
if (frame) return;
frame = requestAnimationFrame(() => {
dispatch({ type: 'APPEND', id, token: acc });
acc = '';
frame = null;
});
}This caps DOM writes at 60fps and keeps the input responsive. For longer sessions, consider virtualizing the message list; streaming text in an off-screen bubble still costs layout.
7. Support branching and resend without losing history
Optimistic UIs often let the user edit a previous message. When they do, truncate all subsequent messages locally and mark them pending re-stream. Keep the old ones in an undo stack.
function branchFrom(state: ChatState, messageId: string): ChatState {
const idx = state.messages.findIndex(m => m.id === messageId);
if (idx === -1) return state;
const kept = state.messages.slice(0, idx + 1).map(m => ({ ...m, status: 'complete' }));
return { ...state, messages: kept };
}This pattern turns the react optimistic ui streaming chat from a linear terminal into a real workspace. The cost is complexity in your reducer; only adopt it if your users actually branch.
8. Meter usage per token on the client too
Even if your gateway provides per-token usage metering server-side, show an approximate live count. Increment a counter on each token for the assistant role. It is a cheap way to surface cost before the bill arrives.
function countTokens(state: ChatState, id: string, added: number): ChatState {
return {
...state,
messages: state.messages.map(m =>
m.id === id ? { ...m, usage: (m.usage ?? 0) + added } : m
),
};
}Do not treat client counts as authoritative. Character length divided by four is fine for a live hint.
Common pitfalls summary
- Generating server IDs client-side and assuming they match: always patch after stream.
- Ignoring
signalaborts: leaves zombie streams consuming bandwidth. - Rendering raw
contentwithout sanitization if you ever inject HTML: usetextContentor a memoized markdown renderer. - Forgetting that
crypto.randomUUIDneeds a secure context; provide a fallback for localhost HTTP.
A react optimistic ui streaming chat is mostly disciplined state management. The streaming part is ten lines; the reconciliation is the product.