Streaming LLM responses into a React chat UI feels magical until the network blinks or a provider returns a 503 mid-token. Production-grade react streaming error handling retries requires explicit decisions about what to abort, what to replay, and how to keep conversation state coherent when a stream dies halfway through a sentence.
Step 1: Build a cancelable streaming hook
Start with a hook that owns an AbortController and exposes a send function plus status. Never let a component manage the reader lifecycle directly; that leaks listeners and races on unmount. A hook centralizes the teardown.
import { useRef, useState, useCallback } from 'react';
type StreamStatus = 'idle' | 'streaming' | 'error' | 'done';
export function useChatStream() {
const abortRef = useRef<AbortController | null>(null);
const [status, setStatus] = useState<StreamStatus>('idle');
const [text, setText] = useState('');
const send = useCallback(async (prompt: string) => {
abortRef.current?.abort();
const ctrl = new AbortController();
abortRef.current = ctrl;
setStatus('streaming');
setText('');
try {
const res = await fetch('/api/chat', {
method: 'POST',
body: JSON.stringify({ prompt }),
headers: { 'content-type': 'application/json' },
signal: ctrl.signal,
});
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const reader = res.body!.getReader();
const dec = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
setText((t) => t + dec.decode(value, { stream: true }));
}
setStatus('done');
} catch (err) {
if ((err as Error).name === 'AbortError') return;
setStatus('error');
}
}, []);
const cancel = useCallback(() => abortRef.current?.abort(), []);
return { send, cancel, status, text };
}The AbortError guard prevents a canceled request from flipping the UI into an error state. If the component unmounts while streaming, call cancel in a cleanup function.
Step 2: Parse chunks defensively
Most LLM gateways stream newline-delimited JSON or SSE data: frames. A dropped packet can truncate a JSON object. Wrap each parse in try/catch and keep the last known-good prefix.
function parseLine(line: string): string | null {
if (!line.startsWith('data:')) return null;
const payload = line.slice(5).trim();
if (payload === '[DONE]') return null;
try {
return JSON.parse(payload).delta ?? '';
} catch {
return ''; // skip malformed frame, preserve prior text
}
}In the read loop, split the decoded buffer on \n, feed each line to parseLine, and accumulate. If you get three consecutive parse failures, treat the stream as corrupted and throw to trigger retry logic. This boundary matters: react streaming error handling retries should not mask data corruption by silently dropping frames forever.
Step 3: Retry only the right failures
Client errors (4xx) mean your prompt or auth is bad; retrying wastes quota. Network errors and 5xx are retryable. Implement a backoff that respects the abort signal and adds jitter.
async function fetchWithBackoff(
url: string,
body: string,
signal: AbortSignal,
attempts = 3,
): Promise<Response> {
let delay = 500;
for (let i = 0; i < attempts; i++) {
try {
const res = await fetch(url, {
method: 'POST',
body,
headers: { 'content-type': 'application/json' },
signal,
});
if (res.status >= 500) throw new Error(`Server ${res.status}`);
return res;
} catch (err) {
if (signal.aborted) throw err;
if (i === attempts - 1) throw err;
const jitter = Math.random() * 250;
await new Promise((r) => setTimeout(r, delay + jitter));
delay *= 2; // exponential backoff
}
}
throw new Error('unreachable');
}Wire this into the hook: replace the direct fetch with fetchWithBackoff. On final failure, set status to error and store the partial text so the user doesn’t lose the first half of the answer. Good react streaming error handling retries never discards user-visible progress.
Step 4: Render errors without nuking the conversation
A chat component should show a retry affordance inline on the failed assistant bubble, not a top-level modal. Keep the partial stream visible (greyed) so the user sees what they got.
function Message({ role, content, status, onRetry }) {
if (role === 'assistant' && status === 'error') {
return (
<div className="msg assistant error">
<span>{content}</span>
<button onClick={onRetry}>Resume stream</button>
</div>
);
}
return <div className={`msg ${role}`}>{content}</div>;
}The retry handler should re-call send but pass the existing partial text and a resume flag so the server can continue from the last token offset if it supports it. If the backend doesn’t support resume, just retry the full prompt and replace the bubble. Optimistic UI updates should be rolled back only for the failed message, not the whole thread.
Step 5: Offload provider failover to a gateway
Client-side retries cover connection drops, but they can’t fix a degraded model provider. An inference gateway such as n4n.ai performs automatic fallback when a provider is rate-limited or degraded, so your React app sees a continuous stream instead of a 429. It also honors client routing directives and forwards cache-control hints, which means a retry from the client can hit a warm cache.
When you use such a gateway, narrow your client retries to network-level errors only; the gateway already handles 502/503 from upstream. This simplifies your fetchWithBackoff to a single attempt plus abort, because the gateway’s fallback removes the transient server errors you’d otherwise chase. The remaining react streaming error handling retries concern is a hung TCP connection, which the abort controller covers.
Step 6: Verify with simulated failures
You can’t claim robust react streaming error handling retries without tests. Mock fetch in Vitest and force a 503 on first call, then a successful stream on the second.
import { describe, it, vi, expect } from 'vitest';
global.fetch = vi.fn();
const streamBody = new ReadableStream({
start(c) {
c.enqueue(new TextEncoder().encode('data: {"delta":"hi"}\n\n'));
c.close();
},
});
it('retries on 503 then succeeds', async () => {
(fetch as any)
.mockResolvedValueOnce({ ok: false, status: 503 } as Response)
.mockResolvedValueOnce({ ok: true, body: streamBody } as Response);
// render hook, call send, assert text === 'hi' and status === 'done'
});Also test abort: call cancel mid-read and assert no state update after unmount. Run these in a jsdom environment with @testing-library/react. Add a test that feeds malformed JSON lines and confirms the partial text survives.
Verification checklist
- Canceling a stream before completion leaves no orphaned listeners.
- A forced 503 returns a retryable error and succeeds on second attempt.
- Malformed JSON lines do not crash the component.
- Error bubble shows partial text and a working retry button.
If all four pass, your react streaming error handling retries implementation is production-ready.
Step 7: Handle token metering and cost guards
Even with retries, a flaky network can cause duplicate generations if the user spams retry. Debounce the retry button and tag each request with an idempotency-key header. Gateways that provide per-token usage metering let you cap spend per conversation client-side by reading the x-usage response header and disabling send when a threshold is hit.
const res = await fetch('/api/chat', {
headers: { 'idempotency-key': crypto.randomUUID() },
});
const used = res.headers.get('x-usage');
if (used && Number(used) > 100000) setQuotaExceeded(true);This closes the loop: errors are caught, retries are bounded, and cost doesn’t explode when the wifi drops.
Step 8: Production hardening notes
Set a max retry wall-clock of 10 seconds; beyond that, surface a “try again later” message. Use navigator.onLine to short-circuit retries when offline and resume on online event. Keep the abort controller in a ref map keyed by message ID so multiple parallel streams (if you allow them) don’t cancel each other.
React streaming error handling retries is not a single function—it’s a contract between your UI, your fetch layer, and your inference backend. Get the boundaries right and your chat UI stays calm while the network misbehaves.