The fastest way to crash a Go service is a go goroutine leak streaming chat endpoint that ignores context cancellation. When you pipe an LLM’s token stream to an HTTP client, every disconnected browser or timed-out mobile app leaves a blocked goroutine behind unless you engineer explicit teardown.
Step 1: Model the stream with a context and a bounded channel
Start by treating the upstream HTTP body as a source of chunks and the client response writer as a sink. Spawn a producer goroutine that reads from resp.Body and pushes to a bounded channel. An unbounded channel hides backpressure and guarantees a go goroutine leak streaming chat pipeline the moment the consumer disappears.
Define a minimal chunk type:
type chunk struct {
Data []byte
Err error
}The producer must never block on a send without also watching ctx.Done(). Use a small buffer (8 is plenty for token frames):
func streamTokens(ctx context.Context, resp *http.Response, out chan<- chunk) {
defer close(out)
buf := make([]byte, 4096)
for {
n, err := resp.Body.Read(buf)
if n > 0 {
select {
case out <- chunk{Data: append([]byte(nil), buf[:n]...)}:
case <-ctx.Done():
return
}
}
if err != nil {
select {
case out <- chunk{Err: err}:
case <-ctx.Done():
}
return
}
}
}If the context cancels while the producer is blocked on out, the select lets it exit instead of hanging on a channel that no one drains.
Step 2: Cancel the upstream request on client disconnect
In an HTTP handler, r.Context() is canceled when the client closes the connection. Pass that context into the upstream request. If you are calling an OpenAI-compatible gateway such as n4n.ai, the cancel signal is forwarded to the provider, so token generation stops immediately instead of streaming into a dead socket.
func handler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
reqBody, _ := json.Marshal(map[string]any{
"model": "gpt-4o-mini",
"messages": []map[string]string{
{"role": "user", "content": "stream a poem"},
},
"stream": true,
})
req, err := http.NewRequestWithContext(ctx, "POST", upstreamURL, bytes.NewReader(reqBody))
if err != nil {
http.Error(w, err.Error(), 500)
return
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := http.DefaultClient.Do(req)
if err != nil {
http.Error(w, err.Error(), 502)
return
}
defer resp.Body.Close()
// ... wire streaming below
}Without NewRequestWithContext, the upstream call keeps running after the client leaves, and the producer goroutine stays alive waiting on resp.Body.Read. That is the classic go goroutine leak streaming chat failure mode.
Step 3: Select on ctx.Done() in every blocking operation
The consumer loop writes chunks to the http.ResponseWriter. It must also abandon ship when the client disconnects. Use http.Flusher for SSE-style output and always select on ctx.Done():
ch := make(chan chunk, 8)
go streamTokens(ctx, resp, ch)
flusher, _ := w.(http.Flusher)
for {
select {
case <-ctx.Done():
return // client gone; producer exits via its own select
case c := <-ch:
if c.Err != nil {
return
}
if _, err := w.Write(c.Data); err != nil {
return
}
flusher.Flush()
}
}No default case in the consumer select—busy-looping wastes CPU. The channel read and context cancel are the only events that matter.
Step 4: Tie goroutine lifetimes together with errgroup
Ad-hoc go statements make it easy to forget a wait. Use golang.org/x/sync/errgroup to bind the producer and consumer to one context:
import "golang.org/x/sync/errgroup"
func streamChat(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
resp := openUpstream(ctx, r) // as above
defer resp.Body.Close()
ch := make(chan chunk, 8)
g, ctx := errgroup.WithContext(ctx)
g.Go(func() error {
streamTokens(ctx, resp, ch)
return nil
})
g.Go(func() error {
flusher := w.(http.Flusher)
for {
select {
case <-ctx.Done():
return ctx.Err()
case c := <-ch:
if c.Err != nil {
return c.Err
}
if _, err := w.Write(c.Data); err != nil {
return err
}
flusher.Flush()
}
}
})
if err := g.Wait(); err != nil && !errors.Is(err, context.Canceled) {
log.Printf("stream error: %v", err)
}
}When the client disconnects, ctx cancels, both goroutines return, and g.Wait() unblocks. No orphaned goroutines.
Step 5: Close the body and drain residual data
Even with cancellation, the producer may have queued a final chunk. Because the channel is bounded and the consumer exits on ctx.Done(), the producer’s last send could block if the buffer is full and consumer is gone. The select on ctx.Done() in streamTokens prevents that block. Still, always call resp.Body.Close() in a defer so the TCP connection returns to the pool.
If you prefer to be extra safe, drain after the group waits:
_ = g.Wait()
for range ch { // channel is closed by producer, so this exits
}Since streamTokens closes out on exit, this loop terminates immediately once the producer finishes.
Step 6: Write a goroutine leak test
A go goroutine leak streaming chat bug hides in production but is obvious in a unit test. Use runtime.NumGoroutine() and httptest to simulate an early client disconnect.
func TestNoGoroutineLeak(t *testing.T) {
before := runtime.NumGoroutine()
srv := httptest.NewServer(http.HandlerFunc(streamChat))
defer srv.Close()
req, _ := http.NewRequest("POST", srv.URL, bytes.NewReader(nil))
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)
defer cancel()
req = req.WithContext(ctx)
_, _ = http.DefaultClient.Do(req)
time.Sleep(50 * time.Millisecond) // allow teardown
after := runtime.NumGoroutine()
if after > before+1 { // +1 tolerates test harness jitter
t.Fatalf("goroutine leak: %d before, %d after", before, after)
}
}The test cancels the client context after 10 ms, forcing the server’s r.Context() to cancel mid-stream. If your teardown is correct, the goroutine count returns to baseline.
Verify success
Run the test with -race and watch GODEBUG=gctrace=1 in a load test. Success means:
runtime.NumGoroutine()stays flat under repeated client disconnects.pprofgoroutine profile shows no stacks stuck inchan sendorsemacquire.- Upstream provider stops billing tokens after cancel (per-token metering confirms the gateway received the abort).
A go goroutine leak streaming chat service is avoidable with three rules: bound your channels, select on context cancellation in every goroutine, and prove it with a leak test. Ship the test, not just the handler.