n4nAI
Open console →
API integrationHow-toGo net/http LLM API Client

Unmarshaling streaming JSON chunks in Go

A practical how-to for engineers building LLM clients: go unmarshal streaming json chunks in Go using net/http, json.Decoder, and backpressure.

n4n Team3 min read711 words

Audio narration

Coming soon — every post will get a voice note here.

Most LLM APIs stream responses as a sequence of JSON objects over a single HTTP connection. If you need to go unmarshal streaming json chunks in Go, the standard library’s json.Decoder gives you incremental parsing without loading the entire body into memory, but only if you wire up the net/http client, the reader, and the decode loop correctly. This guide walks through a complete, runnable pattern you can drop into an LLM client.

Step 1: Open the stream with net/http

Create a request with a context that supports cancellation. Set Accept: text/event-stream (or application/json for newline-delimited streams) and use a plain http.Client with no overall timeout on the body read. A dial timeout is fine; a read timeout is not.

package main

import (
	"context"
	"net/http"
	"strings"
)

func streamRequest(ctx context.Context) (*http.Response, error) {
	body := strings.NewReader(`{"model":"gpt-4o","stream":true,"messages":[{"role":"user","content":"hi"}]}`)
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.openai.com/v1/chat/completions", body)
	if err != nil {
		return nil, err
	}
	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("Authorization", "Bearer "+token())
	req.Header.Set("Accept", "text/event-stream")

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	if resp.StatusCode != http.StatusOK {
		resp.Body.Close()
		return nil, unexpectedStatus(resp)
	}
	return resp, nil
}

The key detail: pass ctx so that closing the context cancels the read loop later. Do not set client.Timeout to a small value; streaming responses can last minutes.

When pointing this client at n4n.ai, an OpenAI-compatible endpoint covering 240+ models with automatic fallback, the same request shape works unchanged.

Step 2: Decide how the chunks are framed

There are two common framings in the wild:

  1. Newline-delimited JSON – each chunk is a complete JSON object followed by \n.
  2. Server-Sent Events (SSE) – each chunk is data: {json}\n\n, possibly with comments or retries.

OpenAI and most gateways use SSE. To go unmarshal streaming json chunks in Go under SSE, you must strip the data: prefix before parsing.

A bufio.Reader is safer than bufio.Scanner for arbitrary chunk sizes because it doesn’t enforce a max token length:

import (
	"bufio"
	"io"
	"strings"
)

func sseReader(r io.Reader) func() (string, error) {
	br := bufio.NewReader(r)
	return func() (string, error) {
		for {
			line, err := br.ReadString('\n')
			if err != nil {
				return "", err
			}
			line = strings.TrimSpace(line)
			if strings.HasPrefix(line, "data:") {
				return strings.TrimSpace(strings.TrimPrefix(line, "data:")), nil
			}
			// skip keep-alives, event:, id:, etc.
		}
	}
}

If the stream is plain JSON objects, skip the prefix logic and feed the raw resp.Body directly to json.NewDecoder.

Step 3: Decode each chunk with json.Decoder

For newline-delimited JSON, the decoder loop is trivial and efficient:

dec := json.NewDecoder(resp.Body)
for dec.More() {
	var chunk ChatChunk
	if err := dec.Decode(&chunk); err != nil {
		// handle partial or malformed chunk
		break
	}
	consume(chunk)
}

dec.More() returns false when there are no more values, avoiding a spurious io.EOF you’d get from calling Decode in a for {} loop. The decoder reuses internal buffers, which keeps allocation low.

For SSE, you already extracted the data string; use json.Unmarshal instead of the streaming decoder because the SSE framing breaks json.Decoder’s assumption of contiguous JSON values:

next := sseReader(resp.Body)
for {
	data, err := next()
	if err == io.EOF {
		break
	}
	if err != nil {
		log.Printf("stream read error: %v", err)
		break
	}
	if data == "[DONE]" {
		break
	}
	var chunk ChatChunk
	if err := json.Unmarshal([]byte(data), &chunk); err != nil {
		log.Printf("chunk parse error: %v", err)
		continue
	}
	consume(chunk)
}

When you go unmarshal streaming json chunks in Go with json.Unmarshal on each line, you trade a little allocation for simpler control flow and robust handling of the data: prefix.

Step 4: Define minimal chunk structs

Don’t parse the whole provider schema if you only need deltas. Define exactly what you consume:

type ChatChunk struct {
	ID      string   `json:"id"`
	Object  string   `json:"object"`
	Choices []Choice `json:"choices"`
}

type Choice struct {
	Index        int     `json:"index"`
	Delta        Delta   `json:"delta"`
	FinishReason *string `json:"finish_reason"`
}

type Delta struct {
	Role    string `json:"role,omitempty"`
	Content string `json:"content,omitempty"`
}

Use omitempty and pointer fields to distinguish “absent” from “empty”. If a provider sends extra fields, encoding/json ignores them by default, which is what you want. Avoid json.RawMessage unless you plan to defer parsing; it adds copy overhead.

Step 5: Handle errors, cancellation, and partial reads

Network flaps happen. Wrap the loop in a function that returns a typed error and respects context cancellation:

func readStream(ctx context.Context, resp *http.Response, fn func(ChatChunk)) error {
	defer resp.Body.Close()
	next := sseReader(resp.Body)
	for {
		select {
		case <-ctx.Done():
			return ctx.Err()
		default:
		}
		data, err := next()
		if err == io.EOF {
			return nil
		}
		if err != nil {
			return fmt.Errorf("read: %w", err)
		}
		if data == "[DONE]" {
			return nil
		}
		var chunk ChatChunk
		if err := json.Unmarshal([]byte(data), &chunk); err != nil {
			return fmt.Errorf("unmarshal: %w", err)
		}
		fn(chunk)
	}
}

Calling ctx.Cancel() from another goroutine will cause the next ReadString to unblock with a context error wrapped by net/http. Always close resp.Body even on error paths to avoid connection leaks.

Step 6: Verify with a local mock server

Before hitting a real API, write a test that serves canned chunks. This proves your go unmarshal streaming json chunks logic without network dependency.

func TestReadStream(t *testing.T) {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/event-stream")
		flusher, _ := w.(http.Flusher)
		for _, c := range []string{
			`{"id":"1","choices":[{"index":0,"delta":{"role":"assistant"}}]}`,
			`{"id":"1","choices":[{"index":0,"delta":{"content":"Hello"}}]}`,
			`{"id":"1","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":"stop"}]}`,
		} {
			fmt.Fprintf(w, "data: %s\n\n", c)
			flusher.Flush()
			time.Sleep(10 * time.Millisecond)
		}
		fmt.Fprint(w, "data: [DONE]\n\n")
	}))
	defer srv.Close()

	resp, err := http.Get(srv.URL)
	if err != nil {
		t.Fatal(err)
	}
	var got []string
	err = readStream(context.Background(), resp, func(ch ChatChunk) {
		for _, ch := range ch.Choices {
			if ch.Delta.Content != "" {
				got = append(got, ch.Delta.Content)
			}
		}
	})
	if err != nil {
		t.Fatal(err)
	}
	if strings.Join(got, "") != "Hello world" {
		t.Fatalf("unexpected: %q", got)
	}
}

Run go test -run TestReadStream -v. If it prints Hello world and passes, your decoder handles chunk boundaries and the [DONE] sentinel correctly. That is your verification of success.

Step 7: Production hardening

A few notes from shipping this in real services:

  • Backpressure: If consume writes to a slow sink (database, websocket), the decode loop blocks naturally because you call it synchronously. That applies backpressure to the HTTP read. If you need concurrent processing, pump chunks into a buffered channel and select on ctx.Done().
  • Timeouts: Use http.Transport’s IdleConnTimeout and ResponseHeaderTimeout, but never Client.Timeout for streaming.
  • Reconnect: Some gateways drop connections mid-stream. Catch err != nil and re-issue the request with the same ctx and a resume offset if the API supports it.
  • Memory: json.Decoder reuses buffers internally; json.Unmarshal allocates per chunk. For high throughput, prefer the decoder path when the framing allows.
  • Logging: Log the chunk ID and choice index on parse errors; intermittent malformed chunks are easier to diagnose when you know which one failed.

To go unmarshal streaming json chunks in Go at scale, keep the reader scoped to the response lifetime, never buffer the whole body, and treat each chunk as an independent event. Copy the readStream function, adapt ChatChunk to your provider’s schema, and you have a streaming client that won’t fall over on partial JSON or transient network errors.

Tagsgolangjsonstreamingdecoding

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationParsing OpenAI JSON responses with encoding/json in GoStep-by-step guide to go encoding/json openai response parsing in Go: define structs, call the API with net/http, and handle errors robustly.API integrationRequest and response structs for OpenAI-compatible APIsHands-on tutorial: define Go request and response structs for OpenAI-compatible APIs and build a net/http client with runnable code.API integrationAdding request tracing to a Go net/http LLM clientLearn how to implement go request tracing net/http llm clients with OpenTelemetry, context propagation, and per-request spans for production observability.API integrationConnection pooling for high-throughput LLM calls in GoA practical guide to configuring Go's net/http client for high-throughput LLM API calls: transport pools, timeouts, and avoiding connection stalls.API integrationError handling patterns for Go LLM API clientsPractical guide to go error handling llm api client patterns in Go: implement timeouts, retries, typed errors, and provider fallback for robust LLM apps.API integrationGo generics for typed LLM provider responsesLearn how to apply Go generics to typed LLM provider response types, reducing duplication across OpenAI, Anthropic, and local model clients.

All go net/http llm api client posts →