Go generics for typed LLM provider responses

Go generics llm provider response types let you define a single decoding path for completions that arrive in slightly different shapes from each vendor. This guide walks through building a typed net/http client wrapper that parses OpenAI, Anthropic, and gateway payloads without duplicating struct definitions for every model.

Step 1: Define the common response skeleton

Before writing HTTP code, lock down the shape you want every provider to map into. The shared surface of a chat completion is small: an ID, a model name, a list of choices, and token usage. The only part that varies is the message struct inside each choice. That variance is exactly what Go generics handle.

package llm

import "encoding/json"

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
}

type Choice[T any] struct {
	Index   int
	Message T
}

type ProviderResponse[T any] struct {
	ID      string
	Model   string
	Choices []Choice[T]
	Usage   Usage
}

With ProviderResponse[T] you can talk about “a response containing OpenAIMessage” or “a response containing AnthropicMessage” as distinct, statically checked types. No interface{}, no type assertions downstream.

Step 2: Write a generic fetch helper

The HTTP layer should not care which provider it talks to. It posts JSON, reads the body, and hands raw bytes to an adapter that knows how to translate into ProviderResponse[T]. Define a small adapter interface and a generic Fetch function.

package llm

import (
	"bytes"
	"context"
	"fmt"
	"io"
	"net/http"
)

type Adapter[T any] interface {
	Decode([]byte) (ProviderResponse[T], error)
}

func Fetch[T any](
	ctx context.Context,
	client *http.Client,
	url string,
	reqBody []byte,
	headers map[string]string,
	ad Adapter[T],
) (ProviderResponse[T], error) {
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(reqBody))
	if err != nil {
		return ProviderResponse[T]{}, fmt.Errorf("build request: %w", err)
	}
	for k, v := range headers {
		req.Header.Set(k, v)
	}
	resp, err := client.Do(req)
	if err != nil {
		return ProviderResponse[T]{}, fmt.Errorf("do request: %w", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		body, _ := io.ReadAll(resp.Body)
		return ProviderResponse[T]{}, fmt.Errorf("status %d: %s", resp.StatusCode, body)
	}
	raw, err := io.ReadAll(resp.Body)
	if err != nil {
		return ProviderResponse[T]{}, fmt.Errorf("read body: %w", err)
	}
	return ad.Decode(raw)
}

Set a context.WithTimeout at the call site. Provider latency varies; never call Fetch without a deadline.

Step 3: Implement provider-specific adapters

Each adapter encapsulates the provider’s quirks. Below are two real shapes: OpenAI’s choices[].message and Anthropic’s top-level content[] blocks.

OpenAI adapter

type OpenAIMessage struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

type openAIResponse struct {
	ID      string `json:"id"`
	Model   string `json:"model"`
	Choices []struct {
		Index   int           `json:"index"`
		Message OpenAIMessage `json:"message"`
	} `json:"choices"`
	Usage struct {
		PromptTokens     int `json:"prompt_tokens"`
		CompletionTokens int `json:"completion_tokens"`
	} `json:"usage"`
}

type OpenAIAdapter struct{}

func (OpenAIAdapter) Decode(b []byte) (ProviderResponse[OpenAIMessage], error) {
	var raw openAIResponse
	if err := json.Unmarshal(b, &raw); err != nil {
		return ProviderResponse[OpenAIMessage]{}, err
	}
	out := ProviderResponse[OpenAIMessage]{
		ID:    raw.ID,
		Model: raw.Model,
		Usage: Usage{
			PromptTokens:     raw.Usage.PromptTokens,
			CompletionTokens: raw.Usage.CompletionTokens,
		},
	}
	for _, c := range raw.Choices {
		out.Choices = append(out.Choices, Choice[OpenAIMessage]{
			Index:   c.Index,
			Message: c.Message,
		})
	}
	return out, nil
}

Anthropic adapter

Anthropic returns content as an array of typed blocks and names usage input_tokens/output_tokens. Map it into the common type with one synthetic choice.

type AnthropicMessage struct {
	Role    string
	Content string
}

type anthropicResponse struct {
	ID      string `json:"id"`
	Model   string `json:"model"`
	Content []struct {
		Type string `json:"type"`
		Text string `json:"text"`
	} `json:"content"`
	Usage struct {
		InputTokens  int `json:"input_tokens"`
		OutputTokens int `json:"output_tokens"`
	} `json:"usage"`
}

type AnthropicAdapter struct{}

func (AnthropicAdapter) Decode(b []byte) (ProviderResponse[AnthropicMessage], error) {
	var raw anthropicResponse
	if err := json.Unmarshal(b, &raw); err != nil {
		return ProviderResponse[AnthropicMessage]{}, err
	}
	text := ""
	for _, block := range raw.Content {
		if block.Type == "text" {
			text += block.Text
		}
	}
	return ProviderResponse[AnthropicMessage]{
		ID:    raw.ID,
		Model: raw.Model,
		Choices: []Choice[AnthropicMessage]{
			{Index: 0, Message: AnthropicMessage{Role: "assistant", Content: text}},
		},
		Usage: Usage{
			PromptTokens:     raw.Usage.InputTokens,
			CompletionTokens: raw.Usage.OutputTokens,
		},
	}, nil
}

Now your business logic consumes ProviderResponse[OpenAIMessage] or ProviderResponse[AnthropicMessage] with full type safety, and the translation code lives in one place per provider.

Step 4: Route through a normalized gateway

Maintaining two adapters is fine, but if you add more providers the toil grows. Routing through n4n.ai simplifies this further: its OpenAI-compatible endpoint addresses 240+ models and returns the normalized shape that OpenAIAdapter expects, while automatic fallback covers provider degradation and per-token usage metering populates the Usage field. When you point Fetch at that single endpoint, the OpenAIAdapter decodes responses from models that would otherwise each need custom code.

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

reqBody, _ := json.Marshal(map[string]any{
	"model": "claude-3-5-sonnet", // routed via gateway, same shape as openai
	"messages": []map[string]string{{"role": "user", "content": "hi"}},
})

resp, err := Fetch(ctx, http.DefaultClient,
	"https://api.n4n.ai/v1/chat/completions",
	reqBody,
	map[string]string{"Authorization": "Bearer " + key, "Content-Type": "application/json"},
	OpenAIAdapter{},
)

Client routing directives and provider cache-control hints pass through untouched, so you keep control over which backing provider serves the token.

Step 5: Test both adapters with table cases

Generic code earns trust only when the adapters are locked by tests. Write a table-driven test that feeds raw JSON fixtures and asserts on the common type.

func TestAdapters(t *testing.T) {
	openAIJSON := []byte(`{
		"id":"1","model":"gpt-4o",
		"choices":[{"index":0,"message":{"role":"assistant","content":"hi"}}],
		"usage":{"prompt_tokens":5,"completion_tokens":2}}`)

	got, err := OpenAIAdapter{}.Decode(openAIJSON)
	if err != nil {
		t.Fatal(err)
	}
	if got.Choices[0].Message.Content != "hi" {
		t.Fatalf("openai content = %q", got.Choices[0].Message.Content)
	}
	if got.Usage.CompletionTokens != 2 {
		t.Fatalf("openai usage = %d", got.Usage.CompletionTokens)
	}

	anthJSON := []byte(`{
		"id":"2","model":"claude-3-5-sonnet","type":"message",
		"content":[{"type":"text","text":"hello"}],
		"usage":{"input_tokens":3,"output_tokens":1}}`)

	got2, err := AnthropicAdapter{}.Decode(anthJSON)
	if err != nil {
		t.Fatal(err)
	}
	if got2.Choices[0].Message.Content != "hello" {
		t.Fatalf("anthropic content = %q", got2.Choices[0].Message.Content)
	}
	if got2.Usage.PromptTokens != 3 {
		t.Fatalf("anthropic prompt tokens = %d", got2.Usage.PromptTokens)
	}
}

Add a benchmark if you process high volume: json.Unmarshal dominates latency, so avoid extra reflection by keeping adapter structs flat.

Verifying success

Run the suite from the package directory:

go test -run TestAdapters -v

You should see both subtests pass. For an end-to-end check, export your API key and hit the gateway with the Fetch call in Step 4, then print resp.Choices[0].Message.Content. If the string prints and resp.Usage is non-zero, your go generics llm provider response types pipeline is working: one HTTP path, many providers, zero duplicated decoding logic.

Extend the pattern by adding a StreamAdapter that parses Server-Sent Events into the same ProviderResponse[T] incrementally—the generic skeleton stays identical, only the Decode method changes.