n4nAI
Open console →
API integrationHow-toGo net/http LLM API Client

Adding request tracing to a Go net/http LLM client

Learn how to implement go request tracing net/http llm clients with OpenTelemetry, context propagation, and per-request spans for production observability.

n4n Team2 min read440 words

Audio narration

Coming soon — every post will get a voice note here.

Building a go request tracing net/http llm client is less about adding logs and more about treating every inference call as a first-class span. When you instrument the transport layer, you capture latency, status, and token counts without polluting business logic. This guide walks through a complete setup you can drop into a production service.

Step 1: Initialize an OpenTelemetry tracer

Pull the OTel SDK and a stdout exporter for local verification. Swap the exporter for OTLP in production.

go get go.opentelemetry.io/otel@latest \
  go.opentelemetry.io/otel/sdk@latest \
  go.opentelemetry.io/otel/exporters/stdout/stdouttrace@latest \
  go.opentelemetry.io/otel/semconv/v1.21.0@latest

Initialize a tracer provider once at process start. The function returned shuts the provider down gracefully.

func initTracer() func(context.Context) error {
    exporter, err := stdouttrace.New(stdouttrace.WithPrettyPrint())
    if err != nil {
        panic(err)
    }
    tp := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(exporter),
        sdktrace.WithResource(resource.Default()),
    )
    otel.SetTracerProvider(tp)
    otel.SetTextMapPropagator(propagation.TraceContext{})
    return tp.Shutdown
}

Call initTracer in main, defer the shutdown. The foundation of any go request tracing net/http llm setup is a global tracer every transport can reach.

Step 2: Wrap the transport with a tracing RoundTripper

http.RoundTripper is the seam where you own the request/response cycle. A custom transport starts a client span, injects context headers, and records attributes.

type tracingTransport struct {
    base   http.RoundTripper
    tracer trace.Tracer
}

func (t *tracingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    ctx, span := t.tracer.Start(req.Context(), "llm.http.request",
        trace.WithSpanKind(trace.SpanKindClient))
    defer span.End()

    otel.GetTextMapPropagator().Inject(ctx, propagation.HeaderCarrier(req.Header))

    traceCtx := httptrace.WithClientTrace(ctx, &httptrace.ClientTrace{
        GotFirstResponseByte: func() {
            span.AddEvent("first_byte_received")
        },
    })
    req = req.WithContext(traceCtx)

    resp, err := t.base.RoundTrip(req)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        return nil, err
    }
    span.SetAttributes(
        semconv.HTTPStatusCode(resp.StatusCode),
        semconv.HTTPMethod(req.Method),
        semconv.HTTPURL(req.URL.String()),
    )
    return resp, nil
}

Use httptrace to grab low-level signals like time-to-first-byte. Those events matter for LLM calls where token streaming delays hide behind a single long response.

Step 3: Build the LLM client around the instrumented transport

Construct an http.Client with the wrapped transport. Keep the chat logic plain; the transport does the observability work.

type LLMClient struct {
    http *http.Client
    url  string
}

func NewLLMClient(baseURL string) *LLMClient {
    base := http.DefaultTransport
    tr := &tracingTransport{base: base, tracer: otel.Tracer("llm/client")}
    return &LLMClient{
        http: &http.Client{Transport: tr, Timeout: 60 * time.Second},
        url:  strings.TrimRight(baseURL, "/") + "/v1/chat/completions",
    }
}

type msg struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type chatResponse struct {
    Model  string `json:"model"`
    Usage  usage  `json:"usage"`
    Choices []struct {
        Message msg `json:"message"`
    } `json:"choices"`
}

type usage struct {
    PromptTokens     int `json:"prompt_tokens"`
    CompletionTokens int `json:"completion_tokens"`
    TotalTokens      int `json:"total_tokens"`
}

func (c *LLMClient) Chat(ctx context.Context, model string, msgs []msg) (string, usage, error) {
    payload, _ := json.Marshal(map[string]any{
        "model":    model,
        "messages": msgs,
    })
    req, _ := http.NewRequestWithContext(ctx, http.MethodPost, c.url, bytes.NewReader(payload))
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", "Bearer "+os.Getenv("LLM_API_KEY"))

    resp, err := c.http.Do(req)
    if err != nil {
        return "", usage{}, err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        body, _ := io.ReadAll(resp.Body)
        return "", usage{}, fmt.Errorf("llm status %d: %s", resp.StatusCode, body)
    }

    var cr chatResponse
    if err := json.NewDecoder(resp.Body).Decode(&cr); err != nil {
        return "", usage{}, err
    }
    if len(cr.Choices) == 0 {
        return "", usage{}, errors.New("empty choices")
    }
    return cr.Choices[0].Message.Content, cr.Usage, nil
}

The usage struct captures token counts. Attach it to the active span after the call if you want per-request cost visibility.

Step 4: Handle retries and multi-model routing

LLM endpoints fail intermittently. Wrap calls in a retry loop, but start a child span per attempt so you can see which try succeeded.

func (c *LLMClient) ChatWithRetry(ctx context.Context, model string, msgs []msg, max int) (string, error) {
    var lastErr error
    for i := 0; i < max; i++ {
        attemptCtx, span := otel.Tracer("llm/client").Start(ctx, fmt.Sprintf("attempt_%d", i))
        _, err := c.Chat(attemptCtx, model, msgs)
        if err == nil {
            span.End()
            return "", nil // simplified
        }
        span.RecordError(err)
        span.End()
        lastErr = err
        time.Sleep(time.Duration(i+1) * 200 * time.Millisecond)
    }
    return "", lastErr
}

If you target a unified gateway such as n4n.ai, a single OpenAI-compatible endpoint fronts 240+ models, so the model field in your request is the only reliable dimension for grouping spans. The gateway’s automatic fallback when a provider is degraded means the served model may differ from the requested one; record the response’s model field too.

Step 5: Export traces and verify

Run the binary with the stdout exporter. Make one call:

LLM_API_KEY=sk-test go run main.go

You should see a span named llm.http.request with http.status_code 200 and an event first_byte_received. A broken key should produce a span with status.error and the recorded error string.

Example trimmed output:

{
  "name": "llm.http.request",
  "kind": "Client",
  "attributes": [
    { "key": "http.method", "value": "POST" },
    { "key": "http.status_code", "value": 200 }
  ],
  "events": [ { "name": "first_byte_received" } ]
}

Verification checklist:

  • Span appears for every Chat call, including retries.
  • Trace context headers (traceparent) leave your process; inspect with a proxy or server log.
  • Token usage from the response is logged or attached as span attributes.
  • A forced timeout surfaces as an error span, not a silent failure.

That closes the loop on a production-ready go request tracing net/http llm client. The transport pattern extends to streaming responses: start the span on the request, end it when the stream hits EOF, and count tokens from SSE chunks.

Tagsgolangtracingobservabilitynet-http

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationConnection pooling for high-throughput LLM calls in GoA practical guide to configuring Go's net/http client for high-throughput LLM API calls: transport pools, timeouts, and avoiding connection stalls.API integrationnet/http vs resty for calling LLM APIs in GoA pragmatic head-to-head comparison of Go's net/http and resty for building LLM API clients: ergonomics, latency, retries, streaming, and verdicts.API integrationA custom Go http.RoundTripper for LLM API retriesImplement a go http.roundtripper retry llm api calls with backoff and body replay; a practical Go guide with runnable code for resilient LLM clients.API integrationBuild a chat completions client with Go net/httpStep-by-step tutorial for building a Go net/http chat completions client against OpenAI-compatible APIs, with structs, streaming, and error handling.API integrationError handling patterns for Go LLM API clientsPractical guide to go error handling llm api client patterns in Go: implement timeouts, retries, typed errors, and provider fallback for robust LLM apps.API integrationGo generics for typed LLM provider responsesLearn how to apply Go generics to typed LLM provider response types, reducing duplication across OpenAI, Anthropic, and local model clients.

All go net/http llm api client posts →