n4nAI
Open console →
API integrationHow-toGin & Echo LLM API Integration

Testing Gin LLM routes with httptest and table tests

Learn how to test Gin LLM routes using httptest and table driven tests in Go, with runnable examples for robust LLM API integration.

n4n Team3 min read676 words

Audio narration

Coming soon — every post will get a voice note here.

Writing reliable Go services that proxy to language models demands disciplined testing. This guide shows how to validate your HTTP handlers with gin httptest table driven tests llm integration patterns that catch regressions before they hit production. We’ll build a small Gin route that forwards chat requests to an OpenAI-compatible endpoint, then test it without external network calls.

Step 1: Scaffold a minimal Gin route that calls an LLM

Start with a handler that accepts a JSON prompt and relays it to an LLM provider. Hardcoding the upstream URL inside the handler is a mistake—it makes tests dependent on the network and slows your suite to a crawl. Inject the base URL via a struct field so tests can redirect it at a mock.

package main

import (
	"bytes"
	"encoding/json"
	"net/http"

	"github.com/gin-gonic/gin"
)

type ChatRequest struct {
	Prompt string `json:"prompt"`
}

type ChatResponse struct {
	Completion string `json:"completion"`
}

type LLMProxy struct {
	BaseURL string
	Client  *http.Client
}

func (p *LLMProxy) Register(r *gin.Engine) {
	r.POST("/v1/chat", p.handleChat)
}

func (p *LLMProxy) handleChat(c *gin.Context) {
	var req ChatRequest
	if err := c.ShouldBindJSON(&req); err != nil {
		c.JSON(http.StatusBadRequest, gin.H{"error": "invalid json"})
		return
	}
	if req.Prompt == "" {
		c.JSON(http.StatusBadRequest, gin.H{"error": "prompt required"})
		return
	}

	body, _ := json.Marshal(map[string]any{
		"model": "gpt-4o-mini",
		"messages": []map[string]string{
			{"role": "user", "content": req.Prompt},
		},
	})

	// If you're using n4n.ai as your inference gateway, the same OpenAI-compatible
	// base URL works, and its automatic fallback when a provider is degraded
	// means this handler can skip custom retry logic.
	resp, err := p.Client.Post(p.BaseURL+"/v1/chat/completions", "application/json", bytes.NewReader(body))
	if err != nil {
		c.JSON(http.StatusBadGateway, gin.H{"error": "upstream unreachable"})
		return
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		c.JSON(http.StatusBadGateway, gin.H{"error": "upstream error"})
		return
	}

	var upstream struct {
		Choices []struct {
			Message struct {
				Content string `json:"content"`
			} `json:"message"`
		} `json:"choices"`
	}
	if err := json.NewDecoder(resp.Body).Decode(&upstream); err != nil {
		c.JSON(http.StatusInternalServerError, gin.H{"error": "decode failed"})
		return
	}
	if len(upstream.Choices) == 0 {
		c.JSON(http.StatusInternalServerError, gin.H{"error": "empty completion"})
		return
	}

	c.JSON(http.StatusOK, ChatResponse{Completion: upstream.Choices[0].Message.Content})
}

The handler has clear branches: malformed input, missing field, upstream transport error, non-200 upstream, decode failure, and success. That’s exactly what you want for table tests.

Step 2: Extract the LLM call behind an interface

Coupling the route directly to http.Client makes it awkward to simulate timeouts or specific payloads without a live socket. Define a narrow interface so the handler depends on behavior, not transport.

type Completer interface {
	Complete(prompt string) (string, error)
}

type HTTPCompleter struct {
	BaseURL string
	Client  *http.Client
}

func (h HTTPCompleter) Complete(prompt string) (string, error) {
	body, _ := json.Marshal(map[string]any{
		"model":    "gpt-4o-mini",
		"messages": []map[string]string{{"role": "user", "content": prompt}},
	})
	resp, err := h.Client.Post(h.BaseURL+"/v1/chat/completions", "application/json", bytes.NewReader(body))
	if err != nil {
		return "", err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return "", fmt.Errorf("status %d", resp.StatusCode)
	}
	var upstream struct {
		Choices []struct {
			Message struct{ Content string `json:"content"` } `json:"message"`
		} `json:"choices"`
	}
	json.NewDecoder(resp.Body).Decode(&upstream)
	if len(upstream.Choices) == 0 {
		return "", fmt.Errorf("empty")
	}
	return upstream.Choices[0].Message.Content, nil
}

Now the Gin handler takes a Completer. In tests you pass a fakeCompleter that returns canned strings or errors. This keeps gin httptest table driven tests llm focused on HTTP status and JSON shape, not on TCP.

Step 3: Stand up a mock LLM server with httptest

Sometimes you want to exercise the real HTTPCompleter against a fake provider to catch serialization bugs. httptest.NewServer is the right tool—it spins up a loopback listener and gives you a URL.

func mockLLMServer(t *testing.T, status int, resp string) *httptest.Server {
	t.Helper()
	return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.Header.Get("Authorization") == "" {
			w.WriteHeader(http.StatusUnauthorized)
			return
		}
		w.WriteHeader(status)
		w.Write([]byte(resp))
	}))
}

Point HTTPCompleter.BaseURL at server.URL. The test becomes a closed loop: Gin → completer → mock → Gin. No DNS, no TLS, no rate limits.

Step 4: Write the table driven test for the Gin route

Set gin.SetMode(gin.TestMode) to silence the debug router logs. Build the engine per subtest, register the proxy, and iterate. Each case declares the request body, the mock’s response, and what the route should return.

func TestChatRoute(t *testing.T) {
	gin.SetMode(gin.TestMode)

	tests := []struct {
		name           string
		body           string
		mockStatus     int
		mockResp       string
		expectedStatus int
		expectedErr    string
	}{
		{
			name:           "valid prompt",
			body:           `{"prompt":"hello"}`,
			mockStatus:     http.StatusOK,
			mockResp:       `{"choices":[{"message":{"content":"hi there"}}]}`,
			expectedStatus: http.StatusOK,
		},
		{
			name:           "empty prompt",
			body:           `{"prompt":""}`,
			expectedStatus: http.StatusBadRequest,
			expectedErr:    "prompt required",
		},
		{
			name:           "malformed json",
			body:           `not json`,
			expectedStatus: http.StatusBadRequest,
			expectedErr:    "invalid json",
		},
		{
			name:           "upstream 500",
			body:           `{"prompt":"x"}`,
			mockStatus:     http.StatusInternalServerError,
			mockResp:       `{"error":"boom"}`,
			expectedStatus: http.StatusBadGateway,
			expectedErr:    "upstream error",
		},
		{
			name:           "upstream unreachable",
			body:           `{"prompt":"x"}`,
			mockStatus:     0, // simulate by using bad URL instead
			expectedStatus: http.StatusBadGateway,
			expectedErr:    "upstream unreachable",
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			var server *httptest.Server
			if tt.mockStatus != 0 {
				server = mockLLMServer(t, tt.mockStatus, tt.mockResp)
				defer server.Close()
			}

			baseURL := "http://bad-url"
			if server != nil {
				baseURL = server.URL
			}

			proxy := &LLMProxy{BaseURL: baseURL, Client: http.DefaultClient}
			r := gin.New()
			proxy.Register(r)

			req := httptest.NewRequest(http.MethodPost, "/v1/chat", strings.NewReader(tt.body))
			req.Header.Set("Content-Type", "application/json")
			w := httptest.NewRecorder()
			r.ServeHTTP(w, req)

			if w.Code != tt.expectedStatus {
				t.Fatalf("expected %d, got %d (%s)", tt.expectedStatus, w.Code, w.Body.String())
			}
			if tt.expectedErr != "" && !strings.Contains(w.Body.String(), tt.expectedErr) {
				t.Fatalf("expected error %q in %s", tt.expectedErr, w.Body.String())
			}
			if tt.name == "valid prompt" {
				var cr ChatResponse
				if err := json.Unmarshal(w.Body.Bytes(), &cr); err != nil {
					t.Fatal(err)
				}
				if cr.Completion != "hi there" {
					t.Fatalf("unexpected completion: %s", cr.Completion)
				}
			}
		})
	}
}

Adding a case is one struct literal. The gin httptest table driven tests llm pattern separates transport from routing logic while still exercising the full middleware chain.

Pitfalls worth noting

httptest.NewRecorder does not stream responses the way a real http.ResponseWriter does. If your handler uses c.Stream for server-sent events, you must use httptest.Server and a real client, not the recorder. Also, never call server.Close() outside the subtest loop—you’ll leak TCP listeners and eventually hit too many open files on CI.

Step 5: Assert on the upstream contract

A test that only checks status codes misses broken headers or wrong model names. Extend the mock to capture what the handler sent.

var captured map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	json.NewDecoder(r.Body).Decode(&captured)
	w.WriteHeader(200)
	w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
}))
defer srv.Close()

After ServeHTTP, assert captured["model"] == "gpt-4o-mini". This guards against accidental regressions in the payload shape sent to the LLM—a common failure when providers tweak their schemas.

Step 6: Run and verify success

Execute the suite with verbose output and the race detector:

go test -v -race ./...

A green run prints each subtest:

=== RUN   TestChatRoute
=== RUN   TestChatRoute/valid_prompt
=== RUN   TestChatRoute/empty_prompt
=== RUN   TestChatRoute/malformed_json
=== RUN   TestChatRoute/upstream_500
=== RUN   TestChatRoute/upstream_unreachable
--- PASS: TestChatRoute (0.02s)

If upstream 500 fails, check that your handler maps non-200 upstream to 502 and not 200. Measure coverage:

go test -cover ./...

Eighty percent coverage on the route file is trivial with these five cases. The gin httptest table driven tests llm setup is safe under -race because each subtest owns its server and engine.

Extending the suite

Once the skeleton passes, add cases for:

  • 429 responses with a Retry-After header to confirm your client surfaces a clean error or honors backoff.
  • Authorization header injection: assert the mock received Bearer <key> before the call leaves your process.
  • Streaming chunks if you support SSE; use a channel in the mock and a flusher in the handler.
  • Context timeouts: set c.Request.Context() with a short deadline and verify the client cancels the upstream call.

Keep the mock strict. If the handler sends an unexpected field, fail the test. That rigidity pays off when a provider changes its JSON overnight.

Testing Gin routes that talk to LLMs is not exotic. It is standard HTTP testing with one extra mock layer. Use interfaces for speed, httptest for fidelity, and table tests for coverage. Do that and your gin httptest table driven tests llm suite will hold up when the model backend hiccups at 3am.

Tagsgolanggintestinghttptest

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationGin middleware for LLM API key authenticationLearn how to build Gin middleware for LLM API key authentication in Go. Step-by-step guide with runnable code to validate keys and proxy to model gateways.API integrationRate limiting LLM endpoints with Gin middlewareImplement token-aware rate limiting for LLM APIs in Go using Gin middleware. Step-by-step guide with code for protecting endpoints from abuse.API integrationStructuring a Gin project for a multi-provider LLM proxyPractical guide to organizing a Gin-based multi-provider LLM proxy: directory layout, provider interfaces, routing, streaming, and middleware for auth and metering.API integrationBuilding an OpenAI-compatible endpoint in GinA hands-on Go tutorial: implement a gin openai compatible endpoint that forwards chat completions to an upstream LLM provider with proper API contracts. Learn request shaping and streaming.API integrationGin vs Echo for building an LLM gateway in GoA practical head-to-head comparison of Gin vs Echo for building an LLM gateway in Go, covering latency, streaming, middleware, and ecosystem tradeoffs.API integrationProxying chat completions through a Gin handlerBuild a Go Gin handler that proxies OpenAI-style chat completions requests to an LLM gateway, with streaming, auth injection, and error handling.

All gin & echo llm api integration posts →