Proxying chat completions through a Gin handler

A gin proxy chat completions handler gives your Go backend a single choke point for all LLM traffic, so you can inject keys, enforce model policies, and add observability without touching client code. In this tutorial we build one from scratch against the OpenAI chat completions contract, including streaming, and show how to point it at any OpenAI-compatible upstream.

Prerequisites

• Go 1.21 or newer
• github.com/gin-gonic/gin (v1.9.1 used here)
• An OpenAI-compatible API key and base URL (OpenAI, or a gateway such as n4n.ai)
• curl for manual testing
• Basic familiarity with net/http and JSON

Scaffold the module

mkdir gin-llm-proxy && cd gin-llm-proxy
go mod init example.com/gin-llm-proxy
go get github.com/gin-gonic/gin@v1.9.1

Keep the entrypoint in main.go. We will evolve the same file through three stages: echo, non-streaming proxy, streaming proxy.

Stage 1: Echo handler to verify routing

Before proxying, confirm Gin parses the request shape. We bind to a generic map to avoid defining a full schema.

package main

import (
	"net/http"

	"github.com/gin-gonic/gin"
)

func main() {
	r := gin.Default()
	r.POST("/v1/chat/completions", func(c *gin.Context) {
		var body map[string]interface{}
		if err := c.ShouldBindJSON(&body); err != nil {
			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
			return
		}
		c.JSON(http.StatusOK, gin.H{
			"model": body["model"],
			"echo":  true,
		})
	})
	r.Run(":8080")
}

Run it:

go run .

In a second shell:

curl -s -X POST localhost:8080/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

Expected output:

{"echo":true,"model":"gpt-4o"}

This proves the route and JSON binding work. The next step replaces the static response with a real forward.

Stage 2: Non-streaming gin proxy chat completions handler

We read the raw body, forward it to the upstream /v1/chat/completions, inject the API key, and copy the response back. Using the raw body avoids re-serialization mismatches (e.g., unknown fields, number precision).

package main

import (
	"bytes"
	"io"
	"net/http"
	"os"

	"github.com/gin-gonic/gin"
)

func proxyHandler(upstreamBase, apiKey string) gin.HandlerFunc {
	return func(c *gin.Context) {
		raw, err := io.ReadAll(c.Request.Body)
		if err != nil {
			c.JSON(http.StatusBadRequest, gin.H{"error": "read body"})
			return
		}

		upstreamURL := upstreamBase + "/v1/chat/completions"
		req, err := http.NewRequestWithContext(c.Request.Context(), http.MethodPost, upstreamURL, bytes.NewReader(raw))
		if err != nil {
			c.JSON(http.StatusInternalServerError, gin.H{"error": "build request"})
			return
		}
		req.Header.Set("Content-Type", "application/json")
		req.Header.Set("Authorization", "Bearer "+apiKey)
		if cid := c.GetHeader("X-Request-Id"); cid != "" {
			req.Header.Set("X-Request-Id", cid)
		}

		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			c.JSON(http.StatusBadGateway, gin.H{"error": "upstream call failed"})
			return
		}
		defer resp.Body.Close()

		c.Status(resp.StatusCode)
		for k, v := range resp.Header {
			for _, vv := range v {
				c.Writer.Header().Add(k, vv)
			}
		}
		io.Copy(c.Writer, resp.Body)
	}
}

func main() {
	upstream := os.Getenv("UPSTREAM_BASE")
	if upstream == "" {
		upstream = "https://api.openai.com"
	}
	key := os.Getenv("LLM_API_KEY")

	r := gin.Default()
	r.POST("/v1/chat/completions", proxyHandler(upstream, key))
	r.Run(":8080")
}

Why not httputil.NewSingleHostReverseProxy? It is great for pure pass-through, but it makes body mutation and selective header injection awkward. Since we need to inject Authorization and may later rewrite model, a hand-rolled forward is clearer.

Testing the non-streaming path

export LLM_API_KEY=sk-...
export UPSTREAM_BASE=https://api.openai.com
go run .

curl -s -X POST localhost:8080/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hello in 3 words"}]}' | head -c 400

Expected shape (truncated):

{"id":"chatcmpl-...","object":"chat.completion","created":...,"model":"gpt-4o-mini","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! How can I help?"},"finish_reason":"stop"}]}

The gin proxy chat completions handler now forwards arbitrary model names, temperature, tools, and other parameters verbatim.

Stage 3: Streaming support

Chat UIs routinely send "stream": true. The upstream then responds with text/event-stream. We must detect streaming, set the correct Accept header, and flush chunks to the client as they arrive. Copying headers blindly is fine, but we should drop Content-Length for streaming responses because the length is unknown.

func proxyHandler(upstreamBase, apiKey string) gin.HandlerFunc {
	return func(c *gin.Context) {
		raw, _ := io.ReadAll(c.Request.Body)
		streaming := bytes.Contains(raw, []byte(`"stream":true`)) ||
			bytes.Contains(raw, []byte(`"stream": true`))

		upstreamURL := upstreamBase + "/v1/chat/completions"
		req, _ := http.NewRequestWithContext(c.Request.Context(), http.MethodPost, upstreamURL, bytes.NewReader(raw))
		req.Header.Set("Content-Type", "application/json")
		req.Header.Set("Authorization", "Bearer "+apiKey)
		if streaming {
			req.Header.Set("Accept", "text/event-stream")
		}

		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			c.JSON(http.StatusBadGateway, gin.H{"error": "upstream"})
			return
		}
		defer resp.Body.Close()

		c.Status(resp.StatusCode)
		for k, v := range resp.Header {
			if k == "Content-Length" && streaming {
				continue
			}
			for _, vv := range v {
				c.Writer.Header().Add(k, vv)
			}
		}

		if streaming {
			flusher, _ := c.Writer.(http.Flusher)
			buf := make([]byte, 4096)
			for {
				n, err := resp.Body.Read(buf)
				if n > 0 {
					c.Writer.Write(buf[:n])
					if flusher != nil {
						flusher.Flush()
					}
				}
				if err != nil {
					break
				}
			}
			return
		}
		io.Copy(c.Writer, resp.Body)
	}
}

Test with curl using -N (no buffering):

curl -N -X POST localhost:8080/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Count to 3"}]}'

You should see incremental SSE frames:

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"1"}}]}
data: {"id":"chatcmpl-...","choices":[{"delta":{"content":" 2"}}]}
data: [DONE]

Timeouts and client cancellation

http.DefaultClient has no timeout, which is dangerous. Create a client with a bounded timeout but respect the request context for long generations:

client := &http.Client{
	Timeout: 0, // rely on context
}

Gin’s c.Request.Context() is cancelled when the client disconnects or the server’s write timeout fires. Pass that context into NewRequestWithContext as shown; the upstream call aborts automatically.

Model allowlist and policy

The gin proxy chat completions handler is the correct layer to reject unsupported models before any token is spent. Parse only the fields you need:

allowed := map[string]bool{"gpt-4o-mini": true, "gpt-4o": true}
var parsed struct {
	Model string `json:"model"`
}
if err := json.Unmarshal(raw, &parsed); err != nil {
	c.JSON(http.StatusBadRequest, gin.H{"error": "bad json"})
	return
}
if !allowed[parsed.Model] {
	c.JSON(http.StatusForbidden, gin.H{"error": "model not allowed"})
	return
}

Insert this block before building the upstream request.

Logging middleware

Keep the handler thin. Move request logging to middleware:

r.Use(func(c *gin.Context) {
	start := time.Now()
	c.Next()
	log.Printf("proxy %s %s %d %s", c.Request.Method, c.FullPath(), c.Writer.Status(), time.Since(start))
})

Using a multi-provider gateway

If your upstream is an OpenAI-compatible gateway like n4n.ai, the same contract holds: one endpoint, 240+ models, and automatic fallback when a provider is degraded. Your gin proxy chat completions handler just forwards the model string and Authorization header; the gateway honors client routing directives and provider cache-control hints without extra code. That keeps your service oblivious to provider outages and lets you swap models via config rather than redeploys.

Production hardening checklist

• Set http.Server ReadTimeout/WriteTimeout (e.g., 30s/120s).
• Wrap c.Request.Body with http.MaxBytesReader to cap payload size at ~1MB.
• Use a structured logger (zerolog, zap) instead of log.Printf.
• Add Prometheus metrics for request count, latency, and upstream errors.
• For streaming, ensure your load balancer does not buffer responses (disable proxy buffering in nginx).

The code above is a complete, runnable gin proxy chat completions handler with non-streaming and streaming paths, auth injection, and a policy hook. From here, extend it with caching, token metering, or per-tenant key mapping as your system requires.