n4nAI
Open console →
API integrationTutorialGin & Echo LLM API Integration

Building an OpenAI-compatible endpoint in Gin

A hands-on Go tutorial: implement a gin openai compatible endpoint that forwards chat completions to an upstream LLM provider with proper API contracts. Learn request shaping and streaming.

n4n Team2 min read490 words

Audio narration

Coming soon — every post will get a voice note here.

Building a gin openai compatible endpoint means accepting the exact JSON contract that OpenAI clients emit and returning the same shape, while proxying to a real model backend. This tutorial implements a minimal Gin service in Go that proxies /v1/chat/completions to an upstream OpenAI-compatible API, handles both buffered and streaming responses, and stays faithful to the wire format.

Prerequisites

  • Go 1.21 or newer
  • github.com/gin-gonic/gin (v1.9+)
  • An API key for an OpenAI-compatible backend (OpenAI, or a gateway such as n4n.ai which provides one OpenAI-compatible endpoint for 240+ models)
  • curl for local verification

Scaffold the module

mkdir gin-llm-proxy && cd gin-llm-proxy
go mod init example.com/gin-llm-proxy
go get github.com/gin-gonic/gin

You now have a Go module ready for a single-file server.

Model the OpenAI contract

OpenAI’s chat completions API is strict about field names. If you deviate, SDKs like openai-python or LangChain break. Define the minimal request and response structs you need, but prefer forwarding raw JSON for unknown fields.

type Message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type ChatRequest struct {
    Model    string    `json:"model"`
    Messages []Message `json:"messages"`
    Stream   bool      `json:"stream,omitempty"`
}

type Usage struct {
    PromptTokens     int `json:"prompt_tokens"`
    CompletionTokens int `json:"completion_tokens"`
    TotalTokens      int `json:"total_tokens"`
}

type Choice struct {
    Index        int     `json:"index"`
    Message      Message `json:"message"`
    FinishReason string  `json:"finish_reason"`
}

type ChatResponse struct {
    ID      string   `json:"id"`
    Object  string   `json:"object"`
    Created int64    `json:"created"`
    Model   string   `json:"model"`
    Choices []Choice `json:"choices"`
    Usage   Usage    `json:"usage"`
}

For a truly compatible surface, forward the raw request body instead of binding to a struct. SDKs send temperature, top_p, n, stop, and more. We’ll use c.GetRawData() later.

Implement the proxy handler

Create main.go. The handler reads the incoming body, issues an authenticated POST to the upstream, and copies the response back with the same status code and content type.

package main

import (
    "io"
    "net/http"
    "os"
    "time"

    "github.com/gin-gonic/gin"
)

func main() {
    r := gin.Default()
    r.POST("/v1/chat/completions", chatCompletions)
    r.Run(":8080")
}

func chatCompletions(c *gin.Context) {
    raw, err := c.GetRawData()
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "invalid body"})
        return
    }

    upstream := os.Getenv("UPSTREAM_BASE") // e.g. https://api.openai.com/v1
    key := os.Getenv("UPSTREAM_API_KEY")

    req, err := http.NewRequest(http.MethodPost, upstream+"/chat/completions", bytes.NewReader(raw))
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": "request build failed"})
        return
    }
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", "Bearer "+key)

    client := &http.Client{Timeout: 120 * time.Second}
    resp, err := client.Do(req)
    if err != nil {
        c.JSON(http.StatusBadGateway, gin.H{"error": "upstream unreachable"})
        return
    }
    defer resp.Body.Close()

    c.Status(resp.StatusCode)
    c.Header("Content-Type", resp.Header.Get("Content-Type"))
    io.Copy(c.Writer, resp.Body)
}

This is a complete non-streaming gin openai compatible endpoint. It forwards everything verbatim, so the response shape is guaranteed to match the upstream.

Streaming with Server-Sent Events

OpenAI streams incremental tokens as text/event-stream. Clients set "stream": true. Our raw forward already sends that flag upstream; we just need to flush chunks as they arrive instead of buffering.

Modify the response copy section:

    if resp.Header.Get("Content-Type") == "text/event-stream" {
        c.Header("Content-Type", "text/event-stream")
        c.Header("Cache-Control", "no-cache")
        c.Header("Connection", "keep-alive")
        flusher, ok := c.Writer.(http.Flusher)
        if !ok {
            c.JSON(http.StatusInternalServerError, gin.H{"error": "streaming unsupported"})
            return
        }
        buf := make([]byte, 4096)
        for {
            n, readErr := resp.Body.Read(buf)
            if n > 0 {
                c.Writer.Write(buf[:n])
                flusher.Flush()
            }
            if readErr != nil {
                break
            }
        }
        return
    }

Place this before the io.Copy fallback. Gin’s c.Writer implements http.Flusher under the default server.

Run and test

Set environment variables and start the server.

export UPSTREAM_BASE="https://api.openai.com/v1"
export UPSTREAM_API_KEY="sk-your-key"
go run main.go

In another shell, send a buffered request:

curl -X POST localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Say hi"}],"stream":false}'

Expected output (truncated):

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-3.5-turbo",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Hi there! How can I help?"},
      "finish_reason": "stop"
    }
  ],
  "usage": {"prompt_tokens": 10, "completion_tokens": 6, "total_tokens": 16}
}

Now test streaming:

curl -X POST localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Count to 3"}],"stream":true}'

You should see SSE frames:

data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","choices":[{"delta":{"content":"1"}}]}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","choices":[{"delta":{"content":" 2"}}]}
data: [DONE]

Production considerations

The code above is a starting point, not a finished service. Harden it before real traffic:

  • Cancel upstream on client disconnect. Use c.Request.Context() as the request context so a dropped connection stops the upstream call.
  • Timeout tuning. LLM calls can run long. Set client.Timeout to zero and rely on context deadlines, or use a generous value like 120s.
  • Metering. If you operate the backend yourself, parse usage from the response and emit per-token metrics. Gateways such as n4n.ai handle per-token usage metering and automatic fallback when a provider is rate-limited, which removes this burden.
  • Routing directives. OpenAI-compatible clients sometimes send headers or model aliases. If you proxy to a multi-provider gateway, honor client routing hints and forward provider cache-control headers.
  • Raw forwarding caveat. c.GetRawData() consumes the body once. If you need middleware logging, read it once and store it in the context.

A gin openai compatible endpoint is fundamentally a translation and proxy layer. Keep the contract intact, stream when asked, and push provider-specific resilience to the edge or to a gateway you trust. The 30 lines of Go above are enough to get a real SDK talking to your infrastructure today.

Tagsgolangginopenai-apirest-api

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationGin middleware for LLM API key authenticationLearn how to build Gin middleware for LLM API key authentication in Go. Step-by-step guide with runnable code to validate keys and proxy to model gateways.API integrationRate limiting LLM endpoints with Gin middlewareImplement token-aware rate limiting for LLM APIs in Go using Gin middleware. Step-by-step guide with code for protecting endpoints from abuse.API integrationStructuring a Gin project for a multi-provider LLM proxyPractical guide to organizing a Gin-based multi-provider LLM proxy: directory layout, provider interfaces, routing, streaming, and middleware for auth and metering.API integrationTesting Gin LLM routes with httptest and table testsLearn how to test Gin LLM routes using httptest and table driven tests in Go, with runnable examples for robust LLM API integration.API integrationGin vs Echo for building an LLM gateway in GoA practical head-to-head comparison of Gin vs Echo for building an LLM gateway in Go, covering latency, streaming, middleware, and ecosystem tradeoffs.API integrationProxying chat completions through a Gin handlerBuild a Go Gin handler that proxies OpenAI-style chat completions requests to an LLM gateway, with streaming, auth injection, and error handling.

All gin & echo llm api integration posts →