Echo context propagation llm calls is a common stumbling block when building Go web services that proxy to language models. If you leak context or ignore cancellation, a slow upstream inference call will hang your HTTP handler and exhaust resources. This guide shows how to thread request-scoped values, deadlines, and trace metadata through Echo handlers into LLM client calls without writing brittle boilerplate.
Step 1: Capture the request context in your Echo handler
Echo gives you echo.Context, but the underlying net/http request context is what cancels when the client disconnects. Always derive your LLM call context from c.Request().Context(), never context.Background().
package main
import (
"net/http"
"time"
"github.com/labstack/echo/v4"
)
func chatHandler(c echo.Context) error {
// Correct: binds to client lifecycle.
ctx := c.Request().Context()
// Wrong: ignores disconnects and timeouts.
// ctx := context.Background()
_ = ctx
return c.String(http.StatusOK, "ack")
}The moment the client closes the connection, ctx.Done() fires. Any LLM client that respects context will abort the in-flight request.
Step 2: Inject request-scoped values with a typed key
You need trace IDs, tenant IDs, or abuse signals available deep inside your LLM call path. Put them in the context using a custom key type to avoid collisions.
type ctxKey string
const traceIDKey ctxKey = "trace_id"
func traceMiddleware(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
traceID := c.Request().Header.Get("X-Trace-Id")
if traceID == "" {
traceID = generateID()
}
ctx := context.WithValue(c.Request().Context(), traceIDKey, traceID)
c.SetRequest(c.Request().WithContext(ctx))
c.Response().Header().Set("X-Trace-Id", traceID)
return next(c)
}
}After this middleware, any code that calls c.Request().Context() gets the trace ID. This is the backbone of echo context propagation llm calls: the handler and the model client share one immutable context tree.
Step 3: Configure an OpenAI-compatible client that takes context
Most Go LLM SDKs follow the openai package pattern where every method accepts a context.Context as the first argument. If you route through n4n.ai, its single OpenAI-compatible endpoint addresses 240+ models and honors the same context cancellation, which keeps your echo context propagation llm calls provider-agnostic.
import "github.com/sashabaranov/go-openai"
func newClient() *openai.Client {
cfg := openai.DefaultConfig("sk-your-key")
cfg.BaseURL = "https://api.n4n.ai/v1" // or your gateway
return openai.NewClientWithConfig(cfg)
}
func askLLM(ctx context.Context, client *openai.Client, prompt string) (string, error) {
req := openai.ChatCompletionRequest{
Model: "gpt-4o-mini",
Messages: []openai.ChatCompletionMessage{
{Role: openai.ChatMessageRoleUser, Content: prompt},
},
}
resp, err := client.CreateChatCompletion(ctx, req)
if err != nil {
return "", err
}
return resp.Choices[0].Message.Content, nil
}The ctx passed here should be the Echo request context, possibly with a timeout wrapped around it.
Step 4: Apply a derived timeout to bound inference latency
Model calls can take 10–60 seconds. You want a hard cap that is stricter than your HTTP server timeout but looser than your SLA. Derive it from the request context so cancellation propagates both ways.
func chatHandler(c echo.Context) error {
baseCtx := c.Request().Context()
ctx, cancel := context.WithTimeout(baseCtx, 8*time.Second)
defer cancel()
client := newClient()
answer, err := askLLM(ctx, client, "Summarize: "+c.QueryParam("q"))
if err != nil {
if ctx.Err() == context.DeadlineExceeded {
return c.JSON(http.StatusGatewayTimeout, map[string]string{"error": "llm timeout"})
}
return c.JSON(http.StatusBadGateway, map[string]string{"error": err.Error()})
}
return c.JSON(http.StatusOK, map[string]string{"answer": answer})
}If the client disconnects before the 8 seconds, baseCtx cancels, ctx cancels, and the SDK aborts the HTTP request to the model provider.
Step 5: Forward trace metadata to the LLM gateway
Providers and gateways use headers for routing, cache hints, or billing tags. Echo context propagation llm calls should carry the trace ID into the outbound request. The go-openai client allows a custom HTTPClient with a roundtripper that injects headers.
type traceInjector struct {
rt http.RoundTripper
}
func (t *traceInjector) RoundTrip(req *http.Request) (*http.Response, error) {
if traceID, ok := req.Context().Value(traceIDKey).(string); ok {
req.Header.Set("X-Trace-Id", traceID)
}
return t.rt.RoundTrip(req)
}
func newClientWithTrace() *openai.Client {
cfg := openai.DefaultConfig("sk-your-key")
cfg.BaseURL = "https://api.n4n.ai/v1"
cfg.HTTPClient = &http.Client{
Transport: &traceInjector{rt: http.DefaultTransport},
}
return openai.NewClientWithConfig(cfg)
}Now every LLM request emits the same trace ID you set in Step 2. If you later query gateway logs, you can correlate a slow completion with a specific Echo request.
Step 6: Implement fallback without losing context
A single provider may rate-limit. You can attempt a secondary model, but both attempts must share the same ctx. Use a helper that loops over model names.
func askWithFallback(ctx context.Context, client *openai.Client, prompt string, models []string) (string, error) {
var lastErr error
for _, m := range models {
req := openai.ChatCompletionRequest{
Model: m,
Messages: []openai.ChatCompletionMessage{
{Role: openai.ChatMessageRoleUser, Content: prompt},
},
}
resp, err := client.CreateChatCompletion(ctx, req)
if err == nil {
return resp.Choices[0].Message.Content, nil
}
lastErr = err
// If context cancelled, stop immediately.
if ctx.Err() != nil {
return "", ctx.Err()
}
}
return "", lastErr
}Because the context is checked each iteration, a deadline or disconnect stops the fallback loop. This pattern keeps echo context propagation llm calls clean under partial degradation.
Step 7: Verify end to end with a local test
Run the server and hit it with curl, then simulate a slow provider by adding a delay in your client or using a mock. Confirm three things: response returns within timeout, trace ID echoes back, and cancelling curl mid-flight stops the upstream call.
# Terminal 1: start server
go run main.go
# Terminal 2: normal request
curl -i "http://localhost:8080/chat?q=hello"
# Expect X-Trace-Id header and JSON answer.
# Terminal 3: abort test
curl -i "http://localhost:8080/chat?q=slow" &
sleep 0.2
kill -TERM %1 # simulates client disconnectTo assert context cancellation in unit tests, use httptest with a handler that sleeps and check ctx.Err().
func TestChatCancellation(t *testing.T) {
e := echo.New()
e.Use(traceMiddleware)
e.GET("/chat", chatHandler)
req := httptest.NewRequest(http.MethodGet, "/chat?q=hi", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
// Immediately cancel request context.
ctx, cancel := context.WithCancel(req.Context())
c.SetRequest(req.WithContext(ctx))
cancel()
_ = chatHandler(c)
if ctx.Err() != context.Canceled {
t.Fatal("expected canceled context")
}
}If the test passes, your echo context propagation llm calls correctly respect lifecycle signals.
Pitfalls to avoid
Do not store the echo.Context itself in a goroutine spawned from the handler. The context is recycled after the response writes. Only the derived context.Context from c.Request() is safe to pass to background work, and even then it cancels when the request ends—so for true background jobs, explicitly detach with context.WithoutCancel (Go 1.21+) or clone values manually.
Another trap: setting values on c.Set() does not propagate to the LLM client. c.Set is Echo-specific. Only context.WithValue on the request context reaches the HTTP transport layer.
Finally, watch out for middleware order. If you wrap c.Request() with a new context after a timeout middleware has already run, you may shadow the deadline. Register traceMiddleware before any timeout or auth middleware that reads the context.
Wrap-up
You now have a request-scoped trace ID, a bounded timeout, and a cancellation path that reaches the model provider. The pattern is small but eliminates the most common production incidents with LLM-backed Echo services. Echo context propagation llm calls is not a framework feature; it is disciplined use of context.Context from the HTTP edge to the inference client.