Structured logging fields every LLM API call should capture

Most teams bolt logging onto their LLM integration as an afterthought, then scramble when a prompt regression ships or a bill spikes. The structured logging fields for LLM APIs that matter go far beyond HTTP status and latency—they must capture routing decisions, token economics, and per-request provenance so you can reconstruct any call weeks later.

Below is the set of fields I emit on every request through our gateway, whether it’s a 20-token classifier or a 32k-token RAG completion. Skip any of these and you will pay for it during the next incident review.

1. Correlation and trace identifiers

A single user action often triggers multiple model calls: a retrieval rewrite, a completion, and a JSON extraction pass. Without a shared trace_id you cannot reconstruct the sequence. At minimum, log trace_id (propagated from your frontend or orchestrator), span_id (unique per call), and parent_span_id if you have nested spans.

Use W3C trace context or OpenTelemetry semantics. Don’t invent your own format; the ecosystem tooling expects traceparent. In practice:

{
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "parent_span_id": "b7ad6b7169203331",
  "request_id": "req_8c1f2e9a"
}

If you’re using Python, bind these to logging.LoggerAdapter so every line in the request scope inherits them. That’s cheaper than threading locals through your call sites.

2. Requested vs. resolved model and provider

Clients usually ask for a capability, not a specific endpoint: "model": "gpt-4o-mini" or "model": "best-cheap-coder". The gateway may rewrite that to a provider, region, and concrete version. Log both requested_model and resolved_model, plus provider, provider_region, and route_reason. When a fallback fires, record fallback_from and fallback_to.

An OpenRouter-class gateway such as n4n.ai will forward the resolved provider and model, and emit a fallback event if the primary was rate-limited or degraded. If you roll your own router, you must log this yourself—silent fallback is how you end up comparing Claude outputs to GPT outputs in the same A/B bucket.

{
  "requested_model": "gpt-4o-mini",
  "resolved_model": "gpt-4o-mini-2024-07-18",
  "provider": "azure-openai",
  "provider_region": "eastus2",
  "route_reason": "explicit",
  "fallback_from": null,
  "fallback_to": null
}

3. Token accounting and cache hints

Token counts are your unit of cost and often your unit of latency. The structured logging fields for LLM APIs must include cached token counts because a missed cache is the most common silent cost leak. Capture prompt_tokens, completion_tokens, total_tokens, and—critically—cached_read_tokens and cached_write_tokens if the provider supports prompt caching.

Forward provider cache-control hints (cache_control on the request) and log whether the provider honored them. A structured logging field like cache_hit (bool) plus cache_key_prefix (truncated hash) lets you spot cache fragmentation caused by volatile system prompts.

{
  "prompt_tokens": 1820,
  "completion_tokens": 64,
  "total_tokens": 1884,
  "cached_read_tokens": 1500,
  "cached_write_tokens": 320,
  "cache_hit": true,
  "cache_key_prefix": "a1b2c3"
}

4. Latency breakdown: TTFT and total

A single “duration_ms” is useless for streaming LLMs. You need time_to_first_token_ms (TTFT) and time_to_last_token_ms, plus stream_duration_ms if you keep the connection open. TTFT tells you if the provider queued your request; total tells you generation speed.

Log them as integers in milliseconds. If you batch, also log inter_token_latency_p50 and p99 when you have enough samples—but per-request, raw timestamps are enough.

log.info("llm.call.complete", extra={
    "ttft_ms": 420,
    "total_ms": 3100,
    "stream_ms": 2680,
})

5. Sampling and generation parameters

Reproducing a bad output requires knowing the knobs. Log the full request config minus secrets: temperature, top_p, max_tokens, stop_sequences, frequency_penalty, presence_penalty, tools (names only), and response_format. If you pass a seed, log it; if not, log seed: null so you know randomness was in play.

Don’t log the prompt and response inline in this field—that’s a separate concern (see §8). Parameters are small and high-value for debugging prompt drift.

{
  "temperature": 0.2,
  "top_p": 1.0,
  "max_tokens": 1024,
  "stop_sequences": ["\n\n"],
  "tools": ["search", "calculator"],
  "seed": null
}

6. Finish reason and error taxonomy

finish_reason (stop, length, tool_call, content_filter) explains why generation ended. On failure, log error_code (provider-specific), http_status, provider_error_type, and retry_attempts. A generic error: true is not actionable; a structured taxonomy is.

Map provider errors to your own enum at the gateway edge so dashboards don’t fragment across “rate_limit_error” vs “429”. Example:

{
  "finish_reason": "length",
  "error_code": "rate_limit",
  "http_status": 429,
  "provider_error_type": "rate_limit_error",
  "retry_attempts": 2
}

7. Cost and tenancy metadata

Even if billing is post-hoc, log cost_usd (computed from token counts and current price sheet) and tenant_id / project_id / user_id (hashed if needed). Per-token usage metering is only useful when you can attribute it. n4n.ai and similar gateways return usage with the response; compute cost synchronously and log it before the next line of code runs.

{
  "cost_usd": 0.0042,
  "tenant_id": "org_123",
  "project_id": "prod-chat",
  "user_id_hash": "sha256:9f86d"
}

8. Content fingerprints and safety signals

You rarely want raw prompts in logs, but you need to detect duplicates and red-flag incidents. Log prompt_hash (sha256 of normalized prompt) and response_hash. If you run moderation, log moderation_flagged and moderation_categories (e.g., ["harassment"]). For streams, hash the concatenated output.

This gives you cardinality without leaking PII. When a customer complains about “that weird answer,” you can grep the hash and pull the full transcript from your secure store.

{
  "prompt_hash": "sha256:3b9e...",
  "response_hash": "sha256:7c1a...",
  "moderation_flagged": false,
  "moderation_categories": []
}

Synthesis

The fields above form a contract between your LLM client and your observability stack. Here’s the minimal schema I’d put in every log line:

Capture these as structured JSON, not concatenated strings. Your future on-call self will query them at 3 a.m. with Splunk or ClickHouse, not grep a text file. The structured logging fields for LLM APIs are not optional hygiene—they are the difference between guessing and knowing.