Logging prompts and completions without leaking PII

Logging LLM prompts without leaking PII is a non-negotiable control for any production system that sends user data to a model. The naive approach of writing raw request and response bodies to stdout will eventually expose emails, tokens, or health records. This guide walks through a concrete pipeline that captures what you need for debugging and cost analysis while stripping sensitive fields before they hit your log store.

Step 1: Define a minimal structured log schema

Start by deciding what fields are actually useful for post-incident analysis. You rarely need the full prompt text to debug a latency spike or a 429. You need model name, request ID, token counts, latency, and a coarse signal about prompt content.

Define a schema:

{
  "request_id": "req_123",
  "model": "gpt-4o-mini",
  "prompt_redacted": "User asked about order status for [EMAIL]",
  "prompt_hash": "a1b2c3...",
  "completion_redacted": "Order [ORDER_ID] shipped",
  "tokens_in": 120,
  "tokens_out": 30,
  "latency_ms": 840,
  "error": null
}

Store the redacted strings only for low-cardinality categories. For anything that could be a unique identifier, store a hash instead of the value.

Step 2: Redact at the call boundary

Intercept the prompt and completion inside your LLM client wrapper, not in a separate log shipper. That ensures redaction happens even when a teammate adds a quick script.

A regex-based redactor covers the common patterns. Extend it with your domain specifics (order IDs, internal account numbers).

import re
import hashlib

PII_PATTERNS = {
    "email": re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"),
    "ssn": re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
    "api_key": re.compile(r"\b(sk|pk)_[A-Za-z0-9]{20,}\b"),
}

def redact(text: str) -> tuple[str, list[str]]:
    found = []
    for name, pat in PII_PATTERNS.items():
        if pat.search(text):
            found.append(name)
            text = pat.sub(f"[{name.upper()}]", text)
    return text, found

Wrap your completion call:

def logged_completion(client, model, prompt, request_id):
    redacted_prompt, _ = redact(prompt)
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    completion = resp.choices[0].message.content
    redacted_completion, _ = redact(completion)
    # emit log here (Step 4)
    return resp

Never log prompt or completion raw. The wrapper is the only place they exist unredacted in your process.

Step 3: Hash unique identifiers for correlation

Sometimes you need to trace all requests from a single user without storing their email or user ID. Use a salted hash computed at the edge.

SALT = b"prod-logging-salt"  # load from secret manager

def pseudonymize(uid: str) -> str:
    return hashlib.sha256(SALT + uid.encode()).hexdigest()[:16]

Log user_pseudo: pseudonymize(user_id) instead of user_id. The salt stays out of logs, so the value cannot be reversed without infrastructure access.

Step 4: Emit a single JSON log line per request

Use the standard library logging with a JSONFormatter or just json.dumps. Avoid multi-line stack traces in the same record; ship those to an error tracker.

import logging
import json

logger = logging.getLogger("llm")

def emit_log(record: dict):
    logger.info(json.dumps(record))

In logged_completion, build the record:

record = {
    "request_id": request_id,
    "model": model,
    "prompt_redacted": redacted_prompt,
    "prompt_hash": hashlib.sha256(prompt.encode()).hexdigest()[:16],
    "completion_redacted": redacted_completion,
    "tokens_in": resp.usage.prompt_tokens,
    "tokens_out": resp.usage.completion_tokens,
    "latency_ms": int(latency * 1000),
    "error": None,
}
emit_log(record)

This gives you token-level cost tracking and enough context to reproduce most bugs from the redacted text alone.

Step 5: Use gateway metering instead of body logging

If you route through an OpenAI-compatible gateway, you can avoid logging prompt text entirely for cost accounting. For example, n4n.ai returns per-token usage in the standard usage object and supports per-token usage metering, so finance can bill by team without you writing prompt bodies to a ledger.

# client is pointed at the gateway's /v1/chat/completions
resp = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[...],
)
# resp.usage holds prompt_tokens, completion_tokens

Because the gateway honors client routing directives and forwards provider cache-control hints, you can also tag requests with a header like x-request-purpose: debug and have the gateway echo it back in metrics, keeping PII out of the metrics pipeline entirely.

Automatic fallback when a provider is rate-limited or degraded means your logging schema should capture route_fallback: true if you detect a retry, but that flag carries no user data.

Step 6: Verify redaction with a replay harness

A redactor is only as good as its test suite. Write a pytest that feeds known PII through redact and asserts the patterns are gone.

def test_redact_email():
    text = "Reach me at john.doe@example.com about the account"
    out, found = redact(text)
    assert "john.doe@example.com" not in out
    assert "EMAIL" in out
    assert "email" in found

For end-to-end verification, run a local proxy that records what your app sends to the log sink. Send a canary prompt containing a fake SSN and API key, then assert the log file contains [SSN] and [API_KEY] but not the literals.

# start app with log capture to /tmp/llm.log
python app.py & 
curl -X POST localhost:8000/ask -d '{"prompt":"my ssn is 123-45-6789 and key sk_abcd1234efgh5678ijkl"}'
grep -c "123-45-6789" /tmp/llm.log  # expect 0
grep -c "\[SSN\]" /tmp/llm.log      # expect 1

If both assertions pass, your pipeline for logging LLM prompts without leaking PII is working. Re-run this harness in CI on every change to the redaction layer.

Step 7: Sample to control log volume and exposure surface

At high traffic, logging every redacted prompt still creates a large surface. Sample 10% of successful requests for redacted text; keep 100% of error and fallback records because they are rare and high-value.

import random

def should_log_full(sample_rate=0.1):
    return random.random() < sample_rate

# in logged_completion
if resp.status == "error" or route_fallback:
    record["prompt_redacted"] = redacted_prompt
else:
    record["prompt_redacted"] = redacted_prompt if should_log_full() else None

This cuts storage cost and reduces the chance that a missed redaction pattern affects a large slice of users.

Operational notes

Keep the salt and redaction patterns in configuration, not code, so compliance can update them without a deploy. Set a retention policy of 14–30 days on redacted logs; hashes can live longer if needed for trend analysis.

Don’t rely on the model provider to redact. Completion filtering is inconsistent and often misses novel formats. Your boundary is the only boundary you control.

Logging LLM prompts without leaking PII becomes a solved problem once you treat redaction as a first-class function in the request path rather than a downstream scrubber. The code above is production-shaped; adapt the patterns to your framework and ship it before your next audit.