Most teams discover they are redacting PII in LLM logs only after a compliance auditor flags plaintext emails and phone numbers in their prompt stores. If you send user data to any LLM API, your request logs are a liability unless you strip identifiers before they hit disk. This guide walks through a concrete pipeline for redacting PII in LLM logs that you can deploy in an afternoon and maintain under real load.
Step 1: Locate every log sink that captures request bodies
You cannot redact what you cannot see. Map the path of a typical chat completion call from your service to the model provider. In a standard Python stack, logs are written in three places: the application logger, any middleware or proxy, and the provider’s own usage dashboard.
Focus on the first two. Provider dashboards are out of your control, so you must redact before the request leaves your boundary. If you call the API directly from multiple services, standardize on a single HTTP client wrapper or a sidecar proxy. Redacting PII in LLM logs is a boundary concern, not a per-service chore.
# example: centralized client wrapper
import httpx
class RedactingClient:
def __init__(self, base_url: str, redactor):
self._client = httpx.Client(base_url=base_url)
self._redactor = redactor
def post(self, path: str, json: dict):
redacted = self._redactor.redact_payload(json)
return self._client.post(path, json=redacted)Step 2: Pick a redaction primitive
Redacting PII in LLM logs is not the same as anonymizing a dataset. You need reversibility only if you must debug production issues, but most teams are better served by irreversible masks.
Three primitives:
- Pattern mask: replace with a fixed token like
[EMAIL]. Cheap, deterministic, destroys debuggability. - Hash with salt:
sha256(email + salt)[:16]. Preserves equality (same email → same hash) so you can correlate logs without exposing value. - Token vault: map original to random UUID in a KMS-backed store. Expensive, but reversible for authorized responders.
For regulated industries, hashing with a per-tenant salt is the pragmatic default. It satisfies “cannot reconstruct the individual” while letting you group all requests from a given user across a trace.
import hashlib
def hash_pii(value: str, salt: str) -> str:
return "hash:" + hashlib.sha256((value + salt).encode()).hexdigest()[:16]Avoid naive replacement with ***. Auditors will ask whether the original is recoverable; a salted hash has a clear answer.
Step 3: Implement a recursive redactor for OpenAI-style payloads
LLM requests are nested JSON. Messages contain content that may be a string or a list of parts. Tool calls and function responses embed JSON too. Write a function that walks the structure and applies your primitive to detected entities.
Start with regex for high-precision fields (email, phone, SSN). For free text, add an ML detector such as Microsoft Presidio. Keep the hot path fast—regex first, ML only when regex misses.
import re
from typing import Any
EMAIL_RE = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")
PHONE_RE = re.compile(r"\b\d{3}[-.]?\d{3}[-.]?\d{4}\b")
def redact_text(text: str, salt: str) -> str:
text = EMAIL_RE.sub(lambda m: hash_pii(m.group(0), salt), text)
text = PHONE_RE.sub(lambda m: hash_pii(m.group(0), salt), text)
return text
def redact_payload(payload: Any, salt: str) -> Any:
if isinstance(payload, dict):
return {k: redact_payload(v, salt) for k, v in payload.items()}
if isinstance(payload, list):
return [redact_payload(i, salt) for i in payload]
if isinstance(payload, str):
return redact_text(payload, salt)
return payloadOptional ML layer:
from presidio_analyzer import AnalyzerEngine
analyzer = AnalyzerEngine()
def redact_with_presidio(text: str, salt: str) -> str:
results = analyzer.analyze(text=text, language="en")
for r in sorted(results, key=lambda x: x.start, reverse=True):
text = text[:r.start] + hash_pii(text[r.start:r.end], salt) + text[r.end:]
return textCall redact_with_presidio inside redact_text after regex. The walker above handles messages[].content and arbitrary metadata without special-casing.
Step 4: Redact streaming responses and async spans
Logs often capture response chunks for latency debugging. If the model echoes PII (e.g., a user asks “repeat my email”), the stream must be redacted too. Attach the same redactor to your response iterator.
def redact_stream(stream, salt):
for chunk in stream:
if hasattr(chunk, "choices") and chunk.choices:
text = chunk.choices[0].delta.content or ""
chunk.choices[0].delta.content = redact_text(text, salt)
yield chunkFor distributed tracing (OpenTelemetry), scrub attributes in a span processor before export. Never rely on the default stdout exporter in prod.
from opentelemetry.sdk.trace import SpanProcessor
class RedactProcessor(SpanProcessor):
def on_end(self, span):
for k, v in list(span.attributes.items()):
if isinstance(v, str) and EMAIL_RE.search(v):
span.set_attribute(k, hash_pii(v, SALT))Redacting PII in LLM logs must cover both directions; response leakage is the more common audit failure because teams only scrub the request.
Step 5: Deploy as a proxy or wrapper, not an afterthought
The redaction layer must be mandatory. If you use a gateway, enforce it there. For example, an OpenAI-compatible endpoint that honors client routing directives lets you insert a redaction sidecar without modifying app code; n4n.ai forwards provider cache-control hints, so pre-redaction won’t break prompt caching as long as you redact deterministically before the cache key is computed.
Otherwise, enforce at the HTTP client level with a unit-tested middleware. Fail closed: if the redactor throws, drop the log line, not the PII.
def safe_redact(payload, salt):
try:
return redact_payload(payload, salt)
except Exception:
return {"redaction_error": True, "model": payload.get("model")}Log the error count separately. A rising error rate means your payload shape changed and the walker needs a patch—not an excuse to disable redaction.
Step 6: Write fixture-based tests that fail on leakage
Redaction regressions are silent. Build a fixture file with real-looking PII (generated, not production) and assert zero matches in redacted output.
import pytest
SAMPLE = {
"messages": [{"role": "user", "content": "Email me at jane@acme.com or call 415-555-2671"}]
}
def test_redact_removes_pii():
out = redact_payload(SAMPLE, salt="test")
assert "jane@acme.com" not in str(out)
assert "415-555-2671" not in str(out)
assert "hash:" in str(out)Add a CI step that runs the detector over the last 1k lines of a staging log file and fails if it finds a valid email pattern:
# fail build if any email-shaped string survives redaction in staging logs
if grep -Eo '"content":"[^"]*@[^"]*\.[^"]*"' staging-logs.jsonl | grep -v 'hash:'; then
echo "PII leakage detected" && exit 1
fiTreat the test suite as the contract. If a new field appears in the API spec, the recursive walker covers it, but the tests prove it.
Step 7: Sample and verify in production
Deploy, then confirm. Pull a random sample of 100 redacted request logs from your sink (Datadog, Loki, S3). Run the same regexes from Step 3 over them.
# example: scan a JSON lines log for leaked emails
grep -Eo '"content":"[^"]*@[^"]*\.[^"]*"' redacted-logs.jsonl | headIf that returns anything, your walker missed a field. Common gaps: metadata, tags, function_call.arguments, and system message injections from third-party plugins. Patch and redeploy.
Also verify that hashes are stable per salt: same input in two logs yields identical hash, proving correlation works without exposure. Rotate the salt on a schedule (quarterly) and document the rotation in your compliance matrix.
How to verify success
Success means three things: (1) automated tests pass on every PR, (2) production log scans return zero PII pattern matches for a rolling 24h window, (3) on-call can still debug because hashed values group related requests.
Set an alert: if the redactor’s error fallback fires more than 1% of requests, page someone. That fallback is a signal you changed the payload shape and the walker needs updating.
Redacting PII in LLM logs is not a one-time script. It is a maintained boundary, like auth. Treat it as such, and your audit will be boring—which is the goal.