How long to retain LLM audit logs for compliance

Most teams treat LLM audit log retention periods as an afterthought until a regulator asks for twelve months of prompts and completions. Getting this wrong means either paying to store terabytes of low-value text or facing gaps during an audit. This guide gives you a concrete process to set, implement, and verify retention policies for LLM inference logs.

Step 1: Map the regulations that actually apply to your logs

Retention is not a single number. A healthcare app under HIPAA must treat logs containing PHI differently from a consumer chatbot under GDPR. Start by listing the regimes you operate under and the specific clauses that mention recordkeeping.

For example, SOC 2 does not mandate a fixed duration but expects you to define one and enforce it consistently. GDPR Article 5(1)(e) requires storage limitation—keep personal data no longer than necessary for the purpose. Financial services under SEC Rule 17a-4 often demand six years for communications and require write-once-read-many (WORM) storage. Your LLM audit log retention periods should be derived from the strictest applicable rule per data class, not a company-wide guess.

Overlaps happen. If a log contains EU personal data and falls under SEC scope, the longer period and the WORM requirement win. Document the precedence order explicitly.

Write this down as code, not a wiki page:

# compliance_map.yaml
regulations:
  - name: GDPR
    scope: EU_personal_data
    max_retention_days: 365
    requires_erasure: true
    worm: false
  - name: HIPAA
    scope: PHI
    max_retention_days: 2555  # 7 years typical for audit controls
    requires_erasure: true
    worm: false
  - name: SEC_17a-4
    scope: financial_comms
    max_retention_days: 2190
    requires_erasure: false  # must be retrievable, not deleted
    worm: true
precedence: [SEC_17a-4, HIPAA, GDPR]

Load this in your retention service so policy changes are reviewable in pull requests and auditable later.

Step 2: Classify your log fields by sensitivity and value

A raw LLM request contains three categories: routing metadata (model, latency, token counts, request ID), prompt text, and completion text. The metadata is low-risk and high-value for debugging and cost allocation. Prompt and completion may contain PII, PHI, or secrets.

Classify each field at ingest. A simple deterministic classifier avoids shipping text to a separate classifier model and keeps latency near zero:

def classify_log_record(record: dict) -> str:
    """Return retention class: 'metadata_only', 'pii', 'phi'."""
    prompt = record.get("prompt", "")
    completion = record.get("completion", "")
    combined = prompt + completion
    if any(tag in combined for tag in ("SSN", "diagnosis", "patient", "ICD-")):
        return "phi"
    if any(tag in combined for tag in ("@example.com", "phone", "address", "email")):
        return "pii"
    return "metadata_only"

Run this synchronously in your logging middleware. The class travels with the record and drives later TTL. If you cannot inspect cleartext (e.g., client-side encryption), force the strictest class—never default to metadata_only on unknown content.

Segregate rather than redact when possible. Store metadata in a hot table; push prompt/completion to a restricted bucket. Redaction at the edge reduces blast radius but complicates later forensic review.

Step 3: Define explicit retention periods per class

With regulations mapped and fields classified, produce a concrete policy. Keep metadata for 90 days; PII for the GDPR max; PHI for the HIPAA max. If a record mixes classes, assign the strictest.

{
  "retention_policy": {
    "metadata_only": {"days": 90, "erase": true, "worm": false},
    "pii": {"days": 365, "erase": true, "worm": false},
    "phi": {"days": 2555, "erase": true, "worm": false},
    "financial_comms": {"days": 2190, "erase": false, "worm": true}
  }
}

Store this as versioned JSON in your config repo. Your LLM audit log retention periods are now declarative and auditable. When a new regulation appears, you add one entry and bump precedence—no code change required.

Step 4: Instrument your pipeline to tag entries at the edge

Capture logs as close to the model call as possible. If you route through an OpenAI-compatible gateway, n4n.ai fronts 240+ models with automatic fallback and per-token usage metering, which gives you a single place to attach retention class and timestamps before the data spreads across services.

Emit a structured log line with mandatory fields:

import json, time, logging

logger = logging.getLogger("audit")

def emit_audit_log(request_id, model, prompt, completion, tokens):
    cls = classify_log_record({"prompt": prompt, "completion": completion})
    entry = {
        "ts": int(time.time()),
        "request_id": request_id,
        "model": model,
        "class": cls,
        "tokens": tokens,
        "prompt": prompt if cls != "metadata_only" else "[redacted]",
        "completion": completion if cls != "metadata_only" else "[redacted]"
    }
    logger.info(json.dumps(entry))

Ship these to an append-only store. For most teams, object storage with daily prefixes is cheapest; for queryability, load into a columnar table with a class and ts column. Keep the raw object as the system of record and the database as an index.

Step 5: Enforce retention with lifecycle rules or jobs

Do not rely on humans to delete. Use bucket lifecycle policies for raw JSON, and scheduled SQL for indexed copies. For WORM classes, use object lock instead of expiration.

S3 example for erasable classes:

aws s3api put-bucket-lifecycle-configuration --bucket my-llm-audit \
  --lifecycle-configuration file://lifecycle.json

{
  "Rules": [
    {"ID": "metadata-90", "Prefix": "class=metadata_only/",
     "Expiration": {"Days": 90}, "Status": "Enabled"},
    {"ID": "pii-365", "Prefix": "class=pii/",
     "Expiration": {"Days": 365}, "Status": "Enabled"},
    {"ID": "phi-2555", "Prefix": "class=phi/",
     "Expiration": {"Days": 2555}, "Status": "Enabled"}
  ]
}

For SEC-class WORM, enable compliance mode object lock at bucket creation; you cannot expire, only retain.

If you use Postgres for the index:

DELETE FROM audit_logs
WHERE ts < extract(epoch from now()) - (retention_days * 86400)
  AND class IN (SELECT class FROM retention_policy WHERE erase);

Run this daily inside a transaction with a advisory lock to avoid concurrent double-deletes. The object store remains the source of truth; the DB delete just shrinks the search surface.

Step 6: Verify retention is actually working

A policy that isn’t verified is a liability. Write a test that asserts no row older than its class limit exists, and run it in CI against a production replica weekly.

def test_retention_enforced(db_conn):
    cur = db_conn.cursor()
    cur.execute("SELECT class, MIN(ts) FROM audit_logs GROUP BY class")
    limits = {"metadata_only": 90, "pii": 365, "phi": 2555}
    now = time.time()
    for cls, min_ts in cur.fetchall():
        age_days = (now - min_ts) / 86400
        assert age_days <= limits[cls], f"Stale {cls} log: {age_days:.1f}d"

Also alert if the count of deleted rows is zero for 30 days—that may indicate the job died or ingestion stopped. For object storage, list the earliest object per prefix:

aws s3 ls s3://my-llm-audit/class=pii/ --recursive | awk '{print $1}' | sort | head -1

Confirm the earliest date is within 365 days. If not, the lifecycle rule failed or the prefix is mislabeled. Add a checksum manifest per day so you can prove nothing was silently altered before deletion.

Step 7: Document the decision and review quarterly

Retention is a living policy. Regulations change; your product may enter a new market or add a new model provider. Put the YAML, JSON, and this runbook in one repo. Quarterly, diff the regulation map against your current product scope and update precedence.

When you shorten a period, run a manual purge and record the attestation in the audit log itself—a meta-entry stating “purged class=pii prior to 2023-01-01 per GDPR update”. That entry is itself subject to the new policy but should carry a legal-hold flag so it survives the purge it describes.

Train on-call engineers to recognize a legal hold request. A hold freezes deletion for matched request IDs regardless of class. Implement it as a blocklist table checked before every DELETE.

Getting LLM audit log retention periods right is less about a magic number and more about traceable, enforced automation. Follow these steps and you can answer a regulator with a commit hash, not a panic.