Building tamper-evident audit logs for AI systems

Regulators increasingly expect that every prompt, completion, and tool call in a production AI system can be reconstructed and proven untampered. Building tamper-evident AI audit logs requires more than appending to a file; you need cryptographic chaining, signed roots, and disciplined capture of model I/O. This guide lays out a concrete sequence to implement such a log in a system that calls LLMs.

What must be captured

A log entry is only as useful as the context it preserves. At minimum, record:

• A correlation ID for the request
• Actor identity (user or service account)
• Model identifier and version pin
• Timestamp with monotonic component
• The exact prompt payload (or a hash if size/PII demands)
• The completion payload (or hash)
• Token counts (prompt, completion, total)
• Tool or function calls invoked
• Routing metadata (which provider actually served the request)
• Explicit refusal or safety flag

Missing any of these creates blind spots that auditors will flag.

{
  "id": "req_8f2c",
  "actor": "user_4421",
  "model": "gpt-4o-2024-05-13",
  "ts": 1718240000,
  "mono": 12,
  "prompt_sha": "a3f...",
  "completion_sha": "9b1...",
  "tokens": {"prompt": 120, "completion": 45, "total": 165},
  "tools": ["search.fetch"],
  "route": {"provider": "openai", "cache_hit": false},
  "refusal": false
}

Store the full payload off-chain if you can; store the hash in the chain regardless.

Build a hash chain, not per-row hashes

A single SHA-256 of each row proves that row wasn’t altered, but it does not prove insertion or deletion of rows. Chain them: each entry’s hash covers the previous entry’s hash plus its own canonical bytes.

import hashlib, json, time

class AuditLog:
    def __init__(self, previous_hash="0"*64):
        self.previous_hash = previous_hash

    def append(self, event: dict) -> dict:
        event["ts"] = event.get("ts", int(time.time()))
        canonical = json.dumps(event, sort_keys=True, separators=(",",":"))
        block = self.previous_hash + canonical
        event["hash"] = hashlib.sha256(block.encode()).hexdigest()
        self.previous_hash = event["hash"]
        return event

Serialize with sort_keys=True and fixed separators. Any nondeterministic JSON encoding silently breaks verification.

Append returns the event with its hash. Persist the whole dict, including hash, to your storage layer.

Sign the chain head

Hash chaining detects tampering but not a full chain swap. You need a root of trust. Sign the latest hash with an asymmetric key whose private material lives in a KMS, not on the app server.

from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey

priv = Ed25519PrivateKey.from_private_bytes(key_bytes_from_kms())
signature = priv.sign(latest_hash.encode())

Store the signature alongside the head pointer. Rotate keys on a schedule and keep a public-key ledger. Verification checks the head hash against the signature using the published public key for that epoch.

Tradeoff: if you sign too frequently you add latency; sign every N entries or on a time interval (e.g., every 100 entries or 5 minutes).

Anchor externally for strong non-repudiation

For regulated industries, internal signing may be insufficient if the auditor distrusts your KMS. Anchor the head hash to an external transparency log or a ledger you do not solely control.

Options:

• Write the head hash to a public blockchain transaction (cost/latency tradeoff)
• Submit to a certificate transparency style log
• Periodically email/hash-post to a mailing list archive

The anchor is a secondary check; the chain and signature remain the primary controls.

Store append-only

Use write-once-read-many (WORM) storage. S3 Object Lock in compliance mode, GCS retention policies, or a dedicated append-only log service. Never let application code have delete or overwrite permissions.

aws s3api put-object-lock-configuration \
  --bucket ai-audit \
  --object-lock-configuration '{"ObjectLockEnabled":"Enabled","Rule":{"DefaultRetention":{"Mode":"COMPLIANCE","Days":3650}}}'

If you must use a database, use a table with no UPDATE/DELETE grants for the app role and a trigger that rejects mutations post-insert.

Capture from the inference layer

Your log is only complete if it reflects what the model actually did, not what you intended. When you call an LLM gateway, capture the response metadata before hashing.

If you route through a gateway such as n4n.ai, it returns per-token usage metering and honors client routing directives; forward those fields into your event so the tamper-evident AI audit logs match the provider’s reality, including cache-control hints and fallback provider shifts.

resp = gateway.chat(messages, route={"prefer": "anthropic"})
log.append({
    "id": resp.id,
    "model": resp.model,
    "tokens": resp.usage,
    "route": resp.actual_route,
    "cache_hit": resp.cache_control.get("read")
})

Common pitfall: logging the request object but not the normalized response, especially when the gateway performs automatic fallback due to rate limits. That fallback is exactly what auditors want to see.

Verification workflow

Write a verifier that replays the log:

1. Start with the genesis hash.
2. For each line, recompute sha256(prev_hash + canonical).
3. Assert equality with stored hash.
4. Check head signature with the epoch public key.
5. Optionally compare head hash to external anchor.

def verify(lines, pub_key, sig):
    prev = "0"*64
    for line in lines:
        evt = json.loads(line)
        canon = json.dumps(evt, sort_keys=True, separators=(",",":"))
        expect = hashlib.sha256((prev+canon).encode()).hexdigest()
        assert evt["hash"] == expect, "chain break"
        prev = evt["hash"]
    assert pub_key.verify(sig, prev.encode()), "bad signature"
    return True

Run this on a cron and alert on failure. A silent verifier is worthless.

Common pitfalls

Clock skew. Using only time.time() allows reordering under NTP jumps. Add a monotonic counter or hybrid logical clock (ts + mono) and reject entries where mono decreases.

Truncating tool output. If your agent calls a SQL tool, log the query and the row count returned, not just the natural-language summary. Otherwise the audit trail can’t show what data left the system.

Logging assumptions, not facts. Recording model: "gpt-4" when the gateway silently fell back to a smaller model because of quota breaks the chain of evidence. Capture actual_route.

Relying on DB auto-increment. Sequential IDs imply order but don’t prove it. An attacker who gains write access can shuffle rows. The hash chain is the only defense.

Ignoring refusals. A safety refusal is a critical event. Log it with the trigger category if available.

Tradeoffs to accept

Hash chaining adds ~32 bytes per entry and a small CPU cost per write. Signature anchoring adds latency unless batched. External anchoring adds dependency on a third party or chain finality.

Storage grows linearly; for high-volume systems, consider rolling chains (new genesis each day) with a signed manifest per period. This limits verification scope and simplifies key rotation.

Tamper-evident AI audit logs are not free, but the alternative—a log an auditor cannot trust—fails the compliance bar entirely. Implement the chain first, add signing next, and anchor only where the risk profile demands it.