Audit logging requirements for AI in financial services

Building AI audit logging in financial services is not about ticking a compliance checkbox; it’s about constructing a forensic record that survives regulatory scrutiny years after the fact. If your LLM pipeline can’t reconstruct who asked what, which model answered, and why, you’re exposed to SEC 17a-4 books-and-records rules and model-risk mandates like SR 11-7.

What Regulators Actually Expect

Regulated financial firms must treat model inputs and outputs as records of business activity. For broker-dealers, FINRA and SEC Rule 17a-4 require writable-once, readable-many (WORM) retention for defined periods—often seven years. MiFID II demands timestamping to milliseconds for investment advice. Beyond storage, examiners want provenance: the exact model version, parameters, and upstream data sources.

The catch is that LLM systems are non-deterministic and multi-tenant. Your audit layer must compensate for that by capturing enough context to make any individual inference reproducible or at least explainable.

Step 1: Define the Audit Scope and Event Taxonomy

Start by enumerating the events that matter. A minimal set for AI audit logging in financial services includes:

• inference.request — prompt sent, user identity, session, model ID.
• inference.response — completion, token counts, latency.
• retrieval.document — sourced chunks with IDs.
• human.override — manual edit of model output.
• config.change — prompt template or model version swap.

Encode these as a strict schema. Example:

{
  "event_type": "inference.request",
  "event_id": "uuid4",
  "ts": "2025-04-12T15:04:05.123Z",
  "actor": {"user_id": "u_839", "role": "teller"},
  "model": {"id": "gpt-4o-2024-08-06", "provider": "openai"},
  "request": {"prompt_hash": "sha256:...", "redacted": true}
}

Treat this schema as a contract. Version it.

Step 2: Capture Provenance Metadata at Request Time

Don’t rely on backend logs scraped after the fact. Instrument the calling code. A Python wrapper around an OpenAI-compatible client can attach trace headers and emit a structured event before the call returns.

import time, uuid, hashlib, json, logging

def logged_chat(client, user_id, prompt, model="gpt-4o"):
    evt = {
        "event_type": "inference.request",
        "event_id": str(uuid.uuid4()),
        "ts": time.strftime("%Y-%m-%dT%H:%M:%S.%fZ", time.gmtime()),
        "actor": {"user_id": user_id},
        "model": {"id": model},
        "prompt_hash": hashlib.sha256(prompt.encode()).hexdigest()
    }
    logging.info(json.dumps(evt))
    return client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}]
    )

The hash lets you prove the prompt content later without storing raw PII in the hot log. Keep the full redacted prompt in cold storage.

Step 3: Ensure Immutable, Time-Synced Storage

WORM isn’t optional. Use object storage with compliance lock (S3 Object Lock in compliance mode) or a hash-chained append-only file. For a lightweight proof-of-concept:

# append event, then seal with previous hash
PREV=$(tail -n1 chain.log | jq -r .hash)
HASH=$(echo "$EVENT" | sha256sum | cut -d' ' -f1)
echo "$EVENT" | jq --arg h "$HASH" --arg p "$PREV" '. + {hash:$h, prev:$p}' >> chain.log

Clock skew destroys forensic value. Sync hosts to NTP and stamp in UTC. Tradeoff: strict chaining adds write latency; batch events if throughput demands, but document the batch window in your policy.

Step 4: Log the Full Prompt/Response with Redaction

AI audit logging in financial services fails when teams store raw prompts containing account numbers. Redact before persistence. A minimal mask:

import re
def redact(text):
    text = re.sub(r"\b\d{9,18}\b", "[ACCT]", text)  # naive account number
    text = re.sub(r"\b\d{3}-\d{2}-\d{4}\b", "[SSN]", text)
    return text

Store the redacted full text alongside the hash. Keep the original only in an access-controlled vault with its own audit trail. Pitfall: redaction regexes drift; test against real transcripts quarterly.

Step 5: Correlate with Model and Provider Telemetry

A single user request may hit multiple providers due to fallback. Capture token usage, cache hits, and routing decisions. An inference gateway that exposes per-token metering and forwards provider cache-control hints simplifies this—n4n.ai, for example, returns usage fields and honors client routing directives, giving you consistent audit columns without custom provider adapters.

{
  "event_type": "inference.response",
  "usage": {"prompt_tokens": 412, "completion_tokens": 88, "cache_read": 300},
  "route": {"primary": "openai", "fallback_used": false}
}

If a provider is degraded and your system fails over, that is itself an audit event. Log it with the same schema.

Step 6: Retention, Access Controls, and Reconstruction

Set retention to the maximum applicable period (seven years for many SEC records). Use IAM policies so only a compliance role can read the raw chain; application services get a write-only principal.

Build a replay tool that, given an event_id, fetches the redacted prompt/response, the model version, and the retrieval documents, then renders a PDF for examiners. If you can’t reconstruct the exact input/output within an hour, your process is immature.

Step 7: Model Risk and Change Management

Log every prompt-template deployment and model version pin. When you swap gpt-4o for claude-3-5-sonnet, emit a config.change event referencing the git commit. SR 11-7 expects documented model inventory; your audit log is that inventory’s backbone.

Common Pitfalls and Tradeoffs

• Under-logging: Skipping retrieval sources makes advice unexplainable. Always log document IDs.
• Over-logging: Writing full PII to hot logs triggers GDPR and GLBA violations. Redact at the edge.
• Non-repudiation gaps: If the signing key lives on the same host as the app, a compromise falsifies history. Use a separate sealing service.
• Latency vs. compliance: Synchronous hash chaining can add 10–20ms per event. Accept it for regulated paths; sample for internal analytics.
• Schema drift: Adding fields without versioning breaks replay. Use explicit schema_version.

A Minimal Reference Implementation

Combine the pieces in a FastAPI endpoint that writes to a chained log and redacts:

from fastapi import FastAPI, Request
import json, hashlib, time, re

app = FastAPI()
CHAIN = "chain.log"

def redact(t): return re.sub(r"\d{9,18}", "[ACCT]", t)

@app.post("/v1/chat")
async def chat(req: Request):
    body = await req.json()
    prompt = redact(body["prompt"])
    evt = {
        "schema_version": 1,
        "event_type": "inference.request",
        "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "prompt_redacted": prompt,
        "model": body["model"]
    }
    with open(CHAIN, "a") as f:
        f.write(json.dumps(evt) + "\n")
    # ... call model, log response similarly
    return {"status": "logged"}

This is not production-complete, but it shows the discipline: structured events, redaction, and append-only persistence from line one.

AI audit logging in financial services is a systems problem, not a logging library choice. Get the taxonomy right, seal the records, and you’ll answer any examiner’s “show me exactly what the model said to the customer on March 3” without panic.