Building an audit trail for legal and healthcare LLM answers

Regulated industries demand proof of how a system reached a conclusion. Building an audit trail for legal healthcare llm answers means capturing every prompt, retrieval, model response, and human correction in an immutable record that survives scrutiny from auditors, bar associations, and hospital compliance officers. Skip this and you will not pass a deposition.

Step 1: Define the audit schema

Before writing code, decide what constitutes a complete record. For legal and healthcare use, the minimum fields are: request ID, timestamp, user ID, input prompt, retrieved context (with source document IDs), model name, provider, response text, token usage, and any subsequent human override. In a healthcare setting, the retrieved context may include protected health information (PHI) references; in legal, it may cite privileged documents. Your schema must preserve those references without silently dropping them.

Store these as a strict JSON schema. Avoid free-form logging; auditors need predictable fields.

{
  "request_id": "uuid4",
  "ts": "2025-04-22T10:12:33Z",
  "user_id": "lawyer_42",
  "prompt": "Does clause 4.2 conflict with NY state law?",
  "retrieval": [{"doc_id": "case_1999_123", "chunk": 42}],
  "model": "gpt-4o",
  "provider": "openai",
  "response": "No conflict because...",
  "tokens": {"prompt": 120, "completion": 80},
  "human_override": null
}

Add an event field defaulting to "inference" so you can mix overrides and purges into the same stream later.

Step 2: Wrap your LLM calls with a logging interceptor

Most teams call the model through a thin client. Inject an interceptor that serializes the schema before and after the call. Below is a minimal Python wrapper using the OpenAI SDK. It logs both success and failure paths—a timeout is also an auditable event.

import openai, json, uuid, time, hashlib

def logged_chat(user_id, prompt, retrieval, model="gpt-4o"):
    req_id = str(uuid.uuid4())
    try:
        resp = openai.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}]
        )
        text = resp.choices[0].message.content
        record = {
            "event": "inference",
            "request_id": req_id,
            "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
            "user_id": user_id,
            "prompt": prompt,
            "retrieval": retrieval,
            "model": model,
            "provider": "openai",
            "response": text,
            "tokens": {
                "prompt": resp.usage.prompt_tokens,
                "completion": resp.usage.completion_tokens
            },
            "human_override": None
        }
    except Exception as e:
        record = {
            "event": "inference_error",
            "request_id": req_id,
            "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
            "user_id": user_id,
            "prompt": prompt,
            "retrieval": retrieval,
            "model": model,
            "error": str(e)
        }
    write_audit(record)
    return text, req_id

The write_audit function must append to an append-only store. Do not use a mutable database row that gets updated in place; that breaks non-repudiation.

Step 3: Implement hash-chained storage

An audit trail legal healthcare llm answers system fails if someone can silently edit a record. Hash chaining makes tampering evident. Each record includes the SHA-256 of the previous record’s hash plus its own payload.

import sqlite3, hashlib, json

conn = sqlite3.connect("audit.db")
conn.execute("""CREATE TABLE IF NOT EXISTS trail (
    id INTEGER PRIMARY KEY,
    hash TEXT,
    payload TEXT
)""")

def write_audit(record):
    cur = conn.cursor()
    last = cur.execute("SELECT hash FROM trail ORDER BY id DESC LIMIT 1").fetchone()
    prev_hash = last[0] if last else "genesis"
    serialized = json.dumps(record, sort_keys=True)
    h = hashlib.sha256((prev_hash + serialized).encode()).hexdigest()
    cur.execute("INSERT INTO trail (hash, payload) VALUES (?, ?)", (h, serialized))
    conn.commit()

Now any deletion or modification shifts every subsequent hash. Verification replays the chain. For stronger guarantees, periodically write the latest hash to an external WORM bucket or a blockchain timestamping service—but the local chain is sufficient for most subpoenas.

Step 4: Capture provider routing and fallback events

In production, models degrade. If you route through a gateway that provides automatic fallback when a provider is rate-limited, you must log the actual provider that served the response. Using n4n.ai as an OpenAI-compatible endpoint gives per-token usage metering and forwards provider cache-control hints, which simplifies capturing those events without custom retry code.

When you call the gateway, point the client at its base URL and read the response headers:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.n4n.ai/v1",
    api_key="YOUR_KEY"
)

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    headers={"X-Route-Prefer": "openai,anthropic"}
)
record["provider"] = resp.headers.get("X-Actual-Provider", "unknown")
record["gateway_req_id"] = resp.headers.get("X-Request-Id")

This ensures your audit trail legal healthcare llm answers reflects the real inference path, not just your intended one. If a fallback to a different provider occurred, that fact is permanently recorded.

Step 5: Record human corrections and approvals

A lawyer or clinician reviewing the output is part of the audit story. When a human edits the response, write a new record linked by request_id with human_override set.

def log_human_override(req_id, corrected_text, editor_id):
    record = {
        "event": "human_override",
        "request_id": req_id,
        "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "user_id": editor_id,
        "corrected_response": corrected_text
    }
    write_audit(record)

Treat overrides as append-only events, never as edits to the original. The original model answer stays in the chain untouched; the override is a sibling event. This pattern defends you when someone claims the model “got it wrong and we fixed it silently.”

Step 6: Enforce retention and access controls

Legal and healthcare retention periods differ by jurisdiction; codify them as policy, not as manual deletes. Use a read-only role for auditors and encrypt the database at rest. For purge, mark records expired but keep the hash chain intact by writing a purge event rather than deleting rows.

def mark_purge(req_id, reason):
    record = {
        "event": "purge",
        "request_id": req_id,
        "reason": reason,
        "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
    }
    write_audit(record)

Never grant the application backend the ability to DELETE from the trail table. The audit trail legal healthcare llm answers is only credible if the threat model includes a compromised app server.

Step 7: Build a verification script

Auditors will ask: “Can you prove this log is complete?” Write a CLI that recomputes hashes and checks ordering.

def verify_chain():
    cur = conn.cursor()
    rows = cur.execute("SELECT hash, payload FROM trail ORDER BY id").fetchall()
    prev = "genesis"
    for h, payload in rows:
        computed = hashlib.sha256((prev + payload).encode()).hexdigest()
        if computed != h:
            raise ValueError("Chain broken at " + h)
        prev = h
    return True

if __name__ == "__main__":
    print("CHAIN VALID" if verify_chain() else "CHAIN INVALID")

Run verify_chain() in CI and on a nightly cron. If it returns True, the trail is intact. Pipe the output to your monitoring system so a broken chain pages someone at 3 a.m.

Step 8: Expose a minimal audit API

Give compliance teams a read-only endpoint that returns a record by request_id and the verification status.

from flask import Flask, jsonify, request
from functools import wraps

app = Flask(__name__)

def require_audit_token(f):
    @wraps(f)
    def dec(*a, **k):
        if request.headers.get("X-Audit-Key") != "READ_ONLY_KEY":
            return jsonify({"error": "forbidden"}), 403
        return f(*a, **k)
    return dec

@app.route("/audit/<req_id>")
@require_audit_token
def get_audit(req_id):
    cur = conn.cursor()
    rows = cur.execute(
        "SELECT payload FROM trail WHERE json_extract(payload, '$.request_id') = ?",
        (req_id,)
    ).fetchall()
    return jsonify([json.loads(r[0]) for r in rows])

Do not expose write methods. The audit trail legal healthcare llm answers is worthless if the API can mutate it.

Verify success

Deploy the wrapper, generate ten test queries through your legal/healthcare Q&A flow, then run verify_chain(). Expect CHAIN VALID. Pull one request_id from the database and call GET /audit/<req_id> with the read key; you should see the original prompt, retrieval sources, model response, and any override. Finally, attempt to alter a payload row directly in SQLite and re-run verification—it must raise ValueError. That failure is the feature.

Building this pipeline takes a day, not a quarter. The hard part is discipline: never log to a mutable store, never skip the hash chain, and never let the audit path share write credentials with the app. Do that, and your audit trail legal healthcare llm answers will hold up under subpoena.