Designing audit logs for healthcare LLM applications

Building audit logs for healthcare LLM apps requires more than a stdout print. You need tamper-evident records that map each inference to a user, a de-identified patient reference, and the exact model version used, while keeping PHI out of the log store.

Prerequisites

• Python 3.11 or newer
• PostgreSQL 15+ with a writable schema
• An OpenAI-compatible LLM endpoint. We’ll point the SDK at a gateway that honors client routing directives and forwards provider cache-control hints; n4n.ai fits because it meters per-token usage and provides automatic fallback when a provider is rate-limited.
• psycopg3 and openai Python packages (pip install psycopg openai)

You should already have a way to authenticate clinicians and resolve a patient identifier (MRN) in your app.

Threat model and logging scope

Healthcare logs fail audits for two reasons: they contain unprotected PHI, or they can’t prove which model answered a question. Audit logs for healthcare LLM apps must record what was asked as a hash, who asked, for which patient (hashed), and which model responded. Store the raw prompt in a separate encrypted blob store if you need replay, never in the same row as identifiers.

Schema design

Create a narrow table. Keep hashes as char(64), tokens as integers, and routing as jsonb.

CREATE TABLE llm_audit_log (
    id BIGSERIAL PRIMARY KEY,
    ts TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    actor_id TEXT NOT NULL,
    patient_ref CHAR(64) NOT NULL,
    model_id TEXT NOT NULL,
    provider TEXT NOT NULL,
    prompt_hash CHAR(64) NOT NULL,
    completion_hash CHAR(64) NOT NULL,
    prompt_tokens INT NOT NULL,
    completion_tokens INT NOT NULL,
    cache_hit BOOLEAN NOT NULL DEFAULT FALSE,
    routing_directive JSONB,
    consent_flag BOOLEAN NOT NULL,
    error_state TEXT
);

CREATE INDEX idx_audit_ts_actor ON llm_audit_log (ts, actor_id);

patient_ref is sha256(mrn)—never the MRN itself. prompt_hash lets you detect duplicate requests without exposing content.

Step 1: Hashing helpers

import hashlib

def sha256(s: str) -> str:
    return hashlib.sha256(s.encode("utf-8")).hexdigest()

Step 2: Wrap the LLM call with logging

We instantiate the OpenAI client against the gateway endpoint. The extra_headers field carries routing directives. The gateway returns provider cache status; we read a header to populate cache_hit.

from openai import OpenAI
import psycopg
import json

client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="YOUR_KEY")

def logged_completion(
    actor_id: str,
    patient_mrn: str,
    model: str,
    prompt: str,
    consent: bool,
    routing: dict | None = None
) -> str:
    patient_ref = sha256(patient_mrn)
    prompt_h = sha256(prompt)
    routing = routing or {}

    try:
        resp = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            extra_headers=routing
        )
        cache_hit = resp.headers.get("x-cache", "MISS").upper() == "HIT"
        usage = resp.usage
        completion_text = resp.choices[0].message.content
        _insert(actor_id, patient_ref, model, resp.model, prompt_h,
                sha256(completion_text), usage.prompt_tokens,
                usage.completion_tokens, cache_hit, routing, consent, None)
        return completion_text
    except Exception as e:
        _insert(actor_id, patient_ref, model, model, prompt_h, sha256(""),
                0, 0, False, routing, consent, str(e)[:500])
        raise

Step 3: Persist the record

Use a connection per call for clarity; in production use a pool.

def _insert(actor_id, patient_ref, model_id, provider, prompt_hash,
            completion_hash, pt, ct, cache_hit, routing, consent, error):
    with psycopg.connect("postgres://app:secret@localhost/audit") as conn:
        conn.execute(
            """INSERT INTO llm_audit_log
               (actor_id, patient_ref, model_id, provider, prompt_hash,
                completion_hash, prompt_tokens, completion_tokens, cache_hit,
                routing_directive, consent_flag, error_state)
               VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
            (actor_id, patient_ref, model_id, provider, prompt_hash,
             completion_hash, pt, ct, cache_hit, json.dumps(routing),
             consent, error)
        )

Step 4: Run and verify

A minimal driver:

if __name__ == "__main__":
    out = logged_completion(
        actor_id="doc_882",
        patient_mrn="MRN-123456",
        model="openai/gpt-4o",
        prompt="Summarize current meds for patient with CKD stage 3.",
        consent=True,
        routing={"x-n4n-route": "openai"}
    )
    print("completion length:", len(out))

Run:

python app.py

Expected stdout:

completion length: 142

Now inspect the database:

SELECT ts, actor_id, model_id, prompt_tokens, completion_tokens, cache_hit
FROM llm_audit_log
ORDER BY ts DESC LIMIT 1;

Sample row:

2024-05-12 14:33:01.442+00 | doc_882 | openai/gpt-4o | 132 | 48 | t

The t confirms a cache hit was logged from the gateway header.

Step 5: Querying audit logs for healthcare LLM apps

Monthly per-clinician volume:

SELECT actor_id,
       COUNT(*) AS calls,
       SUM(prompt_tokens + completion_tokens) AS total_tokens
FROM llm_audit_log
WHERE ts > NOW() - INTERVAL '30 days'
GROUP BY actor_id
ORDER BY total_tokens DESC;

To prove consent was captured for every call tied to a patient:

SELECT patient_ref, COUNT(*) FILTER (WHERE NOT consent_flag) AS missing_consent
FROM llm_audit_log
GROUP BY patient_ref
HAVING COUNT(*) FILTER (WHERE NOT consent_flag) > 0;

Empty result means every logged interaction had consent recorded.

Reliability and fallback

A gateway such as n4n.ai automatically falls back when a provider is degraded, and its per-token metering means the usage numbers in your audit logs for healthcare LLM apps match what you’re billed. That removes a class of reconciliation bugs where your log shows 0 tokens but the invoice shows thousands.

If you self-host models, set provider to your inference server name and still log tokens from the local usage response.

Retention and immutability

Write-once is non-negotiable. Revoke UPDATE/DELETE on llm_audit_log from the app role:

REVOKE UPDATE, DELETE ON llm_audit_log FROM app;

Ship the WAL to an append-only bucket. For 7-year retention typical in healthcare, partition by month:

CREATE TABLE llm_audit_log_2024_05 PARTITION OF llm_audit_log
FOR VALUES FROM ('2024-05-01') TO ('2024-06-01');

What not to log

Never log patient_mrn raw, free-text clinical notes, or full prompt/completion bodies in this table. If you need content for debugging, write it to an encrypted store keyed by prompt_hash and gate access behind a separate break-glass role.

Wrap-up checklist

• Hashes for patient and prompt/completion
• Token counts from provider response
• Cache hit flag from gateway header
• Consent boolean per row
• DB role lacks DELETE/UPDATE
• Monthly audit query returns expected aggregates

That’s the core of a defensible design for audit logs for healthcare LLM apps. Build the wrapper once, enforce it at the gateway layer, and your compliance team stops filing exceptions.