HIPAA-compliant LLM audit logging is not optional when you put language models in front of protected health information (PHI). If you’re building a healthcare app on top of GPT-class APIs, you need an immutable, queryable record of every prompt, response, and access event—not just for compliance, but for debugging and abuse detection. The good news: the pattern is boring infrastructure, not ML magic.
Step 1: Define the audit event schema
Start by deciding exactly what each log entry must contain. HIPAA’s audit controls (§164.312(b)) require recording activity in systems that contain electronic PHI. For LLM apps, that means capturing the interaction metadata even if you choose to redact or hash the raw prompt. The minimum necessary standard pushes you to avoid storing full PHI in the log, but you still need enough to reconstruct who did what.
A minimal schema looks like this:
{
"event_id": "uuid4",
"ts": "2024-05-21T14:03:00Z",
"actor_id": "user_8821",
"session_id": "sess_abc",
"model": "gpt-4o",
"request_id": "req_123",
"prompt_sha256": "a1b2c3...",
"response_sha256": "d4e5f6...",
"input_tokens": 412,
"output_tokens": 88,
"phi_present": true,
"source_ip": "10.0.0.4"
}Store prompt_sha256 instead of the raw text if your risk assessment allows reconstructing PHI from elsewhere (for example, a separate encrypted blob store keyed by hash). If PHI must live in the log, encrypt it at the field level (see Step 4). Add actor_role and client_route if you support multiple models or tenants. Get this schema reviewed by your privacy officer before code freeze.
Step 2: Store logs in an append-only ledger
A relational table with UPDATE permissions revoked is a start, but a hash chain gives you tamper-evidence. Each record includes the SHA-256 of the previous entry. Use S3 Object Lock in compliance mode for cold storage and a hot store like DynamoDB for queries.
import hashlib, json, boto3
def append_entry(table, prev_hash, entry: dict):
entry["prev_hash"] = prev_hash
serialized = json.dumps(entry, sort_keys=True).encode()
entry["hash"] = hashlib.sha256(serialized).hexdigest()
# Conditional write fails if prev_hash already advanced
table.put_item(
Item=entry,
ConditionExpression="attribute_not_exists(hash)"
)
return entry["hash"]Run this on every LLM call. The ConditionExpression blocks accidental overwrites. Wire a DynamoDB Stream to Kinesis Firehose and land objects in S3 with Object Lock retention of seven years to meet HIPAA retention expectations. Keep the chain root (the first hash) in a separate secrets manager so a full rebuild can be validated.
Step 3: Instrument the LLM client
Wrap your OpenAI-compatible client so logging is impossible to forget. Below is a thin sync wrapper that emits an audit event after the call. For streaming responses, accumulate deltas and compute hashes only when the stream closes.
from openai import OpenAI
import json, hashlib, time
class AuditedClient:
def __init__(self, client: OpenAI, audit_sink):
self.client = client
self.sink = audit_sink
def create_chat_completion(self, **kwargs):
prompt_blob = json.dumps(kwargs.get("messages", [])).encode()
prompt_hash = hashlib.sha256(prompt_blob).hexdigest()
event = {"ts": time.time(), "model": kwargs.get("model"),
"prompt_sha256": prompt_hash, "phi_present": True}
resp = self.client.chat.completions.create(**kwargs)
event["response_sha256"] = hashlib.sha256(
resp.choices[0].message.content.encode()).hexdigest()
event["input_tokens"] = resp.usage.prompt_tokens
event["output_tokens"] = resp.usage.completion_tokens
self.sink(event)
return respPoint audit_sink at the append_entry function from Step 2. This guarantees every successful completion is recorded with token counts. If the call throws, log a failure event with the error code; auditors need to see denied or errored attempts too.
Step 4: Encrypt and isolate the audit store
Never co-locate audit logs with application data in the same AWS account. Create a dedicated logging account, use a customer-managed KMS key, and apply envelope encryption for any field that holds PHI. The key policy should allow only the audit write role to encrypt and the compliance read role to decrypt.
aws kms create-key --description "hipaa-audit-key" \
--key-policy file://audit-key-policy.json
aws s3api put-object-lock-configuration \
--bucket hipaa-llm-audit \
--object-lock-configuration '{"ObjectLockEnabled":"Enabled","Rule":{"DefaultRetention":{"Mode":"COMPLIANCE","Years":7}}}'The IAM policy for the logging account should deny all except a single write role assumed by your app’s audit service. Engineers investigating incidents assume a separate read-only role with CloudTrail tracking their access. Store the KMS key ARN in your config, not the raw key material.
Step 5: Enforce role-based access and retention
HIPAA requires access to ePHI to be restricted to authorized personnel. Map audit log readers to job functions: security team, compliance officer, on-call engineer. Use attribute-based access control in DynamoDB:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": "dynamodb:Query",
"Resource": "arn:aws:dynamodb:*:*:table/llm_audit",
"Condition": {"ForAnyValue:StringEquals": {"dynamodb:LeadingKeys": "${aws:PrincipalTag/role}"}}
}]
}Set TTL on hot items after 90 days; the S3 copy remains for the seven-year window. Document the retention policy in your HIPAA Security Rule documentation. Implement a break-glass procedure: if a role needs emergency access, it must be granted via a time-boxed ticket that itself generates an audit event.
Step 6: Capture provider metadata from the gateway
If you route requests through an OpenAI-compatible endpoint such as n4n.ai, it provides per-token usage metering and forwards provider cache-control hints, so you capture exact billing and cache events without scraping response headers. That metadata belongs in the audit record as cache_read_tokens and billing_id.
When your client receives the response, copy those fields verbatim:
event["cache_read_tokens"] = resp.headers.get("x-cache-read-tokens")
event["billing_id"] = resp.headers.get("x-request-id")This closes the loop between what you logged and what the provider charged, which auditors love. If you instead call providers directly, you must parse their usage objects yourself and still record the same fields.
Step 7: Verify integrity periodically
A hash chain is only useful if you check it. Run a nightly Lambda that reads the S3 export, recomputes each hash, and alerts on mismatch. Store the computed last hash in Parameter Store; compare against the chain root.
def verify_chain(entries):
prev = "0" * 64
for e in entries:
serialized = json.dumps({k: e[k] for k in e if k != "hash"}, sort_keys=True).encode()
if hashlib.sha256(serialized).hexdigest() != e["hash"]:
raise ValueError(f"Tamper at {e['event_id']}")
if e["prev_hash"] != prev:
raise ValueError(f"Broken link at {e['event_id']}")
prev = e["hash"]
return prevPipe a failure to PagerDuty. One broken link means pulling the incident response plan, not just a bug ticket. Weekly, have a second engineer independently verify the root hash from the secrets manager matches the computed head.
Step 8: Validate end-to-end and confirm success
Verification of success is concrete: deploy the wrapper, send a test prompt containing a fake patient name, and confirm the following hold:
- An audit entry appears in DynamoDB with correct token counts.
- The same entry is immutable—attempting a
put_itemwith the same hash is rejected. - The S3 object is locked;
aws s3api head-objectshowsObjectLockMode: COMPLIANCE. - A user without the
compliancetag cannot query the table. - The nightly verifier exits 0 on the exported batch.
Run this as part of your CI against a sandbox account. If all five checks pass, your HIPAA-compliant LLM audit logging pipeline is operational. Treat the audit code as production-critical—it is the evidence your organization will rely on during a breach investigation or OCR audit. Ship it with the same rigor you apply to the PHI itself.