How to log LLM requests for SOC 2 compliance

Building SOC 2 compliant LLM request logging into your stack is non-negotiable when enterprise customers ask for a clean audit. You need an immutable record of every prompt, completion, and token count tied to a verified user identity, while keeping PII out of the cleartext trail.

Step 1: Define the audit log schema

SOC 2 compliant LLM request logging starts with a fixed contract. Auditors distrust ad-hoc JSON that changes per service. Lock the fields before you write a line of ingestion code.

{
  "event_time": "2024-05-12T18:22:01Z",
  "user_id": "usr_8f2c",
  "org_id": "org_123",
  "session_id": "sess_ab91",
  "model": "gpt-4o",
  "provider": "openai",
  "request_id": "req_77a",
  "prompt_chars": 412,
  "response_chars": 88,
  "prompt_hash": "sha256:ab12...",
  "response_hash": "sha256:cd34...",
  "tokens_in": 120,
  "tokens_out": 30,
  "latency_ms": 820,
  "client_ip": "203.0.113.9",
  "error": null,
  "consent_flag": true
}

Add consent_flag if you process under a DPA that requires explicit opt-in for model training exclusion. Keep field names stable across deploys; schema drift is a finding waiting to happen.

Step 2: Intercept LLM calls in one place

Scatter logging across controllers and you will miss background jobs, retries, and streaming fallbacks. Wrap the client so every call funnels through one audit path.

import hashlib, time, json, logging
from openai import OpenAI

logger = logging.getLogger("llm_audit")

class AuditedLLMClient:
    def __init__(self, api_key, user_id, org_id, session_id, client_ip):
        self.client = OpenAI(api_key=api_key)
        self.meta = dict(user_id=user_id, org_id=org_id,
                         session_id=session_id, client_ip=client_ip)

    def chat(self, model, messages):
        start = time.time()
        resp = self.client.chat.completions.create(model=model, messages=messages)
        text = resp.choices[0].message.content
        entry = {
            **self.meta,
            "event_time": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
            "model": model,
            "request_id": resp.id,
            "prompt_hash": sha256(json.dumps(messages)),
            "response_hash": sha256(text),
            "tokens_in": resp.usage.prompt_tokens,
            "tokens_out": resp.usage.completion_tokens,
            "latency_ms": int((time.time()-start)*1000),
            "error": None,
        }
        logger.info(json.dumps(entry))
        return text

def sha256(s):
    return "sha256:" + hashlib.sha256(s.encode()).hexdigest()[:8]

For streaming responses, buffer the final text and log once after the stream closes. If you route through a gateway such as n4n.ai, which exposes one OpenAI-compatible endpoint for 240+ models and returns per-token usage metering, the same wrapper captures provider fallback events without extra code.

Step 3: Redact before you log

Hashes alone may not satisfy teams that need to debug prod. Redact first, then hash the original for integrity.

import re

EMAIL_RE = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")
SSN_RE = re.compile(r"\b\d{3}-\d{2}-\d{4}\b")

def redact(text):
    return SSN_RE.sub("[SSN]", EMAIL_RE.sub("[EMAIL]", text))

# Inside AuditedLLMClient.chat, after building entry:
redacted_msgs = [{"role": m["role"], "content": redact(m["content"])} for m in messages]
entry["prompt_redacted"] = redacted_msgs

Treat redaction as best-effort. The hash of the original remains the integrity anchor. SOC 2 compliant LLM request logging accepts that you cannot show cleartext PII in the audit store, but you must show you tried to remove it.

Step 4: Ship to an append-only sink

Local stdout is not durable. For SOC 2 compliant LLM request logging, durability and immutability are separate concerns. Use S3 Object Lock in COMPLIANCE mode.

import boto3, uuid

s3 = boto3.client("s3")

def ship_log(entry: dict, bucket: str):
    key = f"llm-logs/{entry['event_time']}-{uuid.uuid4().hex}.json"
    s3.put_object(
        Bucket=bucket,
        Key=key,
        Body=json.dumps(entry),
        ObjectLockMode="COMPLIANCE",
        ObjectLockRetainUntilDate="2030-01-01T00:00:00Z",
    )

Create the bucket with lock enabled from the start:

aws s3api create-bucket --bucket audit-logs --object-lock-enabled

Then set a default retention policy on the bucket. This prevents overwrite and delete for the window, which is exactly what the auditor wants to see.

Step 5: Enforce retention and access control

The app role should have s3:PutObject only. Never grant s3:DeleteObject.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject"],
      "Resource": "arn:aws:s3:::audit-logs/*"
    }
  ]
}

Enable MFA Delete on the bucket. Schedule a quarterly access review: list principals with read access and confirm they are security or compliance staff. Document the review in your control matrix. Retention periods should match your customer contracts—typically three to seven years—and a lifecycle policy can transition objects to Glacier after two years without expiring them early.

Step 6: Verify your logging pipeline end to end

Write a test that proves logs land and cannot be altered.

import pytest
from moto import mock_aws
from myapp.llm import AuditedLLMClient, ship_log

@mock_aws
def test_log_written():
    # mock S3 setup omitted for brevity
    client = AuditedLLMClient("fake", "u1", "o1", "s1", "1.2.3.4")
    # stub chat.completions.create to return a fake response
    client.chat("gpt-4o", [{"role": "user", "content": "hi"}])
    # assert object exists in bucket with expected key prefix

Manual verification note: run aws s3api delete-object --bucket audit-logs --key <key> against a locked object. Expect AccessDenied. That failure is evidence of WORM behavior.

Pull a sample of request IDs weekly. Confirm each maps to a log entry, a user session, and a token count. SOC 2 compliant LLM request logging is a continuous control, not a launch task. If the log sink accepts a delete, your control has regressed.

Independent evidence beats provider dashboards

Do not rely solely on OpenAI or Anthropic console logs. Those are outside your control perimeter and may not capture your internal user IDs or consent flags. Generating your own trail satisfies the “monitoring of unauthorized access” criterion with direct evidence and survives a provider outage.

What you shipped

You now have a single wrapper that emits redacted, hashed, token-metered events; a WORM sink that rejects deletes; and an IAM boundary that passes audit. Assign an owner to the retention policy and document the schema in your SOC 2 wiki. That is the whole job.