Auditing API key usage for security and cost control

Auditing api key usage security cost is not optional once you have more than one service calling an LLM gateway. A single leaked key can run up unexpected charges or exfiltrate data; without logs you won’t know until the bill arrives. In practice, teams discover abuse only after a provider sends a five-figure invoice or a customer reports odd behavior. This guide walks through building an audit trail from request interception to cost alerting using standard components you can ship this week.

Step 1: Centralize API key validation

Never distribute provider keys to client apps. Put a single reverse proxy or gateway in front of the model endpoint and validate a scoped key there. This gives you one place to intercept traffic, attach an internal key ID, and enforce rate limits. If you skip this step, you will be grepping provider logs with no mapping to your services.

Hash or truncate keys in storage. Store only the first eight characters plus a hash of the rest; never log the full secret. Below is a minimal Express middleware that checks a bearer token against an in-memory map. In production, back this with a database or KMS.

import express from 'express';
import crypto from 'crypto';

const KEY_MAP = new Map<string, { owner: string; scope: string[] }>([
  ['sk-proj-abc', { owner: 'svc-billing', scope: ['chat'] }],
]);

function hashKey(token: string): string {
  return crypto.createHash('sha256').update(token).digest('hex');
}

export function authMiddleware(req: express.Request, res: express.Response, next: express.NextFunction) {
  const auth = req.headers.authorization;
  if (!auth?.startsWith('Bearer ')) return res.status(401).json({ error: 'missing token' });
  const token = auth.slice(7);
  const record = KEY_MAP.get(token);
  if (!record) return res.status(403).json({ error: 'invalid token' });
  (req as any).keyMeta = {
    id: token.slice(0, 8),
    hash: hashKey(token),
    owner: record.owner,
    scope: record.scope,
  };
  next();
}

Verify success

Start the server, call it without a token (expect 401), then with a valid token (expect 200). The keyMeta field should be attached for later logging, and your server should never print the full token.

Step 2: Emit structured request logs

Every request that passes auth must emit a log line with enough detail to reconstruct usage. Capture the key ID, owner, route, model, token counts, latency, and response status. Use JSON logs so they pipe directly into any aggregator without custom parsing.

import json
import time
import logging

logger = logging.getLogger("api_audit")
logger.setLevel(logging.INFO)

def log_request(key_id: str, owner: str, model: str, prompt_tokens: int,
                completion_tokens: int, status: int, latency_ms: float):
    entry = {
        "ts": int(time.time() * 1000),
        "key_id": key_id,
        "owner": owner,
        "model": model,
        "prompt_tokens": prompt_tokens,
        "completion_tokens": completion_tokens,
        "total_tokens": prompt_tokens + completion_tokens,
        "status": status,
        "latency_ms": latency_ms,
    }
    logger.info(json.dumps(entry))

If you proxy to an OpenAI-compatible endpoint, parse the usage field from the response. For streaming responses, capture the final usage chunk emitted at the end of the stream—do not estimate from character counts. That field is part of the standard chat completion response schema.

What to redact

Never include prompt or completion text in audit logs. Token counts are sufficient for cost and security analysis. If you need content tracing, hash the payload separately and store it in a restricted bucket.

Step 3: Store logs in a queryable store

Printing logs is not auditing. Ship them to a database where you can group by key and time. SQLite is enough for most single-tenant setups; Postgres works when you need concurrency or long retention.

CREATE TABLE api_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    ts INTEGER NOT NULL,
    key_id TEXT NOT NULL,
    owner TEXT NOT NULL,
    model TEXT NOT NULL,
    prompt_tokens INTEGER NOT NULL,
    completion_tokens INTEGER NOT NULL,
    total_tokens INTEGER NOT NULL,
    status INTEGER NOT NULL,
    latency_ms REAL NOT NULL
);
CREATE INDEX idx_key_ts ON api_usage(key_id, ts);
CREATE INDEX idx_owner_ts ON api_usage(owner, ts);

Load the JSON logs with a small worker:

import sqlite3, json, sys

conn = sqlite3.connect("audit.db")
cur = conn.cursor()
for line in sys.stdin:
    e = json.loads(line)
    cur.execute(
        "INSERT INTO api_usage (ts, key_id, owner, model, prompt_tokens, completion_tokens, total_tokens, status, latency_ms) "
        "VALUES (:ts, :key_id, :owner, :model, :prompt_tokens, :completion_tokens, :total_tokens, :status, :latency_ms)",
        e,
    )
conn.commit()

Set a retention policy. Drop rows older than 90 days unless compliance requires more. Partition by month if using Postgres.

Verify success

Query SELECT count(*) FROM api_usage; after a test run. You should see rows matching your requests. Run SELECT distinct owner FROM api_usage; to confirm attribution works.

Step 4: Aggregate token usage and cost

Auditing api key usage security cost requires turning tokens into dollars. Maintain a price table keyed by model; pull real numbers from your provider’s pricing page monthly. Never hardcode guessed values—prices change.

-- example price table (USD per 1000 tokens)
CREATE TABLE model_pricing (
    model TEXT PRIMARY KEY,
    prompt_per_1k REAL,
    completion_per_1k REAL
);

SELECT
    u.key_id,
    u.owner,
    u.model,
    SUM(u.prompt_tokens) AS prompt_tokens,
    SUM(u.completion_tokens) AS completion_tokens,
    SUM(u.prompt_tokens * p.prompt_per_1k / 1000.0) AS prompt_cost,
    SUM(u.completion_tokens * p.completion_per_1k / 1000.0) AS completion_cost
FROM api_usage u
JOIN model_pricing p ON u.model = p.model
WHERE u.ts > strftime('%s', 'now', '-1 day') * 1000
GROUP BY u.key_id, u.model;

Run this daily. If a key’s spend exceeds its allocated budget, flag it. The aggregation also exposes which models a key touches—useful for scope tightening. For cost allocation across teams, sum by owner:

import sqlite3

conn = sqlite3.connect("audit.db")
cur = conn.cursor()
rows = cur.execute("""
    SELECT u.owner, SUM(u.total_tokens * (p.prompt_per_1k + p.completion_per_1k) / 1000.0)
    FROM api_usage u JOIN model_pricing p ON u.model = p.model
    WHERE u.ts > strftime('%s','now','-7 day')*1000
    GROUP BY u.owner
""").fetchall()
for owner, cost in rows:
    print(f"{owner}: ${cost:.2f}")

Step 5: Detect anomalies and alert

Static budgets catch overspend after the fact. Add a simple deviation check: compute each key’s daily total tokens over the last 14 days, then alert if today’s volume exceeds mean + 3*stddev. This catches compromised keys that suddenly spike.

import sqlite3, statistics

conn = sqlite3.connect("audit.db")
cur = conn.cursor()
rows = cur.execute(
    "SELECT key_id, date(ts/1000, 'unixepoch') as day, SUM(total_tokens) "
    "FROM api_usage GROUP BY key_id, day"
).fetchall()

series = {}
for key_id, day, tokens in rows:
    series.setdefault(key_id, []).append(tokens)

for key_id, vals in series.items():
    if len(vals) < 7:
        continue  # need baseline
    mean = statistics.mean(vals[:-1])
    stdev = statistics.pstdev(vals[:-1]) or 1
    today = vals[-1]
    if today > mean + 3 * stdev:
        print(f"ALERT: {key_id} used {today} tokens, expected ~{mean:.0f}")

Wire the print to Slack or email. The threshold is tunable; start conservative to avoid noise. Expect false positives during product launches—whitelist those keys temporarily.

Step 6: Rotate and scope keys based on audit findings

The audit data tells you exactly which services need which models. Issue narrowly scoped keys (e.g., svc-extract only allowed llama-3-70b) and rotate every 30–90 days. Revoke any key showing anomalous patterns from Step 5.

If you front requests with n4n.ai, its per-token usage metering and automatic fallback reduce the surface area: one OpenAI-compatible endpoint covering 240+ models, so you audit a single gateway key instead of dozens of provider keys. That simplifies the logging layer but does not remove the need for the anomaly checks above.

Implement rotation by deleting the old key from your KEY_MAP (or database) and publishing the new one via secret manager. Force a redeploy of the consuming service. Automate with a cron job that issues a new key, updates the proxy, and notifies owners.

# pseudo-cron: rotate.sh
#!/usr/bin/env bash
NEW_KEY=$(openssl rand -hex 16)
sqlite3 audit.db "UPDATE keys SET active=0 WHERE owner='svc-extract';"
sqlite3 audit.db "INSERT INTO keys (token, owner, active) VALUES ('$NEW_KEY','svc-extract',1);"
vault kv put secret/svc-extract/api_key value="$NEW_KEY"
kubectl rollout restart deployment/svc-extract

Verify the full pipeline

1. Send a request with a valid key to your proxy. Confirm a log line appears and a row lands in api_usage.
2. Run the Step 4 aggregation for yesterday. Confirm token counts match your test traffic and costs look sane.
3. Inject a synthetic spike (e.g., loop 1000 requests) and confirm Step 5 prints an alert.
4. Rotate the key; confirm old key returns 403 and new key works.
5. Check that no full token appears in logs, database, or error messages.

Auditing api key usage security cost is iterative. Review the aggregates weekly, tighten scopes, and adjust alert thresholds as traffic patterns stabilize. The payoff is a gateway that fails closed, bills predictably, and surfaces abuse before the provider sends a shock invoice.