How often should you rotate LLM API keys?

How often rotate LLM API keys is a question that gets answered with a numbingly generic “every 90 days” in most compliance docs. That advice ignores how LLM keys are actually used: they often carry full account billing rights, get pasted into notebooks, and flow through proxy layers. This guide gives an actionable path to set a rotation policy based on exposure, not superstition.

Why Rotation Matters for LLM Keys

LLM API keys are not like a read-only database credential. A leaked OpenAI, Anthropic, or Google key is a direct line to your billing account and your users’ prompt data. Most providers offer no per-key scoping—one key equals all models, all spend. Rotation is your only circuit breaker after a leak.

The second problem is blast radius. If the same key is used in production, CI, and a developer’s laptop, revoking it to stop a leak takes down everything. You need separation and a rotation rhythm that matches each environment.

There is also a compliance dimension. SOC 2 and ISO 27001 expect evidence of periodic credential rotation. But auditors care that you have a reasoned policy, not that you pick 90 days arbitrarily. Document your tiers and the automation that enforces them.

Threat Model: Where Keys Leak

Before deciding cadence, map how keys escape:

• Source control: A key committed to a repo, even briefly, gets scraped by bots within minutes. GitHub secret scanners help, but they are reactive.
• Logs: Verbose HTTP clients log Authorization headers. LangChain and raw curl -v are common offenders. A single misconfigured logging middleware can broadcast the key to your log aggregator.
• Client-side apps: Shipping a key in a mobile or browser bundle is equivalent to publishing it. Reverse engineering extracts it in seconds.
• Third-party libraries: A malicious or compromised npm package can exfiltrate env vars. The supply chain is now the primary attack vector for many teams.
• Shared notebooks: Colab and Jupyter cells get shared with colleagues or accidentally public. The key persists in the notebook’s execution history.

If your key touches any of the last three, assume it is already burned. That reframes how often rotate LLM API keys should be answered: continuously for high-risk surfaces, periodically for isolated server secrets.

A Practical Rotation Cadence

Forget the universal clock. Use three tiers:

Tier 1: Isolated server-side, secrets manager-backed

Keys live only in AWS Secrets Manager or Vault, injected at runtime, never logged. Rotate every 90 days. This is your baseline. The exposure window is small because the key never leaves the trust boundary.

Tier 2: Shared dev, CI, or multi-tenant scripts

Keys used in CI or by multiple engineers should rotate every 30 days. Better, issue per-engineer keys if the provider allows (OpenAI orgs support this). A key that builds a Docker image and a key that runs prod are different risk classes.

Tier 3: Any client-exposed or notebook usage

Treat as ephemeral. Generate short-lived keys per session if the provider supports OAuth, or rotate manually every 24–48 hours. Realistically, never put a permanent key there; use a proxy that mints short tokens. If you must, automate revocation after each session.

Immediate triggers

Rotate the moment you see a suspicious charge, a key in a public repo, or a dependency alert. Do not wait for the schedule.

Step 1: Inventory and Classify

Run a scan for key patterns in your repos and env files:

grep -rE "sk-[a-zA-Z0-9]{20,}" . --include="*.py" --include="*.env" --include="*.js"

For broader coverage, use a tool like trufflehog to scan git history:

trufflehog git https://github.com/yourorg/yourrepo --only-verified

List every key, its provider, where it’s used, and its tier. If you cannot map a key to a service, revoke it. Spreadsheet is fine; a CMDB entry is better.

Step 2: Store in a Secrets Manager

Never inline keys in code. Use a manager that supports versioning and automatic rotation. Example with AWS Secrets Manager in Python:

import boto3
import os

client = boto3.client("secretsmanager")
secret = client.get_secret_value(SecretId="prod/llm/openai")
os.environ["OPENAI_API_KEY"] = secret["SecretString"]

For HashiCorp Vault, the pattern is similar but uses AppRole or Kubernetes auth:

import hvac
client = hvac.Client(url="https://vault.example.com")
client.auth_kubernetes(role="llm-app", jwt=open("/var/run/secrets/token").read())
secret = client.secrets.kv.v2.read_secret_version(path="llm/openai")
os.environ["OPENAI_API_KEY"] = secret["data"]["data"]["key"]

For local dev, use .env with python-dotenv, but never commit the file. Add .env to .gitignore and use pre-commit hooks to block accidental adds.

Step 3: Automate Rotation with Dual-Key Cutover

Calendar rotation fails when the new key isn’t deployed before the old is revoked. Use a dual-key window: create the new key, deploy it to all services, verify traffic, then revoke the old.

For OpenAI orgs, the admin key can manage keys programmatically:

# Create new key
NEW_KEY=$(curl -s https://api.openai.com/v1/organization/keys \
  -H "Authorization: Bearer $OPENAI_ADMIN_KEY" \
  -d '{"name":"rotated-$(date +%s)"}' | jq -r '.key')

# Push to secrets manager (pseudo)
aws secretsmanager put-secret-value --secret-id prod/llm/openai --secret-string "$NEW_KEY"

# After deploy verified, revoke old key by id
curl -X DELETE https://api.openai.com/v1/organization/keys/$OLD_KEY_ID \
  -H "Authorization: Bearer $OPENAI_ADMIN_KEY"

A robust Python rotation job:

import requests, boto3, time, os

ADMIN = os.environ["OPENAI_ADMIN_KEY"]
OLD_ID = os.environ["OLD_KEY_ID"]

def create_key():
    r = requests.post("https://api.openai.com/v1/organization/keys",
                      headers={"Authorization": f"Bearer {ADMIN}"},
                      json={"name": f"rotated-{int(time.time())}"})
    return r.json()["key"]

def revoke_key(key_id):
    requests.delete(f"https://api.openai.com/v1/organization/keys/{key_id}",
                    headers={"Authorization": f"Bearer {ADMIN}"})

new_key = create_key()
boto3.client("secretsmanager").put_secret_value(
    SecretId="prod/llm/openai", SecretString=new_key)
# TODO: trigger deploy, wait for health check
# if healthy: revoke_key(OLD_ID)

Write a cron or GitHub Action that runs this quarterly for Tier 1, monthly for Tier 2. The script must fail closed: if new key deployment isn’t confirmed, abort revocation.

Step 4: Monitor Usage and Set Alerts

Rotation without monitoring is blind. Most providers expose usage endpoints. Set a daily spend alert and anomaly detection:

# Simple threshold check against OpenAI usage API
import requests
resp = requests.get("https://api.openai.com/v1/organization/usage",
                    headers={"Authorization": f"Bearer {ADMIN_KEY}"})
usage = resp.json()
if usage["total_usage"] > 100_00:  # $100 in cents
    alert_slack("LLM spend spike")

If you see tokens used from a region you don’t operate in, rotate immediately. For providers without usage APIs, parse billing CSVs daily.

Step 5: Incident Rotation Runbook

When a leak is suspected:

1. Issue a new key via admin API.
2. Deploy to all live services via secrets manager.
3. Confirm health checks pass with new key.
4. Revoke old key.
5. Pull logs to estimate exposure window and cost.
6. Purge key from any repos and force push history if needed.

Practice this runbook quarterly so it isn’t theory. A tabletop exercise where you intentionally revoke a canary key surfaces gaps fast.

Common Pitfalls and Tradeoffs

Pitfall: Rotating too often without automation. If you manually rotate monthly, you’ll eventually slip and leave a service on a dead key, causing an outage. Automate or pick a longer interval.

Pitfall: One key for everything. A single key means rotation is all-or-nothing. Issue separate keys per service and per environment.

Pitfall: Logging the key during cutover. Your rotation script must not print the secret. Use jq -r to variables, not echo.

Pitfall: Forgetting downstream caches. If you use a local model cache or a CDN for prompt templates, old auth may linger. Flush on rotation.

Tradeoff: Short-lived keys vs provider support. Some LLM providers lack per-key expiry. You must simulate expiry by revoking and recreating, which requires admin privileges separated from the runtime key.

Tradeoff: Gateway caching. If you front models with a proxy that caches provider responses, rotating the provider key behind the proxy may require cache flush to avoid serving stale auth. Plan for that.

Tradeoff: Rate limits on key creation. Providers may cap how many keys you can create per org. Design your tiering to stay under the limit.

Using a Gateway to Reduce Rotation Burden

If you route through a gateway such as n4n.ai—a single OpenAI-compatible endpoint covering 240+ models with automatic fallback on provider degradation—you only need to rotate the edge key; upstream provider credentials stay sealed in your vault. This collapses the rotation surface from a dozen provider keys to one, and the gateway can honor your routing directives without exposing backend secrets to application code.

Checklist

• Inventory all LLM keys and assign tiers.
• Move every key to a secrets manager.
• Set rotation: 90d Tier 1, 30d Tier 2, ephemeral Tier 3.
• Build dual-key cutover script with fail-closed logic.
• Alert on spend and geographic anomalies.
• Run incident runbook drill.

How often rotate LLM API keys is ultimately a function of where they live and how fast you can cut over. Build the machinery once, then let the calendar or the alarm trigger it.