An LLM API key is a long-lived secret string that identifies and authenticates a client to a model provider’s inference endpoint. It functions like a password for programmatic access, but with finer-grained controls: rate limits, model allowlists, spending caps, and audit trails. Every request to a provider such as OpenAI, Anthropic, or Google includes this key in an Authorization header, and the provider validates it before routing the request to a GPU cluster.
How an LLM API key works
At the protocol level, an LLM API key is a bearer token. The client sends it in the Authorization: Bearer <key> header on every HTTPS request. The provider’s edge layer extracts the key, looks up the associated account and project, checks quotas and permissions, then either forwards the request to the inference fleet or returns a 401/429 response.
POST /v1/chat/completions HTTP/1.1
Host: api.openai.com
Authorization: Bearer sk-proj-abcdef1234567890
Content-Type: application/json
{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}Most providers issue keys with a recognizable prefix — sk- for OpenAI, sk-ant- for Anthropic, AIza for Google — which helps with secret scanning and log redaction. The key itself is typically a base64- or base32-encoded blob containing an identifier, a signature, and sometimes embedded metadata (project ID, environment tag, expiration timestamp).
Key lifecycle
- Creation — Generated in the provider dashboard or via an admin API. You assign a name, optional expiration, and optional scopes (e.g., “chat only,” “embeddings only,” “admin”).
- Distribution — Copied once into a secret manager (Vault, 1Password, AWS Secrets Manager, GitHub Actions secrets). Never commit it to source control.
- Rotation — Replace the key on a schedule (30–90 days) or after a suspected leak. Providers support multiple active keys per project to enable zero-downtime rotation.
- Revocation — Immediate invalidation via dashboard or API. Use this when an employee leaves or a CI log accidentally prints the key.
# Example: rotating an OpenAI key via CLI (pseudo-code)
openai api keys create --name "prod-web-2025-01" --expires 2025-04-01
# deploy new key to secret manager
openai api keys delete <old-key-id>Why the key design matters for engineers
Rate limits and quotas are tied to the key
Providers enforce tiered limits per key: requests per minute (RPM), tokens per minute (TPM), and concurrent requests. A single key shared across ten microservices creates a noisy-neighbor problem — one service’s burst starves the others. The fix is separate keys per service or per environment, each with its own quota bucket.
# Example: per-service keys in a Kubernetes secret manifest
apiVersion: v1
kind: Secret
metadata:
name: llm-keys
stringData:
chat-service: "sk-proj-chat-xxxxx"
embedding-service: "sk-proj-embed-xxxxx"
batch-worker: "sk-proj-batch-xxxxx"Spend control requires key-level visibility
Provider dashboards show usage aggregated by key. If you use one key for everything, you cannot answer “which feature drove the $3,000 spike last Tuesday?” without adding your own instrumentation. Keys tagged by service, environment, or feature give you that breakdown for free.
Audit trails depend on key identity
When a provider investigates abuse (spam generation, extraction attacks, ToS violations), they trace activity to the key. A compromised key used for malicious traffic can get your entire account suspended. Scoped keys limit blast radius: a key that only calls text-embedding-3-small cannot be used to generate disallowed content via gpt-4o.
Concrete example: wiring a key through a gateway
Most production teams do not call provider endpoints directly from application code. They route through an internal gateway or proxy that handles retries, fallbacks, observability, and key injection. This keeps keys out of application logs and lets you swap providers without code changes.
# gateway.py — minimal example of key injection and provider routing
import os
import httpx
from fastapi import FastAPI, HTTPException, Header
from pydantic import BaseModel
app = FastAPI()
PROVIDER_KEYS = {
"openai": os.environ["OPENAI_API_KEY"],
"anthropic": os.environ["ANTHROPIC_API_KEY"],
}
class ChatRequest(BaseModel):
model: str
messages: list[dict]
provider: str | None = None # optional client directive
@app.post("/v1/chat/completions")
async def chat_completions(
req: ChatRequest,
authorization: str = Header(...),
):
# Validate caller's internal token (not the provider key)
if not validate_internal_token(authorization):
raise HTTPException(401, "Invalid internal token")
provider = req.provider or select_provider(req.model)
provider_key = PROVIDER_KEYS[provider]
url = PROVIDER_ENDPOINTS[provider]
async with httpx.AsyncClient(timeout=60.0) as client:
resp = await client.post(
url,
json=req.model_dump(exclude={"provider"}),
headers={"Authorization": f"Bearer {provider_key}"},
)
if resp.status_code >= 400:
# Log provider error without exposing provider key
log_provider_error(provider, resp.status_code, resp.text)
raise HTTPException(resp.status_code, "Upstream error")
return resp.json()In this pattern, the application presents an internal service-to-service token (short-lived, rotating, scoped to the gateway). The gateway holds the provider keys in memory, injected at deploy time from a secret manager. The provider never sees the internal token; the application never sees the provider key.
If you operate a gateway like this at scale, you eventually want automatic fallback when a provider degrades, per-token metering for cost allocation, and the ability to honor client routing directives (e.g., “use Anthropic for this request”). n4n.ai implements these patterns in a single OpenAI-compatible endpoint that addresses 240+ models.
Common misconceptions
“The API key is the model weights”
The key grants access to the model, not the model itself. You cannot download weights, inspect architecture, or run inference locally with an API key. Providers that offer self-hosted options (e.g., Cohere, Mistral, Meta) distribute weights under separate licenses and delivery mechanisms — not via the API key.
“One key per organization is fine”
Sharing a single key across dev, staging, and production is the most common cause of preventable incidents. A staging load test exhausts the production quota. A developer’s script leaks the key in a public gist. The fix is trivial: generate separate keys per environment and per service, store them in a secret manager, and rotate on a calendar.
“Prefixes like sk- are just branding”
The prefix is a security feature. Secret scanners (GitHub, GitLab, TruffleHog, ggshield) ship with regexes for known prefixes. A key without a standard prefix bypasses automated detection. If you build a wrapper service that issues its own keys, give them a unique prefix (e.g., n4n-sk-) and register it with your secret scanning tools.
“Revoking a key is instant everywhere”
Revocation is eventually consistent. The provider’s edge caches may serve requests for a few seconds to a minute after revocation, depending on TTL configuration. Design your rotation to overlap old and new keys rather than assuming hard cutover.
“API keys are the only auth option”
Some providers support OAuth 2.0 client credentials flow for service-to-service calls, returning short-lived access tokens (5–60 minutes) instead of long-lived keys. This reduces blast radius of a leak but adds token refresh complexity. For most teams, long-lived keys with proper rotation and scoping are simpler and equally secure.
Checklist for production use
- Separate keys per service and environment (dev/staging/prod)
- Keys stored in a secret manager, never in code or config files
- Rotation schedule (30–90 days) with overlapping validity
- Scopes restricted to required models and endpoints
- Expiration dates set on every key
- Secret scanning enabled in CI/CD and pre-commit hooks
- Alerting on 401/429 rates per key
- Dashboard or queryable logs showing spend and latency per key
- Runbook for emergency revocation and re-issue
Closing thought
An LLM API key is a capability token, not just a password. Treat it like a production database credential: scoped, rotated, audited, and never shared across trust boundaries. The few minutes you spend setting up per-service keys and a rotation calendar will save you hours of incident response when — not if — something goes wrong.