Securing multi-tenant LLM APIs with OAuth2

Securing multi-tenant LLM APIs with OAuth2 is the only sane way to expose shared model infrastructure to multiple customers without building your own credential protocol. A bearer token issued by a standards-compliant authorization server gives you tenant isolation, scoped access, and revocable credentials with minimal custom code.

1. Define tenant identity and scopes upfront

Before writing any auth code, decide what a tenant means in your system. Typically it is an organization ID, but you may need project-level or sub-tenant isolation inside a larger account. Draw the boundary on paper first; retrofitting isolation into token claims later breaks every issued token.

Map OAuth2 scopes to LLM capabilities, not generic CRUD verbs. A scope like llm:chat:acme limits chat completions to tenant acme. Avoid a shared llm:read scope that crosses tenant boundaries—it will eventually be used by the wrong client.

{
  "sub": "client_123",
  "tenant_id": "acme",
  "scope": "llm:chat:acme llm:embed:acme",
  "aud": "https://api.example.com",
  "exp": 1700000000,
  "jti": "a1b2c3"
}

Put the tenant ID as an explicit custom claim. Relying solely on string-parsed scopes is fragile; a dedicated claim makes authorization logic a dictionary lookup instead of a regex match.

2. Pick the correct OAuth2 grant

For backend services calling your LLM API, use the client credentials grant. It exchanges a client ID and secret for a token with no user in the loop.

curl -X POST https://auth.example.com/oauth/token \
  -d grant_type=client_credentials \
  -d client_id=$CLIENT_ID \
  -d client_secret=$CLIENT_SECRET \
  -d scope="llm:chat:acme"

For user-facing applications, use the authorization code grant with PKCE. Never use the implicit grant; it places tokens in URLs and browser history where they leak. Device flow applies only to CLI tools without a browser.

Tradeoff: client-credentials tokens are often long-lived if you set long expiries. Prefer 5–15 minute TTLs and refetch on a schedule. The refresh overhead is one POST per few minutes per worker, which is cheap compared to a leaked long-lived token.

3. Mint tenant-bound tokens

Your authorization server must embed the tenant claim at issuance. If you roll your own with PyJWT, keep the signing secret off API nodes and use asymmetric keys.

import jwt, time

def issue_token(client_id: str, tenant_id: str, scopes: list[str], private_key: str) -> str:
    payload = {
        "sub": client_id,
        "tenant_id": tenant_id,
        "scope": " ".join(scopes),
        "aud": "https://api.example.com",
        "iat": int(time.time()),
        "exp": int(time.time()) + 900,
        "jti": generate_jti(),
    }
    return jwt.encode(payload, private_key, algorithm="RS256")

Publish the public key via a JWKS endpoint and rotate keys by adding a new kid. API nodes cache JWKS for an hour. This keeps securing multi-tenant LLM APIs with OAuth2 independent of shared secrets.

4. Verify at the edge, not in every service

Terminate OAuth2 validation in a gateway or a shared dependency. Local JWT verification beats introspection for p99 latency, but you lose instant revocation.

from fastapi import Depends, HTTPException, Request
import jwt

def require_tenant(request: Request):
    auth = request.headers.get("Authorization", "")
    if not auth.startswith("Bearer "):
        raise HTTPException(401, "Missing bearer")
    token = auth.split()[1]
    try:
        claims = jwt.decode(
            token,
            PUBLIC_KEY,
            algorithms=["RS256"],
            audience="https://api.example.com",
            options={"require": ["exp", "tenant_id", "aud"]},
        )
    except jwt.PyJWTError:
        raise HTTPException(401, "Invalid token")
    return claims

If you need revocation, call the authorization server’s introspection endpoint, but cache the response keyed by jti with TTL equal to token exp. That avoids a network hop per request while still honoring revocation.

Pitfall: omitting audience validation lets a token minted for another service be replayed against your LLM API. Always pin aud.

5. Map tokens to tenant quotas and routing

Once the tenant claim is trusted, enforce per-tenant rate limits and model routing. Extract the tenant, look up its plan, and set concurrency caps before calling any model.

@app.post("/v1/chat/completions")
def chat(body: dict, claims: dict = Depends(require_tenant)):
    tenant_id = claims["tenant_id"]
    if not quota_enforcer.allow(tenant_id):
        raise HTTPException(429, "Quota exceeded")
    routed_model = router.pick_model(tenant_id, body.get("model"))
    return upstream.call(routed_model, body)

When you front models with an inference gateway, pass the tenant’s routing preference via a header derived from the claim. For example, n4n.ai honors client routing directives and forwards provider cache-control hints, so you can signal X-Route-To: provider-x without exposing provider keys to the tenant. The OAuth2 token proves who is calling; the gateway handles automatic fallback when a provider is rate-limited or degraded.

This separation is the core of securing multi-tenant LLM APIs with OAuth2: identity and scope live in the token, while model selection and failover live in the data plane.

6. Rotate and revoke without downtime

Short token lifetimes (under 15 minutes) shrink the window for stolen tokens. For long-running batch jobs, issue a new client-credentials token on a cron rather than extending expiry.

Maintain a revocation list for incident response. Store jti hashes in Redis with a TTL matching token exp.

def is_revoked(jti: str) -> bool:
    return redis.exists(f"revoked:{jti}") == 1

Tradeoff: revocation checks add a dependency. Use them only for high-risk tenants or post-breach, not as a default path. Key rotation should be continuous; never share a signing key across environments.

7. Common pitfalls we keep seeing

Logging full tokens. Never log Authorization headers. Log jti or sub for traceability.

Wildcard scopes. llm:* seems convenient but destroys tenant isolation. Scope per tenant, per capability.

Trusting proxy headers. If you read X-Tenant from a header, an attacker spoofs it. Only trust claims from the verified token.

Skipping alg check. Pin algorithms=["RS256"]. Accepting alg:none is the classic JWT bypass.

Confusing user and tenant. A user may belong to multiple tenants. Bind the token to the tenant they act as, not the user ID alone.

No clock skew tolerance. Allow 30 seconds of leeway on exp to avoid spurious 401s across nodes.

8. Observe and audit

Emit structured logs with tenant_id, sub, scope, and latency. Meter per-token usage if you bill by consumption.

{"ts": "2024-01-01T00:00:00Z", "tenant_id": "acme", "sub": "client_123", "model": "gpt-4o", "tokens": 1280}

If you use a gateway with per-token usage metering, correlate its records with your OAuth2 sub to reconcile bills. Alert on unusual scope usage—a tenant suddenly calling llm:embed at 100x baseline is a leaked token or a misconfigured client.

Securing multi-tenant LLM APIs with OAuth2 is fundamentally about strict claim validation and scoped issuance. Get the token design right and the rest is plumbing.

9. Migration checklist from API keys

If you currently hand out static API keys, migrate in phases:

1. Issue OAuth2 client credentials alongside existing keys.
2. Mirror key quotas into token scopes.
3. Flip the gateway to require Bearer for new tenants only.
4. Measure error rates; extend to all tenants.
5. Expire old keys after 30 days.

This avoids a big-bang break for existing integrations while closing the multi-tenant gap.

10. Quick reference

• Tenant ID in token claim, not just scope.
• RS256 with JWKS rotation.
• Audience locked to your API.
• Gateway validates locally, introspects on revocation.
• Per-tenant quota from claim.
• No token logging.
• Short TTL + scheduled refresh.

Follow this path and your multi-tenant LLM surface stays closed to cross-tenant leakage while remaining standard enough that any OAuth2 client library can talk to it.