Implementing oauth2 scopes llm api access control lets you restrict which models, endpoints, and operations a client can invoke without provisioning a separate API key per use case. This guide walks through a concrete pattern: define a scope taxonomy, mint tokens with those scopes, enforce them at your gateway, and verify the policy end to end.
Step 1: Define a scope taxonomy that mirrors LLM operations
Start by listing the operations your LLM API exposes. Typical surfaces are chat completions, text completions, embeddings, moderation, fine-tune management, and usage metering. Map them to hierarchical scopes so you can grant least privilege.
{
"scopes": {
"llm:chat": "Invoke chat completion endpoints on standard models",
"llm:chat:gpt-4": "Chat with GPT-4 class models only",
"llm:embed": "Create text embeddings",
"llm:moderate": "Run moderation checks",
"llm:admin:keys": "Rotate and read API keys",
"llm:cache": "Hint provider to use prompt caching"
}
}Keep the namespace prefixed with llm: to avoid collisions with other OAuth resources behind the same authorization server. Fine-grained scopes like llm:chat:gpt-4 let you grant an analytics job access to cheap models without exposing flagship weights. Avoid wildcard scopes such as llm:* in production; they defeat the purpose of oauth2 scopes llm api access control.
Document the taxonomy in your developer portal and treat it as an API contract. When a new model family ships, add a new scope rather than silently expanding an existing one.
Step 2: Issue scoped tokens via client credentials
Stand up a token endpoint that authenticates the client and returns a JWT with a scope claim. Below is a minimal Flask example using PyJWT. In production, sign with an RSA key and publish a JWKS endpoint so gateways can rotate keys without a deploy.
import jwt
import time
from flask import Flask, request, jsonify
app = Flask(__name__)
PRIVATE_KEY = open("rsa_private.pem").read()
CLIENT_SECRETS = {"client-123": "supersecret"}
@app.route("/oauth/token", methods=["POST"])
def token():
cid = request.form.get("client_id")
csec = request.form.get("client_secret")
if CLIENT_SECRETS.get(cid) != csec:
return jsonify(error="invalid_client"), 401
requested = request.form.get("scope", "").split()
allowed = {"llm:chat", "llm:chat:gpt-4", "llm:embed", "llm:cache"}
granted = [s for s in requested if s in allowed]
if not granted:
return jsonify(error="invalid_scope"), 400
now = int(time.time())
payload = {
"iss": "https://auth.example.com",
"sub": cid,
"aud": "llm-api",
"scope": " ".join(granted),
"exp": now + 900,
"iat": now,
}
tok = jwt.encode(payload, PRIVATE_KEY, algorithm="RS256")
return jsonify(access_token=tok, token_type="Bearer", expires_in=900)The client requests scope=llm:chat%20llm:chat:gpt-4 and receives a bearer token. The space-delimited string in the scope claim is RFC 6749 compliant. Set exp short (15 minutes) for server-to-server calls; long-lived user tokens should use refresh tokens.
If you already run an OAuth2 provider (Keycloak, Auth0, Cognito), configure these scopes as client scopes and use the standard token endpoint instead of rolling your own.
Step 3: Enforce scopes at the gateway
Your LLM gateway should reject requests whose token lacks the required scope before spending a single inference token. A Flask middleware example:
from functools import wraps
import jwt
from flask import jsonify, request
PUBLIC_KEY = open("rsa_public.pem").read()
def require_scope(needed):
def decorator(f):
@wraps(f)
def inner(*args, **kwargs):
auth = request.headers.get("Authorization", "")
if not auth.startswith("Bearer "):
return jsonify(error="unauthorized"), 401
tok = auth.split()[1]
try:
claims = jwt.decode(tok, PUBLIC_KEY, algorithms=["RS256"], audience="llm-api")
except jwt.ExpiredSignatureError:
return jsonify(error="token_expired"), 401
except jwt.InvalidTokenError:
return jsonify(error="invalid_token"), 401
token_scopes = claims.get("scope", "").split()
if needed not in token_scopes:
return jsonify(error="insufficient_scope"), 403
return f(claims, *args, **kwargs)
return inner
return decoratorApply it to routes:
@app.route("/v1/chat", methods=["POST"])
@require_scope("llm:chat")
def chat(claims):
# forward to provider
...This pattern gives you oauth2 scopes llm api access control at the edge with sub-millisecond overhead. For high-throughput fleets, implement the same check in Envoy ext_authz or a Kong plugin so Python never becomes the bottleneck.
Step 4: Map scopes to model-level restrictions
A coarse llm:chat scope is not enough if you need to block specific model families. Parse the requested model from the body and cross-check against scopes that carry model suffixes.
MODEL_SCOPE = {
"gpt-4": "llm:chat:gpt-4",
"gpt-4o": "llm:chat:gpt-4",
"gpt-3.5-turbo": "llm:chat",
}
@app.route("/v1/chat", methods=["POST"])
@require_scope("llm:chat")
def chat(claims):
body = request.get_json()
model = body.get("model", "gpt-3.5-turbo")
needed = MODEL_SCOPE.get(model)
if needed and needed not in claims["scope"].split():
return jsonify(error="model_forbidden"), 403
# proceed to providerIf a token has only llm:chat but the client sends model: gpt-4, the gateway returns 403. This is the core of fine-grained oauth2 scopes llm api access control. You can extend the map to regex matching for model series (e.g., ^claude-3- → llm:chat:claude-3).
Step 5: Forward only permitted traffic to the provider
When you proxy to an upstream inference service, strip any admin scopes from the downstream call and pass only the model routing hint. If you front models through a gateway such as n4n.ai, it honors client routing directives and forwards provider cache-control hints, but the scope decision must already be made on your side. Never trust the upstream to enforce your tenant isolation.
import requests
def forward_to_provider(body, claims):
headers = {"Authorization": "Bearer <provider-key>"}
if "llm:cache" in claims["scope"].split():
headers["Cache-Control"] = "max-age=3600"
r = requests.post("https://api.provider.com/v1/chat", json=body, headers=headers)
return r.json()Note that provider API keys remain centralised; the OAuth2 layer is about who can call what, not about hiding the upstream credential.
Step 6: Verify enforcement with a red team curl
Issue a token with limited scope and attempt a forbidden call.
# token with only llm:embed
TOKEN=$(curl -s -X POST https://auth.example.com/oauth/token \
-d "client_id=client-123" -d "client_secret=supersecret" \
-d "scope=llm:embed" | jq -r .access_token)
curl -i https://gateway.example.com/v1/chat \
-H "Authorization: Bearer $TOKEN" \
-d '{"model":"gpt-4","messages":[]}'
# Expect HTTP/1.1 403 insufficient_scopeThen mint a token with llm:chat:gpt-4 and retry:
TOKEN2=$(curl -s -X POST https://auth.example.com/oauth/token \
-d "client_id=client-123" -d "client_secret=supersecret" \
-d "scope=llm:chat%20llm:chat:gpt-4" | jq -r .access_token)
curl -i https://gateway.example.com/v1/chat \
-H "Authorization: Bearer $TOKEN2" \
-d '{"model":"gpt-4","messages":[{"role":"user","content":"hi"}]}'
# Expect HTTP/1.1 200 and a completionAlso test expired and malformed tokens to confirm 401s. If all responses match expectations, your oauth2 scopes llm api access control policy is live.
Step 7: Audit, rotate, and narrow scopes
Scopes accumulate cruft. Review granted scopes per client quarterly. Use introspection logs to find tokens that request llm:admin:keys but only ever call llm:chat. Revoke and reissue with narrower scopes. Store scope grants in your IdP so revocation propagates before the JWT exp via a short TTL or revocation list.
For server-to-server workloads, set exp to 15 minutes and use client credentials refresh. For user-delegated access, pair scopes with PKCE and treat llm:admin:* as high privilege. Add a CI test that asserts new routes declare a require_scope decorator; missing enforcement should fail the build.
That is the full loop: define, issue, enforce, map, forward, verify, audit. Implemented correctly, oauth2 scopes llm api access control removes the need for per-model API keys and gives you a single auditable policy surface.