A single unrestricted API credential is a liability waiting to be exploited. Implementing api key scoping limit access shrinks the blast radius when a token leaks, letting you restrict which models, routes, or operations a given key can invoke. This guide walks through a concrete scoping design and the code to enforce it on an OpenAI-compatible inference endpoint.
Step 1: Inventory surfaces and define scopes
Before writing code, api key scoping limit access requires a clear map of your attack surface. List every endpoint your gateway exposes and the capabilities behind it. For an LLM gateway the typical surfaces are /v1/chat/completions, /v1/embeddings, /v1/models, and administrative routes like /v1/keys.
Decide what a scoped key should be allowed to do. A search-service key probably only needs chat.completions with a cheap model. An analytics job might need embeddings but never chat. Write this down as a policy document:
{
"key_roles": {
"svc-search": {
"scope": ["chat.completions"],
"models": ["gpt-4o-mini", "mistral-7b"],
"max_tokens_per_min": 50000
},
"svc-embed": {
"scope": ["embeddings"],
"models": ["text-embedding-3-small"]
}
}
}Keep the schema flat. Nested wildcard scopes (chat.*) feel convenient but become hard to audit. Explicit strings are easier to enforce and reason about.
Step 2: Choose a token format
You have two real options: opaque random tokens stored in a database, or signed self-describing tokens (JWTs). Opaque tokens give you instant revocation but force every request to hit a lookup service. JWTs move policy to the edge, at the cost of delayed revocation unless you maintain a short exp or a revocation list.
For most LLM proxies, a short-lived JWT (hours, not days) with a scope claim and a models claim is the right tradeoff. You sign with an HMAC secret shared by your auth layer and your gateway.
# policy.py
import jwt, os, time
SECRET = os.environ["KEY_SIGNING_SECRET"]
def mint_scoped_key(key_id: str, scope: str, models: list[str], ttl=3600):
payload = {
"sub": key_id,
"scope": scope,
"models": models,
"exp": int(time.time()) + ttl,
}
return jwt.encode(payload, SECRET, algorithm="HS256")The minted token embodies api key scoping limit access by carrying an explicit model allowlist and a single scope string. A key with scope: "embeddings" cannot be replayed against chat endpoints.
Step 3: Issue keys per service
Never hand a broad admin key to a microservice. Generate a dedicated scoped key for each caller. In a CI pipeline or provisioning script:
from policy import mint_scoped_key
search_token = mint_scoped_key(
key_id="svc-search-01",
scope="chat.completions",
models=["gpt-4o-mini"],
ttl=86400
)
print(search_token)Store the token in your secret manager (Vault, AWS Secrets Manager). The service reads it at boot. If the token leaks, the sub identifies the owner and the tight scope limits damage.
Step 4: Enforce scopes at the gateway
Enforcement is where api key scoping limit access actually protects you. A FastAPI dependency is the cleanest place to check claims before the handler runs. The example below validates the signature, checks the scope against the route, and verifies the requested model is in the key’s allowlist.
from fastapi import Depends, Request, HTTPException
import jwt
SECRET = os.environ["KEY_SIGNING_SECRET"]
def require_scope(scope: str):
async def checker(request: Request):
auth = request.headers.get("authorization", "")
if not auth.startswith("Bearer "):
raise HTTPException(401, "missing bearer")
token = auth.split(" ", 1)[1]
try:
claims = jwt.decode(token, SECRET, algorithms=["HS256"])
except jwt.InvalidTokenError:
raise HTTPException(401, "bad token")
if claims.get("scope") != scope:
raise HTTPException(403, "scope forbidden")
# for chat scope, enforce model allowlist
if scope == "chat.completions":
body = await request.json()
if body.get("model") not in claims.get("models", []):
raise HTTPException(403, "model forbidden")
request.state.claims = claims
return checker
@app.post("/v1/chat/completions")
async def chat(request: Request, _=Depends(require_scope("chat.completions"))):
# handler assumes scope + model already validated
...Note: request.json() can only be read once. In production, cache the body via request.state or use a middleware that replays it. The point is that the model field in the request body is part of the authorization decision, not just the path.
If you front your models with a gateway that already does per-token usage metering, like n4n.ai, scoped keys become even easier to monitor—you can alert on a key that suddenly burns tokens on a model it was never meant to call.
Step 5: Rotate, revoke, and audit
Scopes drift. A team requests “temporary” access to a larger model and never gives it back. Set a calendar: every 30 days, re-mint keys with the minimal scope and rotate the signing secret if you suspect exposure.
Revocation with JWTs is weak unless exp is short. Mitigate by maintaining a denylist of sub values in your dependency:
DENYLIST = {"svc-search-01"} # loaded from Redis
async def checker(request: Request):
...
if claims["sub"] in DENYLIST:
raise HTTPException(403, "revoked")Regular audits of api key scoping limit access policies prevent scope creep. Pull your provisioning log, diff it against actual usage metrics, and delete unused keys.
Step 6: Verify success with tests
A scoping scheme is useless if you can’t prove it works. Write an integration test that exercises both the allowed and denied paths. Using curl against a local stub:
# export a scoped token minted for gpt-4o-mini only
export SCOPED_TOKEN=$(python -c "from policy import mint_scoped_key; print(mint_scoped_key('test','chat.completions',['gpt-4o-mini']))")
# 1. allowed call
curl -s -o /dev/null -w "%{http_code}\n" \
-H "Authorization: Bearer $SCOPED_TOKEN" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}' \
http://localhost:8000/v1/chat/completions
# expect 200
# 2. forbidden model
curl -s -o /dev/null -w "%{http_code}\n" \
-H "Authorization: Bearer $SCOPED_TOKEN" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o","messages":[{"role":"user","content":"ping"}]}' \
http://localhost:8000/v1/chat/completions
# expect 403
# 3. missing scope (use an embeddings token)
export EMBED_TOKEN=$(python -c "from policy import mint_scoped_key; print(mint_scoped_key('test','embeddings',['text-embedding-3-small']))")
curl -s -o /dev/null -w "%{http_code}\n" \
-H "Authorization: Bearer $EMBED_TOKEN" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}' \
http://localhost:8000/v1/chat/completions
# expect 403In pytest, assert the same status codes against your app fixture. If all three cases pass, your api key scoping limit access implementation is correctly rejecting out-of-scope requests.
How to confirm in production
After deploy, watch your gateway logs for 403 scope forbidden or 403 model forbidden on known keys. A sudden spike in those errors from a single sub means either a misconfigured client or a probing attacker. Per-token metering (if available) should show zero usage for forbidden models. That is the definitive signal that scoping holds.
Step 7: Document the contract for clients
Scoped keys shift some responsibility to the caller. Publish a short note: “Keys are bound to a scope and model list; requesting an unlisted model returns 403.” Include the exact error shape so client code can handle it gracefully instead of retrying blindly.
Treat api key scoping limit access as a living control. New models appear, new endpoints ship, and scopes must follow. Build the inventory step into your CI so the policy doc fails if an endpoint exists without a corresponding scope definition.
That’s the full loop: map, mint, enforce, rotate, verify. Do it for every key, and a leaked credential becomes a contained incident instead of a headline.