Shipping a GDPR compliant LLM API is not about adding a consent banner; it is about demonstrable control over where prompts go, who processes them, and how long they persist. If you operate a gateway or resell model access to EU users, the regulation treats you as a data controller with strict obligations toward every upstream model provider. The following checklist is the ordered path we follow to close compliance gaps before they become audit findings.
1. Inventory data flows and classify roles
You cannot protect data you cannot describe. Start by writing a precise inventory of every byte that enters your request path: raw prompt text, inferred embeddings, cached tokens, response streams, and metadata like IP and user agent.
{
"service": "llm-gateway",
"data_in": ["prompt_text", "user_id", "client_ip"],
"data_out": ["completion_text", "token_counts"],
"subprocessors": ["openai", "anthropic", "self-hosted-mistral"],
"storage": ["redis-ttl-60s", "postgres-audit-30d"]
}Classify yourself as controller or processor. If you make independent decisions about which model to call, you are a controller. If you merely forward on behalf of a customer who dictates the model, you may be a processor—but most gateways that add routing logic, caching, or fallback are controllers. This distinction drives every later step, including whether you need a DPA with your own customers.
Common pitfall: teams log full prompts “for debugging” and forget those logs are personal data. Treat prompt text as regulated by default. Embeddings derived from personal data are also personal data under GDPR’s broad definition; do not ship them to an unvetted vector store.
2. Execute DPAs with every upstream model provider
A GDPR compliant LLM API cannot send EU personal data to a subprocessor without a signed Article 28 contract. That includes hosted model vendors, vector stores, and even your error-tracking SaaS. A privacy policy posted on a vendor’s website is not a DPA.
Maintain a versioned folder of DPAs and alert when a provider updates terms.
# simple checksum tracker for DPA pdfs
for f in dpas/*.pdf; do
sha256sum "$f" >> dpa_inventory.txt
done
git diff --quiet dpa_inventory.txt || echo "DPA change detected"If a provider refuses a DPA (or offers only a clickthrough privacy policy), you must block routing to them for EU tenants. No exception. The tradeoff: this reduces model availability, but compliance beats coverage. Also insist on liability clauses and breach notification timelines in the DPA; a contract that shifts all risk to you is a business risk, not just a compliance one.
3. Enforce data residency and explicit routing
EU data must stay in EU-backed infrastructure unless you have explicit, documented transfer mechanisms (SCCs plus a transfer impact assessment). Implement region pinning at the request level so that a misconfigured client cannot silently route to a US GPU.
function selectEndpoint(region: string, userConsent: boolean) {
if (region === "eu" && !userConsent) {
return "https://eu.model-gateway.example/v1";
}
if (region === "eu" && userConsent) {
// SCC-covered US endpoint with DPA
return "https://us.model-gateway.example/v1";
}
throw new Error("unsupported region");
}Some gateways, including n4n.ai, honor client routing directives and forward provider cache-control hints, which lets you pin traffic to EU nodes while still leveraging provider-side prompt caches. The same class of gateway may provide automatic fallback when a provider is rate-limited or degraded, but you must configure region constraints to prevent fallback from silently shipping data to a non-compliant endpoint under load.
Pitfall: assuming a provider’s “European pricing tier” means EU storage. Read the fine print; many route inference to US GPUs but bill from Dublin. Always verify the physical region of processing, not the billing entity.
4. Scrub and minimize prompts at the edge
The least data you send downstream, the smaller your liability. Deploy a redaction middleware that strips obvious PII before the request leaves your trust boundary.
import re
EMAIL_RE = re.compile(r"[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+")
SSN_RE = re.compile(r"\b\d{3}-\d{2}-\d{4}\b")
def redact(text: str) -> str:
text = EMAIL_RE.sub("[REDACTED_EMAIL]", text)
text = SSN_RE.sub("[REDACTED_SSN]", text)
return text
# in request handler
payload["messages"][-1]["content"] = redact(payload["messages"][-1]["content"])Tradeoff: aggressive regex breaks prompts that legitimately contain emails (e.g., “email the report to x@y.com”). For many enterprise use cases, you should instead run a local NER model and only redact high-confidence spans. Over-redaction degrades output quality; under-redaction creates risk. Measure both with a sampled human review queue.
Another tradeoff: redaction at the edge means you cannot use provider-side prompt caching on the original text. If you forward a redacted version, cache hits drop. Decide based on whether the caching vendor is itself a vetted subprocessor with EU residency.
5. Configure retention and cryptographic deletion
GDPR mandates storage limitation. Set TTLs on any transient cache and define hard deletion for persistent stores.
{
"redis": { "prompt_cache_ttl_seconds": 0 },
"postgres": {
"audit_log_ttl_days": 30,
"erase_on_subject_request": true
}
}When a user invokes erasure, you must propagate deletion to subprocessors where feasible. For model providers that keep training data, you cannot delete what you already sent, so the only compliance path is to never send regulated PII there in the first place (see step 4).
Pitfall: soft deletes that leave rows recoverable. Use DELETE or crypto-shredding with key disposal, not is_deleted=true. Example crypto-shred approach:
# on erasure request
kms.schedule_key_deletion(key_id=user_key_id, pending_days=0)
# all rows encrypted with user_key_id become unreadableThis gives you near-instant logical erasure without scanning terabytes of tables.
6. Build subject access and erasure endpoints
Data subjects have the right to export and erase. Expose authenticated endpoints that return all stored artifacts for a user_id.
app.get("/v1/user/:id/data", async (req, res) => {
const records = await auditStore.findByUser(req.params.id);
res.json({ entries: records, exported_at: new Date().toISOString() });
});
app.delete("/v1/user/:id", async (req, res) => {
await auditStore.hardDelete(req.params.id);
await vectorStore.purge(req.params.id);
res.status(204).end();
});A GDPR compliant LLM API must respond to erasure within 30 days, but technical deletion should be near-instant for your own stores. Subprocessor deletion lags are documented as constraints. Provide a machine-readable status so your enterprise customers can prove they propagated requests downward.
7. Immutable audit logging and breach pipeline
You need evidence of who accessed what. Emit structured logs with request ID, tenant ID, model used, and byte size—never the prompt content.
{
"ts": "2025-04-12T08:22:01Z",
"tenant": "acme",
"model": "mistral-eu",
"prompt_bytes": 412,
"completion_bytes": 88,
"region": "eu"
}Wire a breach notification timer: if an incident exposes personal data, you have 72 hours to notify the supervisory authority. Automate the draft notification so it is ready, not crafted under panic. Include a runbook that identifies which subprocessors must be notified concurrently.
Tradeoff: verbose audit logs can become personal data themselves if they contain IPs. Hash or truncate IPs at collection (e.g., keep only the /24). The log schema above intentionally omits user_id from hot paths; map it separately in an access-controlled store.
8. Validate with automated compliance tests
Compliance is a feature; test it. Write integration tests that assert EU requests never hit non-EU endpoints and that erasure removes rows.
def test_eu_routing(client):
r = client.post("/v1/chat", json={"region": "eu", "messages": [{"role": "user", "content": "hi"}]})
assert r.headers["x-resolved-region"] == "eu"
def test_erasure(client):
client.delete("/v1/user/123")
assert client.get("/v1/user/123/data").json()["entries"] == []Run these in CI alongside unit tests. A GDPR compliant LLM API fails the build if a routing regression slips through. Add a periodic live test that calls a canary EU endpoint and verifies the resolved region header.
Common pitfalls we still see
- Treating SCCs as a blanket permit. They require a transfer impact assessment; document it or the DPA is hollow.
- Assuming open-weight self-hosting removes all risk. If the host VM ships logs to a US parent, you still transfer.
- Ignoring metadata. Token counts alone can reconstruct queries under some attacks; minimize what you keep.
- Using a single global API key across regions. Key scoping should match residency boundaries.
The checklist above is not exhaustive, but it is the minimum bar for an enterprise-ready gateway. Execute it in order; each step unblocks the next, and skipping the DPA step to “move faster” will cost you a contract or a fine later.