HIPAA-ready LLM APIs for healthcare enterprise teams

Building a HIPAA compliant LLM API for healthcare enterprise teams means more than ticking a box on encryption. You need a signed Business Associate Agreement (BAA) with the model provider, verifiable audit trails, and architectural controls that prevent PHI from leaking into training pipelines. The good news is that several managed offerings now meet these bars, but their operational trade-offs differ sharply.

1. Azure OpenAI Service

Microsoft was first to offer a generally available LLM service with a standard BAA and HIPAA eligibility for designated regions. You provision a resource in a supported geography, accept the BAA through the Azure portal, and get an endpoint that speaks the OpenAI SDK protocol. The models run on dedicated capacity slices that are logically isolated; Microsoft explicitly states prompts and completions are not used for training.

The practical upside is API compatibility. Your existing OpenAI client code works with a base_url swap:

from openai import OpenAI

client = OpenAI(
    base_url="https://<resource>.openai.azure.com/openai/deployments/gpt-4o",
    api_key="<azure_key>",
)
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize discharge notes"}]
)

Audit logging flows through Azure Monitor, which you can pipe to your own SIEM. The downside is throughput ceilings on per-deployment quotas and the need to manage deployment versions manually. If you need a HIPAA compliant LLM API with minimal legal friction and you already live in Azure, this is the default choice.

2. AWS Bedrock

Bedrock is HIPAA-eligible and covers multiple foundational model providers (Anthropic, Cohere, Mistral, Amazon) under a single AWS BAA. The models are invoked through a unified bedrock-runtime interface, but each provider has distinct prompt formats and token limits. You are not renting a raw OpenAI-compatible endpoint; you are calling managed model containers inside your VPC boundary.

import boto3
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = bedrock.invoke_model(
    modelId="anthropic.claude-v3-sonnet",
    body=b'{"messages":[{"role":"user","content":"Extract ICD codes"}]}'
)

Bedrock’s strength is cross-provider redundancy without negotiating separate BAAs for each model vendor—AWS is the single business associate. The weak spot is latency consistency and the fact that not every model version is eligible for PHI processing; you must check the service authorization matrix. For teams standardized on AWS, a HIPAA compliant LLM API via Bedrock is the path of least resistance.

3. Google Vertex AI

Vertex AI offers a BAA for covered services, and selected Gemini and PaLM models are HIPAA-eligible when used in specific regions with VPC Service Controls enabled. The API is OpenAI-adjacent but uses Google’s predict schema rather than chat completions. You also need to disable Vertex’s request logging and turn on CMEK (customer-managed encryption keys) to satisfy typical healthcare security review.

Google’s differentiator is tight integration with BigQuery and healthcare-specific connectors like Healthcare FHIR APIs. But model eligibility churns: a model that is HIPAA-eligible today may be excluded in a new preview. Treat Vertex as a HIPAA compliant LLM API only after you have confirmed the exact model ID and region in your BAA addendum. The operational burden is higher than Azure or Bedrock if you are not already a GCP shop.

4. Self-hosted open-weight models (vLLM or TGI)

If you run inference on your own compliant infrastructure—bare metal or a HIPAA-eligible cloud account with a BAA for the IaaS layer—no third-party model provider BAA is required. Open-weight models like Llama 3, Mixtral, or Command R+ deployed via vLLM give you full control over request logging, retention, and network egress.

docker run --gpus all -p 8000:8000 \
  vllm/vllm-openai:latest \
  --model meta-llama/Llama-3-8B-Instruct \
  --disable-log-stats

This is the only option where the HIPAA compliant LLM API boundary is entirely yours. The cost is engineering ownership: you patch the stack, scale the GPUs, and validate model behavior. For high-volume internal tools where PHI never leaves your VPC, the math often favors self-hosting despite the ops load.

5. Multi-provider gateways with BAA passthrough

A gateway that aggregates model providers behind one OpenAI-compatible endpoint can simplify client code, but it does not magically make upstream calls HIPAA-compliant. You still need a BAA with every backend that touches PHI. Some gateways, including n4n.ai, honor client routing directives and forward provider cache-control hints, letting you pin traffic to a BAA-covered Azure or Bedrock route while keeping a single integration surface.

The architecture only works if the gateway itself is deployed inside your compliant boundary or is itself a signed business associate. Automatic fallback is useful—if a provider is rate-limited, the gateway shifts to another eligible backend—but you must verify that the fallback target is also under BAA. A multi-provider HIPAA compliant LLM API is a composition of contracts, not a single checkbox.

Synthesis

Pick the HIPAA compliant LLM API that matches your existing cloud trust boundary. The BAA is the real product; the model is just the tenant.