AWS Bedrock vs Azure OpenAI: enterprise LLM pricing compared

Choosing between cloud LLM platforms forces a hard look at the bill. The real story in AWS Bedrock vs Azure OpenAI pricing is not just per-token rates but how each vendor structures commitments, quotas, and hidden throughput costs that surface only after you scale.

Capabilities

Model availability

Bedrock aggregates third-party foundation models behind a single runtime API: Anthropic Claude 3/3.5 families, Meta Llama 3, Amazon Titan, Cohere Command, and a few others. Azure OpenAI exposes OpenAI’s catalog—GPT-4o, GPT-4 Turbo, text-embedding-3, DALL·E—plus Microsoft’s Phi models in some regions. If your stack already depends on OpenAI’s tooling, Azure is the path of least resistance. If you need Llama 3 or Claude with direct AWS IAM, Bedrock wins.

Fine-tuning and customization

Both support fine-tuning, but the ergonomics differ. Bedrock Custom Models imports weights into your account and serves them as a private model ID; Azure OpenAI fine-tuning runs through a training job that stores a derived model in your resource. Neither gives you raw weight access for closed models. Bedrock also supports continued pre-training for Titan; Azure supports distillation for some OpenAI models.

Modalities

Azure OpenAI has first-class image generation and audio transcription endpoints. Bedrock covers text and image via Stable Diffusion and Titan Multimodal, but the API surface is less uniform. For pure text generation, both are equivalent.

Price/Cost Model

On-demand per-token

Both platforms charge per token. AWS Bedrock vs Azure OpenAI pricing on on-demand inference is published per model and per region; Bedrock passes through provider rates with a small AWS margin, Azure applies its own rate card. You pay for input and output tokens separately, and for some models, a separate charge for training or storage. There is no minimum spend on either on-demand tier.

Provisioned throughput and commitments

Azure OpenAI pushes Provisioned Throughput Units (PTUs) for predictable latency. You reserve capacity monthly and pay whether you use it or not. Bedrock offers Provisioned Throughput with similar model-specific hourly commitments, but on some models you can buy one-hour minimum blocks instead of monthly. That granularity matters for batch pipelines that run weekly.

Hidden costs

Data egress inside the same cloud is free; cross-region calls are not. Bedrock charges for model customization storage per GB-month and for training jobs per hour. Azure charges for training compute separately from inference and bills PTU reservations even at zero utilization. Both meter by token, but if you use gateway caching, repeated prefixes can cut bills—something a gateway that honors provider cache-control hints helps with.

Latency and Throughput

Cold starts and regional variance

Bedrock serverless endpoints spin up model copies on demand; first token can lag during scale-up. Azure OpenAI shared instances queue under load. PTUs on either side remove queueing but cost fixed capital.

Measured behavior

In practice, a 2k-token prompt to Claude on Bedrock returns first token in roughly 300–800ms under warm load, with p99 degrading during region saturation. Azure GPT-4o under PTU hits similar ranges; on shared tier, p99 can slip when the region saturates. Neither publishes hard SLA on per-call latency, only on API availability (99.9% typical).

Throughput ceilings

Bedrock documents per-model tokens-per-minute defaults (e.g., 200k TPM for some Claude SKUs) that you raise via support tickets. Azure ties throughput to PTU count: 1 PTU ≈ a fixed concurrent token rate. Exceeding it returns 429s.

Ergonomics

SDK and API shape

Bedrock uses a single InvokeModel RPC with model-specific JSON bodies. Azure OpenAI mirrors the OpenAI REST schema. Code shows the difference:

# Bedrock
import boto3, json
br = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = br.invoke_model(
    modelId="anthropic.claude-3-sonnet-20240229-v1:0",
    body=json.dumps({"messages":[{"role":"user","content":"hi"}],"max_tokens":10})
)
print(json.loads(resp["body"].read())["content"])

# Azure OpenAI
from openai import AzureOpenAI
client = AzureOpenAI(api_key="...", azure_endpoint="https://my.call.windows.net", api_version="2024-02-01")
r = client.chat.completions.create(model="gpt-4o", messages=[{"role":"user","content":"hi"}], max_tokens=10)
print(r.choices[0].message.content)

Bedrock’s unified method means you swap modelId and body schema; Azure’s client is OpenAI-compatible, so existing OpenAI code works with an endpoint swap. For raw HTTP, Bedrock requires SigV4 signing:

aws bedrock-runtime invoke-model \
  --model-id anthropic.claude-3-sonnet-20240229-v1:0 \
  --body '{"messages":[{"role":"user","content":"hi"}],"max_tokens":10}' \
  --cli-binary-format raw-in-base64-out output.json

Auth and config

Bedrock uses AWS SigV4 signed requests and IAM policies per model ARN. Azure uses Entra ID or key auth with resource-scoped RBAC. If you already run Terraform for AWS, Bedrock slots in; Azure shops get the same with Bicep. Both support private network endpoints.

Ecosystem

Integration with cloud services

Bedrock logs to CloudWatch, ties to SageMaker for pipelines, and sits inside VPC endpoints. Azure OpenAI streams to App Insights, ties to Azure ML, and uses Private Link. Both are native to their clouds—no surprise. If your observability stack is Datadog on AWS, Bedrock metrics forward without a bridge; Azure needs the Monitor exporter.

Gateway abstraction

If you want to avoid writing both clients, an OpenAI-compatible gateway such as n4n.ai exposes one endpoint addressing 240+ models with automatic fallback when a provider is degraded, per-token metering, and forwarding of cache-control hints. That collapses the AWS Bedrock vs Azure OpenAI pricing comparison into a single token ledger and lets you shift traffic based on real-time 429 rates.

Limits

Rate limits and quotas

Bedrock applies per-model TPM and RPM defaults that you raise via support tickets. Azure OpenAI enforces PTU or shared quotas with hard caps per deployment. Both throttle on regional capacity; Bedrock’s fallback across regions is manual unless you build it. Azure’s shared tier can drop to zero available capacity during peaks in some regions—engineers report 429 storms with no auto-failover.

Compliance

Both meet SOC2, ISO, and HIPAA with BAA. Bedrock keeps data in-region by default; Azure OpenAI offers data residency opts. For regulated workloads, review the specific model provider’s terms—Anthropic on Bedrock differs from OpenAI on Azure regarding prompt retention.

Model deprecation

Azure gives notice before retiring a model version; Bedrock passes through provider deprecation timelines. If you pin gpt-4-0613 on Azure, you get a sunset window. On Bedrock, a Claude version can be pulled by Anthropic with AWS relaying the notice.

Head-to-Head Summary

Which to Choose

Already on AWS with multi-model needs

Pick Bedrock. You get Llama and Claude without leaving IAM, and the on-demand pricing aligns with sporadic workloads. Use provisioned blocks for batch jobs that need guaranteed capacity for a few hours.

OpenAI-locked product surface

If your codebase assumes OpenAI function calling and SDK, Azure OpenAI removes porting cost. The monthly PTU is worth it when latency consistency drives revenue, such as customer-facing chat with strict p95 targets.

Cost-sensitive prototyping

Both on-demand tiers are comparable; the AWS Bedrock vs Azure OpenAI pricing gap narrows at low volume. Start with whichever cloud you have credits in, and abstract the client early so you can switch without rewriting business logic.

Regulated, multi-cloud

Neither is inherently safer. Use Bedrock for Anthropic models in-region; use Azure for OpenAI models with Microsoft BAA. A gateway normalizes the billing and routing if you run both, letting compliance teams see one meter.

High-throughput, variable

Bedrock’s hourly provisioned throughput gives finer granularity than Azure’s monthly PTU. If your traffic spikes weekly, Bedrock avoids paying for idle months. Azure PTU can be cheaper at constant 80%+ utilization.

Team without cloud-specific Ops

If you lack AWS or Azure specialists, the OpenAI-compatible shape of Azure reduces onboarding. Bedrock’s SigV4 and model-ID sprawl adds a learning curve. A gateway that fronts both removes the gap entirely.

The comparison isn’t about which is cheaper per token—it’s about which commitment model and ecosystem tax fits your architecture. Engineer for the abstraction, then optimize the rate card.