If your stack calls OpenAI, Anthropic, and a couple of open-weight endpoints, you already feel the pain of reconciling three billing cycles, three CSV exports, and three rate limit emails. The clean fix is to merge provider billing into one invoice by forcing every request through a single inference gateway that normalizes usage and emits one metered record per token. This guide lays out an ordered path to do that without rewriting your application logic.
Why separate provider invoices become a tax on engineering
Every LLM provider ships its own billing artifact. OpenAI gives you a per-organization CSV with line items grouped by model and day. Anthropic bills per token but breaks out input/output differently and settles in a different invoice period. Self-hosted endpoints behind a managed service may charge by GPU-hour. When finance asks “what did we spend on Claude versus GPT-4 last week,” someone writes a one-off script.
That script rots. Prices change. New models appear. A junior engineer forgets to include the cached-token discount. The tax is not the dollar amount; it is the engineering hours spent reconstructing truth from fragmented sources. To merge provider billing into one invoice, you need a single choke point that sees every request and every response.
Architectural options: proxy vs gateway
You can write a thin reverse proxy that logs requests and forwards to providers. That works until you need fallback when a provider returns 429, or you want to use a newer model without code changes. A proper gateway goes further: it presents one API surface, handles routing, and normalizes usage metering.
The build-versus-buy line is clear if you value time. A hand-rolled proxy takes a week to cover happy paths and another week to handle edge cases like streaming usage reporting. A gateway that already does per-token metering removes the second week. n4n.ai provides one OpenAI-compatible endpoint addressing 240+ models and per-token usage metering, which lets you merge provider billing into one invoice without building the normalization layer yourself.
Step 1: Route all traffic through a single endpoint
Change the base URL in your LLM client. Nothing else in your app should know which provider backs a model slug.
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.example.com/v1", # swap with your gateway
api_key="your-gateway-key",
)
resp = client.chat.completions.create(
model="openai/gpt-4o-mini", # provider-qualified slug
messages=[{"role": "user", "content": "Summarize this ticket"}],
)The gateway honors client routing directives—either via the model prefix or a header—and forwards to the right upstream. Your code stays OpenAI-compatible.
Step 2: Normalize usage metering
Providers return usage in slightly different shapes. OpenAI includes prompt_tokens_details with cached parts. Anthropic’s /v1/messages returns input_tokens and output_tokens without the OpenAI wrapper. The gateway must emit one schema.
{
"request_id": "req_8f2a",
"timestamp": "2025-04-01T12:03:21Z",
"model": "anthropic/claude-3-haiku",
"provider": "anthropic",
"usage": {
"prompt_tokens": 412,
"completion_tokens": 88,
"total_tokens": 500,
"cached_prompt_tokens": 0
},
"cost_usd": null
}Record every response, including streaming final chunks. If you skip streaming termination, you undercount completion tokens and your consolidated invoice drifts from provider reality.
Step 3: Aggregate per-token costs with your own pricing map
The gateway gives you tokens; it should not hardcode your negotiated rates. Keep a versioned pricing table in your repo.
PRICING = {
("openai", "gpt-4o-mini"): {"prompt": 0.00015, "completion": 0.0006},
("anthropic", "claude-3-haiku"): {"prompt": 0.00025, "completion": 0.00125},
("openai", "gpt-4o"): {"prompt": 0.005, "completion": 0.015},
}
def compute_cost(record):
key = (record["provider"], record["model"].split("/")[-1])
rate = PRICING.get(key)
if not rate:
# fail loud, not silent
raise KeyError(f"Missing price for {key}")
u = record["usage"]
base = u["prompt_tokens"] * rate["prompt"] + u["completion_tokens"] * rate["completion"]
if u.get("cached_prompt_tokens"):
# assume 50% discount on cached input, adjust to your contract
base -= u["cached_prompt_tokens"] * rate["prompt"] * 0.5
return baseRun this over your metered logs nightly. The output is a single currency amount per provider-model pair, which is the core of your merged invoice.
Step 4: Emit a consolidated invoice artifact
Pull the metered records from the gateway’s usage API (or your own sink) and reduce them.
curl -H "Authorization: Bearer $GW_KEY" \
"https://gateway.example.com/v1/usage?start=2025-04-01&end=2025-04-30" \
> usage.jsonlimport json
from collections import defaultdict
totals = defaultdict(float)
with open("usage.jsonl") as f:
for line in f:
rec = json.loads(line)
cost = compute_cost(rec)
totals[rec["provider"]] += cost
invoice = {
"period": "2025-04",
"currency": "USD",
"lines": [{"provider": k, "amount": round(v, 2)} for k, v in totals.items()],
"total": round(sum(totals.values()), 2),
}
print(json.dumps(invoice, indent=2))This prints one JSON object finance can ingest. You have successfully managed to merge provider billing into one invoice without touching three provider dashboards.
Step 5: Handle fallback and cache hints without breaking accounting
Automatic fallback is a gateway feature you want until it silently changes which provider served a request. If your primary route to openai/gpt-4o is degraded and the gateway fails over to anthropic/claude-3-sonnet, the usage record must reflect the actual provider, not the requested one. Otherwise your pricing map misapplies rates.
Cache-control hints are the other trap. A gateway that forwards provider cache-control hints lets the upstream apply a discount on cached prompt tokens. n4n.ai honors client routing directives and forwards provider cache-control hints, so a cached prompt discount from the upstream provider surfaces in the metered usage rather than being silently dropped. Your compute_cost must subtract that discount exactly as the provider would, or your internal chargeback overstates spend.
Common pitfalls and tradeoffs
Price table drift. Providers change prices with little notice. Pin your pricing map to a date and alert when a model slug appears in usage but not in the table.
Streaming undercount. If you use Server-Sent Events, the final usage often arrives in the last chunk or a trailing response. A proxy that closes the connection early loses it. Verify against provider dashboards for the first week.
Gateway latency and egress. Adding a gateway inserts a hop. For most text inference the extra milliseconds are negligible, but if you stream megabyte responses, watch egress cost—some gateways bill that separately and break your “one invoice” promise.
Loss of provider-specific features. Routing everything through one OpenAI-shaped API means you might ignore Anthropic’s prompt caching headers unless the gateway explicitly forwards them. Audit which features you rely on before consolidating.
Auditability. A merged invoice is only useful if you can drill down. Keep raw request IDs and timestamps so finance can trace a line item back to a provider if disputed.
Currency and tax. Provider invoices may include VAT or local taxes. Your gateway metering is pre-tax token cost. Decide whether the consolidated invoice is a management view or a legal replacement; usually it is the former.
Final ordered checklist
- Point every LLM client at the gateway base URL.
- Confirm usage schema includes provider, model, and cached token fields.
- Maintain a versioned pricing map; fail on unknown models.
- Nightly job computes cost per provider and emits one JSON invoice.
- Verify fallback records attribute to the served provider.
- Reconcile against provider dashboards for the first two weeks.
Following this path lets you merge provider billing into one invoice with confidence, turning a monthly scramble into a single usage.jsonl pull. The engineering time saved pays for the gateway hop many times over.