Tracking spend across multiple LLM API credit balances

If you run production workloads against several model vendors, you already know the pain of trying to track spend multiple LLM API credit balances that each expose different units, dashboards, and replenishment rules. This guide lays out a concrete architecture to centralize that accounting without trusting vendor UI graphs, using append-only logging and a thin normalization layer.

Step 1: Inventory your credit systems and their metering formats

Before writing code, list every endpoint you call and how it reports consumption. The credit models differ in ways that break naive summation:

OpenAI returns usage in the chat completion response with prompt_tokens, completion_tokens, and total_tokens. Anthropic’s Messages API returns usage with input_tokens, output_tokens, and separate cache fields. Google’s Vertex AI bills by character or token depending on model. Some resellers issue their own credits that decay or have minimum balances.

The first step to track spend multiple LLM API credit balances is to capture the raw usage object from each response at call time. Do not rely on post-hoc billing dashboards; they lag by hours and rarely expose cached-token discounts.

import openai

def call_openai_and_get_usage(prompt: str, model: str = "gpt-4o-mini"):
    resp = openai.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.usage.model_dump(), resp.choices[0].message.content

For an OpenAI-compatible gateway, the shape is identical. n4n.ai exposes one OpenAI-compatible endpoint covering 240+ models and returns per-token usage in the same structure, which removes the need to write per-vendor parsers.

Step 2: Normalize usage to a common ledger record

Vendors disagree on field names and whether they count cached tokens. Define a single record and map each response into it. When you track spend multiple LLM API credit balances, you must also record the price per token at time of call, because vendors change pricing without notice.

from dataclasses import dataclass
from datetime import datetime, timezone

@dataclass
class UsageEvent:
    ts: datetime
    vendor: str
    model: str
    prompt_tokens: int
    completion_tokens: int
    cached_tokens: int = 0
    request_id: str | None = None

def from_openai(vendor: str, model: str, usage: dict, request_id: str | None) -> UsageEvent:
    cached = usage.get("prompt_tokens_details", {}).get("cached_tokens", 0)
    return UsageEvent(
        ts=datetime.now(timezone.utc),
        vendor=vendor,
        model=model,
        prompt_tokens=usage.get("prompt_tokens", 0),
        completion_tokens=usage.get("completion_tokens", 0),
        cached_tokens=cached,
        request_id=request_id,
    )

def from_anthropic(vendor: str, model: str, usage: dict, request_id: str | None) -> UsageEvent:
    # Anthropic separates cache creation vs read
    cached = usage.get("cache_read_input_tokens", 0)
    return UsageEvent(
        ts=datetime.now(timezone.utc),
        vendor=vendor,
        model=model,
        prompt_tokens=usage.get("input_tokens", 0),
        completion_tokens=usage.get("output_tokens", 0),
        cached_tokens=cached,
        request_id=request_id,
    )

Store a static price table keyed by (vendor, model, date). Cache reads are typically 10% of prompt price; cache creation may be 25% depending on vendor.

{
  "openai": {
    "gpt-4o-mini": {"prompt": 0.00000015, "completion": 0.00000060, "cache_factor": 0.1}
  },
  "anthropic": {
    "claude-3-5-sonnet": {"prompt": 0.000003, "completion": 0.000015, "cache_factor": 0.1}
  }
}

Step 3: Route all traffic through a single metering layer

If you directly call three vendor SDKs, you will maintain three parsers and three failure modes. A better pattern is to front your calls with a thin client that wraps the SDK, extracts usage, and writes the event before returning the completion to your app.

class MeteringClient:
    def __init__(self, ledger):
        self.ledger = ledger

    def complete(self, vendor, model, prompt):
        if vendor == "openai":
            usage, text = call_openai_and_get_usage(prompt, model)
            event = from_openai(vendor, model, usage, None)
        elif vendor == "anthropic":
            # assume similar wrapper around anthropic SDK
            usage, text = call_anthropic_and_get_usage(prompt, model)
            event = from_anthropic(vendor, model, usage, None)
        else:
            raise ValueError(vendor)
        self.ledger.append(event)
        return text

A gateway that honors client routing directives and forwards provider cache-control hints simplifies this further: you send one request format, and the gateway handles fallback when a provider is rate-limited or degraded. That single integration point is where you reliably track spend multiple LLM API credit balances without scattering logic across services.

Step 4: Persist events in an append-only ledger

Use SQLite for a single-node deployment or Postgres if you already have it. The schema should be immutable: no updates, only inserts. Compute cost_usd at insert time using the price table; this avoids later repricing errors and makes reconciliation trivial.

CREATE TABLE usage_events (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    ts TEXT NOT NULL,
    vendor TEXT NOT NULL,
    model TEXT NOT NULL,
    prompt_tokens INTEGER NOT NULL,
    completion_tokens INTEGER NOT NULL,
    cached_tokens INTEGER NOT NULL DEFAULT 0,
    request_id TEXT,
    cost_usd REAL
);

def insert_event(conn, event: UsageEvent, price_table: dict):
    price = price_table[event.vendor][event.model]
    cache_factor = price.get("cache_factor", 0.1)
    billable_prompt = (event.prompt_tokens - event.cached_tokens) * price["prompt"] \
                      + event.cached_tokens * price["prompt"] * cache_factor
    cost = billable_prompt + event.completion_tokens * price["completion"]
    conn.execute(
        "INSERT INTO usage_events (ts, vendor, model, prompt_tokens, completion_tokens, cached_tokens, request_id, cost_usd) "
        "VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
        (event.ts.isoformat(), event.vendor, event.model,
         event.prompt_tokens, event.completion_tokens, event.cached_tokens,
         event.request_id, cost),
    )
    conn.commit()

Add a unique index on request_id if your gateway provides one, so retries don’t double-count.

Step 5: Reconcile against vendor statements weekly

Most providers expose a usage export (CSV or JSON) but not a real-time balance API. Download the statement, sum their reported tokens, and compare to your ledger grouped by day. When you track spend multiple LLM API credit balances, reconciliation catches silent price changes and SDK bugs.

sqlite3 ledger.db "SELECT date(ts) AS day, vendor, SUM(cost_usd) FROM usage_events GROUP BY day, vendor;"

If the delta exceeds 2%, inspect cached token handling and whether the vendor counts system prompts differently. OpenAI’s dashboard often includes moderation calls; your ledger may not if you don’t meter them. Anthropic’s console shows usage but not always broken down by cache read vs creation, making your own ledger the source of truth.

Step 6: Set burn-rate alerts

A simple cron query can flag abnormal spend. Compute the trailing 24h cost and compare to the prior 7-day average. This catches runaway loops or a misconfigured model swap to a expensive SKU.

def burn_rate(conn):
    row = conn.execute(
        "SELECT SUM(cost_usd) FROM usage_events WHERE ts > datetime('now', '-1 day')"
    ).fetchone()
    today = row[0] or 0
    row = conn.execute(
        "SELECT SUM(cost_usd)/7 FROM usage_events WHERE ts BETWEEN datetime('now', '-8 day') AND datetime('now', '-1 day')"
    ).fetchone()
    baseline = row[0] or 0
    return today, baseline

if __name__ == "__main__":
    today, baseline = burn_rate(conn)
    if today > baseline * 2:
        print(f"ALERT: spend {today:.2f} vs baseline {baseline:.2f}")

Wire the print to Slack or PagerDuty. The point is not precise forecasting but early detection.

Verify success

After implementing the steps, run a controlled test: make one call to each vendor (or through your gateway) with a known prompt, then query the ledger.

sqlite3 ledger.db "SELECT vendor, model, prompt_tokens, completion_tokens, cost_usd FROM usage_events ORDER BY id DESC LIMIT 3;"

You should see one row per request with non-zero token counts and a plausible cost_usd (sub-cent for small prompts). Next, sum total cost and confirm it matches the sum of manual price-table math. Finally, wait 24 hours and ensure the burn-rate script runs without errors. If those three checks pass, you have a working system to track spend multiple LLM API credit balances across heterogeneous providers.