Estimating monthly LLM spend from token counts

To estimate monthly LLM spend from raw token counts, you need three things: accurate usage logs, a pricing table that matches the exact model IDs you call, and an aggregation layer that respects calendar months and cached-token discounts. Most teams hack this together after their first surprise invoice, but building a small pipeline now saves arguments later.

Step 1: Capture token usage on every request

OpenAI-compatible APIs return a usage object on each completion. If you are not logging it, you are flying blind. With the official Python client, the field is right there:

from openai import OpenAI

client = OpenAI(base_url="https://api.openai.com/v1", api_key="sk-...")

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize this log"}],
)
usage = resp.usage
print(usage.prompt_tokens, usage.completion_tokens, usage.total_tokens)

For streaming calls, you must explicitly ask for usage in the final chunk:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Stream me a poem"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.usage:
        usage = chunk.usage

Newer models nest cached token counts under prompt_tokens_details.cached_tokens. Capture that separately—it is priced differently. If you route through n4n.ai, its per-token usage metering returns the same usage schema, so you get counted tokens without extra instrumentation.

Step 2: Persist usage events with metadata

A single JSON line per request is enough to start. Write the timestamp, served model, and token breakdown:

import json, time

event = {
    "ts": int(time.time()),
    "model": resp.model,
    "prompt_tokens": usage.prompt_tokens,
    "completion_tokens": usage.completion_tokens,
    "cached_tokens": (
        getattr(usage.prompt_tokens_details, "cached_tokens", 0)
        if hasattr(usage, "prompt_tokens_details")
        else 0
    ),
}
with open("usage.jsonl", "a") as f:
    f.write(json.dumps(event) + "\n")

In production, replace the file append with a partitioned database table keyed by event_date. The critical rule: always log resp.model, not the model you requested. Gateways and fallback logic can serve a different model than the one you asked for, and cost must follow the served model.

Step 3: Build a pricing map for your models

Pricing is volatile. Hard-code a snapshot and review it monthly. Cached input is typically discounted 50%, but confirm per provider:

PRICING = {
    "gpt-4o": {
        "input_per_million": 5.0,
        "cached_input_per_million": 2.5,
        "output_per_million": 15.0,
    },
    "gpt-3.5-turbo": {
        "input_per_million": 0.5,
        "cached_input_per_million": 0.25,
        "output_per_million": 1.5,
    },
    "claude-3-5-sonnet": {
        "input_per_million": 3.0,
        "cached_input_per_million": 0.30,  # Anthropic's 90% cache discount
        "output_per_million": 15.0,
    },
}

Do not invent numbers. Pull them from provider docs or a maintained open-source price list. If a model is missing from your map, raise an alert instead of assuming zero cost.

Step 4: Compute cost per event

Apply the pricing map to each event. Separate cached and non-cached input:

def cost_of(event, pricing):
    p = pricing.get(event["model"])
    if not p:
        return None
    non_cached = max(event["prompt_tokens"] - event["cached_tokens"], 0)
    in_cost = non_cached / 1e6 * p["input_per_million"]
    cache_cost = event["cached_tokens"] / 1e6 * p["cached_input_per_million"]
    out_cost = event["completion_tokens"] / 1e6 * p["output_per_million"]
    return in_cost + cache_cost + out_cost

This function is pure. Unit-test it with known token counts and prices before trusting it.

Step 5: Aggregate by month to estimate monthly LLM spend

Group events by calendar month using UTC dates. This is the core step that lets you estimate monthly LLM spend from historical token counts:

from collections import defaultdict
import datetime

monthly = defaultdict(float)
with open("usage.jsonl") as f:
    for line in f:
        ev = json.loads(line)
        dt = datetime.datetime.utcfromtimestamp(ev["ts"])
        key = f"{dt.year}-{dt.month:02d}"
        c = cost_of(ev, PRICING)
        if c is not None:
            monthly[key] += c

for k, v in sorted(monthly.items()):
    print(k, f"${v:.2f}")

If you need per-team or per-endpoint breakdowns, add those fields to the event in Step 2 and group by them alongside the month.

Step 6: Handle multi-provider routing and fallback

When you call a gateway that abstracts over many providers, model strings may include provider prefixes like openai/gpt-4o or anthropic/claude-3-5-sonnet. Normalize them to the keys in your pricing map. More importantly, automatic fallback changes the served model when a provider is degraded. Always use the response’s model field, never the request’s.

If you use n4n.ai’s OpenAI-compatible endpoint that addresses 240+ models, the returned model reflects the actually served variant after fallback, so your cost attribution stays correct without custom parsing.

Step 7: Forecast next month’s spend

A simple projection beats a guess. Take the trailing 30 days of cost, compute average daily burn, and extrapolate:

def forecast_next_month(events, pricing, days=30):
    cutoff = time.time() - days * 86400
    recent = [e for e in events if e["ts"] >= cutoff]
    total = sum(cost_of(e, pricing) or 0 for e in recent)
    avg_daily = total / days
    return avg_daily * 30

For seasonal traffic, use weekly patterns. The point is to estimate monthly LLM spend before the bill arrives, not after.

Step 8: Verify your estimate matches reality

At the close of a billing period, sum your computed costs for that month and compare to the provider’s invoice. Differences under 1% are normal after rounding and tax. If you used a gateway with built-in metering, export its per-token report and diff it against your JSONL aggregation.

If the numbers diverge, check three things:

Cached token counts were captured (not folded into prompt tokens).
Model IDs in logs match pricing keys exactly.
Streaming usage was actually collected on every call.

Your ability to estimate monthly LLM spend reliably depends on this verification loop. Run it monthly for two cycles and the pipeline becomes trustworthy.

Common pitfalls

Cache discounts are easy to miss. A 90% cached-input discount (Anthropic) versus 50% (OpenAI) changes unit economics massively at scale.

Minimum request fees or per-image pricing are not in token counts. If you use vision or audio models, extend the event schema with those units.

Time zones matter. Billing periods are often UTC months; use UTC in aggregation or you will shift a day of cost across months.

Finally, prices change. A pricing map from six months ago is a lie. Automate a weekly fetch from provider docs or a trusted price API, and fail loud when a model is unknown.

Building this pipeline takes an afternoon. Running it saves you from explaining a four-figure overage to finance.