n4nAI
Open console →
Developer toolsTutorialCost Optimization & Model Routing

Building cost dashboards for multi-provider LLM usage

Step-by-step tutorial to build multi-provider LLM cost dashboards from raw usage logs with Python, SQLite, and matplotlib for shipping LLM apps.

n4n Team3 min read584 words

Audio narration

Coming soon — every post will get a voice note here.

If you’re routing prompts across Anthropic, OpenAI, and open-weight models, you need multi-provider LLM cost dashboards to see where tokens—and dollars—go. This tutorial builds a lightweight usage logger and a local visualization from scratch, using only an OpenAI-compatible endpoint and standard Python libraries. No vendor console required.

Prerequisites

  • Python 3.10+ installed locally
  • pip install openai matplotlib (sqlite3 ships in stdlib)
  • An API key for an OpenAI-compatible inference gateway
  • Basic SQL and Python comfort

We will not use any hosted analytics. The goal is total ownership of your usage data so nothing is rounded, omitted, or delayed.

Step 1: Log every token (non-streaming)

Most gateways return a usage object in the chat completion response. Wrap the client call so each request persists prompt_tokens, completion_tokens, and model to a local SQLite file.

from openai import OpenAI
import sqlite3, time

client = OpenAI(
    base_url="https://api.your-gateway.com/v1",
    api_key="sk-your-key"
)

def log_usage(model, usage):
    conn = sqlite3.connect("llm_costs.db")
    cur = conn.cursor()
    cur.execute("""
        CREATE TABLE IF NOT EXISTS usage (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            ts REAL,
            model TEXT,
            prompt_tokens INTEGER,
            completion_tokens INTEGER,
            total_tokens INTEGER
        )
    """)
    cur.execute(
        "INSERT INTO usage (ts, model, prompt_tokens, completion_tokens, total_tokens) VALUES (?,?,?,?,?)",
        (time.time(), model, usage.prompt_tokens, usage.completion_tokens, usage.total_tokens)
    )
    conn.commit()
    conn.close()

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Ping"}]
)
log_usage(resp.model, resp.usage)
print("Logged", resp.usage.total_tokens, "tokens")

Run it. Expected output:

Logged 12 tokens

Verify the row exists:

sqlite3 llm_costs.db "SELECT * FROM usage;"

You should see one row with a Unix timestamp and token counts. This is the foundation for all multi-provider LLM cost dashboards you’ll build later.

Step 2: Capture streaming usage

Production traffic is usually streamed. OpenAI-compatible APIs hide usage unless you explicitly request it. Pass stream_options={"include_usage": True} and read the final chunk.

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Tell me a long story"}],
    stream=True,
    stream_options={"include_usage": True}
)

final_usage = None
for chunk in stream:
    if chunk.usage:
        final_usage = chunk.usage

if final_usage:
    log_usage("gpt-4o-mini", final_usage)
    print("Streamed usage logged:", final_usage.total_tokens)

Some providers only emit usage on the last chunk. If you forget stream_options, you’ll silently record zero tokens and your dashboards will lie. Treat that flag as mandatory.

Step 3: Map models to providers and rates

A model name alone doesn’t tell you the provider or price. Keep a local JSON file you control. Use placeholder rates; insert your contracted numbers.

{
  "gpt-4o-mini": {"provider": "openai", "in_per_1k": 0.0, "out_per_1k": 0.0},
  "claude-3-haiku": {"provider": "anthropic", "in_per_1k": 0.0, "out_per_1k": 0.0},
  "mistral-7b": {"provider": "mistral", "in_per_1k": 0.0, "out_per_1k": 0.0}
}

Enrich the table with a provider column and computed cost.

import json

with open("prices.json") as f:
    prices = json.load(f)

def backfill_provider_and_cost():
    conn = sqlite3.connect("llm_costs.db")
    cur = conn.cursor()
    cur.execute("ALTER TABLE usage ADD COLUMN provider TEXT")
    cur.execute("ALTER TABLE usage ADD COLUMN cost REAL")
    cur.execute("SELECT id, model, prompt_tokens, completion_tokens FROM usage")
    for row in cur.fetchall():
        id_, model, pt, ct = row
        info = prices.get(model, {"provider": "unknown", "in_per_1k": 0, "out_per_1k": 0})
        cost = (pt/1000)*info["in_per_1k"] + (ct/1000)*info["out_per_1k"]
        cur.execute("UPDATE usage SET provider=?, cost=? WHERE id=?",
                    (info["provider"], cost, id_))
    conn.commit()
    conn.close()

Run backfill_provider_and_cost() once. Now each row knows its provider. Re-run it periodically as new models appear.

Step 4: Aggregate for multi-provider LLM cost dashboards

The raw table is useless for decisions. Group by day and provider to see trends.

import sqlite3

def daily_summary():
    conn = sqlite3.connect("llm_costs.db")
    cur = conn.cursor()
    cur.execute("""
        SELECT date(ts, 'unixepoch') AS day,
               provider,
               SUM(total_tokens) AS tokens,
               SUM(cost) AS cost
        FROM usage
        GROUP BY day, provider
        ORDER BY day, provider
    """)
    rows = cur.fetchall()
    conn.close()
    return rows

for day, provider, tokens, cost in daily_summary():
    print(f"{day} | {provider:10} | {tokens:8} tok | ${cost:.4f}")

Expected output (with zero rates):

2024-05-12 | openai     |      120 tok | $0.0000
2024-05-12 | anthropic  |       88 tok | $0.0000

When you fill real rates, the cost column becomes actionable. This query is the data source for every multi-provider LLM cost dashboard you render.

Step 5: Render the dashboard

A terminal table is fine, but a stacked bar chart makes multi-provider LLM cost dashboards legible at a glance. Use matplotlib.

import matplotlib.pyplot as plt
from collections import defaultdict

rows = daily_summary()
series = defaultdict(lambda: defaultdict(int))
for day, provider, tokens, _ in rows:
    series[day][provider] += tokens

days = sorted(series.keys())
providers = sorted({p for d in series.values() for p in d})
bottom = [0] * len(days)

for p in providers:
    vals = [series[d].get(p, 0) for d in days]
    plt.bar(days, vals, bottom=bottom, label=p)
    bottom = [b + v for b, v in zip(bottom, vals)]

plt.ylabel("Total tokens")
plt.title("Daily token usage by provider")
plt.legend()
plt.xticks(rotation=45)
plt.tight_layout()
plt.savefig("dashboard.png")
print("Wrote dashboard.png")

Open dashboard.png. You’ll see a stacked bar per day, each segment a provider. That’s the core of your local dashboard.

Step 6: Collect automatically via a single gateway

If you route through a gateway such as n4n.ai, the OpenAI-compatible endpoint returns usage for 240+ models and applies automatic fallback when a provider is degraded, so your logger only needs one client and one base_url. You avoid writing per-provider adapters and still get clean per-token metering.

To capture routing hints, forward client headers:

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize this"}],
    extra_headers={"x-routing": "cost-optimized"}
)
log_usage(resp.model, resp.usage)

The gateway resolves auto to a concrete model and returns it in resp.model, which your logger stores verbatim. Your dashboards then reflect what actually served the traffic, not what you hoped would.

Step 7: Expose it as a live web dashboard

For a shared view, wrap the query in Streamlit. Install with pip install streamlit.

import streamlit as st
import sqlite3, pandas as pd

st.title("Multi-provider LLM cost dashboards")
conn = sqlite3.connect("llm_costs.db")
df = pd.read_sql_query("""
    SELECT date(ts,'unixepoch') AS day, provider, SUM(total_tokens) AS tokens
    FROM usage GROUP BY day, provider
""", conn)
st.bar_chart(df.pivot(index="day", columns="provider", values="tokens").fillna(0))

Run streamlit run app.py. The browser shows an interactive chart updated on each refresh. No external service touches your numbers.

What to add next

  • Cache hit tracking: forward provider cache-control hints and log prompt_tokens_details.cached_tokens if your gateway exposes them.
  • Alerting: a cron job that emails if daily spend exceeds a threshold.
  • Per-tenant isolation: add a tenant_id column when you serve multiple customers.
  • Latency columns: store resp.system_fingerprint or gateway latency headers to correlate cost with speed.

Owning the usage pipeline means your multi-provider LLM cost dashboards reflect reality, not a vendor’s idea of it. Build the logger once, point it at your gateway, and iterate.

Tagscost-dashboardobservabilityllm-providersmonitoring

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Developer toolsComparing pay-per-token pricing across LLM providersA practical head-to-head pay-per-token pricing comparison of OpenAI, Anthropic, Google, and Mistral across capabilities, cost, latency, and limits.Developer toolsA cost optimization checklist for LLM API usageA practical LLM cost optimization checklist for engineers: right-size models, cache, batch, route, and meter to cut API spend without hurting quality.Developer toolsBuilding a fallback strategy across LLM providersPractical guide to building a reliable LLM provider fallback strategy: model mapping, explicit chains, gateway fallback, streaming, and cost controls.Developer toolsGPT-4o vs Claude vs Gemini: cost per million tokensPractical head-to-head comparison of GPT-4o vs Claude vs Gemini pricing, capabilities, latency, and ergonomics for engineers optimizing LLM app cost.Developer toolsHow caching prompts cuts LLM inference costsLearn how to implement prompt caching for cost reduction across LLM providers with practical steps, code samples, and verification tips for engineers.Developer toolsHow semantic caching reduces redundant LLM callsLearn how to implement semantic caching for LLM calls to cut redundant model requests, with step-by-step code and integration to a gateway.

All cost optimization & model routing posts →