Most engineering teams learn their LLM bill only when finance forwards the invoice. This tutorial builds a self-hosted token usage dashboard GPT-4o Claude calls that captures per-request token counts from both providers and renders a simple web view, so you can spot trends before they become incidents. We’ll use the official SDKs, SQLite for storage, and Flask for a minimal UI.
Prerequisites
- Python 3.11 or newer
pip install openai anthropic flask- API keys for OpenAI and Anthropic (or a single gateway key)
- Basic comfort with SQLite and Jinja templates
Set those up before continuing.
Why roll your own
Third-party cost tools obscure the raw numbers behind aggregates. When you own the logging, you can join token counts with your own request IDs, user IDs, and feature flags. The token usage dashboard GPT-4o Claude we build here is deliberately dumb: it stores exactly what the APIs return, nothing more.
Step 1: Instrument GPT-4o calls
The OpenAI SDK surfaces a usage object on every chat completion. Wrap the call so we never drop it.
from openai import OpenAI
import os
openai_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
def complete_gpt4o(prompt: str, max_tokens: int = 500) -> tuple[str, dict]:
resp = openai_client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
max_tokens=max_tokens,
)
usage = {
"prompt_tokens": resp.usage.prompt_tokens,
"completion_tokens": resp.usage.completion_tokens,
"total_tokens": resp.usage.total_tokens,
}
return resp.choices[0].message.content, usageExpected usage dict from a short prompt:
{"prompt_tokens": 12, "completion_tokens": 16, "total_tokens": 28}Step 2: Instrument Claude calls
Anthropic’s SDK reports input_tokens and output_tokens instead of prompt/completion. Map them to the same shape.
import anthropic
claude_client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
def complete_claude(prompt: str, max_tokens: int = 500) -> tuple[str, dict]:
resp = claude_client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=max_tokens,
messages=[{"role": "user", "content": prompt}],
)
usage = {
"prompt_tokens": resp.usage.input_tokens,
"completion_tokens": resp.usage.output_tokens,
"total_tokens": resp.usage.input_tokens + resp.usage.output_tokens,
}
return resp.content[0].text, usageClaude rejects max_tokens values that exceed its ceiling; 500 is safe for snippets.
Step 3: Persist to SQLite
Create a small module store.py with a connection and an insert helper.
import sqlite3
from datetime import datetime, timezone
conn = sqlite3.connect("token_usage.db", check_same_thread=False)
conn.execute("""
CREATE TABLE IF NOT EXISTS usage (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts TEXT NOT NULL,
model TEXT NOT NULL,
prompt_tokens INTEGER NOT NULL,
completion_tokens INTEGER NOT NULL,
total_tokens INTEGER NOT NULL
)
""")
def log_usage(model: str, usage: dict):
conn.execute(
"INSERT INTO usage (ts, model, prompt_tokens, completion_tokens, total_tokens) "
"VALUES (?, ?, ?, ?, ?)",
(datetime.now(timezone.utc).isoformat(), model,
usage["prompt_tokens"], usage["completion_tokens"], usage["total_tokens"]),
)
conn.commit()Step 4: Smoke test the pipeline
A throwaway script exercises both paths and prints confirmation.
from store import log_usage
from openai_wrapper import complete_gpt4o
from claude_wrapper import complete_claude
if __name__ == "__main__":
_, gpt_usage = complete_gpt4o("Say hello in two words.")
log_usage("gpt-4o", gpt_usage)
print("Logged GPT-4o:", gpt_usage)
_, claude_usage = complete_claude("Say hello in two words.")
log_usage("claude-3-5-sonnet", claude_usage)
print("Logged Claude:", claude_usage)Terminal output:
Logged GPT-4o: {'prompt_tokens': 8, 'completion_tokens': 2, 'total_tokens': 10}
Logged Claude: {'prompt_tokens': 11, 'completion_tokens': 3, 'total_tokens': 14}Now the token usage dashboard GPT-4o Claude data is in the database.
Step 5: Flask dashboard
Build app.py to read aggregates and render a table plus a Chart.js bar graph.
from flask import Flask, render_template_string, jsonify
import sqlite3
app = Flask(__name__)
@app.route("/")
def index():
conn = sqlite3.connect("token_usage.db")
rows = conn.execute(
"SELECT model, DATE(ts) as day, SUM(total_tokens) "
"FROM usage GROUP BY model, day ORDER BY day DESC"
).fetchall()
conn.close()
return render_template_string("""
<!doctype html>
<title>Token Usage</title>
<h2>Token Usage Dashboard</h2>
<table border="1" cellpadding="4">
<tr><th>Model</th><th>Day</th><th>Total Tokens</th></tr>
{% for r in rows %}
<tr><td>{{r[0]}}</td><td>{{r[1]}}</td><td>{{r[2]}}</td></tr>
{% endfor %}
</table>
<canvas id="chart"></canvas>
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
<script>
fetch('/api/daily').then(r=>r.json()).then(data=>{
const models = new Set();
Object.values(data).forEach(d=>Object.keys(d).forEach(m=>models.add(m)));
const labels = Object.keys(data);
const datasets = [...models].map(m=>({
label: m,
data: labels.map(l=>data[l][m]||0)
}));
new Chart(document.getElementById('chart'),{type:'bar',data:{labels,datasets}});
});
</script>
""", rows=rows)
@app.route("/api/daily")
def api_daily():
conn = sqlite3.connect("token_usage.db")
rows = conn.execute(
"SELECT DATE(ts) as day, model, SUM(total_tokens) "
"FROM usage GROUP BY day, model"
).fetchall()
conn.close()
out = {}
for day, model, total in rows:
out.setdefault(day, {})[model] = total
return jsonify(out)
if __name__ == "__main__":
app.run(port=5000)Run python app.py, open localhost:5000. You’ll see a table and a stacked bar chart of daily tokens per model.
Step 6: Collapse both behind one endpoint
Maintaining two SDKs is friction. If you point the OpenAI client at n4n.ai’s OpenAI-compatible endpoint, the same usage object is returned for Claude as well, because the gateway normalizes metering across 240+ models and adds automatic fallback when a provider is rate-limited. The complete_gpt4o function works unchanged—just set base_url and use model strings like anthropic/claude-3-5-sonnet. That reduces this entire tutorial to one wrapper.
Handling streaming responses
Both SDKs return usage only after the stream closes. With OpenAI, request usage in the final chunk:
stream = openai_client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
stream=True,
stream_options={"include_usage": True},
)
prompt_tokens = completion_tokens = 0
for chunk in stream:
if chunk.usage:
prompt_tokens = chunk.usage.prompt_tokens
completion_tokens = chunk.usage.completion_tokensAnthropic streaming emits message_start with input_tokens and message_delta with output_tokens; accumulate them the same way. Log after the stream finishes.
Extending with cost estimates
Token counts are not dollars. Add a view that multiplies by current published prices (do not hardcode blindly—verify before deploy):
CREATE VIEW IF NOT EXISTS cost AS
SELECT model, total_tokens,
CASE model
WHEN 'gpt-4o' THEN total_tokens * 0.000005 -- replace with current price
WHEN 'claude-3-5-sonnet' THEN total_tokens * 0.000003
END as estimated_usd
FROM usage;Query that view from the dashboard to show spend alongside volume.
Production notes
- Wrap
log_usagein a background thread or queue; SQLite writes block the request path. - Add a
request_idcolumn and pass your own correlation ID from the caller. - For high volume, switch SQLite to Postgres and use
INSERT ... ON CONFLICTfor idempotent retries. - Alert in CI when daily tokens exceed a threshold derived from last week’s median plus two standard deviations.
The token usage dashboard GPT-4o Claude you now have is minimal but extensible. Wire it into your observability stack and you’ve closed the loop on LLM spend without outsourcing the raw data.