Multi-agent systems hide cost in plain sight. Tracking token cost multi-agent orchestration requires more than summing a single chat completion’s usage field—each agent spawns branches, retries, and tool calls that fragment spend across models and providers. If you can’t attribute every token to a trace and an agent, you will over-provision or get blindsided by a recursive planner.
Why naive summation fails
In a single LLM call, the math is trivial: read response.usage.total_tokens, multiply by the model price. In an orchestrated graph of agents, a supervisor might call a researcher agent, which calls a search tool that triggers another summarizer, each issuing three or four completions. The tokens accrue in different shapes: system prompts duplicated per agent, few-shot examples re-sent, tool schemas inflated in every request.
Worse, agents often run on different models. A router might send lightweight classification to a small model and deep reasoning to a frontier model. Your cost basis is no longer one line item; it’s a tree with uneven branch weights.
Step 1: Propagate a correlation trace
Assign a single trace_id at the entrypoint of a user task and thread it through every agent invocation. Use context variables rather than passing strings manually—it survives async hops and avoids signature pollution.
import contextvars, uuid
trace_id = contextvars.ContextVar("trace_id")
def start_trace() -> str:
tid = uuid.uuid4().hex
trace_id.set(tid)
return tidWhen an agent makes an LLM call, attach the trace and agent name as request headers. Most OpenAI-compatible gateways forward arbitrary x- headers to your logging pipeline.
from openai import OpenAI
client = OpenAI(base_url="https://api.openai.com/v1", api_key="sk-...")
def chat(agent_name: str, model: str, messages: list):
tid = trace_id.get()
return client.chat.completions.create(
model=model,
messages=messages,
extra_headers={"x-trace-id": tid, "x-agent": agent_name}
)Why contextvars beat parameter passing
If you hand trace_id as a function argument through eight layers of agent framework, you will eventually lose it when a library calls the model directly. Contextvars are read globally inside the same execution context, so even a third-party tool executor reports under the right trace.
Step 2: Capture usage at the edge
Do not trust agents to report their own tokens. Wrap the client so every completion response passes through one metering function. This is also where you normalize provider schemas—Anthropic returns usage.input_tokens, OpenAI returns usage.prompt_tokens. Convert to a single shape.
import logging
class MeteringWrapper:
def __init__(self, client):
self.client = client
def complete(self, agent: str, model: str, messages: list):
tid = trace_id.get()
resp = self.client.chat.completions.create(
model=model,
messages=messages,
extra_headers={"x-trace-id": tid, "x-agent": agent}
)
u = resp.usage
logging.info({
"trace": tid,
"agent": agent,
"model": model,
"prompt_tokens": getattr(u, "prompt_tokens", u.input_tokens),
"completion_tokens": getattr(u, "completion_tokens", u.output_tokens),
"total_tokens": getattr(u, "total_tokens", u.input_tokens + u.output_tokens)
})
return respIf you sit behind a gateway such as n4n.ai, its per-token usage metering across 240+ models on one OpenAI-compatible endpoint already returns a normalized usage object, so the wrapper shrinks to a logging call. That is the only place this matters technically; the rest is your aggregation.
What to log besides tokens
Log latency and finish reason. A length finish means you paid for a truncated answer that will trigger a retry—hidden cost. Log the model snapshot too; providers rotate versions under the same name.
Step 3: Attribute cost to agents and tasks
Ship the logs to a store you can group by (trace_id, agent, model). For local debugging, an in-memory counter is enough to see the shape.
from collections import defaultdict
spend = defaultdict(lambda: defaultdict(int))
def record(trace: str, agent: str, total_tokens: int):
spend[trace][agent] += total_tokensToken counts are not dollars. Maintain a price table keyed by model and token type. Use published list prices; they change, so load from config.
# Illustrative public list prices per 1M tokens (verify before use)
PRICING = {
"gpt-4o": {"prompt": 5.0, "completion": 15.0},
"claude-3-5-sonnet": {"prompt": 3.0, "completion": 15.0},
}
def estimate_cost(trace: str, agent_model: dict) -> float:
total = 0.0
for agent, tokens in spend[trace].items():
model = agent_model[agent]
rate = PRICING[model]
# simplified: assume half prompt/half completion
total += tokens * (rate["prompt"] + rate["completion"]) / 2 / 1_000_000
return totalQuerying aggregated spend
In production, push logs to a column store and run:
SELECT trace_id, agent, model, SUM(total_tokens) AS tokens
FROM llm_usage
WHERE ts > now() - interval '1 day'
GROUP BY trace_id, agent, model
ORDER BY tokens DESC;The top rows tell you which agent is the money pit. Usually it’s the one with the 2,000-token system prompt called 40 times per trace.
Step 4: Set guardrails before runaway loops
The classic multi-agent failure is a planner that re-enters on failure and burns 400K tokens debugging a malformed JSON schema. Enforce a per-trace token ceiling in the metering layer.
TRACE_BUDGET_TOKENS = 200_000
def check_budget(trace: str):
used = sum(spend[trace].values())
if used > TRACE_BUDGET_TOKENS:
raise RuntimeError(f"trace {trace} exceeded {TRACE_BUDGET_TOKENS} tokens")Call check_budget after every completion. Pair it with an alert webhook when a trace hits 80% of budget. The pitfall here is setting the budget too low for legitimate deep tasks; profile your top five workflows before picking a number.
Alerting thresholds
Page on absolute spend per minute, not just per trace. A deployment bug can spawn 10,000 tiny traces that individually pass but collectively bankrupt you.
Step 5: Cut spend with caching and routing
Prompt caching is the highest-leverage optimization. System prompts and tool definitions are identical across agent calls—mark them cacheable. Providers that support cache-control hints will discount repeated prefix tokens.
{
"model": "claude-3-5-sonnet",
"messages": [
{
"role": "system",
"content": "You are a sub-agent that extracts entities.",
"cache_control": {"type": "ephemeral"}
}
]
}Route aggressively. A gateway that honors client routing directives lets you force cheap models for trivial agents without code changes. Send a header like x-route: cheap and let the gateway map to an appropriate provider. This keeps token cost multi-agent orchestration predictable when a supervisor misclassifies a task.
Routing tradeoffs
Pinning a sub-agent to a small model saves tokens but can increase retries if the model fails the task. Measure task success rate per route; a 30% retry rate on a cheap model often costs more than using the competent one.
Common pitfalls and tradeoffs
Tool schema bloat. Every function definition is re-tokenized each call. If you give all agents the full 20-tool manifest, you pay for it in every branch. Narrow the toolset per agent.
Async logging lag. Fire-and-forget log sends can drop under load. Use a buffered sender with backpressure; missing a usage record silently understates cost.
Cross-model normalization. A token in one tokenizer is not equal in another. Comparing gpt-4o tokens to llama-3 tokens by count is meaningless; always convert to cost via price table.
Retry storms. Automatic retries on 429s multiply tokens. Use exponential backoff and count retried prompt tokens against the trace. Gateways with automatic fallback when a provider is degraded reduce this, but you still pay for the first failed call.
Estimating vs billed. Your internal meter will differ from the provider invoice by a few percent due to rounding and cached token discounts. Reconcile weekly, not never.
Tracking token cost multi-agent orchestration is fundamentally a tracing problem, not a billing problem. Instrument once at the edge, attribute relentlessly, and cap before you scale. The teams that do this ship agents that cost cents instead of surprises.