CFOs do not approve line items based on demos. The CFO questions AI agent budget will center on unit economics, failure cost, and auditability. This guide lays out an ordered path to answer those questions with instrumented data instead of vendor slides.
1. Inventory every agent’s token path
Start by listing each agent, the model it calls, and the full token footprint per run. Most teams underestimate because they count only the user message. You also pay for the system prompt, retrieved context, tool schemas, and generated output.
agents = {
"support_triage": {
"model": "gpt-4o-mini",
"avg_input_tokens": 2400, # system + rag chunks + conversation
"avg_output_tokens": 350,
"runs_per_day": 12000,
},
"code_review": {
"model": "claude-3.5-sonnet",
"avg_input_tokens": 8000,
"avg_output_tokens": 1200,
"runs_per_day": 800,
},
}Turn that into a monthly projection with your negotiated rates:
def monthly_token_cost(agents, price_per_1k_in, price_per_1k_out):
total = 0.0
for cfg in agents.values():
in_cost = cfg["avg_input_tokens"] / 1000 * cfg["runs_per_day"] * 30 * price_per_1k_in
out_cost = cfg["avg_output_tokens"] / 1000 * cfg["runs_per_day"] * 30 * price_per_1k_out
total += in_cost + out_cost
return totalThe pitfall here is ignoring cache discounts. Providers like Anthropic and OpenAI charge less for cached input tokens if you send cache_control markers. If you skip that, your projection is 20–40% low. The tradeoff is real: stuffing more context improves accuracy but scales cost linearly. Measure both.
2. Price the failure modes, not just the happy path
The CFO questions AI agent budget always include “what happens when the primary model is rate-limited or degraded?” Automatic fallback sounds like a free insurance policy. It is not. When your router shifts from a $0.10/1M token model to a $3/1M token model, margin evaporates.
Define routing explicitly so finance can see the cost tiers:
{
"model": "router/auto",
"messages": [{"role": "user", "content": "..."}],
"route": {
"prefer": ["groq/llama-3.1-8b"],
"fallback": ["openai/gpt-4o-mini", "anthropic/claude-3.5-haiku"]
},
"cache_control": {"type": "ephemeral"}
}A gateway such as n4n.ai honors client routing directives and forwards provider cache-control hints, which keeps fallback accounting transparent. You still must meter which model actually served the request.
Common pitfall: assuming fallback triggers are rare. In production, a single provider outage can route 30% of daily traffic to premium models for hours. Build a contingency line item for fallback spikes. The tradeoff is reliability versus margin—pick a fallback chain that stays within 3x your base model cost.
3. Instrument per-token metering from day one
If you cannot measure token consumption per agent, you cannot defend the budget. Use the usage field from any OpenAI-compatible endpoint. Do not aggregate only successes.
from openai import OpenAI
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-...")
resp = client.chat.completions.create(
model="router/auto",
messages=[{"role": "user", "content": "Summarize ticket #8821"}]
)
print(resp.usage.model_dump())
# {'prompt_tokens': 2410, 'completion_tokens': 312, 'total_tokens': 2722}Log every call to a structured table:
| agent | model | prompt_tokens | completion_tokens | cost_cents | fallback | ts |
|---|
Include errors that returned partial usage. Some providers charge for input tokens even on 429s. Exclude them and your CFO will spot the gap in the first audit. Meter at the edge, not just in billing exports, so you can attribute cost to a feature flag or customer tier.
4. Build a unit-economics model the finance team can audit
The CFO questions AI agent budget want payback period and cost per resolved unit. A spreadsheet is fine; a small Python script is better because it is version-controlled.
Separate three cost buckets:
- Variable: token cost per run (steps 1–2).
- Fixed: vector store, compute, gateway subscription.
- Contingency: 15% buffer for fallback and retry storms.
base_run_cost = (2400/1000*0.00015) + (350/1000*0.0006) # plug real numbers
monthly_var = base_run_cost * 12000 * 30
fixed = 400.0 # infra
contingency = monthly_var * 0.15
projected = monthly_var + fixed + contingency
# Compare to replaced labor
fte_savings = 2 * 6000 # $/mo
roi_months = projected / (fte_savings - projected)Do not fake prices. Use your contract rates. The tradeoff: if agent accuracy is 85%, the 15% that need human redo incur double token cost plus labor. Add a retry multiplier of 1.2–1.5 to reflect reality.
5. Present a capped pilot with kill switches
Never ask for unbounded spend on day one. Propose a 6-week pilot with hard caps wired into the gateway:
export AGENT_MAX_TOKENS_PER_RUN=12000
export AGENT_DAILY_SPEND_LIMIT_USD=50
export FALLBACK_MODELS="openai/gpt-4o-mini"If the limit hits, the agent returns a safe canned response instead of burning budget. Alert at 80% via your existing monitoring:
if daily_spend_usd > 40:
slack.post("#finops", f"Agent spend at {daily_spend_usd}, throttle engaged")Pitfall: no rollback plan. If the pilot shows 10x cost overrun, you need a flag to disable the agent per route without a code deploy. Tradeoff: tight caps protect budget but may degrade UX during traffic spikes. Calibrate caps from step 3 actuals, not guesses.
6. Report actuals against projections monthly
Variance analysis is what keeps the budget alive. Produce a one-page memo:
- Projected vs actual token count per agent.
- Fallback rate and incremental cost.
- Cost per resolved ticket vs baseline.
If fallback exceeded 5% of runs, explain the root cause. Common culprits: a misconfigured route.prefer list, or a provider quietly deprecating a model. The CFO wants a process response: you adjusted routing or added a cache layer, not an apology.
The CFO questions AI agent budget are solvable with the same discipline you apply to cloud compute. Meter every token, cap every experiment, and review variance in the open. Treat agent spend as a measurable variable cost, and the approval conversation shifts from faith to finance.