What is a FinOps AI agent? It’s a programmatic system that pairs LLM-driven reasoning with direct access to cloud billing, resource, and policy APIs to continuously monitor, explain, and optimize infrastructure spend. Unlike a static dashboard, the agent acts: it opens tickets, modifies autoscaling groups, or triggers Terraform plans inside boundaries set by finance and platform teams.
How a FinOps AI agent works
The core loop is perception, reasoning, action, observation. The agent polls authoritative data sources, reasons over them with a language model that can call tools, executes constrained mutations, and verifies the result. This is not a prompt that returns markdown; it’s a control loop with credentials.
Perception: pulling ground truth
Dashboards lie because they’re stale or aggregated. An agent pulls raw line items. For AWS, that means Cost Explorer and CUR (Cost and Usage Report) via boto3, plus resource metadata from EC2 or EKS APIs. You do not summarize data for the model—you give it the query primitives and let it decide what to fetch.
import boto3
ce = boto3.client("ce")
resp = ce.get_cost_and_usage(
TimePeriod={"Start": "2024-05-01", "End": "2024-06-01"},
Granularity="MONTHLY",
Metrics=["UnblendedCost"],
GroupBy=[{"Type": "DIMENSION", "Key": "SERVICE"}]
)The agent feeds this into the LLM context alongside utilization metrics from CloudWatch or Prometheus. The key is that perception is programmatic, not a human exporting a CSV.
Reasoning: LLM with tools
The model is not an oracle. It’s a planner that emits structured calls. You give it function schemas for read and write operations. The gateway you use should support stable tool-calling across model swaps. For instance, routing through a single OpenAI-compatible endpoint like n4n.ai lets the agent call 240+ models with automatic fallback when a provider is degraded, without rewriting the tool layer.
tools = [{
"type": "function",
"function": {
"name": "resize_eks_nodegroup",
"description": "Change min/max nodes for an EKS managed node group",
"parameters": {
"type": "object",
"properties": {
"cluster": {"type": "string"},
"nodegroup": {"type": "string"},
"min": {"type": "integer"},
"max": {"type": "integer"}
}
}
}
}]The system prompt enforces policy: “Never resize below min=1 without explicit approval. Always post a Slack message before mutating.” The LLM decides when to call, the code decides whether to execute.
Action: constrained API calls
The agent executes only tools marked safe or approved. Write actions go through a policy engine (e.g., OPA) that checks the proposed diff against guardrails. Never let the model call AWS directly with admin keys; broker it.
def execute_tool(name, args):
if not policy_allows(name, args):
return {"error": "blocked by guardrail"}
if name == "resize_eks_nodegroup":
eks = boto3.client("eks")
eks.update_nodegroup_config(**args)
return {"status": "submitted"}Observation: verify and log
After the call, the agent queries the resource again to confirm state. Every step is appended to an immutable audit log. If cost drops as predicted, the loop closes. If not, it rolls back or escalates. In practice, the agent runs on a cron or event trigger—every hour, or on receipt of a new CUR file. It maintains a rolling window of findings in a vector store so the LLM can reference past decisions and avoid repeating them.
Why engineering teams should care
Cloud waste is not a finance problem; it’s an engineering velocity problem. When a team waits two weeks for a monthly invoice to spot a misconfigured NAT gateway, they’ve already burned cash and context. A FinOps AI agent shrinks that feedback loop to minutes.
It also removes the human bottleneck in tedious cleanup. Engineers hate tagging resources. The agent does it via API and validates compliance. Finance gets real-time accruals instead of forensic reports.
The alternative—manual scripts—breaks because they can’t handle novel situations. An LLM agent adapts when a new instance type appears or a discount plan changes; a hardcoded rule doesn’t. Organizational asymmetry makes this worse: finance sees cost, engineering sees latency, and neither has time to reconcile. The agent speaks both languages because it reads both APIs.
Concrete example: reclaiming idle GPU capacity
A platform team runs a shared EKS node group with four p3.2xlarge GPUs for batch training. Usage telemetry shows 4.8% average GPU utilization over 14 days. The monthly blended cost is roughly $2,100 per node.
The agent’s perception step pulls CloudWatch GPU metrics and Cost Explorer. Its reasoning step emits a finding:
{
"finding": "idle_gpu_nodes",
"resources": ["arn:aws:eks:us-east-1:123:nodegroup/train-gpu"],
"monthly_cost_usd": 8400,
"utilization_avg_pct": 4.8,
"recommended_action": "downsize_nodegroup",
"proposed_min": 0,
"proposed_max": 2,
"confidence": 0.94
}Because the guardrail requires human approval for node group deletion, the agent posts to #finops-approvals with a Terraform plan diff:
resource "aws_eks_node_group" "train_gpu" {
cluster_name = "prod"
node_group_name = "train-gpu"
# current: scaling_config { min_size=4 max_size=4 }
# proposed: scaling_config { min_size=0 max_size=2 }
}An on-call engineer clicks approve. The agent calls resize_eks_nodegroup with min=0, max=2. Next day it verifies the billable node count and logs savings. No one wrote a custom Python script for p3.2xlarge; the agent reused generic tools and reasoned about the specific metric.
Common misconceptions
It’s just a chatbot with cost data
Wrong. A chatbot answers “how much did we spend?” An agent calls modify_db_instance to stop a replica. The LLM is the control plane, not the interface. If it can’t mutate infrastructure, it’s a reporter, not an agent.
It replaces FinOps staff
No. It executes the policies those staff define. The agent can’t decide that growth-stage burn is acceptable; a human sets the threshold. It automates the mechanical layer—tagging, rightsizing, orphan detection—and escalates judgment calls.
It requires custom model training
Unnecessary. Retrieval-augmented prompts plus tool schemas beat fine-tuning for this domain. The knowledge is in your APIs, not in model weights. Use a general model with good tool adherence; swap it when a cheaper one clears your eval bar.
It’s only for hyperscalers
Any metered API qualifies. Snowflake credits, OpenAI token spend, Datadog seats—all expose usage endpoints. A FinOps AI agent can govern them uniformly because the pattern is identical: pull usage, reason, act within policy.
Guardrails you must implement
- Dry-run first. Every write tool has a shadow mode that logs but doesn’t mutate. Run in shadow for a week before enabling actions.
- Policy as code. Express limits in Rego or Cedar. The agent queries it before acting; never trust the prompt alone.
- Human-in-the-loop for blast radius. Anything that terminates stateful resources needs explicit approval via signed message.
- Audit trail. Store each LLM completion, tool call, and API response. You’ll need it when finance asks why a discount was dropped or an instance vanished.
- Cost ceiling on the agent itself. The agent spends tokens. Use per-token metering and alert if its own reasoning cost exceeds a daily cap.
When not to build one
If your monthly cloud bill is under $2k, a spreadsheet beats an agent. If you have no tagging discipline, fix that first—the agent will just amplify chaos. And if your org can’t agree on who approves changes, the agent will stall in pending state.
Minimal stack to build one
- LLM endpoint – an OpenAI-compatible gateway with fallback and per-token metering. Avoid pinning to one vendor so you can chase price/performance.
- Cloud SDKs – boto3, azure-mgmt, gcloud bindings.
- Orchestrator – a small state machine (LangGraph, Temporal, or plain Python) that drives the perceive-reason-act loop.
- Policy engine – OPA or similar.
- Messaging – Slack or MS Teams for approvals.
Start with read-only agents. Let it flag wasted spend for a sprint before granting mutation rights. The first win is usually orphaned EBS volumes; the agent finds them faster than any spreadsheet.
Building a FinOps AI agent is less about ML and more about disciplined API plumbing. The model provides judgment; your code provides safety.