Building an AI FinOps agent cloud cost optimization loop is not about another dashboard. It is about closing the gap between detected waste and a merged fix. This guide gives an ordered path you can ship in a week using real cloud APIs and a small LLM-driven agent that opens pull requests instead of writing slide decks.
1. Inventory every dollar with enforced tagging
Cloud cost leaks start with resources nobody owns. Before any agent can act, you need a queryable inventory where every billable resource carries a cost-center tag.
Run a daily scan for untagged assets. Below is a boto3 check for EC2 instances missing cost-center:
import boto3
ec2 = boto3.client("ec2")
resp = ec2.describe_instances()
untagged = []
for res in resp["Reservations"]:
for inst in res["Instances"]:
tags = {t["Key"]: t["Value"] for t in inst.get("Tags", [])}
if "cost-center" not in tags:
untagged.append(inst["InstanceId"])
print(untagged)On GCP, the equivalent is a label check:
gcloud compute instances list --format="table(name,labels)" | grep -v cost-centerPipe the output to a Slack alert and a DynamoDB table. The agent will later read that table to scope its proposals.
Pitfall: tag enforcement via IAM policy denies creation, but legacy resources stay untagged. Backfill with an SCP or a Lambda that tags defaults; otherwise your AI FinOps agent cloud cost optimization will keep tripping over blind spots.
2. Stream cost and utilization telemetry into one store
Effective AI FinOps agent cloud cost optimization requires joined data, not separate billing and metrics tabs. Billing CSVs alone hide waste. You need utilization (CPU, memory, network) joined with cost. Export the AWS Cost and Usage Report (CUR) to S3, then query it locally with DuckDB to avoid a warehouse bill.
import duckdb
duckdb.sql("""
COPY (
SELECT line_item_resource_id, sum(line_item_unblended_cost) as cost
FROM 's3://my-cur-bucket/*/*.parquet'
GROUP BY 1 ORDER BY cost DESC LIMIT 50
) TO 'top_costly_resources.json' (FORMAT JSON)
""")For utilization, pull CloudWatch metrics into the same DuckDB instance. The join key is resource ID.
# pseudocode for metric join
metrics = cloudwatch.get_metric_data(
MetricDataQueries=[{
"Id": "cpu",
"MetricStat": {"Metric": {"Namespace": "AWS/EC2", "MetricName": "CPUUtilization"},
"Period": 86400, "Stat": "Average"}}
],
StartTime="-14d", EndTime="now"
)
duckdb.sql("INSERT INTO util SELECT resource_id, avg(cpu) FROM metrics GROUP BY 1")Now you have a single SQL surface to ask: “Which resources cost > $100/mo and averaged <5% CPU last 14 days?” That query becomes the agent’s primary signal.
3. Encode optimization rules as policy, not prompts
Do not let the LLM invent savings rules at runtime. Define them as versioned JSON so you can review and roll back. Example policy for idle RDS:
{
"policy_id": "idle-rds-shutdown",
"match": {
"service": "rds",
"metric": "CPUUtilization",
"window_days": 14,
"avg_max_percent": 5
},
"action": {
"type": "stop_instance",
"schedule": "off_hours_only"
},
"risk": "medium"
}The agent loads these policies, evaluates them against the telemetry store, and emits a structured diff. Keeping rules in code means the AI FinOps agent cloud cost optimization is auditable, not a black box.
A minimal evaluator:
def evaluate(policy, db):
return db.sql(f"""
SELECT resource_id FROM util
WHERE service = '{policy['match']['service']}'
AND avg_cpu < {policy['match']['avg_max_percent']}
AND window_days >= {policy['match']['window_days']}
""").df()4. Wire an agent that proposes diffs, not advice
The agent’s job is to open a PR with a Terraform or CLI change. It retrieves matched resources, renders a patch, and asks an LLM to draft the human-readable rationale and double-check the policy applicability.
Use an OpenAI-compatible client. Pointing it at a gateway such as n4n.ai gives you 240+ models with automatic fallback when a provider is degraded, so the nightly run does not stall on a single vendor’s 429s.
from openai import OpenAI
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="YOUR_KEY"
)
def draft_pr_body(resource_id, policy):
resp = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[
{"role": "system", "content": "You write concise FinOps PR descriptions."},
{"role": "user", "content": f"Resource {resource_id} matches {policy['policy_id']}. Suggest shutdown step."}
]
)
return resp.choices[0].message.contentThe agent then uses the GitHub API to create a branch, commit a Terraform edit, and open the PR with that body. No auto-merge.
from github import Github
g = Github("TOKEN")
repo = g.get_repo("org/infra")
ref = repo.create_git_ref(f"refs/heads/finops-{resource_id}",
repo.get_branch("main").commit.sha)
# commit modified terraform file, then:
repo.create_pull(title=f"FinOps: stop {resource_id}",
body=draft_pr_body(rid, policy),
head=f"finops-{resource_id}", base="main")Tradeoff: model calls cost tokens. For 10k resources nightly, batch evaluations and only call the LLM for the top 20 candidates. The per-token metering on the gateway keeps that predictable.
5. Apply changes through review, then measure
Route PRs to the owning cost-center team. Require one approval. Once merged, the agent watches the next CUR drop and confirms savings.
Close the loop by writing realized savings back to the policy store:
duckdb.sql("""
UPDATE policy_runs
SET realized_savings = (SELECT cost FROM prev_month WHERE resource_id = $id)
WHERE resource_id = $id
""")Without this feedback, AI FinOps agent cloud cost optimization decays as architectures drift. Run a canary: apply to non-prod first, verify no alert storms, then expand.
Common pitfalls
- Tag sprawl: 40 tag keys with no owner. Pick three: cost-center, env, service.
- Over-broad IAM: The agent needs stop/modify, not delete. Scope to specific resource ARNs.
- Alert fatigue: If the agent opens 200 PRs on day one, teams mute it. Ramp with a daily cap of 5.
- Metric lag: CloudWatch retains 14 days at 1-min; CUR lags 8 hours. Design for stale data.
- Policy drift: A policy written for RDS misses Aurora Serverless. Review matches quarterly.
Tradeoffs
- Autonomy vs safety: Full auto-apply saves time but risks a bad shutdown. Start at PR-only for a quarter.
- Model choice: A small model drafts PR text fine; a larger one catches policy edge cases. Route by task using client routing directives.
- Coverage vs noise: Scanning every region multiplies findings but also false positives. Begin with one high-spend account.
- Storage cost: DuckDB local is cheap but loses history; a partitioned Parquet lake costs S3 pennies and scales.
Ship the pipeline as code, run it on a cron, and let the agent handle the tedious first draft of every optimization. The savings come from the merge, not the model.