I Built an AWS Cost Anomaly Detector — Here's What I Learned

The $3,000 Surprise

Last quarter, our AWS bill jumped 40% overnight. Nobody noticed for two weeks. By the time someone flagged it in the monthly finance review, we’d already burned through $3,000 more than expected — all because of an orphaned NAT Gateway in a staging VPC that nobody remembered creating.

That was the moment I decided to build CostSentry: an open-source tool that monitors your AWS spending and flags anomalies before they compound into budget disasters.

How It Works

CostSentry pulls daily cost data from the AWS Cost Explorer API, breaks it down by service and account, and runs a simple statistical model to detect deviations from your rolling 30-day baseline. When something looks off — say, a 25% spike in EC2 spend that wasn’t there yesterday — it fires a Slack notification with context: which service, which account, and the estimated monthly impact if the trend continues.

The ML layer is intentionally simple. We’re not doing deep learning here. A z-score model with dynamic thresholds covers 90% of real-world anomalies. The remaining 10% — gradual drift, seasonal patterns — gets handled by a lightweight Prophet-based forecasting module that runs weekly.

Lessons Learned

Building a cost monitoring tool taught me more about AWS billing than years of reading invoices ever did. The Cost Explorer API has quirks — data is delayed by 24-48 hours, linked accounts require specific IAM permissions, and Reserved Instance amortization can throw off your baselines if you don’t account for it.

The biggest lesson? Most cost anomalies aren’t dramatic spikes. They’re slow leaks — a forgotten dev environment here, an oversized RDS instance there. CostSentry’s daily digest catches those too, and that’s where most of the savings come from.