AgentsAWSAWS Cost Anomaly Investigation

AWS Cost Anomaly Investigation

Investigate unexpected AWS cost spikes by analyzing Cost Explorer data, tag drift, and resource lifecycle events.

Tested against aws-cli 2.x · copilot-chat · windsurf-1.6+

DIRECTIVE / aws-cost-anomaly-investigation
You are an AWS FinOps engineer. Your task is to investigate a cost spike, find the billing attribution and the driving service, and produce evidence-backed findings.

Context:
- You have read access to Cost Explorer and tagging APIs.
- A budget or anomaly alert fired and you need to explain the change in spend.
- You will not create cost budgets or modify resources.

Steps:
1. Quantify the spike: use aws ce get-cost-and-usage with a 30-day granularity=DAILY, filtered to the anomalous time window, and group by SERVICE to identify the top contributor.
2. Drill into the top service: run the same query grouped by USAGE_TYPE and by a dim like LINKED_ACCOUNT to find where the money is going.
3. Check tag changes: run get-resources for the relevant tags over time, and list any untagged resources (an untagged instance roams the bill).
4. Look for lifecycle causes: aws ec2 describe-instances for launched instances, aws autoscaling describe-auto-scaling-groups for scale events, and aws cloudtrail lookup-events for CreateVolume or RunInstances in the window.
5. Cross-reference with metric events: note data transfer, NAT gateway, or data egress that correlates with the spike.

Output format:
- Top-5 cost contributors table (service, usage type, delta $, % of spike).
- Root cause hypothesis with the strongest evidence, and a secondary hypothesis if attribution is ambiguous.
- A recommended next action (for example, add a tag or right-size an instance) expressed as a suggestion.

Constraints:
- Read-only; no billing or resource changes.
- If grouping is ambiguous, say so; do not fabricate a single cause.
- Focus on the single highest contributor unless multiple exceed 20% of the delta.

Example:
Input: daily cost jumped $420 on /03; group by service shows EC2-Other (data transfer) up 85%.
Output: top: EC2-Other data transfer +$360 (86%); evidence: NAT gateway egress climbed alongside a new ASG launch on /02; next: add a tag to the new ASG and enable a usage-type budget.

Why this directive matters

A surprise AWS bill is rarely one thing; it is the sum of a change someone made and a billing attribution so poor that the change is invisible. Cost Explorer is the right tool, but using it to answer "why is this spiking" takes a disciplined set of grouped queries rather than one glance at the total. This directive walks the agent from the aggregate to the single line item: a daily, then a service, then a usage-type and account grouping over the anomalous window. It then looks for the physical cause in tag setup, instance launches, autoscaler events, and CloudTrail activity, because a cost spike almost always has a creation event behind it. The agent is asked to name the top contributors, offer a root-cause hypothesis with evidence, and separate findings that are certain from those that are ambiguous. The result is a cost investigation that a FinOps lead can act on instead of a spreadsheet of numbers nobody has parsed.

Test Command

aws ce get-cost-and-usage --time-period Start=$(date -d '-14 days' +%F),End=$(date +%F) --granularity DAILY --metrics UnblendedCost --group-by Type=DIMENSION,Key=SERVICE

Prerequisites

  • AWS CLI with Cost Explorer read access
  • ce:Get* and ec2:Describe permissions
  • A cost/budget alert context

Expected Outputs

  • Top-5 contributor table
  • Root-cause hypothesis
  • Evidence-backed findings

References

THE NEXT STEP

Automate it past the prompt.

This intermediate directive is a manual, read-only run. Devopsify can run the same check continuously across your estate, with policy gates, approvals, and a retained audit trail.