AWS Cost Optimization & FinOps

Systematic workflows for AWS cost optimization and financial operations management.

When to Use This Skill

Use this skill when you need to:

•Find cost savings: Identify unused resources, rightsizing opportunities, or commitment discounts
•Analyze spending: Understand cost trends, detect anomalies, or break down costs
•Optimize architecture: Choose cost-effective services, storage tiers, or instance types
•Implement FinOps: Set up governance, tagging, budgets, or monthly reviews
•Make purchase decisions: Evaluate Reserved Instances, Savings Plans, or Spot instances
•Troubleshoot costs: Investigate unexpected bills or cost spikes
•Plan budgets: Forecast costs or evaluate impact of new projects

Cost Optimization Workflow

Follow this systematic approach for AWS cost optimization:

code

┌─────────────────────────────────────────────┐
│ 1. DISCOVER                                 │
│    What are we spending money on?           │
│    Run: find_unused_resources.py            │
│    Run: cost_anomaly_detector.py            │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│ 2. ANALYZE                                  │
│    Where are the optimization opportunities?│
│    Run: rightsizing_analyzer.py             │
│    Run: detect_old_generations.py           │
│    Run: spot_recommendations.py             │
│    Run: analyze_ri_recommendations.py       │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│ 3. PRIORITIZE                               │
│    What should we optimize first?           │
│    - Quick wins (low risk, high savings)    │
│    - Low-hanging fruit (easy to implement)  │
│    - Strategic improvements                 │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│ 4. IMPLEMENT                                │
│    Execute optimization actions             │
│    - Delete unused resources                │
│    - Rightsize instances                    │
│    - Purchase commitments                   │
│    - Migrate to new generations             │
└─────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│ 5. MONITOR                                  │
│    Verify savings and track metrics         │
│    - Monthly cost reviews                   │
│    - Tag compliance monitoring              │
│    - Budget variance tracking               │
└─────────────────────────────────────────────┘

Core Workflows

Workflow 1: Monthly Cost Optimization Review

Frequency: Run monthly (first week of each month)

Step 1: Find Unused Resources

bash

# Scan for waste across all resources
python3 scripts/find_unused_resources.py

# Expected output:
# - Unattached EBS volumes
# - Old snapshots
# - Unused Elastic IPs
# - Idle NAT Gateways
# - Idle EC2 instances
# - Unused load balancers
# - Estimated monthly savings

Step 2: Analyze Cost Anomalies

bash

# Detect unusual spending patterns
python3 scripts/cost_anomaly_detector.py --days 30

# Expected output:
# - Cost spikes and anomalies
# - Top cost drivers
# - Period-over-period comparison
# - 30-day forecast

Step 3: Identify Rightsizing Opportunities

bash

# Find oversized instances
python3 scripts/rightsizing_analyzer.py --days 30

# Expected output:
# - EC2 instances with low utilization
# - RDS instances with low utilization
# - Recommended smaller instance types
# - Estimated savings

Step 4: Generate Monthly Report

bash

# Use the template to compile findings
cp assets/templates/monthly_cost_report.md reports/$(date +%Y-%m)-cost-report.md

# Fill in:
# - Findings from scripts
# - Action items
# - Team cost breakdowns
# - Optimization wins

Step 5: Team Review Meeting

•Present findings to engineering teams
•Assign optimization tasks
•Track action items to completion

Workflow 2: Commitment Purchase Analysis (RI/Savings Plans)

When: Quarterly or when usage patterns stabilize

Step 1: Analyze Current Usage

bash

# Identify workloads suitable for commitments
python3 scripts/analyze_ri_recommendations.py --days 60

# Looks for:
# - EC2 instances running consistently for 60+ days
# - RDS instances with stable usage
# - Calculates ROI for 1yr vs 3yr commitments

Step 2: Review Recommendations

Evaluate each recommendation:

code

✅ Good candidate if:
  - Running 24/7 for 60+ days
  - Workload is stable and predictable
  - No plans to change architecture
  - Savings > 30%

❌ Poor candidate if:
  - Workload is variable or experimental
  - Architecture changes planned
  - Instance type may change
  - Dev/test environment

Step 3: Choose Commitment Type

Reserved Instances:

•Standard RI: Highest discount (63%), no flexibility
•Convertible RI: Moderate discount (54%), can change instance type
•Best for: Specific instance types, stable workloads

Savings Plans:

•Compute SP: Flexible across instance types, regions (66% savings)
•EC2 Instance SP: Flexible across sizes in same family (72% savings)
•Best for: Variable workloads within constraints

Decision Matrix:

code

Known instance type, won't change → Standard RI
May need to change types → Convertible RI or Compute SP
Variable workloads → Compute Savings Plan
Maximum flexibility → Compute Savings Plan

Step 4: Purchase and Track

•Purchase through AWS Console or CLI
•Tag commitments with purchase date and owner
•Monitor utilization monthly
•Aim for >90% utilization

Reference: See references/best_practices.md for detailed commitment strategies

Workflow 3: Instance Generation Migration

When: During architecture reviews or optimization sprints

Step 1: Detect Old Instances

bash

# Find outdated instance generations
python3 scripts/detect_old_generations.py

# Identifies:
# - t2 → t3 migrations (10% savings)
# - m4 → m5 → m6i migrations
# - Intel → Graviton opportunities (20% savings)

Step 2: Prioritize Migrations

Quick Wins (Low Risk):

code

t2 → t3: Drop-in replacement, 10% savings
m4 → m5: Better performance, 5% savings
gp2 → gp3: No downtime, 20% savings

Medium Effort (Test Required):

code

x86 → Graviton (ARM64): 20% savings
- Requires ARM64 compatibility testing
- Most modern frameworks support ARM64
- Test in staging first

Step 3: Execute Migration

For EC2 (x86 to x86):

•Stop instance
•Change instance type
•Start instance
•Verify application

For Graviton Migration:

•Create ARM64 AMI or Docker image
•Launch new Graviton instance
•Test thoroughly
•Cut over traffic
•Terminate old instance

Step 4: Validate Savings

•Monitor new costs in Cost Explorer
•Verify performance is acceptable
•Document migration for other teams

Reference: See references/best_practices.md → Compute Optimization

Workflow 4: Spot Instance Evaluation

When: For fault-tolerant workloads or Auto Scaling Groups

Step 1: Identify Candidates

bash

# Analyze workloads for Spot suitability
python3 scripts/spot_recommendations.py

# Evaluates:
# - Instances in Auto Scaling Groups (good candidates)
# - Dev/test/staging environments
# - Batch processing workloads
# - CI/CD and build servers

Step 2: Assess Suitability

Excellent for Spot:

•Stateless applications
•Batch jobs
•CI/CD pipelines
•Data processing
•Auto Scaling Groups

NOT suitable for Spot:

•Databases (without replicas)
•Stateful applications
•Real-time services
•Mission-critical workloads

Step 3: Implementation Strategy

Option 1: Fargate Spot (Easiest)

yaml

# ECS task definition
requiresCompatibilities:
  - FARGATE
capacityProviderStrategy:
  - capacityProvider: FARGATE_SPOT
    weight: 70  # 70% Spot
  - capacityProvider: FARGATE
    weight: 30  # 30% On-Demand

Option 2: EC2 Auto Scaling with Spot

yaml

# Mixed instances policy
MixedInstancesPolicy:
  InstancesDistribution:
    OnDemandBaseCapacity: 2
    OnDemandPercentageAboveBaseCapacity: 30
    SpotAllocationStrategy: capacity-optimized
  LaunchTemplate:
    Overrides:
      - InstanceType: m5.large
      - InstanceType: m5a.large
      - InstanceType: m5n.large

Option 3: EC2 Spot Fleet

bash

# Create Spot Fleet with diverse instance types
aws ec2 request-spot-fleet --spot-fleet-request-config file://spot-fleet.json

Step 4: Implement Interruption Handling

bash

# Handle 2-minute termination notice
# Instance metadata: /latest/meta-data/spot/instance-action

# In application:
1. Poll for termination notice
2. Gracefully shutdown (save state)
3. Drain connections
4. Exit

Reference: See references/best_practices.md → Compute Optimization → Spot Instances

Quick Reference: Cost Optimization Scripts

All Scripts Location

bash

ls scripts/
# find_unused_resources.py
# analyze_ri_recommendations.py
# detect_old_generations.py
# spot_recommendations.py
# rightsizing_analyzer.py
# cost_anomaly_detector.py

Script Usage Patterns

Monthly Review (Run all):

bash

python3 scripts/find_unused_resources.py
python3 scripts/cost_anomaly_detector.py --days 30
python3 scripts/rightsizing_analyzer.py --days 30

Quarterly Optimization:

bash

python3 scripts/analyze_ri_recommendations.py --days 60
python3 scripts/detect_old_generations.py
python3 scripts/spot_recommendations.py

Specific Region Only:

bash

python3 scripts/find_unused_resources.py --region us-east-1
python3 scripts/rightsizing_analyzer.py --region us-west-2

Named AWS Profile:

bash

python3 scripts/find_unused_resources.py --profile production
python3 scripts/cost_anomaly_detector.py --profile production --days 60

Script Requirements

bash

# Install dependencies
pip install boto3 tabulate

# AWS credentials required
# Configure via: aws configure
# Or use: --profile PROFILE_NAME

Service-Specific Optimization

Compute Optimization

Key Actions:

•Migrate to Graviton (20% savings)
•Use Spot for fault-tolerant workloads (70% savings)
•Purchase RIs for stable workloads (40-65% savings)
•Right-size oversized instances

Reference: references/best_practices.md → Compute Optimization

Storage Optimization

Key Actions:

•Convert gp2 → gp3 (20% savings)
•Implement S3 lifecycle policies (50-95% savings)
•Delete old snapshots
•Use S3 Intelligent-Tiering

Reference: references/best_practices.md → Storage Optimization

Network Optimization

Key Actions:

•Replace NAT Gateways with VPC Endpoints (save $25-30/month each)
•Use CloudFront to reduce data transfer costs
•Colocate resources in same AZ when possible

Reference: references/best_practices.md → Network Optimization

Database Optimization

Key Actions:

•Right-size RDS instances
•Use gp3 storage (20% cheaper than gp2)
•Evaluate Aurora Serverless for variable workloads
•Purchase RDS Reserved Instances

Reference: references/best_practices.md → Database Optimization

Service Alternatives Decision Guide

Need help choosing between services?

Question: "Should I use EC2, Lambda, or Fargate?" Answer: See references/service_alternatives.md → Compute Alternatives

Question: "Which S3 storage class should I use?" Answer: See references/service_alternatives.md → Storage Alternatives

Question: "Should I use RDS or Aurora?" Answer: See references/service_alternatives.md → Database Alternatives

Question: "NAT Gateway vs VPC Endpoint vs NAT Instance?" Answer: See references/service_alternatives.md → Networking Alternatives

FinOps Governance & Process

Setting Up FinOps

Phase 1: Foundation (Month 1)

•Enable Cost Explorer
•Set up AWS Budgets
•Define tagging strategy
•Activate cost allocation tags

Phase 2: Visibility (Months 2-3)

•Implement tagging enforcement
•Run optimization scripts
•Set up monthly reviews
•Create team cost reports

Phase 3: Culture (Ongoing)

•Cost metrics in engineering KPIs
•Cost review in architecture decisions
•Regular optimization sprints
•FinOps champions in each team

Full Guide: See references/finops_governance.md

Monthly Review Process

Week 1: Data Collection

•Run all optimization scripts
•Export Cost & Usage Reports
•Compile findings

Week 2: Analysis

•Identify trends
•Find opportunities
•Prioritize actions

Week 3: Team Reviews

•Present to engineering teams
•Discuss optimizations
•Assign action items

Week 4: Executive Reporting

•Create executive summary
•Forecast next quarter
•Report optimization wins

Template: See assets/templates/monthly_cost_report.md

Detailed Process: See references/finops_governance.md → Monthly Review Process

Cost Optimization Checklist

Quick Wins (Do First)

• Delete unattached EBS volumes
• Delete old EBS snapshots (>90 days)
• Release unused Elastic IPs
• Convert gp2 → gp3 volumes
• Stop/terminate idle EC2 instances
• Enable S3 Intelligent-Tiering
• Set up AWS Budgets and alerts

Medium Effort (This Quarter)

• Right-size oversized instances
• Migrate to newer instance generations
• Purchase Reserved Instances for stable workloads
• Implement S3 lifecycle policies
• Replace NAT Gateways with VPC Endpoints (where applicable)
• Enable automated resource scheduling (dev/test)
• Implement tagging strategy and enforcement

Strategic Initiatives (Ongoing)

• Migrate to Graviton instances
• Implement Spot for fault-tolerant workloads
• Establish monthly cost review process
• Set up cost allocation by team
• Implement chargeback/showback model
• Create FinOps culture and practices

Troubleshooting Cost Issues

"My bill suddenly increased"

•

Run cost anomaly detection:

bash

python3 scripts/cost_anomaly_detector.py --days 30

•
Check Cost Explorer for service breakdown
•
Review CloudTrail for resource creation events
•
Check for AutoScaling events
•
Verify no Reserved Instances expired

"I need to reduce costs by X%"

Follow the optimization workflow:

•Run all discovery scripts
•Calculate total potential savings
•Prioritize by: Savings Amount × (1 / Effort)
•Focus on quick wins first
•Implement strategic changes for long-term

"How do I know if Reserved Instances make sense?"

Run RI analysis:

bash

python3 scripts/analyze_ri_recommendations.py --days 60

Look for:

•Instances running 60+ days consistently
•Workloads that won't change
•Savings > 30%

"Which resources can I safely delete?"

Run unused resource finder:

bash

python3 scripts/find_unused_resources.py

Safe to delete (usually):

•Unattached EBS volumes (after verifying)
•Snapshots > 90 days (if backups exist elsewhere)
•Unused Elastic IPs (after verifying not in DNS)
•Stopped EC2 instances > 30 days (after confirming abandoned)

Always verify with resource owner before deletion!

Best Practices Summary

•Tag Everything: Consistent tagging enables cost allocation and accountability
•Monitor Continuously: Weekly script runs catch waste early
•Review Monthly: Regular reviews prevent cost drift
•Right-size Proactively: Don't wait for cost issues to optimize
•Use Commitments Wisely: RIs/SPs for stable workloads only
•Test Before Migrating: Especially for Graviton or Spot
•Automate Cleanup: Scheduled shutdown of dev/test resources
•Share Wins: Celebrate cost savings to build FinOps culture

Additional Resources

Detailed References:

•references/best_practices.md: Comprehensive optimization strategies
•references/service_alternatives.md: Cost-effective service selection
•references/finops_governance.md: Organizational FinOps practices

Templates:

•assets/templates/monthly_cost_report.md: Monthly reporting template

Scripts:

•All scripts in scripts/ directory with --help for usage

AWS Documentation:

•AWS Cost Explorer: https://aws.amazon.com/aws-cost-management/aws-cost-explorer/
•AWS Budgets: https://aws.amazon.com/aws-cost-management/aws-budgets/
•FinOps Foundation: https://www.finops.org