Optimizing AWS Costs and Capacity Planning with Wavefront

Wavefront can be used as a tool to help plan capacity and optimize AWS costs. We’ll use Wavefront’s AWS integration, which allows us to get AWS billing and EC2 data. Let’s go through some examples of useful charts that can be created to accomplish these goals. We will look at sample billing data from a Wavefront development environment.

This query shows a stacked chart of our AWS costs. As we can see, most of the costs, in orange, are from EC2. There’s a sawtooth pattern because, at the beginning of each month, billing resets and then slowly builds up to the final price at the end of the month.

To determine how to optimize AWS costs, let’s go deeper into why our EC2 costs are so high. In this chart, we query the number of EC2 instances that are running, grouped by instance type, over an approximate two-month time period.
The query is:

We have not reserved any EC2 instances. To figure out how many instances to reserve using a conservative strategy, we can take the previous chart, and query the moving minimum of the previous chart over the last 60 days. This query checks the past 60 days, at each point, and returns the lowest number of instances, by type, that were running in the past 60 days.

This is a conservative way to figure out how many instances of each type to reserve, going forward. The query is:

If you want to customize this chart even further, you can tweak the query to use a different function for reserving instances. We can insert a variable into the query, called ${function}. That function variable can be set to use moving max, moving average, or moving median.

The variable will show up as a drop-down, and the user will be able to select between these different functions. Variables are simply text replacements in a query. The query is:

Below is a screenshot of the variable editor. The green text in the screenshot shows up in a dropdown, and simply replaces ${function} in the query when selected.

Now, let’s assume that we have some reserved instances. We want to create a chart that shows how many instances we have reserved vs how many total instances we have in an availability zone for a certain machine type: m3.large. Here’s how we display the information on a chart:

Now that you know how to query for reserved and total instances by type, we can create charts for each instance type and availability zone. We can now display reserved vs. total instances on a dashboard. This gives us an overview of what is reserved, and what is not reserved. Additionally, it displays what types of machines are coming up, or going down, over time.

The best part is, you don’t need to write these queries and make these dashboards by yourself — they come pre-packaged with Wavefront. These are just a few examples of what Wavefront can do to help optimize AWS costs. With Wavefront’s built-in functions and visualizations, you can create entire dashboards easily, and answer questions for almost any AWS billing scenario.