Best Practices For Optimizing Amazon EBS Volumes

Amazon Elastic Block Store (EBS) can be the cause of excessive costs when using AWS at scale. However, it’s possible to optimize Amazon EBS volumes without sacrificing availability and response times by applying some simple best practices.

Amazon Elastic Block Store (EBS) can be the cause of excessive costs when using AWS at scale. However, it’s possible to optimize Amazon EBS volumes without sacrificing availability and response times by applying some simple best practices.

What is Amazon Elastic Block Store?

Before diving into how to optimize Amazon EBS, we’ll briefly cover what it is and how it works. Amazon EBS provides block-level storage volumes for use with EC2 instances and in many cases, EBS is the only block storage option available. The reason for this is because Amazon optimizes EC2 instances with a configuration stack that minimizes contention between EBS input and output (I/O) and other traffic in order to provide dedicated capacity.

Amazon offers different EBS volume types depending on your price and performance requirements. The different volume types fall into two primary categories: solid state drives (SSD) and hard disk drives (HDD).

• Solid state drives (SSD): Optimized for transactional workloads with frequent read/write operations with small I/O where the dominant performance attribute is IOPS. SSD-backed EBS volumes can be General Purpose SSD (gp2, gp3), which offer a balance of price and performance and are suitable for most workloads, or Provisioned IOPS SSD (io2, io1), which provide high performance for mission-critical, low-latency, or high throughput workloads.
• Hard disk drives (HDD): Optimized for large streaming workloads where the dominant performance attribute is throughput. HDD-backed EBS volumes can be Throughput Optimized HDD (st1) or Cold HDD (sc1). 

You can see in the table below a breakdown of the different EBS volume types, their primary use cases, parameters, and prices. 

table {
margin-bottom: 15px;
}
table td {
padding: 5px 10px;
}

table {
margin-bottom: 15px;
}
table td {
padding: 5px 10px;
}

Also, in December 2020, AWS announced io2 Block Express EBS volumes in preview. io2 Block Express EBS volumes offer customers even lower latency than io2, with up to 256,000 IOPS/volume, 4,000 MB/s of throughput, and a maximum volume size of 64 TB, all with sub-millisecond, low-variance I/O latency. You can learn more about requesting io2 Block Express here.

Understanding the costs associated with EBS

There are several variables that affect EBS pricing: 

• The EBS volume type
• The amount of storage
•  I/O operations per second, or IOPS, which measures how many requests for I/O per second are being made to the volume
• Throughput, or how quickly large files are read or written to a volume
• The region where the instance attached to the EBS volume is deployed

Additional charges apply if you’re making cross-region data transfers or backing up your volumes with EBS Snapshots, which currently cost $0.05 per GB-month of data stored.

How to optimize Amazon EBS Volumes

When optimizing Amazon EBS volumes, there are a few key questions to consider, which we will explore in further detail below. 

1. Am I using the IOPS I provisioned for my io1/io2 volumes?
2. Can I switch my io1/io2 volumes for gp2/gp3 for a slightly lower service guarantee, and a much lower price?
3. Do I have less frequently accessed workloads that can be switched to sc1?
4. Do I have high throughput volumes that could benefit from st1?
5. Do I have idle or unattached volumes that can be deleted?

Overprovisioning IOPS

Overprovisioning your io1/io2 volumes is a common occurrence. Sometimes it’s very obvious, where maximum read or write operations never approach the IOPS provisioned, but other times it’s not as clear.

To get the best measure of the I/O for an io1 or io2 volume, use the CloudWatch metric VolumeConsumedReadWriteOps. This is the most conservative metric AWS has for io1 and io2 performance, as it combines reads and writes, and compensates for larger reads and writes. Note that I/O operations larger than 256k are counted in 256k capacity units. For example, a 1024k I/O would count as four consumed IOPS.

Using the VolumeConsumedReadWriteOps metric, you can get a good idea of the actual need for I/O, and also how many sustained periods of high I/O a volume encounters. From here, you can adjust the amount you’re provisioning or consider if a different volume type would be suitable at a lower price point. This leads to the second question of what size gp2/gp3 volume you should use to deliver the IOPS you need.

Configuring baseline IOPS in gp2

In most cases, it’s not economical to use io1 or io2 SSDs unless a volume has critical I/O needs, or has sustained volume above 10k IOPS. Gp2 can deliver the same IOPS for a lot less.

Gp2 provisions baseline IOPS at a rate of three per GB up to a limit of 10k. To figure out the baseline IOPS for a gp2 volume, simply multiply the GB by three. For example, 400 GB gp2 will deliver a baseline of 1200 IOPS volume at a much lower cost than a 400 GB io1 drive with 1000 PIOPS. The volume can burst to higher levels until it’s “Burst Balance” is exhausted and the performance reverts to baseline, but it will provide the baseline performance for as long as needed.

Type gp3 is the latest generation of general-purpose SSD-based EBS volumes. The upgrade from Type gp2 to gp3 allows users to provision IOPS and throughput independent of storage capacity and receive up to a 20% lower cost per gigabyte.

Previously, developers had to provision block storage volumes that met both the performance and storage needs of the workload, often resulting in overprovisioning for one of the two. Now, users can scale IOPS and throughput without needing to provision additional block storage capacity.

Finding candidates to migrate to HDDs

You can think of sc1 and st1 as workhorses, which are much less expensive than SSD volumes. They perform well in many applications that are not I/O intensive or do not require single millisecond latency. Some rules of thumb that we use:

1. Volumes with an average I/O of the larger of reads or writes less than 85
2. A max I/O for the larger of reads and writes of less than 200 are good candidates for sc1. Performance characteristics of volumes are described here.

Volumes with high throughput are good candidates for st1, which has a high sustained baseline MB/s of I/O, with the ability to burst above baseline. To evaluate a transition to sc1 or st1, it’s recommended to run volume metric reports for a month, and filter by average I/O levels.

When looking for good candidates to migrate, start by sorting volumes by max read and write MB/H to find your high throughput volumes. Sc1 volumes can sustain 250 MB/s, or 90,000 MB/h. St1 adds headroom to 500 MB/s baseline with a burst capability as described here.

Terminating detached and idle volumes

AWS charges for volumes at a GB/hour rate, whether they’re attached to an EC2 instance or not, and regardless of whether they were accessed. By monitoring for unattached and idle volumes you can avoid paying for drives that aren’t necessary.

To handle unattached volumes automatically using the CloudHealth platform, create a policy similar to this:

To find idle EBS volumes, run volume metrics for the previous month, and add the columns for sum read ops and sum write ops.

Sort by the sum of read or write ops and the idle volumes will sort to the top.

In the data above, the first volume had Write activity, but the other three had no reads or writes and are candidates for termination.

Summary

Optimizing EBS Volumes can result in significant cost savings. High-cost io1 and io2 volumes can often be migrated to gp2 or gp3. Non-boot gp2 volumes can often be migrated to HDD storage with 75% cost savings. CloudHealth provides the data you need to optimize your EBS Volumes, and policies to keep unattached volumes under control.

Establishing a cloud financial management practice can also help establish cost optimization processes. Cloud financial management (CFM), also known as Cloud Cost Management, is a function that helps align and develop financial goals, drive a cost-conscious culture, establish guardrails to meet financial targets, and gain greater business efficiencies. Learn more about establishing a Cloud Financial Management practice here.