In the first article of our series comparing AWS services for cloud cost management and optimization, we addressed key considerations for organizations looking into cloud financial management solutions, especially as they compare native tools offered by their cloud service provider or by third-parties. We also outlined our framework for cloud management maturity as it relates to cloud financial management.
In this article, we’ll dive deeper into the first phase of cloud financial management maturity: cloud cost visibility. We’ll address the primary challenges organizations face when it comes to visibility into their cloud spend, the native tools AWS provides to help meet these challenges, and the capabilities that are important to consider as you evaluate the best solution—or combination of solutions—for your business.
The cloud cost visibility challenge
Gaining visibility is crucial as you grow your cloud footprint. This doesn’t mean just being able to see the assets in your AWS environment. You need to be able to allocate the charges in the AWS bill to the responsible cost center, understand how services are being provisioned and configured, identify inefficiencies, and view all of this data in ways that are relevant and useful for a variety of stakeholders with different priorities.
Many organizations start their cloud journey with AWS Cost Explorer and spreadsheets. As organizations advance in their cloud journey, the need for more granular visibility, operational efficiency, and information management becomes too overwhelming for individuals and teams.
How AWS services address the cloud cost visibility challenge
AWS Cost and Usage Report
AWS directs all billing data to a S3 bucket called the Cost and Usage Report (CUR). This report may have millions of line items tracking cost, utilization, Savings Plans, Reserved Instances, and other relevant information. Customers can use other AWS tools to filter the Cost and Usage Report based on parameters such as tags, budgets, and accounts.
For further customization within the AWS environment, and at an additional cost, the Cost and Usage Report can be ingested into other AWS services, such as Athena, Redshift, and QuickSight.
AWS Cost Explorer
AWS Cost Explorer helps customers visualize, understand, and manage AWS costs over time by providing:
- A set of default, out-of-the-box reports for AWS service costs
- AWS service costs by resource-level filters
- The ability to select and purchase Savings Plans
- The ability to launch other AWS services like AWS Forecasts and AWS Budgets for future costs
AWS Cost Categories
AWS Cost Categories allow customers to group costs stemming from different accounts or tags into a category. Costs for these categories will then be visible in the AWS Cost Explorer, Budgets, and Cost and Usage Reports tools.
AWS Budgets
The AWS Budgets tool allows customers to create budgets for predefined time periods and to track costs, usage, and utilization of Savings Plans or Reserved Instances. The budget tool can also deliver regular budget reports. Budgets are accessible within AWS Cost Explorer.
What you need to consider about AWS tools for cloud cost visibility
These tools help organizations gain visibility and answer simple questions about cloud costs and usage. However, as organizations mature in their cloud journey, more challenging cloud financial management challenges arise. Some of the most common challenges are difficulty performing cost re-allocation, achieving operational efficiency, and understanding total cost of ownership (TCO).
Here are a few points to keep in mind when considering using AWS tools and services for comprehensive visibility into cloud costs:
Piecing together data among disparate tools
While AWS offers various tools to provide visibility into cloud spend, choosing the right combinations of these tools and understanding how to best use them can be extremely complex. This is exacerbated when different teams or departments across the organization are using these tools in different ways.
CloudHealth by VMware, for the first time, creates a single view of our cloud activity, across multiple territories. We’re no longer hunting for information. It’s all there at the click of a button.
Sr. Platform Services Manager, Discovery Holdings
Normalized data with customizable views for cross-functional teams
AWS tools provide cost visibility based on the price of the services leveraged. This is an important distinction, as discounts and adjustments applied to the bill at the end of the month could result in a significant difference between AWS service pricing and actual costs billed to the customer.
For example, many customers use the CloudHealth platform to report on amortized costs, apportioned costs, and discounted costs, among others. They can also create distinct perspectives that focus on the relevant KPIs for different users, so the IT and finance teams are presented with the information that’s most relevant to them. The ability to feed this data into business intelligence dashboards is key for incorporating accurate cloud cost information into other key decisions.
Manual work configuring and maintaining data for reporting
The AWS services and tools for cost visibility have become more advanced over time. However, as cloud usage grows, large customers often find their teams inundated with manual tasks related to maintaining the efficacy of reports based on these tools. One example is ensuring that all resources in the environment adhere to the account setup policies and tags that these tools rely on. Further, writing and editing policies in AWS requires a certain level of coding expertise and knowledge of the AWS operating model, which can limit business agility.
One CloudHealth customer estimates that, since enhancing their cost visibility practices beyond their cloud service providers’ tools with the CloudHealth platform, their teams have saved as much as 20 hours per week on cost management tasks while improving visibility from about 10% of their environment to 100%.
Allocating savings from Reserved Instances and Savings Plans
Many organizations embrace committed use programs, such as AWS Savings Plans or Reserved Instances, for discount pricing compared to on-demand instances. To maximize usage of these discount mechanisms, many organizations with linked or consolidated accounts follow a best practice of purchasing from a top-level account in their hierarchy so that, in the event that the purchasing account does not have eligible services to use up the commitment, other linked accounts will still be able to reap the benefits of the discount.
However, many customers struggle to allocate the savings from Reserved Instances and Savings Plans to the groups that actually used the resources. If a top-level account in the hierarchy made the initial purchase, AWS will still attribute the full cost to the purchasing account, even if that account did not use any of the discounted resources.
On the other hand, CloudHealth customers use the platform to allocate Reserved Instance and Savings Plans benefits based on who used the discount, rather than by who purchased it, thereby providing the visibility needed to perform accurate cost showback or chargeback at a granular level.
Amortizing upfront costs from Savings Plans and Reserved Instances
Similarly, amortization for costs related to Reserved Instances and Savings Plans is another common challenge. Simply put, amortization is a way to spread out a capital expense (in this case, Savings Plans or Reserved Instance purchases) over the period of time that your business will benefit from the expense. Both Savings Plans and Reserved Instances offer options to pay all up front, pay partial up front, or pay nothing up front, with the amount of the discount varying by the different payment options. Since none of the reservation is paid for at the start with the no upfront payment option, only the all upfront and partial upfront payment options are candidates for amortization.
Since AWS tracks costs based on the account that made the purchase, the amortized reservation cost is also attributed to the purchasing account. You end up with the same situation discussed in the point above, where it’s still difficult to know which team is responsible for the usage/credit of the discount; you only know who purchased it.
Since CloudHealth tracks the usage of the resources purchased via these discount programs, the amortized costs can also be applied at the usage level, offering a more granular cost allocation approach.
For more information on how this works, see our article: Amortization by Usage: The Only Right Way to Amortize AWS Reserved Instances and Savings Plans Costs
Customizing views into data based on relevant KPIs
Most organizations need to slice and dice their cost, usage, performance, availability, and security data in a variety of ways, and will change their reporting structures over time in response to the requirements of different stakeholders.
However, many organizations can struggle to provide relevant insights on their cloud management efforts based on several disparate sources of data. This is even more difficult for organizations that don’t have 100% adherence to their resource tagging policies. Improperly tagged resources will not be tracked in AWS cost visibility tools, leaving potential for significant blind spots in the environment. These needs also exacerbate the manual work related to maintaining cost visibility, as AWS customers will need to update their reports manually in response to staff or project changes.
Preserving historical information
If an asset’s tag is modified or that asset is deleted, AWS tools will lose the history of that asset. This can cause real challenges for AWS customers trying to understand TCO or responding to audit and compliance requests.
CloudHealth maintains a historical record of all assets over their lifecycle, even for assets that have been modified or deleted.
Container cost visibility
Organizations embracing AWS services for containerized workloads will need to ensure their processes are able to account for the unique cloud financial management challenges that come with them.
AWS tools provide visibility into the cost of underlying infrastructure for containerized workloads, such as EC2 instances. This can be helpful, but lacks the kind of granular visibility needed to understand who in the organization is consuming cluster resources or whether the cluster is leveraging the right mix of resources.
CloudHealth customers have visibility at the service and cluster level to understand the mix of resources being used within clusters, and then to map that back to the teams using them. You can learn more about containers and Kubernetes management best practices in our article here: The State of Kubernetes: Benefits and Challenges You Need to Know in 2021
And for more in-depth information on container cost optimization, see the complete whitepaper we created in partnership with the FinOps Foundation: FinOps for Kubernetes: Unpacking Container Cost Allocation and Optimization
Cost visibility across multiple clouds
When it comes to budgeting and measuring TCO, public cloud cost data cannot be limited to just the AWS environment. Even if your organization operates in just the AWS cloud today, it may not be long before your teams begin to embrace a multi-cloud approach, leveraging services offered by other cloud providers. In 2020, our global assessment of cloud management maturity found that 46.7% of respondents reported operating in multiple clouds (public and/or private), while just 24.3% reported operating in a single public cloud environment.
While access to a diverse range of options for infrastructure offers enhanced flexibility, it can also increase the complexity of cloud management. Many organizations quickly find the need to aggregate visibility and reporting for costs across all environments.
Final thoughts
In this article, we outlined the key capabilities organizations should consider when evaluating the tools they’ll use to support their cloud financial management practice during the visibility phase of the cloud management maturity framework. In our next article, we’ll focus on the second phase: cloud cost optimization. This will cover the key challenges organizations face when it comes to optimizing cloud costs and finding opportunities to eliminate waste, as well as the primary tools and services AWS provides to help solve these challenges.
You can also download our complete whitepaper to have all the information we’ll cover in this series in a single place.
If you’re looking for more detailed information into CloudHealth capabilities and how this differs from your cloud provider’s native tools, please don’t hesitate to get in touch with us directly. Our team of experts would be happy to answer any questions you may have and demonstrate how the platform can help you with your cloud cost management and optimization practice.
