AUTHOR: Craig Stanley (@benchmarkguru)
This article is the first in a series that will explain VMware’s context for benchmarking virtualization and cloud implementation, and discuss the Key Performance Indexes (KPIs) that are used to quantify, measure, and analyze the organizations journey toward ITaaS.
The first step in the benchmarking process is to determine what needs to be measured and how it will be quantified.
Quantification is all about determining:
- What are the KPIs (Key Performance Metrics) that matter and can I determine what data elements they are based on?
- Can I get the information I need consistently to calculate my KPI?
- Can my KPIs be compared against my competition or industry?
First, what are the KPIs that matter most to your operation? Typically, it may be the ratio of a consumable resource to a quantity of production such as a cost per application developed or a cost per server. Other consumption-based metrics may be expressed as ratios like cost per FTE, FTEs per system, or VMs per Host. There are dozens if not hundreds of metrics that may be interesting, but the ones you track need to:
- Measure your current state – the KPIs should not be based on data elements that are out-of-date by the time you collect them.
- Provide meaningful information that supports your strategy – the KPIs should matter.
- Be easy to understand and explain – the relationship of the underlying data should not be so esoteric that the resulting relationship is dubious.
Next, is the KPI based on information you can get consistently over time? In order for benchmarking to be useful, the KPIs need to be based on apples-to-apples comparative data elements. Having consistent data and KPIs also enables you to evaluate year-over-year performance with relevant metrics. This ability is an invaluable oasis of information to validate, justify and monitor continuous improvement.
Lastly, are the KPIs specific to just your organization? Are they common across your competition or industry and can you get access to the KPI data elements from your competition? The primary benefit of benchmarking is to help identify KPIs and determine where you are and what progress you’ve made. Another major benefit is that it provides you the basis you need to see how you are performing against your competition.
KPIs may be industry-focused such as IT spend per new policy or IT spend per claim for an insurance company for example. Comparing KPIs like this across industry competitors can provide significant insight into the IT / business landscape and valuable feedback on strategy.
In my next blog, I’ll talk about the KPIs used for the VMware Infrastructure Move The Needle and explain how they can help quantify the value of moving from your present virtualization state to the level of the best performing peers.
Craig Stanley is the Benchmarking Practice Lead for the VMware Vision Team. You can follow him on Twitter @benchmarkguru.
