The Express Storage Architecture in VMware vSAN™ 8 allowed us to redesign how compression is implemented. First, new techniques are in place to improve the potential data reduction of the compression feature to as high as an 8:1 compression ratio for every 4KB block written, which is a 4x improvement from the original storage architecture. The new design of the compression feature in vSAN occurs in the upper layers of vSAN as it receives the incoming writes. We are able to not only reduce the amount of CPU effort needed to compress the data, but this also reduces the amount of network traffic, as any replica traffic is always compressed. This highly efficient design allows for it to be enabled by default, but for workloads already performing application-level compression, it can be turned off on a per VM basis through storage policies.
Why the change?
vSAN Original Storage Architecture Compression
Compression was originally implemented in vSAN with the release of VSAN 6.2. This feature first came out in a world where hosts and resource costs looked radically different. Host with six core processors that lacked many modern acceleration features were common, and the cost of the CPU overhead and potential latency or metadata amplification to a first write were strongly considered in implementing a system that compressed data only on cache de-stage. Care was taken to adaptively compress data based on it’s compressibility. This came at the cost of compression savings, but helped reduce the CPU overhead at the time which was contending for comparably to today, modest CPU resource budgets. At the time vSAN data services were added at the “bottom of the stack” at the disk group layer and this design made sense.
vSAN Express Storage Architecture
A key theme of vSAN ESA is “Performance without compromise. Two key changes have enabled a new compression system that is most more capacity efficent (up to a 4x improvement over original storage architecture) but also able to consume less CPU. The vSAN file system along with compression “up the stack” to execute immediately on the host where the virtual machine is running.
Writes – The compression process only needs to be run once vs. once for each copy of the data. This reduces amplification of CPU and data written to the drives. This data being compressed before it hits the network also reduces the network traffic required.
Reads – Data is fetched in a compressed state, so this reduces the amount of data transmitted across the network.
Granular management without complexity
With the vSAN original storage architecture compression was available as a single/simple datastore wide option. This made sense as it was deeply embedded in the lower levels of the IO path, and configuring the file system is an “all or nothing” option. It could be enabled or disabled after the fact but this forced a relatively heavy IO operation of re-writing all data to the cluster in a rolling fashion. With vSAN 8 Express Storage Architecture we wanted to give more flexibility and control, but not add excessive complexity. The new compression option is enabled by default, but can be optionally disabled for workloads though the use of the vCenter Storage Policy Based Management (SPBM) Framework. By adding a policy that uses “No Compression” you can disable compression to a specific disk or virtual machine. If disabled (or re-enabled) by policy, it will not change the compression state of existing data retroactively. Compression will only be effective on subsequent writes. This removes the operational performance impact considerations for enabling or disabling the policy. A few applications may perform their own compression (VMware Tanzu Greenplum/PostgreSQL, video, etc.). In these cases, forcing a disable of the compression will save CPU cycles. But this can be done easily and prescriptively using storage policies.
Conclusion
The Express Storage Architecture in VMware vSAN™ 8 was an opportunity to enhance data services in a way that delivers the best performance, the best capacity costs without compromise. The changes made to the management and well as IO path delivers on this promise by making compression that should not need to require extensive pre-planning, or need to operationally manage impacts on day 2 changes in configuration.
