Author Archives: Sankaran Sivathanu

Whitepaper on vSphere Virtual Machine Encryption Performance

vSphere 6.5 introduces a feature called vSphere VM encryption.  When this feature is enabled for a VM, vSphere protects the VM data by encrypting all its contents.  Encryption is done both for already existing data and for newly written data. Whenever the VM data is read, it is decrypted within ESXi before being served to the VM.  Because of this, vSphere VM encryption can have a performance impact on application I/O and the ESXi host CPU usage.

We have published a whitepaper, VMware vSphere Virtual Machine Encryption Performance, to quantify this performance impact.  We focus on synthetic I/O performance on VMs, as well as VM provisioning operations like clone, snapshot creation, and power on.  From analysis of our experiment results, we see that while VM encryption consumes more CPU resources for encryption and decryption, its impact on I/O performance is minimal when using enterprise-class SSD or VMware vSAN storage.  However, when using ultra-high performance storage like locally attached NVMe drives capable of handling up to 750,000 IOPS, the minor increase in per-I/O latency due to encryption or decryption adds up quickly to have an impact on IOPS.

For more detailed information and data, please refer to the whitepaper

Detailed stats for vSphere Flash Read cache

In the Performance of vFlash Read Cache in VMware vSphere 5.5 whitepaper details about performance in several different workloads is provided [http://www.vmware.com/files/pdf/techpaper/vfrc-perf-vsphere55.pdf].  It also covers details about how to tune and configure vFRC to obtain best performance.  This blog article goes through the details of how to obtain runtime statistics about the vSphere Flash Read Cache like the Cache hit rate, Latency of cached I/Os, and average number of cache blocks evicted as used in the whitepaper.

You can run esxcli to get some of these stats using the following commands:

~ # esxcli storage vflash cache list

This would list the identifiers for the caches that are currently in use.  These are displayed one per vFlash-enabled-VMDKs.   To retrieve the vFRC statistics for a particular vFlash-enabled-VMDK, the following command can be used:

~ # esxcli storage vflash cache get –c <cache-identifier>

However, a few more advanced statistics like the amount of data that is cached at any point of time may be obtained by directly accessing the VSI nodes.  The process to do this is as follows:

Cache identifier may be obtained by either the esxcli command shown above or using

~ # cacheID=`vsish -e ls /vmkModules/vflash/module/vfc/cache/`
~ # vsish -e get /vmkModules/vflash/module/vfc/cache/${cacheID}stats

This displays an output similar to the following:

vFlash per cache instance statistics {
cacheBlockSize:8192
numBlocks:1270976
numBlocksCurrentlyCached:222255
numFailedPrimaryIOs:0
numFailedCacheIOs:0
avgNumBlocksOnCache:172494
read:vFlash per I/O type Statistics {
numIOs:168016
avgNumIOPs:61
maxNumIOPs:1969
avgNumKBs:42143
maxNumKBs:227891
avgLatencyUS:16201
maxLatencyUS:41070
numPrimaryIOs:11442
numCacheIOs:156574
avgCacheLatencyUS:17130
avgPrimaryLatencyUS:239961
cacheHitPercentage:94
}
write:vFlash per I/O type Statistics {
numIOs:102264
avgNumIOPs:307
maxNumIOPs:3982
avgNumKBs:10424
maxNumKBs:12106
avgLatencyUS:3248
maxLatencyUS:31798
numPrimaryIOs:102264
numCacheIOs:0
avgCacheLatencyUS:0
avgPrimaryLatencyUS:3248
cacheHitPercentage:0
}
rwTotal:vFlash per I/O type Statistics {
numIOs:270280
avgNumIOPs:88
maxNumIOPs:2027
avgNumKBs:52568
maxNumKBs:233584
avgLatencyUS:11300
maxLatencyUS:40029
numPrimaryIOs:113706
numCacheIOs:156574
avgCacheLatencyUS:17130
avgPrimaryLatencyUS:27068
cacheHitPercentage:58
}
flush:vFlash per operation type statistics {
lastOpTimeUS:0
numBlocksLastOp:0
nextOpTimeUS:0
numBlocksNextOp:0
avgNumBlocksPerOp:0
}
evict:vFlash per operation type statistics {
lastOpTimeUS:0
numBlocksLastOp:0
nextOpTimeUS:0
numBlocksNextOp:0
avgNumBlocksPerOp:0
}
}

This output contains all of the metrics that are discussed in the vShpere Flash Read Cache whitepaper.  This information can be used for sizing the cache and cache blocks in the most effective way.

Disclaimer: This interface (vsish) is not officially supported by VMware, so please use at your own risk.

vSphere Flash Read Cache Performance on vSphere 5.5

vSphere Flash Read Cache (vFRC) is a new solution to enhance storage I/O performance in vSphere 5.5.  vFRC lets you use flash devices (SSD or PCIe cards) as a read cache for VM I/Os and therefore improve performance. vFRC can improve performance for read-intensive workloads that have a high percentage of data locality. Such workloads are generated by database warehousing applications and enterprise server applications such as Web proxy servers and monitoring servers.

A recent technical white paper studies the performance of vFRC with respect to the following workloads:

  • A decision support system (DSS) database workload running with Oracle 11g R2
  • A DVD store workload running with Microsoft SQL Server 2008
  • Enterprise server-level I/O traces that are used extensively in storage research

The results are presented in the paper, along with performance best practices when using vFRC. The paper also gives an overview of the vFRC architecture. To learn more, read Performance of vSphere Flash Read Cache in VMware vSphere 5.5.