Performance Characterization of NVMe-oF in vSphere 7.0 U1

NVMe continues to become increasingly popular because of its low latency and high throughput. VMware vSphere 7.0 introduced support for NVMe over Fabrics (NVMe-oF™), a protocol specification that connects hosts to high-speed flash storage via network fabrics using the NVMe protocol. The NVMe-oF fabrics that vSphere 7.0 supports include Fibre Channel (FC-NVMe) and RDMA (RoCE v2).

I’m pleased to announce we’ve published a new white paper that compares the performance of the legacy Fibre Channel Protocol (SCSI FCP) to FC-NVMe on vSphere 7.0 U1. The same host bus adapter (HBA) and storage area network (SAN) are used in all benchmark runs. The benchmark results show that FC-NVMe consistently outperforms SCSI FCP in virtualized environments, providing higher throughput and lower latency. These results provide a compelling case for customers to upgrade their existing environments to vSphere 7.0 and gain the benefits of NVMe-oF.

We used two benchmark workloads to measure performance for this study: the fio microbenchmark (to generate different I/O sizes and patterns) and Microsoft’s CDB (a benchmark to generate SQL Server OLTP load).

Using fio, we can see from the figure below that the IOPS from the FC-NVMe runs are double that of the SCSI FCP runs for all IO sizes:

IOPS - microbenchmark Random IO pattern runs — IOPS – microbenchmark Random IO pattern runs

And as you can see from the plot below, the latencies are also clearly lower for FC-NVMe vs. SCSI FCP:

Latency - microbenchmark Random IO pattern runs — Latency – microbenchmark Random IO pattern runs

The chart below shows SQL Server database performance, as measured in transactions per second (Txns/sec). FC-NVMe is higher in every case and is 85% higher performing with 2 VMs. Note also that SCSI FCP never quite reached 10,000 Txns/sec, while FC-NVMe was able to surpass 11,000 Txns/sec with both 4 and 8 VMs.

SQL Server Txns/sec for SCSI FCP vs. FC-NVMe

The benchmark results in this performance study show that customers who upgrade their legacy SAN infrastructure from SCSI FCP to FC-NVMe can achieve noticeably higher storage performance using vSphere 7.0.

For a full analysis, more results, and additional test details, please read the Performance Characterization of NVMe-oF in vSphere 7.0 U1 white paper.