Introducing HCIBench 1.6

Happy 2017!

We just released version 1.6.0.0 of HCIBench. You can find the appliance and the change log HERE. In the past couple of months, the focus has been adding new features and bug fixes. This version contains the following new features:

• HCIBench Authentication
• Clear Read Cache and Write Buffer before Testing
• Easy Run
• Re-Use Existing VMs
• 95th Percentile Calculation

Before we discuss the new features in depth, we need to emphasis some minor changes in the network design to avoid any potential confusion. The HCIBench deployment procedure remains the same as previous versions. However, the mapping the Private Network of HCIBench, requires understanding of the following:

1. What VLAN will the Vdbench client VMs be deployed on?
2. Does the VLAN have DHCP service?
3. If yes, does the network mapped to Public Network of HCIBench have the route to that VLAN?

If the answer of question 2 or 3 is No, the Private Network MUST be mapped to the VLAN

.

Again, to ensure your success, prior to deploying this new version of HCIBench make sure you can answer the above questions for your environment.

New Features

HCIBench Authentication:

Once the deployment is complete, you can access the configuration page by visiting this URL

HTTP://HCIBENCH_IP:8080.

In previous versions, no authentication was necessary to access the configuration page. In version 1.6, the root user ID and password must be used to authenticate to prevent unauthorized access to HCIBench.

Starting in version 1.6, from the configuration page, the vCenter Username can be filled as either the UPN (User principle name) format such as “Username@Domain” or the Down-Level Logon Name format such as “Domain\Username”.

Clear Read Cache and Write Buffer before Testing:

At the end of the vSphere Environment Information section, a new option was added: “Clear Read/Write Cache Before Each Testing”. This is specific to vSAN environments. The user MUST have the vSAN datastore specified in the Datastore Name field.

If this option is checked, the write buffer and read cache of vSAN will be flushed to make vSAN achieve a clean state before each Vdbench test starts, and each Vdbench test case will need more time to accomplish due to the additional time added by the cache/buffer flushing, the time of cache/buffer flushing is depending on the de-staging speed, the cache/buffer usage of the cache tier and the performance of the physical drives.

After checking this box, the Host Username and Host Password fields in the Cluster Hosts Information section must be filled out, and the SSH service must be enabled on all the ESXi hosts in the cluster as well. Please be aware if you are not quite sure what “clearing read cache and write buffer of vSAN” means, just leave this box unchecked.

NOTE: Do NOT interrupt the I/O testing process if you choose to clear the read cache and write buffer before test, interruption of clearing read/write buffer process will cause vSAN issue.

Easy Run:

EASY RUN is a new feature of HCIBench specifically designed for vSAN, which can help vSAN user with the workload configuration and producing the reasonable performance results on vSAN.

EASY RUN can be found at the end of Cluster Hosts Information section. If it’s checked, vSAN datastore must be specified in the Datastore Name filed. When EASY RUN is enabled, HCIBench will decide how many Vdbench client VMs should be deployed, how many data disks and size of each data disk should be configured per VM. Also, HCIBench will decide the way of disk initialization depending on whether the vSAN datastore has Deduplication enabled.

If EASY RUN is checked: The sections including Vdbench Guest VM Specification and Vdbench Testing Configuration will be hidden.

*Testing Configuration workload:

For now, only one Vdbench workload profile will be used for I/O testing. The workload configuration is:

100% working-set of each data disk,

70% read and 30% write,

100% random,

4K block size,

30 minutes’ warm-up time and 60 minutes’ testing time.

The entire HCIBench test run on my testbed, which is a 4-node all-flash vSAN cluster with 2 disk groups per host without deduplication enabled, takes around 3.5 hours.

This includes the 30 minutes for VM deployment and configuration, 1.5 hours for disk initialization, 30 minutes for warm-up and 1 hour for testing. When testing vSAN with deduplication enabled, the disk initialization took around 2.5 hours on the same testbed.

After the testing is finished, you can find the summarized result file and a sub-folder which containing the Vdbench raw results files and vSAN observer page in the auto-generated directory “easy-run-TIMESTAMP”.

If you want to define and configure the specific testing parameters yourself, keep EASY RUN unchecked. You will have the option to configure all the parameters as you want.

Re-Use Existing VMs:

The very end of section of the Vdbench Guest VM Specification, a new feature called:

“Re-Use The Existing VMs If Possible” is added. By having this checked, if there are compatible Vdbench client VMs in the cluster, after those VMs are validated, they could be reused for more Vdbench tests without being destroyed and redeployed.

With this checked, HCIBench will try to find the compatible Vdbench client VMs that match the configuration after the testing begins. If there are pre-deployed Vdbench client VMs and they are compatible, for instance, the number of Vdbench client VMs and VMDKs are not less than them in the configuration and the VMDK size equals the configuration, HCIBench will skip deployment process and go ahead to disk initialization (if specified) or Vdbench I/O testing; otherwise, Vdbench client VMs will be deployed as specified.

95^th Percentile Calculation:

In addition to the aggregated IOPS, throughput, and the average latency, HCIBench version 1.6.0.0 introduces the 95^th percentile calculation of the Vdbench testing. The methodology of 95^th percentile calculation is: for each Vdbench raw result file, sort the latency report of all the testing intervals and use the formula below to get the single interval which represents the 95^th percentile of latency:

Here P means percentile should be 95 and N should be the number of reporting intervals, then we can calculate out n – the index number of the 95^th percentile interval and get the latency and the IOPS of that interval. Then, calculate the average of latency and the sum of IOPS across all the Vdbench VMs.

The output above can help the user to understand that during 95% of the testing time, the average latency is below 2.99ms, and the aggregated IOPS is relatively above 171,140.

So there you have it. HCIBench 1.6.0.0 and all its new features! You can still leverage the scripts introduced in Use HCIBench Like a Pro—Part 2 to integrate with your own framework or platform to accomplish more automation.

Happy Testing!

Questions? Comments?