RHIAN CROSS
PSO Senior Consultant
vSAN Load Testing Tool
HCI Bench Part I: Defining the Problem
This article is part one of a two part blog series on using HCIBench to performance test vSphere. Evaluating performance is an important part of considering any storage solution. Higher performing solutions can support more workloads on a given configuration, can better accommodate applications, minimize potential performance problems, and be more cost-effective. There are strong motivations to prefer higher performing solutions to lesser alternatives.
Unfortunately, obtaining directly comparable performance results from publicly available information is difficult at best. There is an infinite variety of potential test scenarios—and many vendors discourage publishing for marketing and competitive reasons. This leaves IT professionals in the position of having to run their own tests, and interpreting the results. This has long been a standard practice in evaluating external storage arrays, but the newer generation of hyper-converged solutions – such as VMware vSAN™—presents new testing challenges.
In a hyper-converged architecture, each server is intended to support both many application VMs, as well as contribute to the pool of storage available to applications. This is best modeled by invoking many dozens of test VMs, each accessing multiple stored VMDKs. The goal is to simulate a very busy cluster.
Unfortunately, popular storage performance testing tools do not directly support this kind of model. To achieve a simulation of a busy production cluster, much effort is required to automate load generation, monitoring and data collection after the fact. These steps waste so much valuable time available to do actual testing, even worse may introduce errors into the process.
To address this situation, VMware released a storage performance testing automation tool— HCIBench—that automates the use of the popular Vdbench testing tool in larger clusters. Users simply specify the testing parameters they would like to run, and HCIBench instructs Vdbench what to do on each node in the cluster.
HCIBench aims to simplify and accelerate customer Proof of Concept (POC) performance testing in a consistent and controlled way. This tool fully automates the end-to-end process of deploying test VMs, coordinating workload runs, aggregating test results, and collecting necessary data for troubleshooting purposes. Evaluators choose the profiles they are interested in; HCIBench does the rest quickly and easily.
HCIBench is not only a benchmark tool designed for vSAN, but also could be used to evaluate the performance of all kinds of Hyper-Converged Infrastructure Storage in the vSphere environment.
HCIBench can be run with VSphere 5.5 and later environments for both HCIBench and its client VMs deployment as a replacement for the vSAN Proactive test which is currently built into vSAN. In my lab, am running HCIBench with vSphere 6.5/vSAN 6.6. HCIBench provides greater flexibility in defining a target performance profile as input and test results from HCIBench can be viewed in a web browser and saved to disk.
HCIBench helps simplify the stress testing task, as it allows users to specify the testing parameters (size of working set, IO profile, number of VMs and VMDKs, etc.) and tests multiple instances of Vdbench on multiple servers. If you don’t want to configure the tool manually, users can select the Easyrun option which will automatically configure the tool. After the test run is completed, HCIBench gathers all the results in one place for easy review and resets itself for the next test run.
Where to find HCI Bench?
There is a dedicated fling page which will provide access to HCIBench and its associated documentation. A zip file containing the Vdbench binaries from Oracle will also be required to be downloaded which can be done through the configuration page after the appliance is installed. You will need to register an account with Oracle to download this file but this doesn’t take long.
HCIBench Download: labs.vmware.com/flings/hcibench
HCIBench User Guide: https://download3.vmware.com/software/vmw-tools/hcibench/HCIBench_User_Guide.pdf
Requirements
- Web Browser: IE8+, Firefox or Chrome
- vSphere 5.5 and later environments for both HCIBench and its client VMs deployment
HCIBench Tool Architecture
The tool is specifically designed for running performance tests using Vdbench against a vSAN datastore. It is delivered in the form of Open Virtualization Appliance (OVA) that includes the following components:
The test Controller VM is installed with:
- Ruby vSphere Console (RVC)
- vSAN Observer
- Automation bundle
- Configuration files
- Linux test VM template
The Controller VM has all the needed components installed. The core component is RVC (https://github.com/vmware/rvc) with some extended features enabled. RVC is the engine of this performance test tool, responsible for deploying Vdbench Guest VMs, conducting Vdbench runs, collecting results, and monitoring vSAN by using vSAN Observer.
Pre-requisites
Before deploying this performance test tool packaged as OVA, make sure the environment meets the following requirements:
- The vSAN Cluster is created and configured properly
- The network for Vdbench Guest VMs is ready, and needs to have DHCP service enabled; if the network doesn’t have DHCP service, “Private Network” must be mapped to the same network when HCIBench being deployed.
- The vSphere environment where the tool is deployed can access the vSAN Cluster environment to be tested
The tool can be deployed into any vSphere environment. However, we do not recommend deploying it into the vSAN Cluster that is tested to avoid unnecessary resource consumption by the tool.
What are you benchmarking?
This is my home lab which runs vSAN 6.6 on 3 x Dell Poweredge T710 servers each with
- 2 x 6 core X5650 2.66Ghz processors
- 128GB RAM
- 6 x Dell Enterprise 2TB SATA 7.2k hot plug drives
- 1 x Samsung 256GB SSD Enterprise 6.0Gbps
- Perc 6i RAID BBWC battery-backed cache
- iDRAC 6 Enterprise Remote Card
- NetXtreme II 5709c Gigabit Ethernet NIC
Controller VM
- CPU: 8 vCPU
- RAM: 4GB
- OS VMDK: 16GB
- Operating system: Photon OS 1.0
- OS Credential: user is responsible for creating the root password when deploying the VM.
- Software installed: Ruby 2.3.0, Rubygem 2.5.1, Rbvmomi 1.8.2, RVC 1.8.0, sshpass 1.05, Apache 4.18, Tomcat 8.54, JDK 1.8u102
Vdbench Guest VM
- CPU: 4 vCPU
- RAM: 4GB
- OS VMDK: 16GB
- OS: Photon OS 1.0
- OS Credential: root/vdbench
- Software installed: JDK 1.8u102, fio 2.13
- SCSI Controller Type: VMware Paravirtual
- Data VMDK: number and size to be defined by user
Rhian Cross is a VMware Professional Services Senior Consultant based in the UK. She is an IT Infrastructure Architect specialising in VMware SDDC and Microsoft Environments alongside Blade/SAN/Fibre technology.
Rhian brings the VMware an in-depth knowledge and experience in handling all aspects of complex information systems and software projects including infrastructure migration and management, hardware and software installation and configuration, implementation, training and ongoing management of support issues. She specialities in VMware SDDC, Windows 2003/2008/2012, SAN storage and Disaster Recovery.
Note: The introduction was from the HCIBench User Guide which was written by Chen Wei and Victor Chen in the VMware Product Enablement team and edit by Catherine Xu, technical writer in the VMware Product Enablement team, to ensure that the contents conform to the VMware writing style.
