HCIBench stands for “Hyper-converged Infrastructure Benchmark”, which is an automation wrapper around the popular and proven VDBench open source benchmark tool that makes it far easier to automate testing across a hyper-converged cluster.

HCIBench was released last August, one year later, we published another major release and moved it from Dropbox to Fling. If you are new to HCIBench, you are more welcome to catch up by reading these posts written by Chuck Hollis, Cormac Hogan and Kevin O’Brien.

If you already have HCIBench installed, you may want to update to the latest version to take advantage of all the new features!

What’s new:

  • HCIBench service and the VDBench client have moved to Photon OS 1.0, which is an operating system developed and maintained by VMware.
  • HCIBench now has added another disk initialization option specifically for the storage that has deduplication enabled.

NOTE: The previous version doesn’t support disk initialization on a deduplication enabled storage. Updating the HCIBench is REQUIRED if you need to test storage that has deduplication enabled.

  • HCIBench now supports multiple datastores deployment. Multiple datastores in the same cluster could be tested at the same time for performance comparison.
  • HCIBench now is able to provide DHCP service if the test VLAN doesn’t have it.
  • HCIBench now creates a VM folder dynamically instead of creating a folder named “perf-test-vms”, this allows the user to have multiple HCIBench Appliances to test multiple clusters in the same vSphere data center at the same time.

Ok, now please allow me, as the writer of HCIBench, to briefly go through the installation and configuration process and provide some tips. For the complete set of installation instructions please refer to the  HCIBench user guide.

First, download the HCIBench OVA from https://labs.vmware.com/flings/hcibench and deploy it to vSphere 5.5 or later environment. Because the vApp option is used for deployment, HCIBench doesn’t support deployment on a standalone ESXi host, the ESXi host needs to be managed by a vCenter server.

When configuring the network, if you don’t have DHCP service on the VLAN that the VDBench client VMs will be deployed on, the “Private Network” needs to be mapped to the same VLAN because HCIBench will be able to provide the DHCP service.

hcibench_blog1

After configuring the public network and the root password, you can start to deploy the HCIBench OVA.

If the public network needs to be reconfigured after deployment, refer to the administration guide. NOTE: do NOT edit the file /etc/systemd/network/eth1.network or edit eth1 of HCIBench.

After HCIBench is booted up, open the browser and go to http://HCIBench_IP:8080, the page may take up-to 3 minutes to load after boot-up so be patient.

In the configuration page, the first section is the vSphere environment, the value of vCenter Username needs to be the UPN (User principle name) format e.g. “Username@Domain” rather than Down-Level Logon Name format e.g. “Domain\Username”.

hcibench_blog2

If the network (here we use Private_VLAN_NO_DHCP as example) specified does not have DHCP service, make sure:

  1. Map HCIBench eth1 to “Private_VLAN_NO_DHCP”
  2. Check Enable DHCP Service on the Network

hcibench_blog3

Then HCIBench will provide DHCP service on VLAN “Private_VLAN_NO_DHCP” during the testing.

In Cluster Hosts Information section, if the type of network specification above is using a Distributed Switch PortGroup, then Deploy on Hosts must be UNCHECKED. Having Directly Deploy on Hosts configured is able to speed up the deployment process but can be used in conjunction with a Virtual Standard Switch.

hcibench_blog4

In the section of Vdbench Guest VM Specification, the Number of VMs must be integral multiple of the number of datastores specified above. For example, if there are 3 datastores specified above, the number of VMs must be 3*N.

Now scroll down to the Vdbench Testing Configuration section, you can either Generate Vdbench Parameter File by Yourself or Upload a Vdbench parameter file. Either way, the parameter file is located in /opt/automation/vdbench-param-files. After generating or uploading the VDBench file, click Refresh then you will be able to see all the files generated or uploaded. Select one particular parameter file you want or just Use All, if you choose to run all the parameter files they will be executed sequentially.

hcibench_blog5

NOTE: When creating the VDBench parameter file by yourself, make sure the number of disks are not more than the Number of Data Disks you specified earlier, disks starts from sda, so if you specify 4 disks for example, you should specify /dev/sda through /dev/sdd.

Disk initialization is designed to avoid storage first-write penalty during the performance testing. For the Disk initialization, you have 3 options: NONE, ZERO and RANDOM. NONE means you choose not to do disk initialization before testing, this could be used if you just want to do a quick testing and have a brief idea how the storage performance looks like, however the first-write penalty may apply which might impact the results. ZERO could be used for initializing a storage without deduplication enabled while RANDOM is particularly designed for storage with deduplication enabled.

hcibench_blog6

We highly recommend to do disk initialization for a serious performance testing, even if this might take a couple of hours longer than not doing that.

If you are deploying the HCIBench appliance for the first time you will be asked to upload the VDBench binary file. We are not allowed to re-distribute VDBench. Just download the VDBench zip file from Oracle website, and upload it directly. This is a one-time effort, once uploaded, this section will become invisible. After uploading, you should be able to see your VDBench zip file in your browser: http://HCIBench_IP:80/vdbench-source/

In case you need to update the VDBench zip file, you can simply go to HCIBench console and delete the old zip file from /opt/output/vdbench-source and upload the new one through the page.

hcibench_blog7

Here are some tips that I think are really important to use HCIBench:

  1. Make sure the VDBench client VM network has enough IPs available if DHCP service is provided by your environment.
  2. Make sure the VDBench client VM network can have DHCP and SSH services to go through.
  3. If the test failed e.g. no results showed up or the results are showing all 0, check the logs in HCIBench:/opt/automation/logs and check the deployment log first.
  4. On the configuration page there are two places for uploading files, the first one is for VDBench parameter files and the second one is for VDBench binary zip file. The second one will be invisible after uploading.
  5. All the results are stored in /opt/output/results, which has 200GB space, in case it’s running out of space, you should migrate the old results to somewhere else or delete some of them to free up the space.

In the part 2, I will share some advanced operations from the HCIBench console, including how to re-use the existing VDBench client VMs for more tests, how to debug HCIBench from the console and how to leverage the scripts of HCIBench to perform other automation tasks. Stay tuned!

Comments

10 comments have been added so far

  1. Hi Chen,

    This is an impressive release as all the configurations could be set while importing the OVA file such as:
    1. IP address configurations
    2. Root password

    It seems that the HCIBench OVA file of the new version could not be imported directly to ESXi host. Is this as designed?
    If only one ESXi host is running, how to import the OVA file to the ESXi host?

  2. If you only have one host, the workaround could be:
    1. unzip the ova to ovf and vmdk files.
    2. edit HCIBench.ovf file and remove the section:
    3. remove HCIBench.mf file to avoid integrity check.
    4. use the ovf for single host (hostd) deployment.
    5. after deployment, you should access the VM using password “vmware” and edit the file /etc/systemd/network/eth0.network following this guide: https://github.com/vmware/photon/blob/master/docs/photon-admin-guide.md#configuring-network-interfaces.
    6. reboot the HCIbench server and you should be good to go.

  3. I’m evaluating HCI systems from different vendors and have no access to the equipment. Could this tool be used to ask them for performance results on their equipment? If so, what would be the best way to ensure they all run similar tests? (send a common config file?)

    I guess I could later in the evaluation I can validate those results.

  4. 1. make sure the Hypervisor is vSphere.
    2. make sure they all use same workload profiles.
    3. their storage capacity might be different, you can try to limit the data-set around 20% of the capacity or so to design how many vms and vmdks and size of vmdk for each of them.

    the only variable they can change is the number of vms, all the vdbench workload profiles and other numbers(# of vmdk, size of vmdk, disk initialization method) should be the same

  5. You show how to use the backend scripts to make the performance testing more efficient as well as troubleshooting HCIBench. I was confused about where core executables are located and now you have cleared my confusion and the correct location is “/opt/automation”. We are using HCIBENCH and if faced any issue while using the HCIBENCH then we will contact to you.
    The way you explained each and everything in this 1nd part is really great. Thanks once again.

  6. Hi Chen,

    Your blog is really helpful. I am running HCIBench for one of my customers with private network of 10.xx.xx.xx series. How should I choose Network values? Can you suggest? To mention all my private vlans have 10.xx.xx.xx IPs. Do I have to map my HCIBench eth1 to private VLAN and enable DHCP option as mentioned in your blog above or there is any other workaround for it.
    Please suggest.

  7. Chen,

    I am referring to same user guide but still getting validation error.
    My ESXi servers are on different network vlan and vcenter on different. Thats where my apprehension begins while choosing Private and Public networks. I have choosen vlan for my vCenter for both networks i.e. 10.xx.xx.xx.
    Also when I check logs I find this::

    networks: Pub-Net-1284 = VM Network
    Did not find 10.51.169.21 in the cluster, will deploy and clone VMs across the cluster
    Deploying VM on lp-vdixdesx20.co.ihc.com…
    DEBUG: Timeout: 300
    Iteration 1: Trying to get host’s IP address …
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed

    0 0 0 0 0 0 0 0 –:–:– –:–:– –:–:– 0
    100 14.4M 100 14.4M 0 0 35.8M 0 –:–:– –:–:– –:–:– 35.8M
    Iteration 1: Trying to access nfcLease.info.entity …
    HttpNfcLeaseComplete succeeded
    Powering on VMs …
    [KPowerOnVM hci-tvm-vsanDatastore-1-1: running ❲ ❳
    [1A[KPowerOnVM hci-tvm-vsanDatastore-1-1: running ❲▬ ❳
    [1A[KPowerOnVM hci-tvm-vsanDatastore-1-1: running ❲▬▬▬ ❳
    [1A[KPowerOnVM hci-tvm-vsanDatastore-1-1: running ❲▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ❳
    [1A[KPowerOnVM hci-tvm-vsanDatastore-1-1: running ❲▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬❳
    [1A[KPowerOnVM hci-tvm-vsanDatastore-1-1: success
    [1A[1BWaiting for VMs to boot …
    green
    hci-tvm-vsanDatastore-1-1: 192.168.3.1
    PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
    From 192.168.2.1 icmp_seq=1 Destination Host Unreachable
    From 192.168.2.1 icmp_seq=2 Destination Host Unreachable

    — 192.168.3.1 ping statistics —
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1006ms
    pipe 2
    PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
    From 192.168.2.1 icmp_seq=1 Destination Host Unreachable
    From 192.168.2.1 icmp_seq=2 Destination Host Unreachable

    — 192.168.3.1 ping statistics —
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1006ms
    pipe 2
    PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
    From 192.168.2.1 icmp_seq=1 Destination Host Unreachable
    From 192.168.2.1 icmp_seq=2 Destination Host Unreachable

    — 192.168.3.1 ping statistics —
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1006ms
    pipe 2
    PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
    From 192.168.2.1 icmp_seq=1 Destination Host Unreachable
    From 192.168.2.1 icmp_seq=2 Destination Host Unreachable

    — 192.168.3.1 ping statistics —
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1006ms
    pipe 2
    PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
    From 192.168.2.1 icmp_seq=1 Destination Host Unreachable
    From 192.168.2.1 icmp_seq=2 Destination Host Unreachable

    — 192.168.3.1 ping statistics —
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1006ms
    pipe 2
    [KPowerOffVM hci-tvm-vsanDatastore-1-1: running ❲ ❳
    [1A[KPowerOffVM hci-tvm-vsanDatastore-1-1: running ❲▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬❳
    [1A[KPowerOffVM hci-tvm-vsanDatastore-1-1: success
    [1A[1B[KDestroy hci-tvm-vsanDatastore-1-1: running ❲ ❳
    [1A[KDestroy hci-tvm-vsanDatastore-1-1: running ❲▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬❳
    [1A[KDestroy hci-tvm-vsanDatastore-1-1: success
    [1A[1BCan’t Ping VM VirtualMachine(“vm-107”) by IP 192.168.3.1
    254
    IP Assignment failed or IP not reachable

    Please suggest.

  8. Priya,

    Please move this comment to fling comment site.
    also, the host you specified doenst match the way how they added into the cluster.

    you may also want to check 3 things:
    1. was hcibench eth1 mapped to VM Network too?
    2. can the physical hosts talk to each other through VM Network?
    3. is HCIBench appliance in the same cluster?

    thanks
    Chen

Leave a Reply

Your email address will not be published.