Product Announcements

Ops changes part 3 – Local disk vs USB vs BFS

When I was planning this series of articles one of the things that was on the top of my list was to "compare" the type of install destinations that could be used as I guess the variety of options is the strength of ESXi. The reason, as hopefully all of you know by now, that ESXi can be installed on a large range of different devices is the size of the system image.The system image is not only tiny, but ESXi is loaded into memory and will only write its configuration roughly once every 10 minutes when it is booted.

Today I want to discuss the variety of options which I have combined in three major categories which I have listed the possibilities below:

  • Local disk (including SSD)
  • Removable Media
    • USB
    • SD
  • Boot From San
    • FC
    • iSCSI

As you can see there is a large variety of options which of course leaves you with a decision to make. We cannot make this decision for you but we can inform you about some of the challenges that you might face as some of the options have some technical restrictions of for instance restrictions from a support perspective.

Local Disk

Local disk is one of the options, together with removable devices, that is the most used. Local disk installations provide two huge advantages over removable devices; those are resiliency and the level of automation.

Resiliency refers to the ability to run two local disks in RAID-1. As mentioned, although ESXi is loaded into memory it will need to write it's configuration once every 10 minutes and of course in the case of a single disk failure a reboot can still be successfully completed without the need to immediately replace a component. On top of that Local Disk enables you to implement a scripted installation mechanism. This is also supported for Boot From SAN (iSCSI/FC) but currently not supported for removable devices like USB and SD media.

Requirements:

  • Minimum disk space required is 5GB

Constraints/Impact:

  • Expensive compared to removable devices like USB

Removable Devices

Removable devices like USB and SD have always been one of the favorite ESXi install destinations for many. The main reason for it being is the "flexibility" and of course the costs associated with a USB/SD device. Flexibility meaning that usually USB/SD devices are easy to swap when an issue arises. As mentioned in the Local Disk section a removable device offers less resiliency, however some hardware vendors have found a solution for that by offering a so called "Dual SD Module". (Dell in this case.)

From an operational perspective the flexibility, and resiliency depending on the hardware vendor, seems to offer a lot of benefits. One thing to note, and this is something that many don't realize, is that it is currently not supported to do a scripted install of ESXi on a removable device. Of course it is fairly easy to convert your kickstart script into a PowerCLI script or combine it with Host Profiles (if your license permits) but it still should be noted.

Requirements (src kb):

  • VMware supports Removable Devices only under these conditions: 
    • The server on which you want to install ESXi 4.x is on the ESXi 4.x Hardware Compatibility Guide.

      AND

    • You have purchased a server with ESXi 4.x Embedded on the server from a certified vendor.

      OR

    • You have used a USB or SD flash device that is approved by the server vendor for the particular server model on which you want to install ESXi 4.x on a USB or SD flash storage device.

Constraints/Impact:

  • Scripted installations of ESXi on removable devices is not supported

I hope this should make it crystal clear that in the case of USB/SD devices it is really important to ensure that your environment is fully supported.

Boot From SAN

The last but definitely not the least option is Boot From SAN. Boot From SAN is an option that was added when 4.1 was released. Not only is FC supported but also iSCSI. Boot From SAN is kind of the best of both worlds, where in this case both worlds would be Local Disks and USB. Boot From SAN enables you to run ESXi on a disk-less server while still providing you with the option to do a scripted install and provide you with resiliency.

Requirement:

  • Support for Boot From SAN for storage device and adapters

Constraints/Impact:

  • Budget, SAN storage can be expensive compared to local disks / removable media

Conclusion

Every option has its benefits and depending on the type of environment all should be considered. Based on requirements and constraints around budget, licensing, array capabilities a decision will need to be made on a per case basis. Generally speaking using "local disks" is the most compelling option as it enables you to script your installation and is relatively cheap compared to Boot From SAN.