UPDATE: Courtesy of Dirk Hohndel, VP and Chief Open Source Officer, and Andrew Hancock, Managing Director and Technical Architect at Cyrus Computer Consultants Ltd, we have the low-down on Synology.
Ever wonder if ESXi-Arm can be installed to an iSCSI LUN? Wonder no more! This short guide will show you how to do this with the Raspberry Pi 4B, although it also applies to any supported Arm system with firmware based on TianoCore.
For the Pi, this is a way to build a Pi cluster with ESXi, without relying on USB sticks and to achieve better I/O performance, as the NIC is gigabit, while the Pi’s USB implementation will max out with ESXi-Arm at around 200 megabit.
Sorting out your LUNs
Everyone’s SAN is a bit different.
QNAP
I’ll show you how I configured my QNAP. It’s not proscriptive, I’m just describing what worked for me.
Note: QNAP’s LUNs start at 0.
The most important item here is the IQN. I also don’t bother with authentication. Perhaps it works great, but in my home environment, it’s just one less thing to worry about.
I do set a few advanced settings and these appear to be okay.
Synology
Synology tells you what number to use for your created volumes. If you have multiple LUNs in the same target, you need to match the value in Number column.
Note: Synology LUNs start at 1.
Others
FreeNAS, etc, configuration will be different. You may have different or additional settings. Experiment, especially if you start seeing “strange” behavior, such as the target not visible in UEFI, or having “read-only” access in ESXi, such as the installer failing to format the LUN. Have a ready recipe for your SAN? Let us know in a comment and we’d love to reference your blog post or comment!
Configuring firmware
This is the most interesting part, and fairly unintuitive. Now some environments may support automatic detection of iSCSI configuration via DHCP, but I’ll be showing how to manually add the LUNs as boot options. TianoCore edk2 UEFI of course. For any “official” server OEM firmware follow the vendor instructions, as usually the firmware interface is radically different.
Of course you’ll start at the main UEFI setup page. On the Pi, you can reach this screen by mashing the ESC key. Use arrow keys to select Device Manager.
Press ENTER. Now select iSCSI Configuration.
Press ENTER. Now select the iSCSI Initiator Name field. Press ENTER. A text entry box will pop up. Enter something here that matches IQN format. Press ENTER.
Now select Add an Attempt and press ENTER.
Select the NIC that you will use for the iSCSI boot attempt. On the Pi, there’s only one on-board NIC, so just press ENTER.
Now, navigate to the iSCSI Mode field, press ENTER, and select Enabled, pressing ENTER to complete selection. Now in my setup, I am using IPv4. Navigate to Enable DHCP and press SPACEBAR to enable.
This is the most important info: enter the IQN correctly under Target Name. If you mistype it here, don’t correct it as it won’t “stick”. Exit out and try re-entering the entire form (I think it’s a TianoCore bug). You also need the server connection info and the LUN ID.
I disable CHAP as that’s how I configured my QNAP.
Now navigate to Save Changes.
Press ENTER. You should see your added iSCSI boot attempt listed.
Exit out (ESC) all the way to the main setup screen and select Reset.
Press ENTER. Your Pi will reboot. Hit ESC to enter UEFI setup again. It will take a bit of time. Select Boot Manager.
Press ENTER. You should see the iSCSI target listed here.
Of course now you can boot the ESXi installer (which in my case is the USB drive).
Installing ESXi
This is fairly normal. You should see your iSCSI LUN under Remote devices. Any errors during installation at this point are highly likely iSCSI target misconfiguration on the SAN, so heed VMware best practices!
Note: Yeah, my LUN is ridiculously small as I use NFSv3 for my VM storage. But you could be using another iSCSI LUN for that, sharing it with all hosts in the cluster.
Finishing up
Like with any other installation on the Pi, you will need to change the boot order to make sure your LUN is the first in the list.
Navigate to Boot Maintenance Manager from the main UEFI setup screen.
Press ENTER. Select Boot Options.
Press ENTER. Select Change Boot Order.
Press ENTER. Select Change the order.
Press ENTER. Now select your iSCSI LUN entry and use the + key to move it up the list.
Press ENTER. Select Commit Changes and Exit.
Press ENTER.
What could go wrong?
If there are networking connectivity issues the iSCSI boot option will not be listed in the Boot Manager. This includes obvious stuff (NIC cable fell out, SAN is off) but also invalid IQN configuration.
Note that the attempt is still listed in the iSCSI Configuration portion of UEFI setup!
Important: if you made any errors in the config, delete and re-create the attempt. There’s a UEFI bug where the attempt configuration won’t be updated.
As soon as the problem is fixed (i.e. NIC cable is back), the boot option will re-appear, even at the same spot/ordering as it was before.
Hope that helps folks out!
P.S.: if you have any field reports of other SANs (no matter how big or small), we’d love to cover them. It seems like some folks are struggling with Synology ones, so if you have things working, let us know via a comment 👍.
