In cases where customers are using numerous pRDMs in their environment, host boot times or storage rescans can take a long time. The reason for the longer scan times is each LUN attached to a host is scanned at boot or during a storage rescan. Typically, RDMs are provisioned to VMs for Microsoft WSFC and are not directly used by the host. During the scan, ESXi attempts to read the partitions on all the disks but it is unable to for devices persistently reserved by the WSFC. Subsequently, the longer it can take a host to boot or rescan storage. The WSFC uses SCSI-3 persistent reservation to control locking between the nodes of the WSFC which, blocks the hosts from being able to read them.
VMware recommends implementing perennial reservation for all ESXi hosts hosting VM nodes with pRDMs. Check the following kb1016106 or more details
The question then arises; How can you get the host not to scan these RDMs and reduce boot or rescan times? I’m glad you asked!
There is a device flag called “Perennially Reserved” which tells the host the RDM should not be scanned and is used elsewhere (perennially) in the environment. Before vSphere 7, this flag is enabled via CLI and requires the UUID (naa.ID).
The command to set the flag to true:
esxcli storage core device setconfig -d naa.id --perennially-reserved=true
To verify the setting:
esxcli storage core device list -d naa.id
In the device list you should see:
Is Perennially Reserved: true
When setting this option, it must be run for each relevant RDM used by the WSFC and on every host with access to that RDM. You can also set the Perennially Reserved flag in Host Profiles.
With the release of vSphere 7, setting the Perennially Reserved flag to true was added to the UI under storage devices. There has also been a field added to show the current setting for the Perennially Reserved flag.
Once you select the RDM to be Perennially Reserved, you have the option to “Mark the selected device as perennially reserved” for the current host or multiple hosts in the cluster. This eliminates the manual process of setting the option per host via CLI. If preferred, you can still use ESXCLI, PowerCLI or Host Profiles.
Once you click YES, the flag will be set to true and the UI updated.
You may also unmark the device using the same process.
Short Clip showing process:
Setting the Perennially Reserved flag on your pRDMs used by your WSFC is recommend in the clustering guides. When set, ESXi no longer tries to scan the devices and this can reduce boot and storage rescan times. I have added links below to resources on clustering guides and the use of this flag. Another benefit of flagging RDMs is you can easily see which devices are RDMs and which are not.
Resources:
- ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS/WSFC nodes with RDMs may take a long time to start or during LUN rescan (1016106)
- Guidelines for Microsoft Clustering on vSphere (1037959)
- Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 7.x: Guidelines for supported configurations (79616)
- Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 6.x: Guidelines for supported configurations (2147661)
