posted

38 Comments

In 5.0, VMware has added a new UI interface to make it much easier to configure multipathing for Software iSCSI. The UI is a major change from 4.x. Previously users needed to use the command-line to get an optimal multi-path configuration with Software iSCSI.

The UI allows one to select different network interfaces for iSCSI use, check them for compliance and configure them with the Software iSCSI Adapter. This multipathing configuration is also referred to as iSCSI Port Binding.

Why should I use iSCSI Multipathing?

The primary use case of this feature is to create a multipath configuration with storage that only presents a single storage portal, such as the DELL EqualLogic and the HP/Lefthand. Without iSCSI multipathing, these type of storage would only have one path between the ESX host and each volume.  iSCSI multipathing allows us to multipath to this type of clustered storage.

Another benefit is the ability to use alternate VMkernel networks outside of the ESXi Management network. This means that if the management network suffers an outage, you continue to have iSCSI connectivity via the VMkernel ports participating in the iSCSI bindings.

Let's see how you go about setting this up. In this example, I have configured a Software iSCSI adapter, vmhba32.

No-devices

At present, no targets have been added, so no devices or paths have been discovered. Before implementing the iSCSI bindings, I need to create a number of additional VMkernel ports (vmk) for port binding to the Software iSCSI adapter.

Vmkernel-nw

As you can see from the above diagram, these vmnics are on trunked VLAN ports, allowing them to participate in multiple VLANs. For port binding to work correctly, the initiator must be able to reach the target directly on the same subnet – iSCSI port binding in vSphere 5.0 does not support routing. In this configuration, if I place my VMkernel ports on VLAN 74, they will be able to reach the iSCSI target without the need of a router. This is an important point, and needs further elaboration as it causes some confusion. If I do not implement port binding, and use a standard VMkernel port, then my initiator can reach the targets through a routed network. This is supported and works just fine. It is only when iSCSI binding is implemented that a direct, non-routed network between the initiators and targets is required, i.e. initiators and targets must be on the same subnet.

There is another important point to note when it comes to the configuration of iSCSI port bindings. On vSwitches which contain multiple vmnic uplinks, each VMkernel (vmk) port used for iSCSI bindings must be associated with a single vmnic uplink. The other uplink(s) on the vSwitch must be placed into an unused state. See below for an example of such a configuration:

Unused-vmnic

This is only a requirement when you have multiple vmnic uplinks on the same vSwitch. If you are using multiple vSwitches with their own vmnic uplinks, then this isn't an issue. Continuing with the network configuration,  we create a second VMkernel (vmk) port. I now have two vmk ports, labeled iscsi1 & iscsi2. These will be used for my iSCSI binding. Note below that one of the physical adapters, vmnic1, appeares disconnected from the vSwitch. This is because both of my VMkernel ports will be bound to vmnic0 only, so vmnic1 has been set to unused across the whole of the vSwitch.

Vmkernel-nw4Next, I return to the properties of my Software iSCSI adapter, and configure the bindings and iSCSI targets. There is now a new Network Configuration tab in the Software iSCSI Adapter properties window. This is where you add the VMkernel ports that will be used for binding to the iSCSI adapter. Click on the Software iSCSI adapater properties, then select the Network configuration tab, and you will see something similar to the screenshot shown below:

Bindings1

After selecting the VMkernel adapters for use with the Software iSCSI Adapter, the Port Group Policy tab will tell you whether or not these adapters are compliant or not for binding. If you have more than one active uplink on a vSwitch that has multiple vmnic uplinks, the vmk interfaces will not show up as compliant. Only one uplink should be active, all other uplinks should be placed into an unused state.

Bindings2
You then proceed to the Dynamic Discovery tab, where the iSCSI targets can now be added. Now, because we are using port binding, you must ensure that these targets are reachable by the Software iSCSI Adapter through a non-routeable network, i.e the storage controller ports are on the same subnet as the VMkernel NICs:

Dynamic-discovery
At this point, I have two vmkernel ports bound to the Software iSCSI Adapter, and 4 targets. These 4 targets are all going to the same storage array, so if I present a LUN out on all 4 targets, this should give me a total of 8 paths. Let's see what happens when I present a single LUN (ID 0):

Devices-8paths
So it does indeed look like I have 8 paths to that 1 device. Let's verify by looking at the paths view:

8paths
And if I take a look at a CLI multipath output for this device, I should see it presented on 8 different targets:

~ # esxcfg-mpath -l -d naa.6006016094602800c8e3e1c5d3c8e011 | grep "Target Identifier"
   Target Identifier: 00023d000002,iqn.1992-04.com.emc:cx.ckm00100900477.a2,t,1
   Target Identifier: 00023d000001,iqn.1992-04.com.emc:cx.ckm00100900477.a2,t,1
   Target Identifier: 00023d000002,iqn.1992-04.com.emc:cx.ckm00100900477.b3,t,4
   Target Identifier: 00023d000001,iqn.1992-04.com.emc:cx.ckm00100900477.b3,t,4
   Target Identifier: 00023d000002,iqn.1992-04.com.emc:cx.ckm00100900477.b2,t,3
   Target Identifier: 00023d000001,iqn.1992-04.com.emc:cx.ckm00100900477.b2,t,3
   Target Identifier: 00023d000002,iqn.1992-04.com.emc:cx.ckm00100900477.a3,t,2
   Target Identifier: 00023d000001,iqn.1992-04.com.emc:cx.ckm00100900477.a3,t,2

 
This new UI for iSCSI Bindings certainly makes configuring multipathing for the Software iSCSI Adapter so much easier. But do keep in mind the requirements to have a non-routable network between the initiator and target, and the fact that vmkernel ports must have only a single active vmnic uplink in vSwitches that have multiple vmnic uplinks.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: Twitter VMwareStorage

About the Author

Cormac Hogan

Cormac Hogan is a Senior Staff Engineer in the Office of the CTO in the Storage and Availability Business Unit (SABU) at VMware. He has been with VMware since April 2005 and has previously held roles in VMware’s Technical Marketing and Technical Support organizations. He has written a number of storage related white papers and have given numerous presentations on storage best practices and vSphere storage features. He is also the co-author of the “Essential Virtual SAN” book published by VMware Press.