Useful VXLAN commands in ESXCLI 5.1

Recently I have been doing some work with VXLAN with my colleagues Venky Deshpande who is responsible for vCloud Networking and Ranga Maddipudi who is responsible for vCloud Security within our technical marketing team (I call them the vCloud Networking & Security Duo). While working in our lab, I came across several VXLAN commands in ESXCLI that I thought might come in handy when configuring or troubleshooting a VMware VXLAN environment. The new VXLAN namespace in ESXCLI 5.1 provides both VXLAN configuration details as well network statistics for an individual ESXi host.

Before jumping straight into the VXLAN commands, here is a quick diagram of our VXLAN lab environment. There are a total of four ESXi hosts, two in Cluster1 on VLAN 2000 and two in Cluster2 on VLAN 2010. There are also three vWire’s with segment ID 5000, 5001 and 5002 which are consumed by several virtual machines.

Here is a quick table of ESXi host configurations

To view the available VXLAN commands, run the following command:

esxcli network vswitch dvs vmware vxlan

Here is an example output of the various sub-namespaces that are available:

~ # esxcli network vswitch dvs vmware vxlan
Usage: esxcli network vswitch dvs vmware vxlan {cmd} [cmd options]

Available Namespaces:
config                A set of commands for VXLAN config related operations
network             A set of commands for VXLAN network related operations
vmknic               A set of commands for VXLAN vmknic related operations
stats                  A set of commands for VXLAN switch statistics related operations

Available Commands:
list                     Get VXLAN switches info on the system.

To list all available VXLAN VDS, run the following command:

esxcli network vswitch dvs vmware vxlan list

Here is a screenshot output:

In addition to the VDS details, you will be able to see the number of unique vWires (Virtual Wires) that are currently being consumed by powered on virtual machines which is denoted by the “Network Counter” property. From the screenshot above, you can see there are currently three virtual machines consuming three unique vWires. This counter will automatically adjust itself as virtual machines are powered on and off.

Configuring VXLAN Statistics Level

By default, VXLAN statistics level is set to 0 which will only display error stats (all drops and mapping full/flood). To be able to see all statistics, you will need to set the statistics level to 1.

To view the current VXLAN statistics level, run the following command:

esxcli network vswitch dvs vmware vxlan config stats get

To change the statistics level from 0 to 1, run the following command:

esxcli network vswitch dvs vmware vxlan config stats set –level 1

Here is a screenshot output:

You will be able to view VXLAN statistics for the following :

• On a per VDS (localized to ESXi host)
• On a per VTEP VMkernel interface
• On a per VXLAN Segment ID
• On a per VDS Port ID

VXLAN Statistics for a VDS

To view statistics for a particular VDS, run the following command and specifying –vds-name which is the name of your VDS:

esxcli network vswitch dvs vmware vxlan stats list –vds-name Cluster01-VXLAN-VDS

Here is a screenshot output:

VXLAN Statistics for a VTEP VMkernel Interface

To view statistics for a particular VTEP (VXLAN Tunnel End Point) VMkernel interface, we will first need to list the available interfaces. To do so, so run the following command and specify –vds-name option:

esxcli network vswitch dvs vmware vxlan vmknic list –vds-name Cluster01-VXLAN-VDS

Here is a screenshot output:

In addition to the VMkernel interface the VTEP is using, we can also see the VLAN ID as well as the VTEP’s IP Address. With the VMKernel details, we can now view the statistics by running the following command and specify –vds-name and –vlan-id option:

esxcli network vswitch dvs vmware vxlan vmknic stats list –vds-name Cluster01-VXLAN-VDS –vlan-id 2000

Here is a screenshot output:

If you want to see all the multicast group joins, you can do so by running the following command and specify –vds-name and –vlan-id option:

esxcli network vswitch dvs vmware vxlan vmknic multicastgroup list –vds-name Cluster01-VXLAN-VDS –vlan-id 2000

Here is a screenshot output:

VXLAN Statistics for a VXLAN Segment ID

To view statistics for a particular VXLAN Segment ID, we will first need to list the available segment IDs. To do so, so run the following command and specify –vds-name option:

esxcli network vswitch dvs vmware vxlan network list –vds-name Cluster01-VXLAN-VDS

Here is a screenshot output:

To view the network statistics for a particular segment ID, run the following command and specify –vds-name and –vxlan-id option:

esxcli network vswitch dvs vmware vxlan network stats list –vds-name Cluster01-VXLAN-VDS –vxlan-id 5000

Here is a screenshot output:

If there is some virtual machine communication occurring, you can easily retrieve the network mapping by running the following command and specify –vds-name and –vxlan-id

esxcli network vswitch dvs vmware vxlan network mapping list –vds-name Cluster01-VXLAN-VDS –vxlan-id 5000

Here is a screenshot output:
The Inner MAC is the MAC Address of the virtual machine on the ESXi host and the Outer MAC is the VTEP’s VMkernel MAC Address along with it’s IP Address and VLAN ID.

VXLAN Statistics for a VDS Port ID

Lastly, to view statistics on individual VDS port ID, we will need to first list all available VDS port ID’s. To do so, so run the following command and specify –vds-name and –vxlan-id option:

esxcli network vswitch dvs vmware vxlan network port list –vds-name Cluster01-VXLAN-VDS –vxlan-id 5000

Here is a screenshot output:

Note: If you wish to map the VDS Port ID to a particular Virtual Machine, you can use the vSphere Web Client under Manage->Ports section of the particular VDS.

To view network statistics for a particular VDS Port ID, run the following command and specify –vds-name, –vxlan-id and –vdsport-id:

esxcli network vswitch dvs vmware vxlan network port stats list –vds-name Cluster01-VXLAN-VDS –vxlan-id 5000 –vdsport-id 968

Here is a screenshot output:

If you are interested in more details about VXLAN, I would highly recommend you follow both Venky and Ranga who blogs under the Networking section of the vSphere Blog.

Get notification of new blog postings and more by following lamw on Twitter:  @lamw