In the last six months, I have talked to many customers and partners on Virtual eXtensible Local Area Network (VXLAN). One of the things I felt was challenging was how to explain the technology to two different type of audience. On one hand, there are Virtual Infrastructure administrators who want to know what problems this new technology is going to solve for them and what are the use cases. While on the other hand, there are Networking folks who want to dig into packet flows and all the innate protocol level details, how this technology compares with others, and what is the impact of this on the physical devices in the network etc.
The papers that we have made available “Network virtualization Design Guide” and “VXLAN Deployment Guide”, provides some basic knowledge about the technology, Use cases, and step-by-step deployment instructions. However, some of the detailed packet flow scenarios are not explained in these papers. So I thought it would be a good idea to put together a series of post discussing the packet flows in a VXLAN environment. Also, there are many common questions that I would like to address as part of this series.
To start this series, I will first describe the different components of the VMware’s VXLAN implementation.
VXLAN Components
The diagram above shows a deployment of two compute clusters that is configured with VXLAN components running on each vSphere host.
VXLAN is an overlay network technology. Overlay network can be defined as any logical network that is created on top of the existing physical networks. VXLAN creates Layer 2 logical networks on top of the IP network. The following two are key traits of an overlay technology:
– It encapsulates original packets into a new header. For example, IPSec VPN, an overlay technology, encapsulates original IP frame in another IP header.
– Communication is typically established between two tunnel end points. For example, in an IPSec based VPN, which runs on the public internet, the tunnels are established between two sites.
When you apply those overlay technology traits to VXLAN, you will see that VXLAN encapsulates original MAC frames in to a UDP header (shown below), and all vSphere hosts participating in VXLAN acts as tunnel end points. They are called Virtual Tunnel Endpoints (VTEPs).
VXLAN – Encapsulation Header
VTEPs are the nodes that provide the encapsulation and de-encapsulation function. When we will go through the detail packet flows it will be clear how these VTEPs encapsulate and de-encapsulate traffic from any virtual machine connected to a VXLAN based Layer 2 logical network or virtual wire. The virtual tunnel endpoint (VTEP) configured on every vSphere host consists of the following three modules:
1) VMware Installation Bundle (VIB) or vmkernel module – VTEP functionality is part of the VDS and is installed as a VMware Installation Bundle (VIB). This module is responsible for VXLAN data path processing, which includes maintenance of forwarding tables and encapsulation and de-encapsulation of packets.
2) vmknic virtual adapter – This adapter is used to carry control traffic, which includes response to multicast join, DHCP, and ARP requests. As with any vmknic, a unique IP address is assigned per host. The IP address is used as the VTEP IP while establishing host-to-host tunnels to carry VXLAN traffic.
3) VXLAN port group – This is configured during the initial VXLAN configuration process. It includes physical NICs, VLAN information, teaming policy, and so on. These port group parameters dictate how VXLAN traffic is carried in and out of the host VTEP through the physical NICs. As shown in the diagram, VLAN 2000 is used as the transport VLAN for VXLAN traffic. The transport VLAN has no relation to the logical Layer 2 networks or virtual wires that you will create.
The configuration of the VTEP on each vSphere host is managed through a central place called vCloud Networking and Security Manager. One of the common questions I get is whether this manager acts as a controller similar to the Openflow controller. The answer is No. In VXLAN there is no special controller or control plane required. So then the question is how in VXLAN a forwarding table is created ? In physical switch infrastructure the forwarding table information helps deliver packets to the right destination.
In VXLAN all the learning about the virtual machine MAC address and its association with VTEP IP is performed through the support of physical network. One of the protocols utilized in the physical network is IP multicast. VXLAN makes use of this IP multicast protocol to populate the forwarding tables in the VTEP.
Before we dig into how IP multicast is utilized in VXLAN, in the next blog, we will take a look at some basics on IP Multicast.
