VMware vSphere Blog

In this post I am going to describe how VTEPs learn about the virtual machines connected to the logical Layer 2 networks. The learning process is quite similar to a transparent bridge function. As transparent bridges learn based on the packets received on the bridge ports, the VTEP also learn based on the inner and outer header of the packets received.

Let’s take an example to illustrate the VTEP learning process.

Example Deployment with Two Hosts

As shown in the diagram above there are two Hosts (Host1, Host 2) on which VTEPs are configured, and each host has one virtual machine connected to logical layer 2 network, identified as VXLAN 5001. Both the virtual machines are powered on and both VTEPs have joined the multicast group 239.1.1.100. Each VTEP has its own forwarding table, which is initially empty as shown in the diagram below.

Initial State of the Forwarding Table

How do the forwarding tables get populated?

We will take an example of virtual machine on Host 1 trying to communicate with the virtual machine on the Host 2. First, an ARP request is sent from the virtual machine MAC1 to find the MAC address of the virtual machine on Host 2. The ARP request is a broadcast packet.

Host 2 VTEP – Forwarding table entry

The diagram above shows the packet flow:

1. Virtual machine on Host1 sends ARP packet with Destination MAC as “FFFFFFFFFFF”
2. VTEP on Host 1 encapsulates the Ethernet broadcast packet into a UDP header with Multicast address “239.1.1.100” as the destination IP address and VTEP address “10.20.10.10” as the Source IP address.
3. The physical network delivers the multicast packet to the hosts that joined the multicast group address “239.1.1.10”.
4. The VTEP on Host 2 receives the encapsulated packet. Based on the outer and inner header, it makes an entry in the forwarding table that shows the mapping of the virtual machine MAC address and the VTEP. In this example, the virtual machine MAC1 running on Host 1 is associated with VTEP IP “10.20.10.10”. VTEP also checks the segment ID or VXLAN logical network ID (5001) in the external header to decide if the packet has to be delivered on the host or not.
5. The packet is de-encapsulated and delivered to the virtual machine connected on that logical network VXLAN 5001.

The entry in the forwarding table of Host 2 VTEP is used during lookup process. The packet flow shown in the diagram below explains the forwarding table lookup for a unicast packet sent from a virtual machine on Host2.

Host 2 VTEP – Forwarding table Lookup

1. Virtual Machine MAC2 on Host 2 responds to the ARP request by sending a unicast packet with Destination Ethernet MAC address as MAC1.
2. After receiving the unicast packet, the VTEP on Host 2 performs a lookup in the forwarding table and gets a match for the destination MAC address “MAC1”. The VTEP now knows that to deliver the packet to virtual machine MAC1 it has to send it to VTEP with IP address “10.20.10.10”.
3. The VTEP creates unicast packet with destination IP address as “10.20.10.10” and sends it out.

The Host1 VTEP receives the unicast packet and it also learns about the location of the virtual machine MAC2 as shown in the diagram below.

Host 1 VTEP – Forwarding table entry

1. The packet is delivered to Host1
2. The VTEP on Host 1 receives the encapsulated packet. Based on the outer and inner header, it makes an entry in the forwarding table that shows the mapping of the virtual machine MAC address and the VTEP. In this example, the virtual machine MAC2 running on Host 2 is associated with VTEP IP “10.20.10.11”. The VTEP also checks segment ID or VXLAN logical network ID (5001) in the external header to decide if the packet has to be delivered on the host or not.
3. The packet is de-encapsulated and delivered to the virtual machine connected on that logical network VXLAN 5001.

As you can see the forwarding table entries are populated based on the inner and outer header fields of the encapsulated packet. Similar to the transparent bridge the forwarding table entries are removed after aging timer expires. One of the common questions I get is what happens after a virtual machine is vMotioned.

In the next few posts I will cover how the forwarding table entries get modified after vMotion of a virtual machine from one host to another.

Here are the links to Part 1, Part 2, Part 3, Part 4.

Get notification of these blogs postings and more VMware Networking information by following me on Twitter:  @VMWNetworking