I am sorry that it took me a while to do this post on VXLAN. I was on summer vacation to India for three weeks and then got busy with the VMworld 2013 preparation. I hope you have registered for VMworld US conference. If you are interested in learning more about network virtualization please checkout the catalogue. I have couple of sessions in the networking track so hope to see you there.
After that shameless promotion about my sessions and before I jump in to explain the vMotion packet flow in VXLAN deployment, I want to address some of the questions that were raised in the comments section of the last blog. Please refer to the diagrams in the blog here.
Q1) I have a doubt. What happens if an unicast packet is sent from VM1 to VM2 and VTEP doesn’t have any entry for VM2?
Answer – If the VTEP hasn’t learned any association of VMs to VTEP, it will send a multicast traffic out on the multicast address associated with that logical network (where VM1 is connected). This is similar to a Switch sending a broadcast when it doesn’t find the destination MAC address in the forwarding table. Only difference is that Multicast packets are used instead of Broadcast.
Q2) Is it possible to have one VM connected to two logical layer 2 network and if yes how will be VTEP table after learning?
Answer – Yes, it is possible. The two vNICs on the VM will have unique MAC addresses – MAC-a and MAC-b and those will be associated with different logical segment IDs, but same VTEP.
Q3) In third pic from top. Let say if Host-2 receive multicast packet from Host-1 and Host-2 doesn’t have any VM on VXLAN 5001. Still VTEP on Host-2 will update forwarding table with VXLAN 5001 info?
Answer – If the Host 2 doesn’t have any VMs, the host VTEP will not join any multicast group. Thus it will not receive any multicast traffic. This is how the traffic optimization happens. Unless the Host has VMs connected to logical networks that VTEP will not join the multicast group and also will not receive any un-necessary traffic.
Hope this helps. Let’s now take a look at the packet flow when a VM connected on a logical network is moved from one host to another. We will look at the changes that happen in each Host’s VTEP forwarding table entries while going through this packet flow.
As shown in the diagram below, Host 1 and Host 2 VTEPs have learnt about the VMs connected to the logical network (VXLAN 5001).
Example Deployment with two Hosts – Forwarding table entries
When the VM-MAC1 is moved from Host 1 to Host 2 through vMotion process a Reverse ARP (RARP) is issued by Host2’s kernel module on behalf of the VM-MAC1.
RARP packet sent from Host 2
The diagram above shows the packet flow:
Once the vMotion process is complete the Host2 sends RARP packet with Destination MAC as “FFFFFFFFFFF”.
VTEP on Host 2 encapsulates the Ethernet broadcast packet into a UDP header with Multicast address “239.1.1.100” as the destination IP address and VTEP address “10.20.10.11” as the Source IP address.
The physical network delivers the multicast packet to the hosts that joined the multicast group address “239.1.1.100”.
Updated VTEP forwarding tables on Host 1 and Host 2
The VTEP on Host 1 receives the encapsulated packet. Based on the outer and inner header, it makes an entry in the forwarding table that shows the mapping of the virtual machine MAC address and the VTEP. In this example, the virtual machine MAC1 running on Host 1 is associated with VTEP IP “10.20.10.11”. VTEP also checks the segment ID or VXLAN logical network ID (5001) in the external header to decide if the packet has to be delivered on the host or not.
The packet is dropped because the virtual machine MAC 1 is not connected on that logical network VXLAN 5001.
The entries in the VTEPs will age out if no traffic is seen on that particular logical network as it happens in any layer 2 switches.
I hope you found this VXLAN series helpful and please post your questions in the comments section. Also, please let me know if you want me to cover any other scenario in the VXLAN environment.
