In earlier releases of ESXi, a VMkernel interface could transport three types of traffic: Management, vMotion and Fault Tolerance. To enable a particular traffic type, one would use either the vSphere Web/C# Client or the vSphere API. Some of you may have recalled using an undocumented command-line tool called vim-cmd in the ESXi Shell to enable vMotion and Fault Tolerance traffic. An issue with this tool is it does not support the Management traffic type. This made it a challenge to completely automate the provisioning of VMkernel interfaces from a scripted installation (kickstart) perspective and required the use of remote CLI/APIs to enable the Management traffic type.
Tag Archives: vMotion
My colleague Frank Denneman recently asked me whether there was a new vSphere API for the new enhancement made to vMotion in vSphere 5.1, which would allow you to migrate a live running virtual machine without the need of shared storage. The simple answer is no, the new enhancement to vMotion operation has been integrated into the existing Storage vMotion API call which is the RelocateVM_Task method.
If you specify both the datastore and host property in the VirtualMachineRelocateSpec to the RelocateVM_Task method, you will be performing a combined vMotion + Storage vMotion. If you just specify the datastore property, then a regular Storage vMotion will be performed. For a regular vMotion operation, you will need to use the MigrateVM_Task method. To get more details, please refer to the vSphere API Reference Guide.
To demonstrate how the RelocateVM_Task method works for a combined vMotion + Storage vMotion, I created a simple vSphere SDK for Perl script called migrateVM.pl. You will need a system with the vCLI installed or use the vMA appliance.
Disclaimer: These script are provided for informational/educational purposes only. It should be thoroughly tested before attempting to use in a production environment.
In the example below, I have two ESXi 5.1 hosts (172.30.0.243 & 172.30.0.251) and they both contain only local datastores (local-datastore-2 & local-datastore-3). I will be migrating a virtual machine called FRANK which resides on one of the ESXi host on local storage to the other ESXi host with only local storage.
FRANK currently resides on ESXi host 172.30.0.251 and datastore local-datastore-3 as seen in the screenshot of our vSphere Web Client.
Now we perform the combined vMotion + Storage vMotion by connecting to our vCenter Server and specifying the –vmname (Virtual Machine to migrate), –vihost (ESXi host to migrate to) and –datastore (datastore to migrate to ) property.
Get notification of new blog postings and more by following lamw on Twitter: @lamw
In the last post Demystifying port limits… I discussed the vitual port limits on the vSphere Standard (VSS) and Distributed switch (VDS). While discussing the VDS limits, I talked about the three different port-binding options available when you configure a port group on VDS. The port binding option describes how a virtual port on the virtual switch binds with virtual machine or a vmkernel nic. In this post, I would like to highlight why you should choose Static Port binding over Ephemeral port binding.
As per the definition of Ephemeral binding, there is no port binding with this choice. When you choose this option the behavior is similar to a standard virtual switch (VSS). The number of virtual ports on the switch is automatically set to 0, and the port group allocates one port for each connected virtual machine or vmkernel nic, up to the maximum number of ports available on that port group. For Example, if a virtual machine 1 is connected to a port group and is powered on, it connects to a virtual port ID #1. Now, if you power off that virtual machine the connection to the virtual port ID #1 is lost. And if you start another virtual machine 2, it will get the virtual port ID #1. So, there is no static one-to-one relation between the virtual machine and virtual port ID #s when it comes to Ephemeral port binding.
Now the question is why do you need to have this static one-to-one relationship between a virtual machine and virtual port IDs? Think of a physical switch, where you connect a server or a pc to switch port. This environment is not dynamic as virtual infrastructure with virtual machines and virtual switch. The network and security administrators like this physical setup, because they can state fully monitor the physical switch ports for any network or security issues that could be caused by the servers or pcs. So, having a static binding between virtual machine and virtual ports helps you better troubleshoot any network issues and also identify any potential security issues.
With VDS, when you keep the default static binding configuration on a port group, you get the following benefits
- Port state persistence helps in troubleshooting network issues
- Helps Firewall/IDS/IPS devices that need state full ports
- Monitoring and Accounting application traffic
- Port state migrated with vMotion
Thus, provides the same benefits that you get in the physical network.
In the Following section, I would like to illustrate how VDS helps you continuously monitor a virtual machine that gets moved from one host to another. Let’s take an example deployment with two hosts, in one deployment we will take a VSS with ephemeral port binding, and in another a VDS with static port binding configuration.
As you can see in the above diagram there are two hosts, each with its own vSphere Standard Switch (VSS). Both VSS have two port groups configured PG-A and PG-B with similar properties. A virtual machine that is hooked to PG-A on Host 1 has virtual port ID vport 1 and another has virtual port ID vport2. Similarly, Host 2 virtual machines have virtual port ID #s that are similar to Host 1. Now, if the red virtual machine on Host 1 is moved to Host 2, it will be assigned with a new virtual port ID by the VSS on Host 2. In this situation all the port statistics of the red VM, when it was connected on Host 1 on vport 2, is lost. The VSS on Host 2 will start collecting the data, a fresh, when the red VM is powered on port group PG-B with vport 3 as its new virtual port ID
Now let’s look at a VDS deployment.The main difference I would like point out here is that the virtual port ID assignment is handled centrally by the vCenter Server. As you can see in the diagram, the virtual port IDs assigned to each virtual machine is unique. When the red virtual machine is moved through the vMotion process from Host 1 to Host 2 the virtual port ID is maintained as dvport 2. Also, all the port statistics related to dvpot 2 is transferred from the Host 1 to Host 2. This allows VDS deployment not to miss any of the information recorded on Host 1 and continue recording the virtual port statistics on Host 2. Please let me know if you have any more questions on this topic. Thanks for reading.
I recently posted on how how vMotion works and figured it would be good to follow-up with a similar blog covering Storage vMotion (svMotion).
Many people think svMotion is new, but the ability to migrate a running VMs disk files to a new datastore (DS) was first introduced in VI 3.0 as an upgrade tool to help with VMFS upgrades. In VI 3.5 it was officially given the name Storage vMotion, but only had CLI support. GUI support was finally added in 4.0 and with 4.1 there were several performance improvements.
Under the covers svMotion is a 6-step process. Once a VM has been selected to have its disk files moved to a new DS using svMotion the following will take place:
1. The VM home directory (config, log, swap, snapshots) is copied to the destination DS
2. A “shadow” VM gets started on the destination DS using the copied files. The “shadow” VM idles waiting for the copying of the VM disk file(s) to complete
3. An initial copy of the VMs disk file(s) is made to the target DS, during the copy changes made to the source are tracked (change block tracking)
4. svMotion iteratively repeats this process of copying the changed blocks from the source DS to the destination DS
5. When the amount of outstanding changed blocks is small enough, svMotion invokes a Fast Suspend and Resume (FSR) of the VM (similar to vMotion) to transfer the running VM over to the idling shadow VM. Like regular vMotion, this transfer will normally happen so quickly that it will be completely transparent to the VM.
6. After the FSR completes the old home directory and VM disk files are deleted from the source DS.
With that let’s consider a few common questions:
Q. Do I need to backup my VM prior to using svMotion?
A. No, svMotion is as safe and reliable as it's vMotion counterpart. Where vMotion transfers memory, svMotion transfer storage. They employ the same FSR logic to transfer control of the running VM.
Q: Do we check for adequate free disk space on the destination DS before beginning Storage vMotion?
A: Yes, checks are made before we initiate the svMotion to ensure that adequate disk space is available on the destination DS. If there is not enough space, the svMotion request fails with no impact to the running VM.
Q: What happens to the VM if you run out of storage on the destination DS?
A: If an out-of-space condition is hit svMotion will clean up the destination DS and the VM will continue to run on the source.
Q: What happens if the iterative coping of the changed blocks fails (i.e the VM is very write intensive)?
A: If the VM is generating disk I/O faster than svMotion can copy, the svMotion will eventually fail leaving the VM running on the source.
Q: Does all the svMotion traffic get copied over the vMotion network?
A. No, a DataMover (DM) module built into the VMkernel, which is optimized for transferring large disk files, is used to copy the disk data. The swap files are also copied using the DM.
Q: What is the threshold when the number of outstanding changed blocks is small enough to proceed with stunning the VM and switching to the destination DS?
A: It depends on the disk copy throughput. svMotion continuously monitors the copy throughput during each copy iteration, when it detects that the time needed to copy the outstanding dirty blocks is less than the target downtime (default 5 secs), it proceeds with the switchover. Note that the 5 seconds is only the time to copy the remaining disk blocks, it does not include the FSR time.