In this blog entry, I will spend some time discussing the configuration maximums related to vSphere standard switch (VSS) and vSphere distributed switch (VDS). I always get this question, what will happen when you cross those configuration maximum limits? Especially, with the vSphere Distributed Switch configuration maximums where there are vCenter Server level limits as well as host level limits. I would like to clarify some of the things regarding these limits in this post. Here are the configuration maximums for vSPhere 5.0 as it pertains to Hosts, VSS, and VDS.
Host Maximums (These apply to both VSS and VDS):
- Total virtual network switch ports per host : 4096
- Maximum Active ports per host : 1016
VSS Maximums
- Port groups per standard switch : 256
VDS Maximums (These are all vCenter Server maximums as vCenter server controls the configuration of VDS)
- Hosts per VDS : 350
- Total distributed virtual network switch ports : 30,000
- Total number of Static Distributed Port groups : 5000
- Total number of Ephemeral Port groups : 256
After taking a look at the limits, let’s focus our attention on the VSS deployments first. In such deployments you have to configure VSS on each host and in some cases there might be multiple VSSs on the same host. When you create a VSS you have an option to define the number of virtual ports on that specific virtual switch (default is 128). Next step is to create port groups. VSS only supports Ephemeral binding and allocates zero virtual ports when a port group is created. The virtual port is allocated only when a virtual machine or vmknic is connected to the port group. In this deployment by keeping the count of the number of VMs and vmknics on a host you can tell how many virtual ports are used. You can then compare the number of virtual ports with host limits.
The host limits are Hard limits. Hard limit means that the host will enforce the limit and you will not be allowed to create more than 4096 virtual ports or have more than 1016 active virtual ports. If you have multiple VSSs on the host this port maximum numbers don’t change. You might have some VSS with more VMs connected and some with less. As long as the total number of VMs and vmknics on the VSSs are within the maximum range you are fine. Also, in my opinion there are enough virtual ports available as per the host maximums, and you should not have any problems regarding scaling your environment and achieving higher consolidation ratios.
Now let’s look at the VDS deployments where there are additional limits placed by the vCenter Server. Before we dive into the limits discussion on VDS, I would like to point out one main difference between the port group configuration on VSS and the distributed port group configuration on VDS. On VSS port group there is only support for Ephemeral port binding. While on VDS distributed port group, you have an option to choose from the following three different port binding types:
1) Static binding: Assigns a distributed port when a virtual machine is connected to distributed port group
2) Dynamic binding: Assigns a distributed port when a powered on virtual machine is connected to distributed port group. This option is deprecated and won’t be available in the future vSphere releases.
3) Ephemeral binding: There is no port binding with this choice. When you choose this option the behavior is similar to a standard virtual switch (VSS). The number of ports is automatically set to 0, and the port group allocates one port for each connected virtual machine, up to the maximum number of ports available on that port group.
The choice of port binding type on a distributed port group determines how the distributed virtual ports are allocated.
For example, if you choose static port binding for distributed port groups by default 128 virtual ports are allocated by vCenter Server. As you can see, this is different from the VSS deployment where no virtual ports are allocated when a port group is created. Some customers have concerns that they will run out of virtual ports as they create large number of distributed port groups OR they have to manually mange the number of virtual ports per distributed port group to overcome the limits.
To illustrate through an example, if you want to create 400 distributed port groups with default number of virtual ports then you would need 51,200 virtual ports. This number is above the vCenter server limit of 30,000 virtual ports. Even though the number of virtual ports are higher than the limit, vCenter Server will allow you to create 400 distributed port groups because vCenter server limits are Soft limits. Soft limit means that the limit is not enforced and you can create more number of distributed port group or virtual ports beyond the specified limits.
However, it is important to note that VMware has tested these maximum limits. If you go beyond those limits, things still should work but you might encounter other challenges in such big environments that are more related to manageability and performance of the management system.We are trying to simplify the workflow for customers where they don’t have to manually manage the number of ports available on a distributed port group or worry about the limits. To that respect the Auto Expand feature that is available in vSphere 5.0 helps grow the number of virtual ports on a distributed port group automatically. For more details on how to configure this feature please take a look at the following blog entry by William Lam here
Finally, I just want to reiterate that the vCenter server limits are soft limits and doesn’t stop you from going beyond the tested limits. And the Host limits are the one that will be enforced. Given the 1016 virtual port limits per host I am sure it provides enough capacity to grow as far as consolidation ratio goes. Would love to hear your comment on this topic. In the next post I will talk more about the Static port binding advantages and the Auto Expand capability.
