One of the things that make DRS so elegant is that is so simple to configure. There are not many knobs that an administrator needs to turn when enabling DRS, and for a majority of environments the default settings do not need to be changed. For other environments changing the DRS migration threshold may provide additional benefits, but it’s important to understand how this could impact the environment.
What is the Migration Threshold?
The DRS migration threshold is a user-defined option that controls how conservative or aggressive DRS runs. Migration recommendations are generated by DRS and they are given a priority rating of 1 (highest) to 5 (lowest). The priority rating for each recommendation must meet the migration threshold for DRS to migrate the VM. If the priority rating does not meet the threshold, it is not migrated.
What is the Priority Rating?
DRS constantly monitors several CPU and memory performance indicators to determine the resource demand of each VM. When a host is not able to satisfy VM demand, DRS generates a migration recommendation to move the VM to another host that can provide more resources. Each recommendation is assigned a priority rating of 1 (highest) to 5 (lowest).
Now let’s take a closer look at the priority ratings.
These are recommendations that are generated to enforce a rule. Priority 1 recommendations are not generated due to cluster imbalance or VM demand. Examples of Priority 1 recommendations include VM (anti-)affinity rules or a host entering Maintenance Mode.
Priority 2 to 5
DRS performs a cost/benefit analysis to calculate a priority rating for each recommendation. The current demand of the VM is examined and DRS evaluates whether a migration would be worth the effort. There is a cost associated with every migration, and each vMotion adds an increased demand on CPU, memory, and network to the source and destination hosts. DRS calculates the performance improvement and weighs that with the cost of performing the migration. The more significant the benefit, the higher the priority rating.
Let that all sink in. Now to the migration threshold
Migration Threshold 1
This is the most conservative setting for the migration threshold, and only priority 1 ratings meet this threshold. Remember, the only way DRS generates a priority 1 rating is when DRS needs to enforce an (anti-)affinity rule or the host enters maintenance mode. That means if the migration threshold is set to 1, no migration recommendations are generated due to resource contention or cluster imbalance. This threshold should only be used when rebalancing cluster load is not needed, but the increased benefits performing operational tasks and enforcing compliance rules is required. Performing rolling ESXi updates to a cluster of hosts with DRS rules is a good example of this scenario. Before a host is upgraded, it enters maintenance mode. Every VM on the host is assigned a priority 1 rating, and very pragmatic host evacuation is done. DRS evaluates the performance data to determine the best destination host for each VM. In addition to choosing the best host based on VM demand, DRS also continues to enforce any (anti-)affinity rules. Keep in mind that once the last host is updated, the cluster load is not automatically redistributed.
Migration Threshold 2 to 5
These thresholds are used when the redistribution of resources is required as VM demand fluctuates. Migration threshold 2 is the most conservative and 5 is the most aggressive. As the Migration Threshold moves upward from 2 to 5, less and less benefit is required before a migration recommendation is applied. Another way to think about this is that the higher the migration threshold, the more migrations are allowed, even when only the slightest benefit to performance or cluster balance will result. Take a look here to learn how DRS can be tuned to improve load balancing.
As a general rule, the default setting is the best setting unless a reason warrants a change. The default setting weighs good performance, low contention, and minimal migrations. Moving the migration threshold setting to a lower setting reduces the number of migrations to only times a significant benefit would result. A higher setting favors a slight benefit to a balanced cluster at the expense of more frequent migrations.
Which migration threshold do you use? Leave a message in the comments below.