Home > Blogs > VMware VROOM! Blog

Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5

VMware vSphere ensures that virtualization overhead is minimized so that it is not noticeable for a wide range of applications including most business critical applications such as database systems, Web applications, and messaging systems. vSphere also supports well applications with millisecond-level latency constraints, including VoIP services. However, performance demands of latency-sensitive applications with very low latency requirements such as distributed in-memory data management, stock trading, and high-performance computing have long been thought to be incompatible with virtualization.

vSphere 5.5 includes a new feature for setting latency sensitivity in order to support virtual machines with strict latency requirements. This per-VM feature allows virtual machines to exclusively own physical cores, thus avoiding overhead related to CPU scheduling and contention. A recent performance study shows that using this feature combined with pass-through mechanisms such as SR-IOV and DirectPath I/O helps to achieve near-native performance in terms of both response time and jitter.

The paper explains major sources of latency increase due to virtualization in vSphere and presents details of how the latency-sensitivity feature improves performance along with evaluation results of the feature. It also presents some best practices that were concluded from the performance evaluation.

For more information, please read the full paper: Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5.


13 thoughts on “Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5

  1. Scott Bailey

    On page 6 you say,
    Also, we recommend over-provisioning CPU to reduce contention; the number of VCPUs in the host should be less than the number of PCPUs to leave one or more PCPUs for VMkernel threads for I/O processing and system management

    Is that correct? You recommend over-provisioning?

    1. Jin Heo Post author

      Yes, we are recommending over-provisioning. This is to achieve to two things:

      1. Latency-sensitive feature performs best when a latency-sensitive VM gets exclusive access to PCPUs. Over-provisioning increases the chances of this. However, as explained in the whtie paper, the best way is to give 100% CPU reservation for a given latency-sensitive VM that guarantees exclusive PCPU access to a VM.

      2. Once PCPUs are exclusively owned by latency-sensitive VMs, they cannot be used by VMkernel threads and user-level processes. So it is recommended to leave one or more PCPUs for those.

      1. Amj

        I think it’s badly worded. Surely you mean that you reccomend over-provisioning the PHYSICAL host with more PCPU, not the Virtual Machine?

  2. Pingback: VMware vSphere 5.5 Enhancements That DBAs Should Care About | Technobabble by Klee

  3. ipad dlna apps

    Do you have a spam issue on this site; I also am a blogger, and I
    was curious about your situation; many of us have created some nice procedures and we are looking
    to trade solutions with others, be sure to shoot me an e-mail if interested.

  4. Bill Bach

    Is it possible that I’ve done this incorrectly? When I set the CPU and memory reservations to their maximum setting, I can run a specific database process (one that reads 50000 records) on my VM in 11 seconds. When I change the Latency Sensitivity mode from NORMAL to HIGH, I notice two things:
    1) The CPU Used, in the vSphere Client Summary Screen, changes from 5600 (two full cores at 2.8GHz) to a low number, like 56. It is almost like the reservation that I gave it disappears.
    2) The same database process now takes 19s to process 50000 requests.

    Now, I believe that the increase in time is dealing more with the lack of CPU reservation, but I cannot figure out exactly WHY this is an issue. Overall, my box has 32GB of RAM (24 in use), and I have 12 logical processors, with 16 total vCPU’s in all the guest VM’s in total. Of these, I have created reservations in only ONE VM (the database VM) — all other reservations are 0. Further, most other boxes are pretty idle — vSphere reports that I am using 1.1GHz out of 33.60GHz.

    Has anyone provided any additional testing or clarification regarding this feature?

    1. Jin Heo

      I’m assuming that you don’t use a pass-through mechanism such as SR-IOV. If that’s the case, the performance degradation you are observing might be because the workload you are running has a high packet rate or the packet size is large (larger than MTU size). The feature disables both VNIC coalescing and LRO that can badly affect the performance in such cases (high packet rate and/or large packet size).

      However, it’s hard to conclusively tell what’s going on with the given information. If you’d like to figure out what’s exactly happening on your system, you can file a support request: https://www.vmware.com/support/file-sr/. Our support team has a systematic way of collecting necessary stats and diagnosing the problem.


  5. Pingback: Dave Stork's IMHO : Exchange 2013 DAG, Windows Server 2012 R2 and VMware vSphere? Check support!

  6. Cristian Seres

    Could you explain how memory and CPU reservation actually effect together with latency sensitivity setting? According to the document “Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5″, latency sensitivity feature requires memory reservation, but does it mean 100% “Reserve all guest memory” option checked or is lower value enough? When you switch latency sensitivity option to High, vSphere web client gives yellow warning “Check CPU reservation – VM needs enough CPU reservation to power on when the level is set to High”. Above-mentioned VMware document strongly recommends maximizing CPU reservations, but does it need to be 100% (total vCPU of VM * clock frequency of CPU) or would latency sensitivity setting work with a lower setting?

    The document “Deploying Extremely Latency-Sensitive Applications in VMware vSphere 5.5″ states that with the latency
    sensitivity feature enabled, the CPU scheduler determines whether exclusive access to PCPUs can be given or not considering various factors including whether PCPUs are over-committed or not. Is there any way to check if the conditions are met or not?

  7. Jin Heo

    1. You need to reserve all guest memory to use the latency-sensitivity feature. Otherwise, you won’t be able to power on the VM.

    2. About CPU, you don’t have to reserve 100% CPU to use the feature. However, reserving 100% CPU increases the chance of having exclusive access to PCPUs.

    3. The information about whether exclusive access to PCPUs can be given or not is kept internally in the CPU scheduler. It’s not external to the user.

  8. eric mui


    Does it help to enable reservation on CPU or memory IF the ESX only run 2 VMs which host the same apps/functions with same priority ? While both are just as aggressive and CPU intensive and are large memory VMs. Thanks

    Best Regards,

  9. Pingback: Updated – Exchange 2013 DAG, Windows Server 2012 R2 and VMware vSphere? Check support! » Dave Stork's IMHO

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>