Being able to work from distributed locations has become more important in recent years, of that there is no doubt. The events of 2020, however, have made Work From Home (WFH) supremely important for countless organizations worldwide. Secure, reliable and resilient networking is a foundational aspect of WFH.
To that end, VMware and Intel collaborated with The Tolly Group to benchmark key performance and end-user experience metrics of VMware’s WFH solution with a single WAN link deployment.
Voice quality in the face of WAN (Internet) packet loss
Throughput benchmarking between an edge and a hub appliance
Download speed in the face of WAN packet loss
VMware SD-WAN™ demonstrated that it could successfully remediate significant packet loss in both the VoIP and file download scenarios and matched its data sheet throughput specifications with respect to raw edge-to-hub throughput.
Here are the results in a nutshell:
Impact of packet loss on VoIP (voice) quality
In a baseline test without VMware’s remediation, engineers demonstrated that even a small amount of packet loss quickly degrades the quality of voice conversations (as measured by mean opinion scores [MOS]). With 1% loss, the conversation quality is degraded. At 2% and higher, the quality becomes unacceptable to most users.
VMware SD-WAN remediates packet loss by detecting that loss and dynamically sending duplicate packets. Tests showed that even with 10% packet loss, the voice quality on the VMware SD-WAN protected link was still rated as good (over 3.50) according to the MOS scale.
With remote users relying on VoIP, packet loss of any kind – without remediation – can almost immediately cause their work to cease. With VoIP services effectively down, they would have to fall back to using their cell phone to interact with internal and external contacts. Similarly, that same packet loss could degrade other essential unified communication functions like group chats and audio/video conferencing. See Figure 1.
Bidirectional throughput: VMware SD-WAN Edge to hub
In this test, engineers benchmarked the throughput between a VMware SD-WAN Edge device and hub device connected directly with a Gigabit Ethernet connection.
The VMware SD-WAN results matched the product specifications for the Gigabit device. The pair of network nodes delivered 43 Mbps at 64-byte packets, an average of 205 Mbps with an iMix traffic stream with a 400-byte average packet size and 350 Mbps with 1300-byte packets.
These throughput rates assure the user that there is no performance bottleneck in the VMware SD-WAN appliances. Users have ample performance capacity to handle common scenarios like collaborative editing in the cloud during a video session. See Figure 2.
Microsoft 365: Work-from-home download speed
A reliable, high-quality internet connection is essential for making a productive environment — but connections are not always reliable. As noted earlier, packet loss on the internet — which is beyond the control of the end user — can have a negative impact directly and indirectly on productivity. Long waits for downloads to complete delay users from working with those files.
As with the VoIP test, packet loss can negatively impact the download time of files. This test used a ~250 MB file and engineers demonstrated that download times got dramatically worse even with just 2% packet loss. A file that required only 37 seconds to download with no packet loss required 12.5 minutes with 2% loss.
The same tests were run with the VMware SD-WAN solution remediating the packet loss. Dynamic Multipath Optimization™ (DMPO) automatically detects the packet loss and the M365 download that is taking place. In this situation, TCP applications such as file transfer would benefit from Negative Acknowledgement (NACK). With on-demand remediation in place VMware SD-WAN detects packet loss in the network and retransmits the missing packets to remediate that loss.
As with the VoIP quality test VMware SD-WAN remediation improves the download times dramatically compared to the baseline.
With 2% loss, VMware SD-WAN improves the download speed by 71% over the baseline. In practical terms, this is very significant. Instead of waiting 12 minutes to be able to work on a cloud file, that file is downloaded and available in little more than 3.5 minutes. For users having to interact with cloud-based files on their local workstation this improves that productivity by more than a factor of 3. See Figure 3.Test Environment
The tests were run using live internet WAN connections for all of the packet loss tests. For an overview of the test environment, see Figure 4.
Advantages from Intel Developer Tools and Technologies
Co-engineering by VMware and Intel® has built optimizations into the solution using the Intel developer tool set, taking advantage of capabilities built into the Intel platforms used for the SD-WAN appliances. Intel-based optimizations help improve performance and security of VMware SD-WAN solutions including the specific offerings for WFH use cases. The Intel developer tools and technologies include the following:
Data Plane Development Kit (DPDK) is a library of open standard software drivers originally developed by Intel that drive up packet-processing performance by routing network packets around the Linux kernel.
Intel QuickAssist Technology (Intel QAT) provides a software-enabled foundation for security, encryption and decryption, authentication, and compression, significantly increasing performance and efficiency.
Intel AES New Instructions (Intel AES-NI) accelerates key parts of the encryption algorithm in hardware, making pervasive, end-to-end encryption possible without degrading performance. These instructions can execute using significantly fewer clock cycles than a software solution.
WFH will only continue to gain in importance going forward. It is essential that solutions work as advertised. Third-party tests like these provide additional assurance for users that their environments are robust and resilient.
Test Environment
