Guest post SD-WAN Technical

Tolly and VMware Report: VMware SD-WAN Evaluation for Work from Anywhere with Dual WAN Links

Guest post by Kevin Tolly, Founder, The Tolly Group

The flexibility and ease-of-use of SD-WAN has made it extraordinarily popular as the “next-gen” WAN for organizations large and small. SD-WAN’s network overlay topology can also provide significant benefits for end-user experience while maintaining solid throughput performance. VMware SD-WAN™,, a service of VMware SASE™, automatically remediates packet loss and delivers a high-quality experience even under adverse network conditions. Dual WAN link deployments are very economical and further enhance the user experience by providing even greater flexibility, robustness, and higher aggregate throughput.

In this blog post, I will give you an overview of the test results. You can also read our full Tolly report #221128 for all of the details.

The testing encompassed three areas:

  1. Voice quality in the face of WAN (Internet) packet loss and latency/jitter on one or both links
  2. Dual WAN link optimization
  3. 10GbE packet throughput benchmarking between an edge and a hub appliance

VMware SD-WAN demonstrated that it could successfully remediate significant packet loss and latency/jitter for VoIP calls, completed a file download despite a link failure, and was able to aggregate both the links to support a large file transfer greater than the actual bandwidth of the link. This confirms that VMware SD-WAN delivers significant resiliency and performance benefits using dual WAN links and matched its data sheet throughput specifications with respect to raw edge-to-hub throughput.

Here are some highlights. Please see the full Tolly report for all of the test results:

A MOS score of 3.5 out of 5.0 and above is considered very good.

Impact of packet loss on VoIP (voice) quality

VMware SD-WAN results due to packet loss

After running a set of baseline tests, the same set of tests were run through the VMware SD-WAN solution using the default policies that provide on-demand remediation for packet loss. See Figure 1.

With packet loss present on a single link, the voice quality without any loss of 4.37 was maintained throughout all the loss scenarios. Even with loss on a single link of 15%, the voice quality score was 4.31 as VMware SD-WAN dynamically sent traffic over the better link.

With packet loss present on both links, the results remained impressive.  Even with 5% loss, the MOS is 4.16 — not far below the zero-loss score of 4.37.

With 10% loss, the MOS of 3.36 remains very close to the “good” level of 3.50.

Chart title: VoIP quality vs. packet loss, single/dual WAN link impairments, VMware SD-WAN Edge to hub. Vertical axis: Mean opinion score (MOS)Horizontal axis: Packet loss, in percentNote: Users generally consider conversations with mean opinion scores of 3.50 or greater to be good quality. At 0 percent packet loss:-MOS for VMware remediation with single link impairment, VMware remediation with dual link impairment, and baseline without remediation are the same, 4.37At 1 percent packet loss:- MOS for VMware remediation with single link impairment, is 4.37-MOS for VMware remediation with dual link impairment, is 4.29- MOS for baseline without remediation is 3.52At 2 percent packet loss: - MOS for VMware remediation with single link impairment, is 4.35-MOS for VMware remediation with dual link impairment, is 4.20- MOS for baseline without remediation is 3.08At 5 percent packet loss:- MOS for VMware remediation with single link impairment, is 4.35-MOS for VMware remediation with dual link impairment, is 4.16- MOS for baseline without remediation is 1.86At 10 percent packet loss:- MOS for VMware remediation with single link impairment, is 4.36-MOS for VMware remediation with dual link impairment, is 3.36- MOS for baseline without remediation is 1.08At 15 percent packet loss: - MOS for VMware remediation with single link impairment, is 4.31-MOS for VMware remediation with dual link impairment, is 2.52- MOS for baseline without remediation is 1.01

Dual WAN link optimization

Prefer low latency link

Uninterrupted download with failed link

Some data transfers can take an extended amount of time — file backups are a common example. For this test, engineers started transferring a large file and confirmed link activity. The file was a video (mp4) file of approximately 250 MB and was downloaded from Microsoft OneDrive. While the transfer was still in progress, that link was physically disconnected. Tolly engineers confirmed that the file transfer continued without interruption as VMware SD-WAN dynamically used the alternate link to complete the transfer.

Dual WAN link aggregate bandwidth for a single app

An important benefit of leveraging two links is the ability to harness additional bandwidth. Traditionally, application traffic would be bound to a single link. VMware SD-WAN can dynamically and transparently aggregate dual links to benefit a single application. For this test, Tolly engineers started a large file transfer and confirmed that the aggregate application throughput exceeded the capacity of either single link.

Table title: Dual WAN Link Optimization Scenario Summary. Table shows the results of Tolly testing. Row 1: Scenario: prefer low-latency link. Benefit: Automatically use optimal link. Methodology: Determine link carrying applications traffic. Use impairment tool to add 250 ms of latency to that link. Monitor traffic statistics to confirm automatic migration of app traffic away from high-latency link. Results: Pass. Row 2: Scenario: uninterrupted download with failed link. Benefit: automatically continue in-progress download in the event of link failure. Methodology: determine link carrying application traffic. Disconnect that link. Monitor application for uninterrupted download completion. Results: Pass. Row 3: Scenario: dual-link aggregate bandwidth for single app. Benefit: leverage available bandwidth of multiple links for a single application stream. Methodology: Run high-bandwidth file transfer application. Confirm throughout is greater than single link and that traffic is traversing both links. Results: Pass. Note: Apposite Technologies Netropy N91 used to simulate WAN environment and introduce impairments. Source: Tolly, July 2001.

Working from anywhere will only continue to gain in importance going forward. It is essential that solutions work as advertised. Third-party tests like these provide additional assurance for users that their environments are robust and resilient.

Learn more

  • Visualize how VMware SD-WAN, VMware SASE and Intel developer tools work together for a smooth user experience in Tolly Group’s infographic.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *