Home > Blogs > The Network Virtualization Blog > Tag Archives: network virtualization

Tag Archives: network virtualization

Using Differentiated Services to Tame Elephants

This post was co-authored by Justin Pettit, Staff Engineer, Networking & Security Business Unit at VMware, and Ravi Shekhar, Distinguished Engineer, S3BU at Juniper Networks.

********************

As discussed in other blog posts and presentations, long-lived, high-bandwidth flows (elephants) can negatively affect short-lived flows (mice). Elephant flows send more data, which can lead to queuing delays for latency-sensitive mice.

VMware demonstrated the ability to use a central controller to manage all the forwarding elements in the underlay when elephant flows are detected.  In environments that do not have an SDN-controlled fabric, an alternate approach is needed.  Ideally, the edge can identify elephants in such a way that the fabric can use existing mechanisms to treat mice and elephants differently.

Differentiated services (diffserv) were introduced to bring scalable service discrimination to IP traffic. This is done using Differentiated Services Code Point (DSCP) bits in the IP header to signal different classes of service (CoS). There is wide support in network fabrics to treat traffic differently based on the DSCP value.

A modified version of Open vSwitch allows us to identify elephant flows and mark the DSCP value of the outer IP header.  The fabric is then configured to handle packets with the “elephant” DSCP value differently from the mice.

Figure 1: Elephants are detected at the edge of the network and signaled to the fabric through DSCP.  Based on these code points, the fabric can treat elephant traffic differently from mice

Figure 1: Elephants are detected at the edge of the network and signaled to the fabric through DSCP. Based on these code points, the fabric can treat elephant traffic differently from mice

Detecting and Marking Elephants with Open vSwitch

Open vSwitch’s location at the edge of the network gives it visibility into every packet in and out of each guest.  As such, the vSwitch is in the ideal location to make per-flow decisions such as elephant flow detection. Because environments are different, our approach provides multiple detection mechanisms and actions so that they can be used and evolve independently.

An obvious approach to detection is to just keep track of how many bytes each flow has generated.  By this definition, if a flow has sent a large amount of data, it is an elephant. In Open vSwitch, the number of bytes and an optional duration can be configured. By using a duration, we can ensure that we don’t classify very short-lived flows as elephants. We can also avoid identifying low-bandwidth but long-lived flows as elephants.

An alternate approach looks at the size of the packet that is being given to the NIC.  Most NICs today support TCP Segmentation Offload (TSO), which allows the transmitter (e.g., the guest) to give the NIC TCP segments up to 64KB, which the NIC chops into MSS-sized packets to be placed on the wire.

Because of TCP’s slow start, the transmitter does not immediately begin sending maximum-sized packets to the NIC.  Due to our unique location, we can see the TCP window as it opens, and tag elephants earlier and more definitively. This is not possible at the top-of-rack (TOR) or anywhere else in the fabric, since they only see the segmented version of the traffic.

Open vSwitch may be configured to track all flows with packets of a specified size. For example, by looking for only packets larger than 32KB (which is much larger than jumbo frames), we know the transmitter is out of slow-start and making use of TSO. There is also an optional count, which will trigger when the configured number of packets with the specified size is seen.

Some new networking hardware provides some elephant flow mitigation by giving higher priority to small flows. This is achieved by tracking all flows and placing new flows in a special high-priority queue. When the number of packets in the flow has crossed a threshold, the flow’s packets from then on are placed into the standard priority queue.

This same effect can be achieved using the modified Open vSwitch and a standard fabric.  For example, by choosing a packet size of zero and threshold of ten packets, each flow will be tracked in a hash table in the kernel and tagged with the configured DSCP value when that flow has generated ten packets.  Whether mice are given a high priority or elephants are given a low priority, the same effect is achieved without the need to replace the entire fabric.

Handling Elephants with Juniper Devices

Juniper TOR devices (such as QFX5100) and aggregation devices (such as MX, EX9200) provide a rich diffserv model CoS to to achieve these goals in the underlay.  These include:

  • Elaborate controls for packet admittance with dedicated and shared limits. Dedicated limits provide a minimum service guarantee, and shared limits allow statistical sharing of buffers across different ports and priorities.
  • A large number of flexibly assigned queues; up to 2960 unicast queues at the TOR and 512K at the aggregation device.
  • Enhanced and varied scheduling methods to drain these queues: strict and round-robin scheduling with up to 4-levels of hierarchical schedulers.
  • Shaping and metering to control the rate of injection of traffic from different queues of a TOR in the underlay network. By doing this, bursty traffic at the edge of the physical network can be leveled out before it reaches the more centrally shared aggregation devices.
  • Sophisticated controls to detect and notify congestion, and set drop thresholds. These mechanisms detect possible congestion in the network sooner and notify the source to slow down (e.g. using ECN).

With this level of flexibility, it is possible to configure these devices to:

  • Enforce minimum bandwidth allocation for mice flows and/or maximum bandwidth allocation for elephant flows on a shared link.
  • When experiencing congestion, drop (or ECN mark) packets of elephant flows more aggressively than mice flows.  This will result in TCP connections of elephant flows to back off sooner, which alleviates congestion in the network.
  • Take a different forwarding path for elephant flows from that of mice flows.  For example, a TOR can forward elephant flows towards aggregation switches with big buffers and spread mice flows towards multiple aggregation switches that support low-latency forwarding.

Conclusion

By inserting some intelligence at the edge and using diffserv, network operators can use their existing fabric to differentiate between elephant flows and mice. Most networking gear provides some capabilities, and Juniper, in particular, provides a rich set of operations that can be used based on the DSCP.  Thus, it is possible to reduce the impact of heavy hitters without the need to replace hardware. Decoupling detection from mitigation allows each to evolve independently without requiring wholesale hardware upgrades.

 

Physical Networks in the Virtualized Networking World

[This post was co-authored by VMware's Bruce Davie and Ken Duda from Arista Networks, and originally appeared on Network Heresy]

Almost a year ago, we wrote a first post about our efforts to build virtual networks that span both virtual and physical resources. As we’ve moved beyond the first proofs of concept to customer trials for our combined solution, this post serves to provide an update on where we see the interaction between virtual and physical worlds heading.

Our overall approach to connecting physical and virtual resources can be viewed in two main categories:

  • terminating the overlay on physical devices, such as top-of-rack switches, routers, appliances, etc.
  • managing interactions between the overlay and the physical devices that provide the underlay.

The latter topic is something we’ve addressed in some other recent posts (herehere and here) — in this blog we’ll focus more on how we deal with physical devices at the edge of the overlay. Continue reading

Geneve, VXLAN, and Network Virtualization Encapsulations

In this post, Bruce Davie and T. Sridhar of VMware’s Networking and Security Business Unit take a look at a proposed a new encapsulation protocol that would standardize how traffic is tunneled over the physical infrastructure by network overlay software.

++++

For as long as we’ve been doing Network Virtualization, there has been debate about how best to encapsulate the data. As we pointed out in an earlier post, it’s entirely reasonable for multiple encapsulations (e.g. VXLAN and STT) to co-exist in a single network. With the recent publication of “Geneve”, a new proposed encapsulation co-authored by VMware, Microsoft, Red Hat and Intel, we thought it would be helpful to clarify a few points regarding encapsulation for network virtualization. First, with all the investment made by us and our partners in developing support for VXLAN (described here), we very much intend to continue supporting VXLAN — indeed, we’ll be enhancing our VXLAN capabilities. Second, we want to explain why we believe Geneve is a necessary and useful addition to the network virtualization landscape.

Read the rest of Bruce’s blog on the Office of the CTO blog here.

Juniper and VMware: Collaborating to Enable The Software-Defined Data Center

The need for businesses to enhance the efficiency of IT and increase application agility is overwhelming. Embracing operational models such as cloud computing helps, but in order to fully leverage these new models companies must explore new ways of handling network connectivity. Network virtualization solutions such as VMware NSX provide an answer for the new cloud-centric networking models. As with any technology, though, network virtualization doesn’t solve some existing challenges by itself: consistent, efficient performance for business-critical applications that span virtual and physical worlds; correlated and integrated management; and enhancing data sharing between the network virtualization solution and the underlying physical network are all critical elements to successful cloud deployments. To address these challenges, we are pleased to announce that Juniper and VMware are expanding our partnership to help our joint customers achieve better application agility for their cloud environments. Continue reading

The Goldilocks Zone: Security In The Software-Defined Data Center Era

Last week, we spoke at the RSA Conference about a new concept in security – the Goldilocks zone.  With the help of Art Coviello, Executive Chairman of RSA, Chris Young, senior vice president and GM of Cisco’s Security business unit, and Lee Klarich, senior vice president of product management from Palo Alto Networks, we departed from the typical discussions about new controls or the latest threats.  We took the opportunity to lay out what we believe is a fundamental architectural issue holding back substantial progress in cyber security, and how virtualization may just provide the answer. The growing use of virtualization and the move towards software-defined data centers enable huge benefits in speed, scalability and agility; those benefits are undeniable. It may turn out, however, that one of virtualization’s biggest benefits is security. Continue reading

VMware at RSA Conference 2014 (#RSAC)

Summary:logo_rsac

  • Company outlines vision for security in the Software-Defined Data Center
  • Product and partner demonstrations in Booth #1615 to showcase growing security portfolio
  • New PCI-DSS 3.0 and FedRAMP reference architectures to be presented

Throughout its history, RSA Conference has consistently attracted the world’s best and brightest in the security field, creating opportunities for attendees to learn about IT security’s most important issues through first-hand interactions with peers, luminaries and emerging and established companies. Continue reading

Deploying VMware NSX with Cisco UCS and Nexus 7000

VMware NSX network virtualization software makes it possible for IT organizations to obtain new levels of business agility, allowing you to deploy advanced policy based network services for your applications as quickly as a virtual machine, today, leveraging your existing physical network infrastructure or next generation network hardware from any vendor.

Back in September, I wrote about the value of deploying the VMware NSX network virtualization platform on Cisco UCS and Nexus infrastructure. We had an overwhelming amount of customer response and request for more content about this deployment scenario. As such, we’ve created a new design guide which you can download here that describes a simple and straight forward example of how to deploy VMware NSX for vSphere on an infrastructure consisting of Cisco UCS and Nexus 7000 series switches. This basic guide should serve as a starting point from which to tailor a design for your specific environment.

We want to offer our thanks to the content creation team here at VMware for collaborating on this effort:

• Dmitri Kalintsev
• Bruce Davie
• Ray Budavari
• Venky Deshpande
• Nikhil Kelshikar
• Rod Stuhlmuller
• Scott Lowe
• Marcos Hernandez
• Chris King

Cheers,
Brad

Elephant Flow Mitigation via Virtual-Physical Communication

Note: this post was developed jointly by Justin Pettit of VMware and Mark Pearson of HP, with additional content from VMware’s Martin Casado and Bruce Davie.

A recent Network Heresy post “Of Mice and Elephants” discussed the impact long-lived flows (elephants) have on their short-lived peers (mice).  A quick summary is that, in a datacenter, it is believed that the majority of flows are short-lived (mice), but the majority of packets are long-lived (elephants). Mice flows tend to be bursty and latency-sensitive, whereas elephant flows tend to transfer large amounts of data, with per-packet latency being of less concern.  These elephants can fill up network buffers, which can introduce latency for mice.

At the HP 2013 Discover Conference, HP and VMware demonstrated a technology preview of detecting and handling elephant flows in an overlay network. The demonstration featured the joint HP-VMware solution announced at VMworld 2013.  VMware NSX provided an overlay network using HP switches as the underlay along with the HP VAN SDN controller. Through controller federation interfaces, the overlay and the underlay co-operated to mitigate the effects of the elephant flows on the mice. The solution shows the power of integration between network virtualization and SDN solutions. Continue reading

Distributed virtual and physical routing in VMware NSX for vSphere

NOTE – this post originally appeared on Bradhedlund.com

This post is intended to be a primer on the distributed routing in VMware NSX for vSphere, using a basic scenario of L3 forwarding between both virtual and physical subnets. I’m not going to bore you with all of the laborious details, just the stuff that matters for the purpose of this discussion. Continue reading

Network Security: The VMware NSX Network Virtualization Platform’s Hidden Gem

This week, we announced a new joint solution with our partner Palo Alto Networks that will

Best-In-Class Partners

automate and accelerate the deployment of next-generation network security with centralized management across physical and virtual domains. You can read the full announcement about the forthcoming integrated solution from our companies in our press release here.

For most data center operators, the idea of achieving the operational model of a VM for their data center networks is a top of mind benefit associated with the VMware NSX network virtualization platform. Through this model they can gain greater agility, efficiency and provisioning speed while reducing complexity as they implement a software-defined data center architecture. An often-overlooked feature set, fundamental to VMware NSX, is network security. Continue reading