Home > Blogs > The Network Virtualization Blog > Category Archives: Products

Category Archives: Products

Talking Tech Series: VMware NSX Edge Scale Out with Equal-Cost Multi-Path Routing

This post was written by Roie Ben Haim and Max Ardica, with a special thanks to Jerome Catrouillet, Michael Haines, Tiran Efrat and Ofir Nissim for their valuable input.

****

The modern data center design is changing, following a shift in the habits of consumers using mobile devices, the number of new applications that appear every day and the rate of end-user browsing which has grown exponentially. Planning a new data center requires meeting certain fundamental design guidelines. The principal goals in data center design are: Scalability, Redundancy and High-bandwidth.

In this blog we will describe the Equal Cost Multi-Path functionality (ECMP) introduced in VMware NSX release 6.1 and discuss how it addresses the requirements of scalability, redundancy and high bandwidth. ECMP has the potential to offer substantial increases in bandwidth by load-balancing traffic over multiple paths as well as providing fault tolerance for failed paths. This is a feature which is available on physical networks but we are now introducing this capability for virtual networking as well. ECMP uses a dynamic routing protocol to learn the next-hop towards a final destination and to converge in case of failures. For a great demo of how this works, you can start by watching this video, which walks you through these capabilities in VMware NSX.

Scalability and Redundancy and ECMP

To keep pace with the growing demand for bandwidth, the data center must meet scale out requirements, which provide the capability for a business or technology to accept increased volume without redesign of the overall infrastructure. The ultimate goal is avoiding the “rip and replace” of the existing physical infrastructure in order to keep up with the growing demands of the applications. Data centers running business critical applications need to achieve near 100 percent uptime. In order to achieve this goal, we need the ability to quickly recover from failures affecting the main core components. Recovery from catastrophic events needs to be transparent to end user experiences.

ECMP with VMware NSX 6.1 allows you to use upto a maximum of 8 ECMP Paths simultaneously. In a specific VMware NSX deployment, those scalability and resilience improvements are applied to the “on-ramp/off-ramp” routing function offered by the Edge Services Gateway (ESG) functional component, which allows communication between the logical networks and the external physical infrastructure.

ECMP Image 1

External user’s traffic arriving from the physical core routers can use up to 8 different paths (E1-E8) to reach the virtual servers (Web, App, DB).

In the same way, traffic returning from the virtual server’s hit the Distributed Logical Router (DLR), which can choose up to 8 different paths to get to the core network.

How is the path determined:

NSX for vSphere Edge Services Gateway device:

When a traffic flow needs to be routed, the round robin algorithm is used to pick up one of the links as the path for all traffic of this flow. The algorithm ensures to keep in order all the packets related to this flow by sending them through the same path. Once the next-hop is selected for a particular Source IP and Destination IP pair, the route cache stores this. Once a path has been chosen, all packets related to this flow will follow the same path.

There is a default IPv4 route cache timeout, which is 300 seconds. If an entry is inactive for this period of time, it is then eligible to be removed from route cache. Note that these settings can be tuned for your environment.

Distributed Logical Router (DLR):

The DLR will choose a path based on a Hashing algorithm of Source IP and Destination IP.

What happens in case of a failure on one of Edge Devices?

In order to work with ECMP the requirement is to use a dynamic routing protocol: OSPF or BGP. If we take OSPF for example, the main factor influencing the traffic outage experience is the tuning of the OSPF timers.

OSPF will send hello messages between neighbors, the OSPF “Hello” protocol is used and determines the Interval as to how often an OSPF Hello is sent.

Another OSPF timer called “Dead” Interval is used, which is how long to wait before we consider an OSPF neighbor as “down”. The OSPF Dead Interval is the main factor that influences the convergence time. Dead Interval is usually 4 times the Hello Interval but the OSPF (and BGP) timers can be set as low as 1 second (for Hello interval) and 3 seconds (for Dead interval) to speed up the traffic recovery.

ECMP Image 2

In the example above, the E1 NSX Edge has a failure; the physical routers and DLR detect E1 as Dead at the expiration of the Dead timer and remove their OSPF neighborship with him. As a consequence, the DLR and the physical router remove the routing table entries that originally pointed to the specific next-hop IP address of the failed ESG.

As a result, all corresponding flows on the affected path are re-hashed through the remaining active units. It’s important to emphasize that network traffic that was forwarded across the non-affected paths remains unaffected.

Troubleshooting and visibility

With ECMP it’s important to have introspection and visibility tools in order to troubleshoot optional point of failure. Let’s look at the following topology.

ECMP Image 3

A user outside our Data Center would like to access the Web Server service inside the Data Center. The user IP address is 192.168.100.86 and the web server IP address is 172.16.10.10.

This User traffic will hit the Physical Router (R1), which has established OSPF adjacencies with E1 and E2 (the Edge devices). As a result R1 will learn how to get to the Web server from both E1 and E2 and will get two different active paths towards 172.16.10.10. R1 will pick one of the paths to forward the traffic to reach the Web server and will advertise the user network subnet 192.168.100.0/24 to both E1 and E2 with OSPF.

E1 and E2 are NSX for vSphere Edge devices that also establish OSPF adjacencies with the DLR. E1 and E2 will learn how to get to the Web server via OSPF control plane communication with the DLR.

From the DLR perspective, it acts as a default gateway for the Web server. This DLR will form an OSPF adjacency with E1 and E2 and have 2 different OSPF routes to reach the user network.

From the DLR we can verify OSPF adjacency with E1, E2.

We can use the command: “show ip ospf neighbor”

 

ECMP Image 4

 

From this output we can see that the DLR has two Edge neighbors: 198.168.100.3 and 192.168.100.10.The next step will be to verify that ECMP is actually working.

We can use the command: “show ip route”

 

ECMP Image 5

 

The output from this command shows that the DLR learned the user network 192.168.100.0/24 via two different paths, one via E1 = 192.168.10.1 and the other via E2 = 192.168.10.10.

Now we want to display all the packets which were captured by an NSX for vSphere Edge interface.

In the example below and in order to display the traffic passing through interface vNic_1, and which is not OSPF protocol control packets, we need to type this command: “debug packet display interface vNic_1 not_ip_proto_ospf”

We can see an example with a ping running from host 192.168.100.86 to host 172.16.10.10

 

ECMP Image 6

 

If we would like to display the captured traffic to a specific ip address 172.16.10.10, the command capture would look like: “debug packet display interface vNic_1 dst_172.16.10.10”

 

ECMP Image 7

 

Useful CLI for Debugging ECMP

  • To check which ECMP path is chosen for a flow
    • debug packet display interface IFNAME
  • To check the ECMP configuration
    • show configuration routing-global
  • To check the routing table
    • show ip route
  • To check the forwarding table
    • show ip forwarding

Useful CLI for Dynamic Routing

  • show ip ospf neighbor
  • show ip ospf database
  • show ip ospf interface
  • show ip bgp neighbors
  • show ip bgp

ECMP Deployment Consideration

ECMP currently implies stateless behavior. This means that there is no support for stateful services such as the Firewall, Load Balancing or NAT on the NSX Edge Services Gateway. The Edge Firewall gets automatically disabled on ESG when ECMP is enabled. In the current NSX 6.1 release, the Edge Firewall and ECMP cannot be turned on at the same time on NSX edge device. Note however, that the Distributed Firewall (DFW) is unaffected by this.

For more in-depth information, you can also read our VMware® NSX for vSphere (NSX-V) Network Virtualization Design Guide

About The Authors

VMware-Roie Ben HaimRoie Ben Haim works as a professional services consultant at VMware, focusing on design and implementation of VMware’s software-defined data center products.  Roie has more than 12 years in data center architecture, with a focus on network and security solutions for global enterprises. An enthusiastic M.Sc. graduate, Roie holds a wide range of industry leading certifications including Cisco CCIE x2 # 22755 (Data Center, CCIE Security), Juniper Networks JNCIE – Service Provider #849, and VMware vExpert 2014, VCP-NV, VCP-DCV. Follow his personal blog at http://roie9876.wordpress.com/

VMware-Max ArdicaMax Ardica is a senior technical product manager in VMware’s networking and security business unit (NSBU). Certified as VCDX #171, his primary task is helping to drive the evolution of the VMware NSX platform, building the VMware NSX architecture and providing validated design guidance for the software-defined data center, specifically focusing on network virtualization. Prior to joining VMware, Max worked for almost 15 years at Cisco, covering different roles, from software development to product management. Max owns also a CCIE certification (#13808).

VMware NSX Runs Great on VCE Vblock Systems

Vblock SystemsLast week at EMC World in Las Vegas, one of the industry’s best offerings in converged infrastructure was on display. The adoption of converged infrastructure is becoming increasingly common in many organizations. In fact, research estimates that the total addressable market for converged infrastructure will reach $402B by 2017. Companies are taking advantage of converged infrastructure to accelerate cloud and software-defined data center deployments. Converged infrastructure is used by IT organizations to reduce provisioning times, centralize the management of IT resources, and increase resource utilization rates – resulting in lower costs. These objectives are enabled by the creation of pools of compute, storage and networking resources that can be shared by multiple applications and managed in a collective manner using policy driven processes. Continue reading

VMware at RSA Conference 2014 (#RSAC)

Summary:logo_rsac

  • Company outlines vision for security in the Software-Defined Data Center
  • Product and partner demonstrations in Booth #1615 to showcase growing security portfolio
  • New PCI-DSS 3.0 and FedRAMP reference architectures to be presented

Throughout its history, RSA Conference has consistently attracted the world’s best and brightest in the security field, creating opportunities for attendees to learn about IT security’s most important issues through first-hand interactions with peers, luminaries and emerging and established companies. Continue reading

Elephant Flow Mitigation via Virtual-Physical Communication

Note: this post was developed jointly by Justin Pettit of VMware and Mark Pearson of HP, with additional content from VMware’s Martin Casado and Bruce Davie.

A recent Network Heresy post “Of Mice and Elephants” discussed the impact long-lived flows (elephants) have on their short-lived peers (mice).  A quick summary is that, in a datacenter, it is believed that the majority of flows are short-lived (mice), but the majority of packets are long-lived (elephants). Mice flows tend to be bursty and latency-sensitive, whereas elephant flows tend to transfer large amounts of data, with per-packet latency being of less concern.  These elephants can fill up network buffers, which can introduce latency for mice.

At the HP 2013 Discover Conference, HP and VMware demonstrated a technology preview of detecting and handling elephant flows in an overlay network. The demonstration featured the joint HP-VMware solution announced at VMworld 2013.  VMware NSX provided an overlay network using HP switches as the underlay along with the HP VAN SDN controller. Through controller federation interfaces, the overlay and the underlay co-operated to mitigate the effects of the elephant flows on the mice. The solution shows the power of integration between network virtualization and SDN solutions. Continue reading

VMware NSX is Off and Running In 2014

This afternoon, VMware announced Q4 2013 and FY2013 earnings. The results can be found here in our earnings press release. During the earnings call, we provided an update on VMware NSX momentum which we want to share with you.

In the fourth quarter, we delivered VMware NSX, our network virtualization platform, which we believe will do for networking what vSphere and server virtualization did for compute. The VMware NSX customers we highlighted for the fourth quarter are great examples of innovative companies that have made the architectural decision to deploy network virtualization and the Software-Defined Data Center as the heart of their data center strategies. They are virtualizing their networks to deliver the speed and agility they need today. These deals in the quarter included three of the top five investment banks, as well as several of the most respected enterprises and Telcos from around the world, including McKesson, Starbucks, Medtronic and China Telecom.

Additionally, this quarter we announced VMware and Palo Alto Networks will deliver a jointly-developed solution for network security. The integrated solution will enable customers to use the VMware NSX network virtualization platform to automate provisioning and distribution of Palo Alto Networks’ next-generation network security in their software-defined data centers. Side Note: if you are planning to attend the upcoming RSA Conference, read about our session which will take place on Monday, February 24 at 1pm titled, “The Goldilocks Zone: Security in the Era of the SDDC.”

Network virtualization with VMware NSX is a key enabler for the software-defined data center. In 2013, VMware took big steps to present a vision for the future of the data center and the transformation of the network. In 2014 we expect to see an accelerated pace of network virtualization adoption as companies move from consideration to decision on the software-defined data center, and we feel very strong in opportunity to capture this market transformation.

Chris King, VP Product Marketing

Networking & Security Business Unit

VMware Announces General Availability of VMware NSX Network Virtualization Platform

Today at VMworld® in Barcelona, we once again highlighted VMware NSX, the platform for network virtualization. More importantly, we announced general availability of VMware NSX. Interested customers should contact their VMware representative who can put them in touch directly with a VMware NSX specialist.

Originally announced at VMworld in San Francisco, VMware NSX represents another giant step for VMware customers as they look to bring the operational benefits of server virtualization to the network. To read more about the launch, see our full blog post from the August announcement.  We also encourage you to read what our broad set of ecosystem partners had to say.

If you are interested in a deeper dive on VMware NSX, here is a great overview video from VMworld San Francisco in August.

Additionally, make sure you take a look at VMware NSX labs available in Hands-On Labs online portal. You can learn more about these labs in our blog post here.

Roger Fortier

VMware NSX Virtualizes the Network to Transform Network Operations

Summary:

  • VMware announces VMware NSX™, the platform for network virtualization
  • Leading Companies to Virtualize Their Networks to Speed Innovation
  • Partner Ecosystem Aligns with VMware to Support Customer Transition to Virtual Networking

Today at VMworld®, we announced VMware NSX, the platform for network virtualization. This announcement is another giant step for VMware as we evolve from being a server virtualization vendor into a supplier of an entire solution for the data center. At the show, our CEO Pat Gelsinger talked about how VMware is helping to transform the network to radically simplify IT as part of his VMworld keynote presentation.  He was joined on stage by several leading companies, including CITI, eBay and GE, to discuss the value of network virtualization. Additionally, more than 20 partners announced support for VMware NSX. Continue reading

Introducing VMware NSX – The Platform For Network Virtualization

Executive Overview: Today’s data center is largely virtualized from a compute perspective, and has unleashed unprecedented benefits of agility, efficiency and capex/opex savings. What is less known is that virtual network access ports have exceeded physical network access ports in number, and this trend is accelerating. In fact, today, 40% of vAdmins manage virtual networks. Beyond virtual switching, the time is ripe to virtualize the rest of the networking stack, and accelerate our customer’s journey to the software-defined data center.

The VMware NSX platform delivers the entire networking and security model in software, decoupled from traditional networking hardware, representing a transformative leap forward in data center networking architecture.

Picking The Right Abstrations For Your Network Virtualization Solution

In my travels around the internet, I became increasingly frustrated by the fact that most descriptions of SDN and network virtualisation solutions dive right down into the specifics of how stuff works. While I’m all for the details, I feel that there is an opportunity here to step back a bit and talk about the abstractions, which is what the end-user will see and deal with. For this post, (and yes, by association) I will talk about the abstractions used by perhaps the most mature network virtualisation solution on the market today. And yes, this means that I won’t be talking

Dmitri Kalintsev, Solution Architect, VMware

Note – this post appeared on the Telecom Occasionally blog. Read the entire post here.

 

Network Virtualization Production Deployments Continue to Rise

It was interesting to read this post on TechTarget that posed the question, “Is network virtualization real or just vaporware?” The facts are pretty clear that network virtualization is being deployed in production networks today to create business opportunities and solve infrastructure challenges for real-world customers.

For example, NTT Communications, one of the world’s largest telecommunications providers, announced it is deploying network virtualization as foundation for its new service enterprise cloud service.

WorldPay has deployed network virtualization to enable user self-provisioning and support around-the-clock development to speed the delivery of new line-of-business applications. By virtualizing the network, WorldPay gains operational simplicity to quickly create development environments in a non-disruptive way, and has on-demand access to network services such as load balancing and firewalls.

How about ViaWest, one of the largest privately-held data center, cloud computing and managed service providers in North America? Network virtualization provides ViaWest with a more efficient way of deploying cloud services than traditional hardware-defined networking approaches, leading to lower overall costs and a more consistent customer experience.

Or Logicworks, which specializes in private, public and hybrid cloud solutions for a broad array of industries, including a cloud offering focused specifically on the emerging healthcare market. Network virtualization enables Logicworks to be flexible in meeting app-specific infrastructure requirements for healthcare customers in a fraction of the time it would take to deploy applications using traditional, physical infrastructure.

To his credit, the author took all of the feedback and comments in stride. He wrote a follow up post on his own blog where he answers his own question, saying, There’s no question that network virtualization is real and is a serious force in the market.”

I’m glad the industry as a whole is starting to see potential for our VMware NSX network virtualization platform. We look forward to delivering on both the potential and the excitement which the author highlighted.

Roger Fortier, VMware, Inc.