We are introducing a new updated version of the NSX design guide just in time for the holiday break to add to your yearend reading list. This design guide incorporates tons of feedback we have received from our readers and is based on the learnings of over 200+ production customer deployments of NSX. Continue reading →
As part of the recent launch of Horizon 6, Tony Paikeday, senior product line manager, End-User Computing, VMware, takes a look at the value proposition of deploying the VMware NSX network virtualization platform together with Horzon.
Deploying VMware NSX with Horizon
VMware NSX, deployed with Horizon, offers a better alternative to securing east-west traffic between VMs, turning data center security from a perimeter-centric view to one that gives each individual desktop VM its own virtual network container – creating if you will, a network of “one.” This approach, also known as micro-segmentation, has been an ideal for network teams, but traditionally unachievable due to the cost, and the operational complexity involved. With the number of user VM’s introduced by desktop virtualization, and the sprawl of firewall rules needing to be manually added, deleted or modified every time a new VM is introduced, this has been untenable in the past. With VMware NSX, we have a completely new model for networking and security, delivering virtualization of the network, much as we did for server virtualization – reproducing it in software, with a logical library of networking elements and services including switches, routers, firewalls, load-balancers and more that can be deployed over any existing network.
You may have seen Joey Logano speed to his first Daytona 500 win this week. Keeping your network in racing shape takes a similar level of patience, stamina, and quick reflexes. Using VMware NSX network virtualization means that you can unlock the full potential of a Software-Defined Data Center, to create and run entire networks on top of existing network hardware, resulting in faster deployment of workloads, as well as greater agility in the face of increasingly dynamic data centers. Watch this overview to learn how VMware NSX reduces the time to provision multi-tier networking and security services from weeks to seconds to win your race. This one-hour overview of VMware NSX outlines how you can bring virtualization to your existing network, transforming both its operations and economics. You’ll learn how several of the largest service providers, global financial, and enterprise data centers in the world are using NSX to reduce costs and provisioning times to improve agility and establish a new model of network security.
This post was written by Roie Ben Haim and Max Ardica, with a special thanks to Jerome Catrouillet, Michael Haines, Tiran Efrat and Ofir Nissim for their valuable input.
The modern data center design is changing, following a shift in the habits of consumers using mobile devices, the number of new applications that appear every day and the rate of end-user browsing which has grown exponentially. Planning a new data center requires meeting certain fundamental design guidelines. The principal goals in data center design are: Scalability, Redundancy and High-bandwidth.
In this blog we will describe the Equal Cost Multi-Path functionality (ECMP) introduced in VMware NSX release 6.1 and discuss how it addresses the requirements of scalability, redundancy and high bandwidth. ECMP has the potential to offer substantial increases in bandwidth by load-balancing traffic over multiple paths as well as providing fault tolerance for failed paths. This is a feature which is available on physical networks but we are now introducing this capability for virtual networking as well. ECMP uses a dynamic routing protocol to learn the next-hop towards a final destination and to converge in case of failures. For a great demo of how this works, you can start by watching this video, which walks you through these capabilities in VMware NSX.
Scalability and Redundancy and ECMP
To keep pace with the growing demand for bandwidth, the data center must meet scale out requirements, which provide the capability for a business or technology to accept increased volume without redesign of the overall infrastructure. The ultimate goal is avoiding the “rip and replace” of the existing physical infrastructure in order to keep up with the growing demands of the applications. Data centers running business critical applications need to achieve near 100 percent uptime. In order to achieve this goal, we need the ability to quickly recover from failures affecting the main core components. Recovery from catastrophic events needs to be transparent to end user experiences.
ECMP with VMware NSX 6.1 allows you to use upto a maximum of 8 ECMP Paths simultaneously. In a specific VMware NSX deployment, those scalability and resilience improvements are applied to the “on-ramp/off-ramp” routing function offered by the Edge Services Gateway (ESG) functional component, which allows communication between the logical networks and the external physical infrastructure.
External user’s traffic arriving from the physical core routers can use up to 8 different paths (E1-E8) to reach the virtual servers (Web, App, DB).
In the same way, traffic returning from the virtual server’s hit the Distributed Logical Router (DLR), which can choose up to 8 different paths to get to the core network.
How is the path determined:
NSX for vSphere Edge Services Gateway device:
When a traffic flow needs to be routed, the round robin algorithm is used to pick up one of the links as the path for all traffic of this flow. The algorithm ensures to keep in order all the packets related to this flow by sending them through the same path. Once the next-hop is selected for a particular Source IP and Destination IP pair, the route cache stores this. Once a path has been chosen, all packets related to this flow will follow the same path.
There is a default IPv4 route cache timeout, which is 300 seconds. If an entry is inactive for this period of time, it is then eligible to be removed from route cache. Note that these settings can be tuned for your environment.
Distributed Logical Router (DLR):
The DLR will choose a path based on a Hashing algorithm of Source IP and Destination IP.
What happens in case of a failure on one of Edge Devices?
In order to work with ECMP the requirement is to use a dynamic routing protocol: OSPF or BGP. If we take OSPF for example, the main factor influencing the traffic outage experience is the tuning of the OSPF timers.
OSPF will send hello messages between neighbors, the OSPF “Hello” protocol is used and determines the Interval as to how often an OSPF Hello is sent.
Another OSPF timer called “Dead” Interval is used, which is how long to wait before we consider an OSPF neighbor as “down”. The OSPF Dead Interval is the main factor that influences the convergence time. Dead Interval is usually 4 times the Hello Interval but the OSPF (and BGP) timers can be set as low as 1 second (for Hello interval) and 3 seconds (for Dead interval) to speed up the traffic recovery.
In the example above, the E1 NSX Edge has a failure; the physical routers and DLR detect E1 as Dead at the expiration of the Dead timer and remove their OSPF neighborship with him. As a consequence, the DLR and the physical router remove the routing table entries that originally pointed to the specific next-hop IP address of the failed ESG.
As a result, all corresponding flows on the affected path are re-hashed through the remaining active units. It’s important to emphasize that network traffic that was forwarded across the non-affected paths remains unaffected.
Troubleshooting and visibility
With ECMP it’s important to have introspection and visibility tools in order to troubleshoot optional point of failure. Let’s look at the following topology.
A user outside our Data Center would like to access the Web Server service inside the Data Center. The user IP address is 192.168.100.86 and the web server IP address is 172.16.10.10.
This User traffic will hit the Physical Router (R1), which has established OSPF adjacencies with E1 and E2 (the Edge devices). As a result R1 will learn how to get to the Web server from both E1 and E2 and will get two different active paths towards 172.16.10.10. R1 will pick one of the paths to forward the traffic to reach the Web server and will advertise the user network subnet 192.168.100.0/24 to both E1 and E2 with OSPF.
E1 and E2 are NSX for vSphere Edge devices that also establish OSPF adjacencies with the DLR. E1 and E2 will learn how to get to the Web server via OSPF control plane communication with the DLR.
From the DLR perspective, it acts as a default gateway for the Web server. This DLR will form an OSPF adjacency with E1 and E2 and have 2 different OSPF routes to reach the user network.
From the DLR we can verify OSPF adjacency with E1, E2.
We can use the command: “show ip ospf neighbor”
From this output we can see that the DLR has two Edge neighbors: 220.127.116.11 and 192.168.100.10.The next step will be to verify that ECMP is actually working.
We can use the command: “show ip route”
The output from this command shows that the DLR learned the user network 192.168.100.0/24 via two different paths, one via E1 = 192.168.10.1 and the other via E2 = 192.168.10.10.
Now we want to display all the packets which were captured by an NSX for vSphere Edge interface.
In the example below and in order to display the traffic passing through interface vNic_1, and which is not OSPF protocol control packets, we need to type this command: “debug packet display interface vNic_1 not_ip_proto_ospf”
We can see an example with a ping running from host 192.168.100.86 to host 172.16.10.10
If we would like to display the captured traffic to a specific ip address 172.16.10.10, the command capture would look like: “debug packet display interface vNic_1 dst_172.16.10.10”
Useful CLI for Debugging ECMP
To check which ECMP path is chosen for a flow
debug packet display interface IFNAME
To check the ECMP configuration
show configuration routing-global
To check the routing table
show ip route
To check the forwarding table
show ip forwarding
Useful CLI for Dynamic Routing
show ip ospf neighbor
show ip ospf database
show ip ospf interface
show ip bgp neighbors
show ip bgp
ECMP Deployment Consideration
ECMP currently implies stateless behavior. This means that there is no support for stateful services such as the Firewall, Load Balancing or NAT on the NSX Edge Services Gateway. The Edge Firewall gets automatically disabled on ESG when ECMP is enabled. In the current NSX 6.1 release, the Edge Firewall and ECMP cannot be turned on at the same time on NSX edge device. Note however, that the Distributed Firewall (DFW) is unaffected by this.
Roie Ben Haim works as a professional services consultant at VMware, focusing on design and implementation of VMware’s software-defined data center products. Roie has more than 12 years in data center architecture, with a focus on network and security solutions for global enterprises. An enthusiastic M.Sc. graduate, Roie holds a wide range of industry leading certifications including Cisco CCIE x2 # 22755 (Data Center, CCIE Security), Juniper Networks JNCIE – Service Provider #849, and VMware vExpert 2014, VCP-NV, VCP-DCV. Follow his personal blog at http://roie9876.wordpress.com/
Max Ardica is a senior technical product manager in VMware’s networking and security business unit (NSBU). Certified as VCDX #171, his primary task is helping to drive the evolution of the VMware NSX platform, building the VMware NSX architecture and providing validated design guidance for the software-defined data center, specifically focusing on network virtualization. Prior to joining VMware, Max worked for almost 15 years at Cisco, covering different roles, from software development to product management. Max owns also a CCIE certification (#13808).
Last week at EMC World in Las Vegas, one of the industry’s best offerings in converged infrastructure was on display. The adoption of converged infrastructure is becoming increasingly common in many organizations. In fact, research estimates that the total addressable market for converged infrastructure will reach $402B by 2017. Companies are taking advantage of converged infrastructure to accelerate cloud and software-defined data center deployments. Converged infrastructure is used by IT organizations to reduce provisioning times, centralize the management of IT resources, and increase resource utilization rates – resulting in lower costs. These objectives are enabled by the creation of pools of compute, storage and networking resources that can be shared by multiple applications and managed in a collective manner using policy driven processes. Continue reading →
Company outlines vision for security in the Software-Defined Data Center
Product and partner demonstrations in Booth #1615 to showcase growing security portfolio
New PCI-DSS 3.0 and FedRAMP reference architectures to be presented
Throughout its history, RSA Conference has consistently attracted the world’s best and brightest in the security field, creating opportunities for attendees to learn about IT security’s most important issues through first-hand interactions with peers, luminaries and emerging and established companies. Continue reading →
Note: this post was developed jointly by Justin Pettit of VMware and Mark Pearson of HP, with additional content from VMware’s Martin Casado and Bruce Davie.
A recent Network Heresy post “Of Mice and Elephants” discussed the impact long-lived flows (elephants) have on their short-lived peers (mice). A quick summary is that, in a datacenter, it is believed that the majority of flows are short-lived (mice), but the majority of packets are long-lived (elephants). Mice flows tend to be bursty and latency-sensitive, whereas elephant flows tend to transfer large amounts of data, with per-packet latency being of less concern. These elephants can fill up network buffers, which can introduce latency for mice.
At the HP 2013 Discover Conference, HP and VMware demonstrated a technology preview of detecting and handling elephant flows in an overlay network. The demonstration featured the joint HP-VMware solution announced at VMworld 2013. VMware NSX provided an overlay network using HP switches as the underlay along with the HP VAN SDN controller. Through controller federation interfaces, the overlay and the underlay co-operated to mitigate the effects of the elephant flows on the mice. The solution shows the power of integration between network virtualization and SDN solutions. Continue reading →
This afternoon, VMware announced Q4 2013 and FY2013 earnings. The results can be found here in our earnings press release. During the earnings call, we provided an update on VMware NSX momentum which we want to share with you.
In the fourth quarter, we delivered VMware NSX, our network virtualization platform, which we believe will do for networking what vSphere and server virtualization did for compute. The VMware NSX customers we highlighted for the fourth quarter are great examples of innovative companies that have made the architectural decision to deploy network virtualization and the Software-Defined Data Center as the heart of their data center strategies. They are virtualizing their networks to deliver the speed and agility they need today. These deals in the quarter included three of the top five investment banks, as well as several of the most respected enterprises and Telcos from around the world, including McKesson, Starbucks, Medtronic and China Telecom.
Additionally, this quarter we announced VMware and Palo Alto Networks will deliver a jointly-developed solution for network security. The integrated solution will enable customers to use the VMware NSX network virtualization platform to automate provisioning and distribution of Palo Alto Networks’ next-generation network security in their software-defined data centers. Side Note: if you are planning to attend the upcoming RSA Conference, read about our session which will take place on Monday, February 24 at 1pm titled, “The Goldilocks Zone: Security in the Era of the SDDC.”
Network virtualization with VMware NSX is a key enabler for the software-defined data center. In 2013, VMware took big steps to present a vision for the future of the data center and the transformation of the network. In 2014 we expect to see an accelerated pace of network virtualization adoption as companies move from consideration to decision on the software-defined data center, and we feel very strong in opportunity to capture this market transformation.
Today at VMworld® in Barcelona, we once again highlighted VMware NSX, the platform for network virtualization. More importantly, we announced general availability of VMware NSX. Interested customers should contact their VMware representative who can put them in touch directly with a VMware NSX specialist.
Originally announced at VMworld in San Francisco, VMware NSX represents another giant step for VMware customers as they look to bring the operational benefits of server virtualization to the network. To read more about the launch, see our full blog post from the August announcement. We also encourage you to read what our broad set of ecosystem partners had to say.
If you are interested in a deeper dive on VMware NSX, here is a great overview video from VMworld San Francisco in August.
Additionally, make sure you take a look at VMware NSX labs available in Hands-On Labs online portal. You can learn more about these labs in our blog post here.
VMware announces VMware NSX™, the platform for network virtualization
Leading Companies to Virtualize Their Networks to Speed Innovation
Partner Ecosystem Aligns with VMware to Support Customer Transition to Virtual Networking
Today at VMworld®, we announced VMware NSX, the platform for network virtualization. This announcement is another giant step for VMware as we evolve from being a server virtualization vendor into a supplier of an entire solution for the data center. At the show, our CEO Pat Gelsinger talked about how VMware is helping to transform the network to radically simplify IT as part of his VMworld keynote presentation. He was joined on stage by several leading companies, including CITI, eBay and GE, to discuss the value of network virtualization. Additionally, more than 20 partners announced support for VMware NSX. Continue reading →