We’re excited to take to the road for another edition of our VMware Software-Defined Data Center Seminar Series. Only this time, we’ll be joined by some great company.
VMware & Palo Alto Networks invite you along for a complementary, half-day educational event for IT professionals interested in learning about how Palo Alto Networks and VMware are transforming data center security.
Thousands of IT professionals attended our first SDDC seminar series earlier this year in more than 20 cities around the globe. Visit #VirtualizeYourNetwork.com to browse the presentations, videos, and other content we gathered.
This free seminar will highlight:
The Software-Defined Data Center approach
Lessons learned from real production customers
Using VMware NSX to deliver never before possible data center security and micro-segmentation
Who should attend?
People who will benefit from attending this session include:
IT, Infrastructure and Data Center Managers
Network professionals, including CCIEs
Security & Compliance professionals
Networking Managers and Administrators
Security Managers and Administrators
8:30 a.m. Registration & Breakfast
9:00 a.m. VMware: Better Security with Micro-segmentation
10:00 a.m. Palo Alto Networks: Next Generation Security Services for the SDDC
This post was written by Roie Ben Haim and Max Ardica, with a special thanks to Jerome Catrouillet, Michael Haines, Tiran Efrat and Ofir Nissim for their valuable input.
The modern data center design is changing, following a shift in the habits of consumers using mobile devices, the number of new applications that appear every day and the rate of end-user browsing which has grown exponentially. Planning a new data center requires meeting certain fundamental design guidelines. The principal goals in data center design are: Scalability, Redundancy and High-bandwidth.
In this blog we will describe the Equal Cost Multi-Path functionality (ECMP) introduced in VMware NSX release 6.1 and discuss how it addresses the requirements of scalability, redundancy and high bandwidth. ECMP has the potential to offer substantial increases in bandwidth by load-balancing traffic over multiple paths as well as providing fault tolerance for failed paths. This is a feature which is available on physical networks but we are now introducing this capability for virtual networking as well. ECMP uses a dynamic routing protocol to learn the next-hop towards a final destination and to converge in case of failures. For a great demo of how this works, you can start by watching this video, which walks you through these capabilities in VMware NSX.
Scalability and Redundancy and ECMP
To keep pace with the growing demand for bandwidth, the data center must meet scale out requirements, which provide the capability for a business or technology to accept increased volume without redesign of the overall infrastructure. The ultimate goal is avoiding the “rip and replace” of the existing physical infrastructure in order to keep up with the growing demands of the applications. Data centers running business critical applications need to achieve near 100 percent uptime. In order to achieve this goal, we need the ability to quickly recover from failures affecting the main core components. Recovery from catastrophic events needs to be transparent to end user experiences.
ECMP with VMware NSX 6.1 allows you to use upto a maximum of 8 ECMP Paths simultaneously. In a specific VMware NSX deployment, those scalability and resilience improvements are applied to the “on-ramp/off-ramp” routing function offered by the Edge Services Gateway (ESG) functional component, which allows communication between the logical networks and the external physical infrastructure.
External user’s traffic arriving from the physical core routers can use up to 8 different paths (E1-E8) to reach the virtual servers (Web, App, DB).
In the same way, traffic returning from the virtual server’s hit the Distributed Logical Router (DLR), which can choose up to 8 different paths to get to the core network.
How is the path determined:
NSX for vSphere Edge Services Gateway device:
When a traffic flow needs to be routed, the round robin algorithm is used to pick up one of the links as the path for all traffic of this flow. The algorithm ensures to keep in order all the packets related to this flow by sending them through the same path. Once the next-hop is selected for a particular Source IP and Destination IP pair, the route cache stores this. Once a path has been chosen, all packets related to this flow will follow the same path.
There is a default IPv4 route cache timeout, which is 300 seconds. If an entry is inactive for this period of time, it is then eligible to be removed from route cache. Note that these settings can be tuned for your environment.
Distributed Logical Router (DLR):
The DLR will choose a path based on a Hashing algorithm of Source IP and Destination IP.
What happens in case of a failure on one of Edge Devices?
In order to work with ECMP the requirement is to use a dynamic routing protocol: OSPF or BGP. If we take OSPF for example, the main factor influencing the traffic outage experience is the tuning of the OSPF timers.
OSPF will send hello messages between neighbors, the OSPF “Hello” protocol is used and determines the Interval as to how often an OSPF Hello is sent.
Another OSPF timer called “Dead” Interval is used, which is how long to wait before we consider an OSPF neighbor as “down”. The OSPF Dead Interval is the main factor that influences the convergence time. Dead Interval is usually 4 times the Hello Interval but the OSPF (and BGP) timers can be set as low as 1 second (for Hello interval) and 3 seconds (for Dead interval) to speed up the traffic recovery.
In the example above, the E1 NSX Edge has a failure; the physical routers and DLR detect E1 as Dead at the expiration of the Dead timer and remove their OSPF neighborship with him. As a consequence, the DLR and the physical router remove the routing table entries that originally pointed to the specific next-hop IP address of the failed ESG.
As a result, all corresponding flows on the affected path are re-hashed through the remaining active units. It’s important to emphasize that network traffic that was forwarded across the non-affected paths remains unaffected.
Troubleshooting and visibility
With ECMP it’s important to have introspection and visibility tools in order to troubleshoot optional point of failure. Let’s look at the following topology.
A user outside our Data Center would like to access the Web Server service inside the Data Center. The user IP address is 192.168.100.86 and the web server IP address is 172.16.10.10.
This User traffic will hit the Physical Router (R1), which has established OSPF adjacencies with E1 and E2 (the Edge devices). As a result R1 will learn how to get to the Web server from both E1 and E2 and will get two different active paths towards 172.16.10.10. R1 will pick one of the paths to forward the traffic to reach the Web server and will advertise the user network subnet 192.168.100.0/24 to both E1 and E2 with OSPF.
E1 and E2 are NSX for vSphere Edge devices that also establish OSPF adjacencies with the DLR. E1 and E2 will learn how to get to the Web server via OSPF control plane communication with the DLR.
From the DLR perspective, it acts as a default gateway for the Web server. This DLR will form an OSPF adjacency with E1 and E2 and have 2 different OSPF routes to reach the user network.
From the DLR we can verify OSPF adjacency with E1, E2.
We can use the command: “show ip ospf neighbor”
From this output we can see that the DLR has two Edge neighbors: 188.8.131.52 and 192.168.100.10.The next step will be to verify that ECMP is actually working.
We can use the command: “show ip route”
The output from this command shows that the DLR learned the user network 192.168.100.0/24 via two different paths, one via E1 = 192.168.10.1 and the other via E2 = 192.168.10.10.
Now we want to display all the packets which were captured by an NSX for vSphere Edge interface.
In the example below and in order to display the traffic passing through interface vNic_1, and which is not OSPF protocol control packets, we need to type this command: “debug packet display interface vNic_1 not_ip_proto_ospf”
We can see an example with a ping running from host 192.168.100.86 to host 172.16.10.10
If we would like to display the captured traffic to a specific ip address 172.16.10.10, the command capture would look like: “debug packet display interface vNic_1 dst_172.16.10.10”
Useful CLI for Debugging ECMP
To check which ECMP path is chosen for a flow
debug packet display interface IFNAME
To check the ECMP configuration
show configuration routing-global
To check the routing table
show ip route
To check the forwarding table
show ip forwarding
Useful CLI for Dynamic Routing
show ip ospf neighbor
show ip ospf database
show ip ospf interface
show ip bgp neighbors
show ip bgp
ECMP Deployment Consideration
ECMP currently implies stateless behavior. This means that there is no support for stateful services such as the Firewall, Load Balancing or NAT on the NSX Edge Services Gateway. The Edge Firewall gets automatically disabled on ESG when ECMP is enabled. In the current NSX 6.1 release, the Edge Firewall and ECMP cannot be turned on at the same time on NSX edge device. Note however, that the Distributed Firewall (DFW) is unaffected by this.
Roie Ben Haim works as a professional services consultant at VMware, focusing on design and implementation of VMware’s software-defined data center products. Roie has more than 12 years in data center architecture, with a focus on network and security solutions for global enterprises. An enthusiastic M.Sc. graduate, Roie holds a wide range of industry leading certifications including Cisco CCIE x2 # 22755 (Data Center, CCIE Security), Juniper Networks JNCIE – Service Provider #849, and VMware vExpert 2014, VCP-NV, VCP-DCV. Follow his personal blog at http://roie9876.wordpress.com/
Max Ardica is a senior technical product manager in VMware’s networking and security business unit (NSBU). Certified as VCDX #171, his primary task is helping to drive the evolution of the VMware NSX platform, building the VMware NSX architecture and providing validated design guidance for the software-defined data center, specifically focusing on network virtualization. Prior to joining VMware, Max worked for almost 15 years at Cisco, covering different roles, from software development to product management. Max owns also a CCIE certification (#13808).
In 2013 we introduced VMware NSX Hands-on-Labs for the first time. The NSX 1303 Hands-on-lab has been by far one of the most popular labs, giving you an in-depth view of VMware NSX. Hands-on-labs are one of the best ways to get a good tour of the product. You can take all of these labs online at http://labs.hol.vmware.com/HOL/catalogs/ . It requires a registration, but is open to everyone. .
This year at VMworld we introduced several new NSX labs to give you a deeper look at NSX, and to showcase the depth of integration NSX provides with 3rd party partners and other VMware products. All of the new 2014 Hands-on-labs have been published and are available to you. Here is a quick tour of the labs and what you can expect to see.
If you are just getting started with NSX and want to know what Network Virtualization is all about, we recommend you start here.
This lab will walk you through five modules of exercises:
NSX Logical Switching – building VXLAN logical switches
NSX Logical Routing - Distributed Routing, Dynamic Routing with OSPF
NSX Distributed Firewall – Micro-segmentation with NSX
NSX Edge Services – Load-balancing, SSL VPN
Once you have completed the introductory lab, we recommend taking the advanced lab which is designed to showcase some of the new features in NSX 6.1. You can read and excellent summary of these new capabilities in Chris Wahl’s blog, “NSX 6.1 Announced, Contains Plethora of Enhancements.”
This lab covers the following areas:
Configuring DHCP Relay so that you can use NSX with external IPAM Services
Scaling out Layer 3 routing with Equal Cost Multi-Pathing (ECMP) and Dynamic Routing Protocols. Yes we actually build out the topology below in the lab! That’s the power of network virtualization.
Building out L2VPN services for multi-site and hybrid cloud connectivity services
Integration with 3rd parties using Service Composer and Trend Micro AV & IPS with NSX. You will see how to register services and how NSX is a platform to integrate with 3rd party services in this exercise.
Networking Monitoring with NSX & Riverbed Cascade – we will even show you how you can monitor with NetFlow in this exercise
The two labs above will surely give you a good view of NSX as a network virtualization platform. Next, let’s see how NSX integrates with other VMware products to build out a complete Software-Defined Data Center. This lab shows the integration capabilities offered by NSX with VMware management solutions.
First up, we will learn about Self-service IT, and how you can deliver applications quickly to your end-users with the integration of vCloud Automation Center and NSX. You will build out a multi-machine blueprint with networking and security, and then deploy it.
Next, if you want to learn about automation and the NSX API, we will walk you through exercises in using vCenter Orchestrator and using the NSX REST API to create a security group. This will give you the fundamentals of NSX automation which you can easily extend upon as you deploy NSX in your own environment.
The third exercise is about operations. We will show you the new NSX Management Pack in vCenter Operations. We will walk you through the dashboards and you will learn how you can actually not just monitor but also troubleshoot you network.
At this point you are surely on your way to become a NSX Ninja
If you want to use OpenStack with NSX and vSphere – we’ve got you covered too! We will walk you through OpenStack on vSphere itself and then show you how to connect it to deploy networks with NSX from OpenStack.
And Of Course There Are More
Those are the main labs I would recommend, but there are others too. There’s a lab where you can learn more about the IT Outcome of Fast Infrastructure Delivery and Application Automation (HOL-SDC-1413) which has some NSX goodness with vCloud Automation Center, or learn about the IT Outcome of Policy-based Compliance and Network Security (HOL-SDC-1414).
If you want to learn about NSX and the partner integration framework you can take HOL-PRT-1462 which will walk you through the NSX and Palo Alto Networks next-gen firewall integration labs and HOL-PRT-1464 which is focused on how you can use NSX Service Composer and Symantec Data Center Security: Server.
In all we have well over 24 hours of labs, and you can sign-up even if you did not go to VMworld. It is always available 24/7, so if you have a few spare hours and want to learn about NSX you can take the lab.
And I will let you in on a little secret. We actually run the labs on NSX. So as you learn, you are also a user of NSX!!!
In the previous post we took a look at the simplicity of deploying VMware NSX into a new or existing VMware environment. This post looks to develop upon our existing infrastructure and build out a three-tier application with a Web, App and Database tier.
This application displayed above highlights what this blog seeks to build out. Note that there are three logical network segments – Web, App, DB and Uplink (Transport) – routing functionality provided by the Logical Distributed Router and an NSX Edge Services Gateway that is used to connect the logical network topology to the physical infrastructure. Continue reading →
VMware NSX network virtualization software makes it possible for IT organizations to obtain new levels of business agility, allowing you to deploy advanced policy based network services for your applications as quickly as a virtual machine, today, leveraging your existing physical network infrastructure or next generation network hardware from any vendor.
Back in September, I wrote about the value of deploying the VMware NSX network virtualization platform on Cisco UCS and Nexus infrastructure. We had an overwhelming amount of customer response and request for more content about this deployment scenario. As such, we’ve created a new design guide which you can download here that describes a simple and straight forward example of how to deploy VMware NSX for vSphere on an infrastructure consisting of Cisco UCS and Nexus 7000 series switches. This basic guide should serve as a starting point from which to tailor a design for your specific environment.
We want to offer our thanks to the content creation team here at VMware for collaborating on this effort:
• Dmitri Kalintsev
• Bruce Davie
• Ray Budavari
• Venky Deshpande
• Nikhil Kelshikar
• Rod Stuhlmuller
• Scott Lowe
• Marcos Hernandez
• Chris King
Software is the foundation that is powering the next evolution of networks and data center infrastructure in today’s digital age. The manifestation of this trend is the software-defined data center, which gains momentum in the market on a daily basis. VMware is committed to providing the knowledge required for the adoption of the new operating model for the network in the era of the software-defined data center. To help the industry take advantage of the opportunity to virtualize their infrastructure, and specifically the network, VMware is providing the programs, curriculum and blueprints to help you capture this transformational opportunity. At VMware Partner Exchange (PEX) in San Francisco this week, we outlined three ways we’re helping to make the software-defined data center real in 2014. Continue reading →
At VMworld 2013 in San Francisco, we launched the VMware NSX network virtualization platform to the world. During the keynote, our CEO Pat Gelsinger was joined by representatives from CITI, GE and eBay to discuss the promise of network virtualization and VMware NSX, and more than 20 partners announced support for the platform.
But perhaps the most successful part of our launch were the VMware NSX Hands-On Labs. These labs were by far the most successful at the show. Attendees consumed more than 2,000 sessions, totaling 124,000 lab minutes during the four days of VMworld. That is roughly equivalent to locking yourself in a room with your laptop and doing nothing but take this lab 24 hours a day, seven days a week, for three months straight.
HOL-SDC-1303 – VMware NSX: The Network Virtualization Platform
A Tech Preview of the exciting new VMware NSX for vSphere product announced at VMworld. Learn how VMware NSX virtualizes your network and simplifies your datacenter operations. This lab is currently based on a beta version of code and you may encounter some user interface issues during the lab exercises. The lab will be improved with newer code as the product moves closer to release. For now, brave the rapids, jump in with both feet and have a go at at VMware NSX, the network virtualization platform.