Today’s blog post is the final entry in a series by Marcos Hernandez, one of the OpenStack specialists in VMware’s Networking & Security Business Unit (NSBU). Part 1 discussed the basics of the Neutron integration for VMware NSX. Part 2 discussed foundational integrations for L2 and L3 network services.

Security Groups

Neutron Security Groups have historically implemented either Linux IPTables or Open vSwitch stateless matches to filter traffic at the hypervisor level. Both approaches have presented challenges to operations teams there is serious work underway aimed at improving the experience (VMware is a contributor to these efforts).

When using NSX and vSphere, we deploy a stateful firewall on each and every ESXi host. That means that every hypervisor will protect the microcosm of virtual machines that it hosts, providing the notion of a distributed data plane. We call this a distributed firewall, or DFW. The NSX DFW runs in the kernel of ESXi and enables granular security controls at the VM vNIC level. When using Neutron Security Groups, the NSX DFW is configured, via the plugin integration. Neutron Security Groups are mapped to instances, meaning the NSX DFW will protect the VM unit.

Running an actual firewall on each hypervisor within your OpenStack cloud has the following benefits:

• The attributes of micro-segmentation are readily available to the OpenStack admins and tenants.
• Compliance requirements (like PCI, or HIPAA) can be met, without sacrificing the openness of OpenStack as your IaaS layer.
• Advanced L4-L7 security services via service insertion/service chaining.

The NSX firewall scales as your ESXi footprint grows. The mere act of increasing your compute capacity due to the organic growth of your business, automatically means you are also adding security and compliance to your virtual infrastructure.

It is important to note that Neutron Security Groups and NSX micro-segmentation can be used standalone, without adopting L2 overlays and L3 virtualization. While not as flexible as a full network virtualization implementation, the micro-segmentation use case is very popular with our customers and provides a great on ramp for customers to introduce OpenStack and NSX without disrupting whatever VLAN operational model may already be in place.

Load Balancing

As of the Kilo version of the NSX plugin, Neutron LBaaS v1.0 support was incorporated. The workflow includes the following capabilities:

• Tenants are able to create application pools (initially empty).
• Tenants add several members to the pool (instance IP address).
• Tenants create one or several health monitors.
• Tenants associate the health monitors with the pool.
• The tenant finally creates a virtual IP (VIP) with the pool.
• Supported protocols: TCP, HTTP and HTTPS.

As with other network services in our implementation, we leverage the NSX Edge Services Gateway (ESG) as an inline load balancer as well as a Neutron router. The NSX load balancer is very feature-rich, and it is ready to support the Neutron LBaaS 2.0 API spec in a future version of the plugin.

Summary – Supported Topologies

The table below summarizes the topologies supported by the NSX-Neutron plugin:

Conclusion – Why NSX-v with OpenStack Neutron?

The benefits of NSX align with the requirements of a robust OpenStack private cloud implementation, which are:

• Agility – Networking at the speed of apps.
• Mobility – Provision anywhere, move anywhere.
• Security – Micro-segment, detect anywhere, detect early
• Multi-tenancy – Share hardware across multiple tenants.
• Simplified operations – Centrally manage, monitor everywhere.

By leveraging the NSX Neutron Plugin for vSphere developed by VMware, cloud administrators can introduce NSX into their OpenStack environment and offer their users and developers the open APIs they require, all without compromising uptime, stability and scalability.

This concludes the series discussing OpenStack Neutron integrations with VMware NSX. You can get some hands-on experience with VMware Integrated OpenStack with our revamped VMworld Hands-on Lab (HOL-1620) featuring VMware Integrated OpenStack and NSX Optimized for vSphere. You can also check out Part 1 and Part 2 of this blog series to read more about NSX integrations with OpenStack.

Marcos Hernandez is a Staff Systems Engineer in the Network and Security Business Unit (NSBU). He is responsible for supporting large Global Enterprise accounts and providing technical guidance around VMware’s suite of networking and cloud solutions, including NSX and OpenStack. Marcos has a background in datacenter networking design and expert knowledge in routing and switching technologies. Marcos holds the CCIE (#8283) and VCIX certifications, and he has a Masters Degree in Telecommunications from Universidad Politécnica de Madrid.

Today’s blog post is the second entry in a series by Marcos Hernandez, one of the OpenStack specialists in VMware’s Networking & Security Business Unit (NSBU). Part 1 discussed the basics of the Neutron integration for VMware NSX. Part 3 will be published in the upcoming weeks. So, check back for more on this topic!

L2 Services

As we discussed in our previous article, when a tenant creates a Neutron network (or networks), the plugin signals NSX Manager to provision a logical switch (or switches), which are overlay constructs that utilize VXLAN to create L2 segments over L3 physical networks. VXLAN is an industry standard, co-developed by VMware and others and supported across the board. Over the past couple of years, VXLAN overlays have been largely demystified and the initial objections (like performance, lack of visibility, etc.) have given way to more practical concerns (like changes in operational processes, automation, etc.). Customers and vendors are getting more educated about each other’s vision and together are making VXLAN, and Software Defined Networking for that matter, a reality in their environments.

These L2 segments, overlays as they may be, are just that: L2 segments. Without a router to connect them together or to other networks, they are completely isolated from each other. An OpenStack Cloud administrator can control, via quotas, the number of Neutron networks allowed per tenant.

L3 Services

Tenants in OpenStack are allowed, by default, to create their own IP subnets and routers. We will cover some of the NSX capabilities available with the Neutron plugin. Before we do that though, just a quick parenthesis about self-service in general: As OpenStack gains more traction in the Enterprise, we are learning that these self-service capabilities may not be desirable. Admins may want to remain in control of the IP subnetting, for example, especially if the use case calls for routable IP address space everywhere. OpenStack lacks the necessary controls to enforce this type of restrictions, so short of forbidding API access to specific functions or simply relying on the good-old honor system, customers have little to no choice when it comes to the built-in OpenStack governance. Projects like OpenStack Congress are attempting to bridge this gap, and some commercial products are already providing the controls that IT requires. vRealize Automation (vRA) is a VMware platform that offers comprehensive, scalable governance and could potentially leverage extension packages to drive provisioning workflows in OpenStack.

Back to the L3 services discussion, we stated that a tenant could create Neutron routers. The NSX-Neutron plugin will translate this provisioning request and signal NSX Manager to create an NSX Edge Services Gateway, or ESG. The ESG is a network appliance that supports a vast number of network features (not all of which are visible by OpenStack, by the way) and that is broadly used in our integration.

Once the Neutron router is created, our previously provisioned Web and App Neutron networks (L2 segments) can be connected to it and routing between them will be available.

The uplink of a Neutron router can be connected to an External network. This is also known as setting the gateway. This External network must sit on routable IP address space within the organization and is also the network where floating IPs reside. If the tenant networks sit on RFC1918 space, then the Neutron router must do Network Address Translation, or NAT (source NAT for internal to external access and DNAT for floating IPs). If the tenant networks sit on routable subnets, then the router does not have to do NAT.

The tenant networks can also be VLAN-backed, instead of VXLAN-backed. If the tenant wants to or has to use VLANs instead of VXLANs, then the admin must create these networks on behalf of the tenant.

Tenant routers can be exclusive (defined at provisioning time using an API extension) or shared (default behavior). Depending on your performance and scalability expectations, you will choose one or the other.

When using NSX, the Neutron L3 services may include a distributed router, which is a very powerful capability in NSX that allows for the optimization of East-West traffic in routed topologies. This is a good example of an enterprise-grade capability of NSX and differentiator from the reference implementation, which can be leveraged without compromising the basic tenet of OpenStack in keeping the API open. A distributed router sends traffic from the source hypervisor to the destination hypervisor without hairpinning the packets through an NSX ESG or a physical router SVI. This increases performance significantly and streamlines traffic engineering within the data center.

Finally, Neutron only supports static routing, which means that when using NSX with your OpenStack implementation, dynamic routing is not an option. NSX supports both OSPF and BGP, but until Neutron supports either one, tenants won’t be able to use dynamic routing. Efforts to implement a BGP speaker in OpenStack began during the Juno cycle and are still ongoing. When this work is complete, the NSX platform, due to its native support of BGP, will be ready to support dynamic routing once the Neutron plugin has been updated.

The picture below shows the basic topologies supported  by the NSX-Neutron plugin:

DHCP Services

In our implementation of DHCP, we replace the dnsmasq process that is used by the reference implementation with an NSX Edge Services Gateway configured with static DHCP bindings. This approach has proven to be very reliable at scale (thousands of VMs).

There is logic in the NSX-Neutron plugin that will automatically determine how to use an Edge Services Gateway for DHCP services. Depending on the use case (overlapping IPs vs. non-overlapping IPs) the same ESG may be reused for multiple tenant networks, as the picture below shows:

In Part 3 of this article series, we will discuss the implementation of critical Neutron services such as security groups and Load-Balancing-as-a-Service. In the meantime, check out our revamped VMworld Hands-on Lab (HOL-1620) featuring VMware Integrated OpenStack and NSX Optimized for vSphere.

Marcos Hernandez is a Staff Systems Engineer in the Network and Security Business Unit (NSBU). He is responsible for supporting large Global Enterprise accounts and providing technical guidance around VMware’s suite of networking and cloud solutions, including NSX and OpenStack. Marcos has a background in datacenter networking design and expert knowledge in routing and switching technologies. Marcos holds the CCIE (#8283) and VCIX certifications, and he has a Masters Degree in Telecommunications from Universidad Politécnica de Madrid.

Today’s blog post is the start of a series by Marcos Hernandez, one of the OpenStack specialists in VMware’s Networking & Security Business Unit (NSBU). Part 2 and Part 3 will be published in the upcoming weeks. So, check back for more on this topic!

As OpenStack becomes more ubiquitous in the industry, cloud architects are looking for ways to provide enterprise-grade network and security services to their consumers without compromising the primary objectives of an OpenStack-based private cloud, which include:

• Vendor-neutral APIs.
• Infrastructure choice and flexibility.
• A public cloud experience with on-premises infrastructure.

Neutron, the networking project in OpenStack, has come a long way in the last few years, adding powerful capabilities at a very fast pace while enabling rich network workflows and a variety of use cases. According to the latest user survey data, Neutron is favored over Nova networking in ~60% of the production OpenStack deployments, which suggests an increased interest to move off flat (VLAN-based) topologies. There is general consensus in the OpenStack community that a cloud that lacks rich network functionality is a mediocre cloud.

As more features are added to Neutron, its architecture becomes more complex (a universal perception amongst OpenStack users). As of the Kilo release, the Neutron community has wisely decided to “decompose” Neutron. The general idea is that Neutron will remain focused on core L2 and L3 services, while L4-L7 services will be “pluggable” and abide by a well-known extensible data model. For vendors like VMware, this is great news. We now have a reference architecture to develop against, promote our value-add, and expose the unique capabilities of NSX.  We can do this and still honor a consumption model that prioritizes the desirable northbound OpenStack APIs.

With all that said, it is important to note (and know) that Neutron core is developed using a reference implementation based on open source components, including:

• Open vSwitch – hypervisor-level networking.
• dnsmasq – DHCP/DNS services.
• Linux iptables – for security groups.
• L3-agent – routing services.
• HAProxy – load balancing.

In a scaled-out production deployment, the reference implementation typically encounters challenges. As a result, OpenStack operators will either scale back the features deployed in an attempt to gain stability, or they will utilize a software-defined network provider.  This is widely recognized as factual and comes at no surprise once you understand how Neutron core is developed and tested.

The community is doing a great job at defining the core APIs and the extensibility model, while many vendors have also taken it upon themselves to test the scalability, reliability and upgradeability of an OpenStack-based solution using their own “productized” distributions. Often, as it is the case with VMware NSX, vendors will replace the reference open source components with their own technology. OpenStack consumers don’t notice the difference; they still interact with the northbound Neutron APIs by means of plugins and drivers, which “translate” the Neutron API calls into private, southbound calls targeted at the vendor’s infrastructure. In the case of VMware NSX Optimized for vSphere, such interactions look like this:

Neutron services leverage a VMware-developed plugin that makes API calls to NSX Manager, which is the API provider and management plane of NSX.  This Neutron plugin is itself an open source project and can be used with ANY OpenStack implementation (DIY and/or off-the-shelf). VMware offers its own OpenStack distribution, VMware Integrated OpenStack, which natively integrates the NSX-Neutron plugin, in addition to other plugins and drivers that connect OpenStack compute and storage services to vSphere.  Leveraging enterprise-grade server virtualization with vSphere and enterprise-grade networking with NSX, customers will enjoy an enterprise-grade OpenStack infrastructure layer, while leveraging the vSphere skillset and tools (vMotion, host maintenance mode, DRS, etc.) which typically are available in IT groups today.

In this post, we will double-click on the Neutron-NSX interactions and will describe how NSX ultimately brings stability to Neutron. You can also see a recorded version of this content that was presented at the OpenStack Summit in Vancouver.

Basic Neutron Workflows and NSX Equivalents
Neutron consists of a number of basic network workflows that are considered “table stakes”. These include:

• L2 services: Ability for tenants to create and consume their own L2 networks.
• L3 services: Ability for tenants to create and consume their own IP subnets and routers. These routers can connect intra-tenant application tiers and can also connect to the external world via NATed and non-NATed topologies.
• Floating IPs: A “Floating IP” is nothing more than a DNAT rule, living on the Neutron router, that maps a routable IP sitting on the external side of that router (External network) to a private IP on the internal side (Tenant network). This floating IP forwards all ports and protocols to the corresponding private IP of the “instance” (VM) and is typically used in cases where there is IP overlap in tenant space.
• DHCP Services: Ability for tenants to create their own DHCP address scopes.
• Security Groups: Ability for tenants to create their own firewall policies (L3/L4) and apply them directly to an instance or a group of instances.
• Load Balancing as-a-Service (LBaaS): Ability for tenants to create their own load balancing rules, virtual IPs and load balancing pools.

The picture below shows these basic workflows and their situation as it relates to the application, as well as the corresponding NSX element that is leveraged each time. For more information about NSX, please visit the official VMware NSX product page.

In Part 2 of this article series, we will dig deeper into the inner workings of the Neutron plugin implementation for NSX. In the meantime, check out our revamped VMworld Hands-on Lab (HOL-1620) featuring VMware Integrated OpenStack and NSX Optimized for vSphere.

Marcos Hernandez is a Staff Systems Engineer in the Network and Security Business Unit (NSBU). He is responsible for supporting large Global Enterprise accounts and providing technical guidance around VMware’s suite of networking and cloud solutions, including NSX and OpenStack. Marcos has a background in datacenter networking design and expert knowledge in routing and switching technologies. Marcos holds the CCIE (#8283) and VCIX certifications, and he has a Masters Degree in Telecommunications from Universidad Politécnica de Madrid.