Home > Blogs > The Network Virtualization Blog > Tag Archives: VMware NSX

Tag Archives: VMware NSX

VCDX-NV Interview: Chris Wahl

Chris Wahl is a Senior Solutions Architect at Ahead, located in Chicago, Ill.  He has more than 14 years of experience as an IT Pro. Chris originally went to school for networking, and has a bachelor’s degree in networking and communications chris-wahl-redmanagement. More recently he’s been doing sys admin work in sys admin engineering, architecture, and data center focused projects. His certifications include VMware VCDX #104, Cisco CCNA data center and CCNP router and switch certifications for which he also teaches classes, and several other VMware, Cisco, Microsoft, and HP certifications. He is also one of the first VCDX-NV certified professionals

What excites you about network virtualization?

I spent quite a few of years managing every type of virtualized infrastructure you can imagine, ranging from very small and medium sized businesses, to a 16,000 person enterprise with over 1,000 virtual machines. In every instance, the roadblock was always the network to the point where in the large deployment that I managed, we would just plan that any network change would take three weeks even if it was just a VLAN on a port. We could pretty much guarantee that it would be about two weeks to make the change, and another week to fix it because it wouldn’t be made correctly. So, the idea of making the physical infrastructure more like plumbing which we can just make work, and then using network virtualization overlay technology is extremely attractive, because it eliminates days and weeks of real world issues that I have run into as a data center focused engineer and completely hated.

How can networking professionals benefit from network virtualization? Why should they not be concerned this will devalue their skills or make them less important?

In my mind, having gone through this as a sys admin originally focused on Novell and microcomputers and mainframes, and then transitioning to Windows and Active Directory, it’s pretty much the same story all over again. You have a base set of skills and experiences that feed into problem solving, the ability to abstract requirements or constraints out of a design. Then there’s that fundamental understanding of how things should be put together, regardless of the operating system or the network in this case. So as a networking professional, it’s more of the same. You’ve been exposed to a number of network architectures from different vendors and the protocols that go with them. None of that is really changing. It’s just that now there are new ways to make that particular piece of the data center better and faster. I actually view this as an opportunity to increase value, make yourself much more integrated in the workflow for the application or the stack, and really offer some ways to differentiate your business or if you’re a consultant your practice from others that don’t ride on this wagon.

As you’ve gone through network virtualization training, has anything surprised you?

Two things stand out. The first is there’s a cardinal rule you can’t route within the hypervisor. With network virtualization you can finally go beyond just Layer 2 switching and really focus on Layer 2/3 routing and offer dynamic flows to the network within a single hypervisor across hypervisors. That to me is huge.  It really opens up a lot of opportunity to go back to the drawing board on the design.  In the same vein, I feel that the ability to do source-based firewalling is extremely impressive. I was just blown away to the fact that we can apply policy basically ACLs at the source port of the Hypervisor and even prevent the VM from putting traffic on the wire if it doesn’t pass policy. That is extremely powerful. When I work with customers, it’s always been a challenge working around firewalls and how we’re going to logically and physically separate the network into these different segments. Firewalling capabilities from with a network virtualization platform puts the whole design on its head. It lets you step back and really reanalyze how you’re doing design and architecture.

What would you say to someone who said “I don’t need to learn about network virtualization?

Well in reality, they may just be bogged down spending 80% of the time keeping the lights on, and I can understand the personal investment that it takes to work on these skills outside of work. A lot of us don’t get the opportunity or the support we would like from our employer to really stretch our legs on these new technologies at work. In addition, some IT shops probably just don’t care. They’re just going to say, we’re not interested in this. I need you to continue being a router or switch jockey because that’s what I hired you to do. My advice would be that’s total nearsightedness; that’s only looking at today’s wants and needs. Network virtualization, it’s a huge game changer. The companies that embrace it are going to be infinitely more dynamic and scalable and able to complete at a whole different level. Therefore network virtualization is going to happen, and getting on the train right now is better than standing in front of the train because you’re going to get hit by it. I would say get on it now while there’s a lot of opportunity to learn and really understand while things are so new. That way when your company says, “Man, we’d really like to do something with network virtualization,” or another opportunity comes up at a different company, you can jump right on it and land with both feet firmly on the ground and start running.

Anything else that you think someone should know?

I would recommend that IT pros not focus too much on the individual technologies, or all of the hype between this vendor and that vendor. I think it’s important for everyone to take a breath, take a step back, look at the ecosystem, look at the open source products that are coming out, look at the vendor products that are coming out and really understand the differences and the similarities. Don’t ask “which product?” Ask “what would benefit my design” and then pick a starting point. Because if you look at SDN and network virtualization, and try to learn everything at once, it’s overwhelming and you’re going to feel like there’s just no way you can learn all of this. But if you pick a starting point of one project or one particular way to implement it, and use that as a landing point to gain education around the technology, it’s going to be a lot easier.

Deploying VMware NSX with Horizon

As part of the recent launch of Horizon 6, Tony Paikeday, senior product line manager, End-User Computing, VMware, takes a look at the value proposition of deploying the VMware NSX network virtualization platform together with Horzon.

VMware NSX

Deploying VMware NSX with Horizon

VMware NSX, deployed with Horizon, offers a better alternative to securing east-west traffic between VMs, turning data center security from a perimeter-centric view to one that gives each individual desktop VM its own virtual network container – creating if you will, a network of “one.” This approach, also known as micro-segmentation, has been an ideal for network teams, but traditionally unachievable due to the cost, and the operational complexity involved. With the number of user VM’s introduced by desktop virtualization, and the sprawl of firewall rules needing to be manually added, deleted or modified every time a new VM is introduced, this has been untenable in the past. With VMware NSX, we have a completely new model for networking and security, delivering virtualization of the network, much as we did for server virtualization – reproducing it in software, with a logical library of networking elements and services including switches, routers, firewalls, load-balancers and more that can be deployed over any existing network.

Read Tony’s full blog post here at http://blogs.vmware.com/euc/2015/03/securing-virtual-desktops-east-west-threats-data-center.html

Roger

Introducing New VCE VxBlock Systems with Integrated VMware NSX

Last month, we outlined VMware’s vision for helping customers achieve one cloud for any application and any device. We believe the prevailing model for cloud adoption will be the hybrid cloud, and the best architecture for achieving the hybrid cloud is through a software-defined data center architecture. The fastest path to building reliable infrastructure for the hybrid cloud is through the use of converged infrastructure systems, and no company has been more successful at delivering on the promise of converged infrastructure than our partner VCE.

Now, the ability to procure and deploy the VMware NSX network virtualization platform with VCE converged infrastructure is about to get whole lot easier.

Today, VCE launched VCE VxBlock Systems, a new family of converged infrastructure systems that will factory-integrate VMware NSX for software-defined data center deployments. The new VxBlock Systems will include VCE pre-integration, pre-testing and pre-validation of VMware NSX, with seamless component-level updates, ongoing lifecycle assurance, and unified single-call support from VCE.

As I wrote previously, VMware NSX already runs great on existing Vblock Systems. Customers today are deploying VMware NSX with their existing Vblocks, and customers will be able to extend VMware NSX environments across their entire VCE converged infrastructure environment as they move to the new VxBlock Systems.

This solution will be a powerful building block for the software-defined data center, delivering unparalleled IT agility through automation, and unparalleled security through micro-segmentation.

Agility through IT Automation

  • Reduce time to provision multi-tier networking and security services from weeks to minutes.
  • Achieve faster development, testing and deployment of new applications by aligning network and security provisioning with compute and storage provisioning.
  • Streamline IT operations through programmatic creation, provisioning, snapshotting, deleting and restoration of complex software-based networks.
  • Build advanced workflows through cloud management platforms to automate provisioning of networking and security, including switching, routing, firewalling, and load balancing without manually reconfiguring physical network devices.

Unparalleled Security

  • Use micro-segmentation and isolation capabilities of VMware NSX to build security directly into the data center infrastructure.
  • Insert advanced partner services from leading security vendors to improve threat protection, reduce risk and help address their compliance requirements.
  • Achieve better security inside the data center through fine-grained policies that enable firewall controls and advanced security down to the level of the virtual NIC.
  • Create dynamic security policies that are automatically applied when a virtual machine spins up, are moved when a virtual machine is migrated and are removed when a virtual machine is de-provisioned

VMware NSX is the ideal platform for virtualizing the network running on top of VCE converged infrastructure.

Hatem

VMware NSX Webcast – Creating Agile Networks

You may have seen Joey Logano speed to his first Daytona 500 win this week. Keeping your network in racing shape takes a similar level of NSX: Wanna Go Fastpatience, stamina, and quick reflexes. Using VMware NSX network virtualization means that you can unlock the full potential of a Software-Defined Data Center, to create and run entire networks on top of existing network hardware, resulting in faster deployment of workloads, as well as greater agility in the face of increasingly dynamic data centers. Watch this overview to learn how VMware NSX reduces the time to provision multi-tier networking and security services from weeks to seconds to win your race. This one-hour overview of VMware NSX outlines how you can bring virtualization to your existing network, transforming both its operations and economics. You’ll learn how several of the largest service providers, global financial, and enterprise data centers in the world are using NSX to reduce costs and provisioning times to improve agility and establish a new model of network security.

Click here to watch this webcast and find out:

  • What the NSX architecture looks like
  • How switching, routing, firewalling, load-balancing and other services are managed with NSX
  • How overlay networks and logical networks all come together with your physical infrastructure

Roger

Network Virtualization at VMware Partner Exchange 2015

VMware Partner Exchange (PEX) is your one-stop shop when it comes to learning about network virtualization and the technology extends VMware’s vision of the software-defined data center. At this year’s event, we are offering both an executive track and a technical track to help partners build their businesses and advance their knowledge, as you take customers on the path to Virtualizing the Network.

PEX Image

If you are a partner that is new to network virtualization, we have a program/learning path where you can send two people to PEX and to achieve their network virtualization competency by attending the 3-Day NSX Install, Configure and Manage Boot Camp prior to the start of the conference. Participants can then attend the free instructor-led VSP-NV and VTSP-NV boot camps during the conference. Continue reading

VMware NSX Loves Hardware

One of the core value propositions of VMware NSX is ability to take advantage of any underlying hardware infrastructure and deliver a fully decoupled virtualized network in software. VMware NSX loves a Modern Infrastructuregood hardware fabric,.

But that’s not the only hardware VMware NSX loves.

The votes have been cast and counted, and we are pleased to announce that VMware NSX was selected as the winner in the “Best Software Defined Infrastructure” category in the 2015 Modern Infrastructure Impact Awards. The awards were judged by the Modern Infrastructure e-zine editorial staff, in conjunction with users, readers, and industry experts.

The Modern Infrastructure Impact Awards recognize the top products, technologies and services in the essential areas of technology that Modern Infrastructure covers. The award-winning tools are those helping to run enterprise businesses with efficiency and insight — whether they’re used inside the data center or out.

VMware NSX delivers secure network services to applications running in the data center, resulting in instant and programmatic provisioning, fast and highly available infrastructure, and increased security and micro segmentation capabilities.

Read about the award here and to learn more about the business value of VMware NSX visit  vmware.com/products/nsx.

Roger

3 Ways To Get Started With VMware NSX

Over the past 12 months, VMware NSX momentum has continued to grow, as we’ve added VMware NSXnew platform capabilities, expanded our partner ecosystem, and of course, had more than 250+ customers purchase NSX for deployment. And as interest in VMware NSX has grown with both customers and IT professionals looking to evolve their careers by adding certification in network virtualization, one of the most common questions that we get is “How can I get started with NSX?.”

We understand that there is a strong demand for individuals and organizations to get their hands on the NSX technology. Many of you are working towards your initial VCP-NV certification. Others of you are exploring NSX as a way to improve your organization’s agility and security while reducing overall costs.

Here are three ways individuals and companies can get started with NSX. Continue reading

Talking Tech Series: VMware NSX Edge Scale Out with Equal-Cost Multi-Path Routing

This post was written by Roie Ben Haim and Max Ardica, with a special thanks to Jerome Catrouillet, Michael Haines, Tiran Efrat and Ofir Nissim for their valuable input.

****

The modern data center design is changing, following a shift in the habits of consumers using mobile devices, the number of new applications that appear every day and the rate of end-user browsing which has grown exponentially. Planning a new data center requires meeting certain fundamental design guidelines. The principal goals in data center design are: Scalability, Redundancy and High-bandwidth.

In this blog we will describe the Equal Cost Multi-Path functionality (ECMP) introduced in VMware NSX release 6.1 and discuss how it addresses the requirements of scalability, redundancy and high bandwidth. ECMP has the potential to offer substantial increases in bandwidth by load-balancing traffic over multiple paths as well as providing fault tolerance for failed paths. This is a feature which is available on physical networks but we are now introducing this capability for virtual networking as well. ECMP uses a dynamic routing protocol to learn the next-hop towards a final destination and to converge in case of failures. For a great demo of how this works, you can start by watching this video, which walks you through these capabilities in VMware NSX.

Scalability and Redundancy and ECMP

To keep pace with the growing demand for bandwidth, the data center must meet scale out requirements, which provide the capability for a business or technology to accept increased volume without redesign of the overall infrastructure. The ultimate goal is avoiding the “rip and replace” of the existing physical infrastructure in order to keep up with the growing demands of the applications. Data centers running business critical applications need to achieve near 100 percent uptime. In order to achieve this goal, we need the ability to quickly recover from failures affecting the main core components. Recovery from catastrophic events needs to be transparent to end user experiences.

ECMP with VMware NSX 6.1 allows you to use upto a maximum of 8 ECMP Paths simultaneously. In a specific VMware NSX deployment, those scalability and resilience improvements are applied to the “on-ramp/off-ramp” routing function offered by the Edge Services Gateway (ESG) functional component, which allows communication between the logical networks and the external physical infrastructure.

ECMP Image 1

External user’s traffic arriving from the physical core routers can use up to 8 different paths (E1-E8) to reach the virtual servers (Web, App, DB).

In the same way, traffic returning from the virtual server’s hit the Distributed Logical Router (DLR), which can choose up to 8 different paths to get to the core network.

How is the path determined:

NSX for vSphere Edge Services Gateway device:

When a traffic flow needs to be routed, the round robin algorithm is used to pick up one of the links as the path for all traffic of this flow. The algorithm ensures to keep in order all the packets related to this flow by sending them through the same path. Once the next-hop is selected for a particular Source IP and Destination IP pair, the route cache stores this. Once a path has been chosen, all packets related to this flow will follow the same path.

There is a default IPv4 route cache timeout, which is 300 seconds. If an entry is inactive for this period of time, it is then eligible to be removed from route cache. Note that these settings can be tuned for your environment.

Distributed Logical Router (DLR):

The DLR will choose a path based on a Hashing algorithm of Source IP and Destination IP.

What happens in case of a failure on one of Edge Devices?

In order to work with ECMP the requirement is to use a dynamic routing protocol: OSPF or BGP. If we take OSPF for example, the main factor influencing the traffic outage experience is the tuning of the OSPF timers.

OSPF will send hello messages between neighbors, the OSPF “Hello” protocol is used and determines the Interval as to how often an OSPF Hello is sent.

Another OSPF timer called “Dead” Interval is used, which is how long to wait before we consider an OSPF neighbor as “down”. The OSPF Dead Interval is the main factor that influences the convergence time. Dead Interval is usually 4 times the Hello Interval but the OSPF (and BGP) timers can be set as low as 1 second (for Hello interval) and 3 seconds (for Dead interval) to speed up the traffic recovery.

ECMP Image 2

In the example above, the E1 NSX Edge has a failure; the physical routers and DLR detect E1 as Dead at the expiration of the Dead timer and remove their OSPF neighborship with him. As a consequence, the DLR and the physical router remove the routing table entries that originally pointed to the specific next-hop IP address of the failed ESG.

As a result, all corresponding flows on the affected path are re-hashed through the remaining active units. It’s important to emphasize that network traffic that was forwarded across the non-affected paths remains unaffected.

Troubleshooting and visibility

With ECMP it’s important to have introspection and visibility tools in order to troubleshoot optional point of failure. Let’s look at the following topology.

ECMP Image 3

A user outside our Data Center would like to access the Web Server service inside the Data Center. The user IP address is 192.168.100.86 and the web server IP address is 172.16.10.10.

This User traffic will hit the Physical Router (R1), which has established OSPF adjacencies with E1 and E2 (the Edge devices). As a result R1 will learn how to get to the Web server from both E1 and E2 and will get two different active paths towards 172.16.10.10. R1 will pick one of the paths to forward the traffic to reach the Web server and will advertise the user network subnet 192.168.100.0/24 to both E1 and E2 with OSPF.

E1 and E2 are NSX for vSphere Edge devices that also establish OSPF adjacencies with the DLR. E1 and E2 will learn how to get to the Web server via OSPF control plane communication with the DLR.

From the DLR perspective, it acts as a default gateway for the Web server. This DLR will form an OSPF adjacency with E1 and E2 and have 2 different OSPF routes to reach the user network.

From the DLR we can verify OSPF adjacency with E1, E2.

We can use the command: “show ip ospf neighbor”

 

ECMP Image 4

 

From this output we can see that the DLR has two Edge neighbors: 198.168.100.3 and 192.168.100.10.The next step will be to verify that ECMP is actually working.

We can use the command: “show ip route”

 

ECMP Image 5

 

The output from this command shows that the DLR learned the user network 192.168.100.0/24 via two different paths, one via E1 = 192.168.10.1 and the other via E2 = 192.168.10.10.

Now we want to display all the packets which were captured by an NSX for vSphere Edge interface.

In the example below and in order to display the traffic passing through interface vNic_1, and which is not OSPF protocol control packets, we need to type this command: “debug packet display interface vNic_1 not_ip_proto_ospf”

We can see an example with a ping running from host 192.168.100.86 to host 172.16.10.10

 

ECMP Image 6

 

If we would like to display the captured traffic to a specific ip address 172.16.10.10, the command capture would look like: “debug packet display interface vNic_1 dst_172.16.10.10”

 

ECMP Image 7

 

Useful CLI for Debugging ECMP

  • To check which ECMP path is chosen for a flow
    • debug packet display interface IFNAME
  • To check the ECMP configuration
    • show configuration routing-global
  • To check the routing table
    • show ip route
  • To check the forwarding table
    • show ip forwarding

Useful CLI for Dynamic Routing

  • show ip ospf neighbor
  • show ip ospf database
  • show ip ospf interface
  • show ip bgp neighbors
  • show ip bgp

ECMP Deployment Consideration

ECMP currently implies stateless behavior. This means that there is no support for stateful services such as the Firewall, Load Balancing or NAT on the NSX Edge Services Gateway. The Edge Firewall gets automatically disabled on ESG when ECMP is enabled. In the current NSX 6.1 release, the Edge Firewall and ECMP cannot be turned on at the same time on NSX edge device. Note however, that the Distributed Firewall (DFW) is unaffected by this.

For more in-depth information, you can also read our VMware® NSX for vSphere (NSX-V) Network Virtualization Design Guide

About The Authors

VMware-Roie Ben HaimRoie Ben Haim works as a professional services consultant at VMware, focusing on design and implementation of VMware’s software-defined data center products.  Roie has more than 12 years in data center architecture, with a focus on network and security solutions for global enterprises. An enthusiastic M.Sc. graduate, Roie holds a wide range of industry leading certifications including Cisco CCIE x2 # 22755 (Data Center, CCIE Security), Juniper Networks JNCIE – Service Provider #849, and VMware vExpert 2014, VCP-NV, VCP-DCV. Follow his personal blog at http://roie9876.wordpress.com/

VMware-Max ArdicaMax Ardica is a senior technical product manager in VMware’s networking and security business unit (NSBU). Certified as VCDX #171, his primary task is helping to drive the evolution of the VMware NSX platform, building the VMware NSX architecture and providing validated design guidance for the software-defined data center, specifically focusing on network virtualization. Prior to joining VMware, Max worked for almost 15 years at Cisco, covering different roles, from software development to product management. Max owns also a CCIE certification (#13808).

New VMworld 2014 Hands-on Labs with VMware NSX Goodness

In 2013 we introduced VMware NSX Hands-on-Labs for the first time. The NSX 1303 Hands-on-lab has been by far one of the most popular labs, giving you an in-depth view of VMware NSX. Hands-on-labs are one of the best ways to get a good tour of the product. You can take all of these labs online at http://labs.hol.vmware.com/HOL/catalogs/ . It requires a registration, but is open to everyone. .

This year at VMworld we introduced several new NSX labs to give you a deeper look at NSX, and to showcase the depth of integration NSX provides with 3rd party partners and other VMware products. All of the new 2014 Hands-on-labs have been published and are available to you. Here is a quick tour of the labs and what you can expect to see.

 

HOL-SDC-1403

If you are just getting started with NSX and want to know what Network Virtualization is all about, we recommend you start here.

HOL-SDC-1403-2nd Image

This lab will walk you through five modules of exercises:

  • NSX Components – Host Preparation, Controller deployment
  • NSX Logical Switching – building VXLAN logical switches
  • NSX Logical Routing  - Distributed Routing, Dynamic Routing with OSPF
  • NSX Distributed Firewall – Micro-segmentation with NSX
  • NSX Edge Services – Load-balancing, SSL VPN

 

HOL-SDC-1425

Once you have completed the introductory lab, we recommend taking the advanced lab which is designed to showcase some of the new features in NSX 6.1. You can read and excellent summary of these new capabilities in Chris Wahl’s blog, “NSX 6.1 Announced, Contains Plethora of Enhancements.”

This lab covers the following areas:

  • Configuring DHCP Relay so that you can use NSX with external IPAM Services
  • Scaling out Layer 3 routing with Equal Cost Multi-Pathing (ECMP) and Dynamic Routing Protocols. Yes we actually build out the topology below in the lab! That’s the power of network virtualization.

HOL-SDC-1425-2nd Image

  • Building out L2VPN services for multi-site and hybrid cloud connectivity services
  • Integration with 3rd parties using Service Composer and Trend Micro AV & IPS with NSX. You will see how to register services and how NSX is a platform to integrate with 3rd party services in this exercise.
  • Networking Monitoring with NSX & Riverbed Cascade – we will even show you how you can monitor with NetFlow in this exercise 

HOL-SDC-1424

The two labs above will surely give you a good view of NSX as a network virtualization platform. Next, let’s see how NSX integrates with other VMware products to build out a complete Software-Defined Data Center. This lab shows the integration capabilities offered by NSX with VMware management solutions.

First up, we will learn about Self-service IT, and how you can deliver applications quickly to your end-users with the integration of vCloud Automation Center and NSX. You will build out a multi-machine blueprint with networking and security, and then deploy it.

Next, if you want to learn about automation and the NSX API, we will walk you through exercises in using vCenter Orchestrator and using the NSX REST API to create a security group. This will give you the fundamentals of NSX automation which you can easily extend upon as you deploy NSX in your own environment.

The third exercise is about operations. We will show you the new NSX Management Pack in vCenter Operations. We will walk you through the dashboards and you will learn how you can actually not just monitor but also troubleshoot you network.

At this point you are surely on your way to become a NSX Ninja

HOL-SDC-1420

If you want to use OpenStack with NSX and vSphere – we’ve got you covered too! We will walk you through OpenStack on vSphere itself and then show you how to connect it to deploy networks with NSX from OpenStack.

And Of Course There Are More

Those are the main labs I would recommend, but there are others too. There’s a lab where you can learn more about the IT Outcome of Fast Infrastructure Delivery and Application Automation (HOL-SDC-1413) which has some NSX goodness with vCloud Automation Center, or learn about the IT Outcome of Policy-based Compliance and Network Security (HOL-SDC-1414).

If you want to learn about NSX and the partner integration framework you can take HOL-PRT-1462 which will walk you through the NSX and Palo Alto Networks next-gen firewall integration labs and HOL-PRT-1464 which is focused on how you can use NSX Service Composer and Symantec Data Center Security: Server.

In all we have well over 24 hours of labs, and you can sign-up even if you did not go to VMworld. It is always available 24/7, so if you have a few spare hours and want to learn about NSX you can take the lab.

And I will let you in on a little secret. We actually run the labs on NSX. So as you learn, you are also a user of NSX!!!

You can always sign-up for a NSX class offered by VMware Education.

Happy learning!

Nikhil

Automating a Multi-Action Security Workflow with VMware NSX

This post was written by VMware’s John Dias, (VCP-DCV), Sr. Systems Engineer, Cloud Management Solutions Engineering Team, and Hadar Freehling, Security & Compliance Systems Engineer Specialist

***

Through a joint effort with Hadar Freehling, one of my esteemed peers here at VMware, we co-developed a proof-of-concept workflow for a network security use case.  Hadar created a short video showing and explaining the use case, but in summary this is a workflow that reacts to and remediates a security issue flagged by third-party integration with VMware NSX. In the video, TrendMicro is used but it could be any other partner integration with vShield Endpoint.

Here’s what happens:

  • A virus is detected on a VM and is quarantined by the AV solution
  • The AV solution tags the VM with an NSX security tag
  • VMware NSX places the VM in a new Security Group, whose network policies steer all VM traffic through an intrusion prevention system (IPS)
  • vCenter Orchestrator (vCO) monitors the security group for changes and when a VM is added
    • a snapshot of the VM is taken for forensic purposes
    • a vSpan session (RSPAN) is set up on the Distributed Virtual Switch to begin capturing inbound/outbound traffic on the VM
    • once the VM has been removed from the security group, the vSpan session is removed

Watch the video below for a walk-through by Hadar:

You will note that there is a portion of the workflow that is handled natively by VMware NSX (Security Tag reaction, Security Group policy) but the snapshot and RSPAN are done via vCO workflow.

If you are interested in exploring this capability, I have provided the vCO workflow package for download. This is provided as-is and you should fully test it (and modify as needed) before using in your environment.

Assuming you have VMware NSX, vShield Endpoint and some third party integration already set up, you will need the following:

  • vCO 5.5.2
  • The NSX plugin for vCO (installed and configured)
  • The REST plugin with your NSX manager added as a REST host
  • vCenter plugin configured

The workflow package includes a good number of “helper” workflows which you will not need to run directly. The master workflow is in the root folder Security Reaction and is named “Set up VM Forensics RUN THIS” (just in case you had any doubt as to which one to run).

Multi-Site Security

The Security Reaction Master Workflow

Running the master workflow will prompt you for three items:

  • The NSX Security Group to monitor – This is why the NSX plugin is required, so that you can browse the vCO managed objects and locate the desired Security Group.
  • A time to sleep in seconds – The master workflow will run continuously until manually stopped and will use a REST call to NSX to get the current membership for the Security Group.  We have no recommendation on this poll time, although in testing we used 5-10 seconds.  It would have been better to use some external event to kick off the vCO workflow but we could not find a way to do this from NSX.  It may be possible to do via the partner solution, but we wanted this workflow package to be “partner neutral.”
  • Destination IPv4 address – This is the destination for the RSPAN (or vSpan session in vSphere API terms).  The vSpan session is created with some defaults (for example sampling rate, normal traffic allowed, etc).  If you want to change any of those properties, you will need to modify the Helper workflow named “Configure encapRemoteMirrorSource vSpan Session on DVS” (modify the “Create Port Mirror” script task).

Also note that this workflow doesn’t support VMs with multiple vNICs. Specifically, it will only create an RSPAN that includes the first vNIC found on a VM.  You can modify the Helper workflow “Implement Forensics” and adjust the script task “Prep for Mirror Creation” so that the additional NICs (if any) are added to the sourcePorts array. It’s something we intended to fix but forgot about until after our final testing and video production – so as they say in the textbooks “this is left as an exercise for the reader.”

Of course, there are many other actions that can be taken besides setting up an RSPAN and getting a snapshot. This solution can be extended to practically any task required during such an event such as creating a ticket in your service desk software, spinning up additional workloads to replace the compromised VM, sending emails, guest OS file system operations…all of these and more can be accomplished using vCO in conjunction with NSX.