This post was written by VMware’s John Dias, (VCP-DCV), Sr. Systems Engineer, Cloud Management Solutions Engineering Team, and Hadar Freehling, Security & Compliance Systems Engineer Specialist

***

Through a joint effort with Hadar Freehling, one of my esteemed peers here at VMware, we co-developed a proof-of-concept workflow for a network security use case.  Hadar created a short video showing and explaining the use case, but in summary this is a workflow that reacts to and remediates a security issue flagged by third-party integration with VMware NSX. In the video, TrendMicro is used but it could be any other partner integration with vShield Endpoint.

Here’s what happens:

  • A virus is detected on a VM and is quarantined by the AV solution
  • The AV solution tags the VM with an NSX security tag
  • VMware NSX places the VM in a new Security Group, whose network policies steer all VM traffic through an intrusion prevention system (IPS)
  • vCenter Orchestrator (vCO) monitors the security group for changes and when a VM is added
    • a snapshot of the VM is taken for forensic purposes
    • a vSpan session (RSPAN) is set up on the Distributed Virtual Switch to begin capturing inbound/outbound traffic on the VM
    • once the VM has been removed from the security group, the vSpan session is removed

Watch the video below for a walk-through by Hadar:

You will note that there is a portion of the workflow that is handled natively by VMware NSX (Security Tag reaction, Security Group policy) but the snapshot and RSPAN are done via vCO workflow.

If you are interested in exploring this capability, I have provided the vCO workflow package for download. This is provided as-is and you should fully test it (and modify as needed) before using in your environment.

Assuming you have VMware NSX, vShield Endpoint and some third party integration already set up, you will need the following:

  • vCO 5.5.2
  • The NSX plugin for vCO (installed and configured)
  • The REST plugin with your NSX manager added as a REST host
  • vCenter plugin configured

The workflow package includes a good number of “helper” workflows which you will not need to run directly. The master workflow is in the root folder Security Reaction and is named “Set up VM Forensics RUN THIS” (just in case you had any doubt as to which one to run).

Multi-Site Security

The Security Reaction Master Workflow

Running the master workflow will prompt you for three items:

  • The NSX Security Group to monitor – This is why the NSX plugin is required, so that you can browse the vCO managed objects and locate the desired Security Group.
  • A time to sleep in seconds – The master workflow will run continuously until manually stopped and will use a REST call to NSX to get the current membership for the Security Group.  We have no recommendation on this poll time, although in testing we used 5-10 seconds.  It would have been better to use some external event to kick off the vCO workflow but we could not find a way to do this from NSX.  It may be possible to do via the partner solution, but we wanted this workflow package to be “partner neutral.”
  • Destination IPv4 address – This is the destination for the RSPAN (or vSpan session in vSphere API terms).  The vSpan session is created with some defaults (for example sampling rate, normal traffic allowed, etc).  If you want to change any of those properties, you will need to modify the Helper workflow named “Configure encapRemoteMirrorSource vSpan Session on DVS” (modify the “Create Port Mirror” script task).

Also note that this workflow doesn’t support VMs with multiple vNICs. Specifically, it will only create an RSPAN that includes the first vNIC found on a VM.  You can modify the Helper workflow “Implement Forensics” and adjust the script task “Prep for Mirror Creation” so that the additional NICs (if any) are added to the sourcePorts array. It’s something we intended to fix but forgot about until after our final testing and video production – so as they say in the textbooks “this is left as an exercise for the reader.”

Of course, there are many other actions that can be taken besides setting up an RSPAN and getting a snapshot. This solution can be extended to practically any task required during such an event such as creating a ticket in your service desk software, spinning up additional workloads to replace the compromised VM, sending emails, guest OS file system operations…all of these and more can be accomplished using vCO in conjunction with NSX.