Welcome to part 6 of the Micro-segmentation Defined– NSX Securing “Anywhere”  blog series. Previous topics covered in this series include

• Part I –    Micro-segmentation Defined
• Part II –  Securing Physical Environments
• Part III – Operationalizing Micro-segmentation
• Part IV – Service Insertion
• Part V – Context, Visibility, and Containment

Previous posts set the stage by introducing and defining the characteristics of micro-segmentation; showing how it has utility in the modern data center; how we might apply it to our existing software-defined and physical networks; how policy-driven NSX management may be used to deliver comprehensive security; and, that we can use physical and virtual third-party security appliances in conjunction with NSX to create a service chain and apply special processing to our vital network flows.

In this sixth part of the NSX Securing “Anywhere” blog, Chris Krueger of Coalfire Systems will preview some of our work in comprehensively benchmarking VMware NSX micro-segmentation. The Micro-segmentation Benchmark is a project being delivered by Coalfire Systems, Inc. an internationally recognized third party audit organization (3PAO) and leading provider of IT advisory services for security in retail, payments, healthcare, financial services, higher education, hospitality, government,and utilities. Coalfire has provided VMware independent security validation of much of the VMware product line against regulatory compliance objectives such as HIPAA, PCI DSS, FedRAMP, FISMA, NERC CIP, CJIS, etc. through the VMware Reference Architecture Framework series of papers available on VMware Partner Exchange.

The VMware NSX Micro-segmentation Benchmark is an industry first and we hope it encourages scientific review of security products from all vendors. This post presents a preview of the upcoming “Coalfire Research and Opinion Series” paper titled, VMware NSX Micro-segmentation Benchmark – A Micro-audit of NSX Threat Mitigation Effectiveness. If attending VMworld, be sure to check out session SEC10019  and Group Discussion NET10712-GD, where we will dive further into the NSX Micro-segmentation Benchmark and findings.

Guest Co-Author – Chris Krueger, Principal, Cloud and Virtualization, Coalfire Systems, Inc., Coalfire Labs

Objectives of the NSX Micro-audit

The objectives of the NSX micro-audit are to take real-world examples of likely network topologies (network design patterns) and test them against actual threat scenarios taken from the “playbook” of actual “hackers” and penetration testers. Using a reference NSX installation constructed on a multi-cluster VMware ESXi 6.0 test-bed, we wanted to determine the following:

  • Does VMware NSX functionally satisfy NIST SP 800-125B recommendations VM-FW-R1, VM-FW-R2, VM-FW-R3 and VM-FW-R4?
  • Are the precepts of micro-segmentation, as defined in the complete definition, satisfied conceptually and in testing, by NSX?
  • Can real-world threats be stopped by NSX in E-W (peer transits on the L2 network) and N-S (network to network transits via L3), using industry-standard Penetration Testing tools?

In this blog, we will focus on the “heart” of the micro-segmentation benchmark, the determination of NSX’s capacity to stop real-word threats, specifically in the E-W (L2) transit direction. In the complete paper, multiple network design patterns, depicting five network scenarios are explored; and, the NIST SP 800-125B and micro-segmentation complete definition topics are addressed.
Owing to the brevity of this blog, our sampling will concentrate on Design Pattern 1, the Flat Network Segment with Physical Router scenario.

Threat Simulation Methodology

Our examination and testing of the NSX technology is based on simulated exploits that depict likely malware and virus behavior in actual production network scenarios. Our testing uses the Rapid 7 free-edition of MetaSploit, running on a Kali Linux VM. This Kali Linux VM performs the function of an exploited machine, being used as a vector to attack other machines on the network(s).

Our methodology encompasses several traditional aspects of actual attack techniques used by both autonomous threats, and human-coordinated exploits. Brief review of the following cyber kill chain diagram will help illustrate our threat simulation methodology:

Picture1

Our threat simulation focuses on an abbreviated attack scenario based the Reconnaissance and Exploitation stages of the kill chain. Specifically, we:

  • Recon via use of the “db_nmap –v –A {IP Range}” command on the Kali Linux MetaSploit console
  • Presume Weaponization and Delivery, with the particular MetaSploit exploit scenario (see below) chosen with knowledge of its lethality on the target machine(s)
  • Invoke Exploitation by running the MetaSploit attack and observing the results via the msconsole. Successful exploitation is evident by MetaSploit dropping into the Meterpreter console (SMB and Magento) or other indication of delivery of lethal payload, in the case of the Java ARA
  • Abort our threat simulation with an expectation that subsequent Installation, Command and Control and Actions on Target events would follow an actual Exploitation

Attack via MetaSploit Toolkit

In our particular exploitation scenarios, we are instigating the events through manual use of the MetaSploit console, following these basic steps:

PREPARATION / Recon (2 steps)

  • DB_NMAP Scanner – All targets – Our MetaSploit console running on the Kali Linux require a set of target hosts in it’s database. Pentesters typically use NMAP either externally or the DB_NMAP to generate the list of target IPs to prepare for MetaSploit exploits and use of other utilities. We ran DB_NMAP –v –A before each exploit. Results from the DB_NMAP command generally create an “early revision” of hosts detail, which has a number of inaccuracies in the table, with approximation of OS version, and other details, which a tester typically corrects by loading and running the auxiliary tool SMB_VERSION.Picture2
  • Auxiliary Tool SMB_VERSION – All Targets — was used to further refine the version, language and option details on all targets. This is a customary step in many PenTests to correct inaccuracies in machine OS typing, versions, etc. Our process run auxiliary/scanner/smb/smb_version before each exploit.Picture3

EXPLOITS / Weaponization (one or several used)

We use the following exploits, which are listed here with reference information that is used to locate them in various threat databases, or is commonly the “handle” for the particular weapon.

  • Classic Worm Exploit – SMB MS08-067 Remote Code Execution
    Targeting Windows Servers and XP workstations and possibly the most (in-)famous exploit of all time for Windows machines, this vulnerability in an SMB Server Service allows remote code execution on the target machine, with full administrator rights. Once exploited, the target machine becomes completely available for hijacking and total domination. In our SMB exploit, our Kali Linux machine acts as the infected PC, and it launches the exploit at other machines in the Network pattern.

Reference to this exploit is found at: https://www.rapid7.com/db/vulnerabilities/WINDOWS-HOTFIX-MS08-067Picture4

  • Browser-based Java Atomic Reference Array – CVE-2012-0507 JRE Sandbox Escape
    Lethal to Windows, Linux, Appliances, etc. and one of the most recognized and pervasive exposures found in 2012, this pervasive exploit leverages an unsafe class of Java variable storage to inject malicious code which may cause the Java SE/RE to run privileged code on whatever platform might have browsed the deadly source. Our Java attack creates the deadly payload on the Kali Linux machine and makes it available to all vulnerable machines via either browser or machine automated access to the “URL of death.” This URL contains the poisonous Java JAR, which is then consumed by the client containing the vulnerable JSE/JRE and this code escapes the JRE sandbox to do its’ business.Picture5 When the target machine (10.0.2.83 in this case) browses to the provided web server, the MetaSploit console confirms delivery of the lethal JAR, by the confirming message “Sending Java AtomicReferenceArray Type Violation Vulnerability” and a generated JAR messagePicture6

The MetaSploit references for this exploit are: https://www.rapid7.com/db/modules/exploit/multi/browser/java_atomicreferencearray

  • Service-based Magento PHP Unserialize – CVE 2016-4010 Remote Code Execution
    Windows servers and workstations that are hosting the Magento storefront application are targeted in this prime example of an exploit being created on an otherwise secure Windows server/workstation by installation of an application with a vulnerability in it. The Magento_unserialize Remote Code Execution exploit takes advantage of insecure PHP object injection and subsequent execution of that object as a trusted process. Frequent use of Windows administrator group credentials for this service install, give the intruder a platform rapid compromise of the target network.Picture7

The threat profile and links for this exploit are here: https://www.rapid7.com/db/modules/exploit/multi/http/magento_unserialize

Real-world Scenario that this Threat Simulation Mimics

These exploits are representative of likely E-W threat traffic, arising from three distinct attack profiles that follow sequentially a phishing/spear-fishing attack that has successfully compromised a system on a trusted network behind a perimeter firewall. Once compromised by a successful attack our “Exploited System” (the Kali Linux MetaSploit box, with other attack tools) then participates in one of these attack profiles:

  • Launches a “worm exploit” and acts as the command relay for the SMB_RCE attack
  • Provides the lethal JAR payload via an Apache server on port 8080 for the JavaARA attack
  • Launches the PHP vulnerability and acts as the command relay for the Magento attack

In addition, the Kali Linux box could be expected to continue Recon operations, be used to run Denial of Service, Man in the Middle, and continue to exploit the target network(s), just as an actual attacker would.

Example of Validation Exercises and Findings from Pattern 1a and 1b Testing

The process of validation consisted of running the aforementioned Kali Linux VM-based reconnaissance (Recon) and MetaSploit exploits (Exploit) against the listed servers, in the Flat Network with Physical Router pattern. This test consisted of a control “a”, where no NSX protections were employed and an NSX protected “b” variation.
Our validation exercises and the results are illustrated with supporting screen-shots of NSX consoles and of the actual VMs under test. Some test results had nuanced outcomes, which when listed in the table, appear with a yellow color-code and a reference to more information which appears in the text below the table.
Our tables use these color-codes with the meanings described:

KEYPicture8

Unimpeded – No action of NSX on network flow
NSX Controlled – NSX Firewalled network flows
“ “ w/ caveat – Modified network flows by NSX

Kill Chain designation matches chart on right

Design Patterns 1a and 1b – Flat Network with Physical Router

In the simplest configuration, our single-segment flat network topology, we observed this behavior:

Picture10 Picture9

 

 

 

NSX distributed firewall (DFW) functionality was constructed to protect the E-W traffic in this environment, yet left in the “Allow” action state, for the control scenario 1a. Using the service composer, Security Group SG-VDI had was configured to include the exploited Kali, the Win8.1 and Server 2003R2 VMs…Picture11

…by application of membership to a security policy VDI:

Which contained the following firewall rule to be applied to the E-W traffic patterns and was subsequently used to NSX Reject (and Block) DFW action in benchmark 1b (after being used in Allow action for 1a):Picture13
Shown in the results table above, the 1a behavior matched the expected result for unimpeded recon and exploit, which we see in the following series of screen shots which show the actual recon and SMB exploit events, as performed on the nsx-svr03r2-01 VM:

Kali Linux launch of msfconsole… issuing the initial DB_NMAP of target, which is the first Recon step…Picture14

… resulting in an addition to the MS_DB of the hosts, and after Recon step 2, the auxiliary SMB_VERSION tool, gets refined to include fine-grained details of the host profile(s). With intact target host information in the MS_DB, we can proceed with the SMB Exploit against the vulnerable Windows target machine…Picture15

… which we now have unfettered access to through the meterpreter console. Exploited success in 1a, followed by similar success in our subsequent JavaARA and Magento exploits.

Scenario 1b begins with the modification of our Firewall rule to Reject, which immediately eliminates all access to the target machines with this “Destination Host Prohibited” ping response, and similar impacts to the reconnaissance DB_NMAP and Aux SMB_Version MetaSploit activities:Picture16Picture17

All subsequent exploits are impossible without useful recon. Should a HOST_DB table be constructed or otherwise created, the exploit similarly fails, without E-W access to the target:Picture18

Summary and Conclusion

The purpose of our review of the VMware NSX networking and security platform was to sort out the facts from the hype, in an actual “micro audit”, where we could present representative E-W/N-S threats against typical network topologies (patterns), and scientifically measure the results. As stated in our “Objectives” section, we wanted to know if NSX functionally satisfied: the NIST SP 800-126 recommendations R1-R4; the defined precepts of micro-segmentation; and could real-world threats be prevented, when launched using actual hacking/penetration testing tools.
In the abbreviated sample based on design pattern 1a/b, provided in this blog, our findings were:

  • NSX provided significant distributed firewall (DFW) protections against E-W in design pattern 1a/b testing
  • Policy-based controls, tight integration with VMware objects and meta-data, the utility of using tools like Service Composer and the support provided by flow tracing and other tools made deployment and operation of NSX very easy and efficient
  • Nuanced controls presented by the firewall actions “Block” and “Reject” deliver granular control to threat scenarios relevant and representative of modern attack surface exploitation

We hope you found this sample tantalizing and encourage you to read the complete paper to get the final word on our Micro-segmentation Benchmark results. The complete report will include 5 network design patterns benchmarked, service insertion scenarios, ALG / Stateful flow control validation, more comprehensive coverage of NIST recommendation adherence, and much more.