ETL Integration Logical Architecture

How to Implement NSX Micro-Segmentation for Informatica (ETL) using VMware NSX Data Center

by: VMware IT Manager Rupak Acharjee

To read more about how VMware IT has implemented micro-segmentation using NSX Data Center in various applications, click here.

VMware IT uses Informatica as a unified enterprise data integration platform for accessing, discovering, and integrating data from virtually any business system, in any format, and delivering that data throughout the enterprise at any speed. Numerous system deployments and improvements can occur continuously.

As an ETL (extract, transform, load) tool, Informatica is used to integrate VMware’s many enterprise applications and databases, including those that process business-critical data from external sources. Integrating this data from multiple systems and sources is a core component of the VMware data integration toolbox. A consolidated view of data makes it easier for business users to analyze and report on data relevant to their initiatives. Informatica improves productivity by codifying and reusing processes to move data without requiring users to write code or scripts.

ETL Integration Logical Architecture

East-West Security Protection

Informatica had to be secured to prevent all manner of threats from any location within or outside of the VMware network. We used NSX Data Center to micro-segment both the Informatica and the internal source and target systems. Best practices developed during other micro-segmentation projects helped streamline this potentially complex process.

This diagram shows the ETL landscape prior to micro-segmentation, when the nodes were protected with a firewall. If one VM was compromised, the threat could easily put the entire ETL landscape at risk.

ETL no NSX microsegmentation

Micro-Segmentation Process

Micro-segmentation for this application was unique in several ways: multiple integrations; systems that could act as both sources and targets; and a high processing volume, with almost 21,000+ jobs handled per day in our production environment. The project was high profile because any issues could potentially impact our customers. Maintaining availability and performance were key goals.

Here are the steps we followed for the micro-segmentation of this application:

  • Server/host identification. Coordinated with the NSX team to capture server names and group them according to the server functionality, such as applications, databases, and third-party software. We checked pre-requisites such as firewall exceptions and node, client, and console settings so they could be configured at the ESXi host level.

The sys logs were forwarded to log management tools such as VMware Log Insight. (We now use vRealize Network Insight for our micro-segmentation projects.) It accelerates micro-segmentation deployment and minimizes business risk during application migration so we can confidently manage and scale NSX deployments.

  • Group security policy implementation. Defined separate firewall rules for each group, then we mapped the security policies to the security groups. By applying the rules to a group instead of individual VMs, management was simplified and centralized. We used the NSX Distributed Firewall to capture the policy.
  • Firewall rule testing. Reviewed the firewall rules, including the usual traffic between the load balancer, application servers, and databases, as well as traffic from middleware and other tools, with the support/development teams for each module. We also reviewed the firewall rules with the Security team to ensure they met corporate policies, then implemented the rules in a pre-production environment.
  • Analysis of traffic and enabling discovery. Enabled discovery to capture all the source and target system IP addresses and port numbers, including those connecting to and from ETL. During the discovery process, the NSX team created a firewall rule that allowed all the traffic coming to the application servers to be captured and uploaded to VMware Log Insight and vRealize Network Insight.
  • Pre-production testing, then production. If any issues were reported during pre-production testing, the relevant source-target IPs and port numbers were reviewed and the required modifications were made. We followed the same process to implement the rules into production.

This diagram shows ETL with micro-segmentation. The entire application is secured using NSX firewalls. Security policies are aligned with logical groups, blocking unauthorized traffic. Each VM is protected. Micro-segmentation policies follow a VM as it moves. This approach protects the application throughout its lifecycle and radically reduces administrative overhead.

Micro-Segmentation Rules

Below is the complete rule set for the ETL micro-segmentation, excluding monitoring and security management tools.

What We Learned

During the micro-segmentation process, we did extensive testing in a non-production environment to ensure it behaved normally. This helped us sort out issues before they could impact our customers. It was also easier to fix issues before the module was rolled out to production.

During the implementation, a few SSH connections and scripts failed as source systems were not able to connect to Informatica due to NSX. Some source and target systems had not been captured with discovery because the reports were scheduled to run monthly or quarterly. We quickly identified and allowed those IP addresses. In a few cases, jobs and scripts failed because database ports were blocked. We then worked with the Security and NSX teams to immediately allow SSH to access those hosts. We were able to connect the source and target systems which had been blocked by NSX.

Because of its business-critical nature, securing Informatica was a high priority.  Micro-segmenting provides the granular level of security required to isolate and protect all parts of our applications from any type of threat. In turn, we have been able to reduce our organizational risk. To read more about the deployment of micro-segmentation with NSX Data Center in various IT applications, click here.

VMware on VMware blogs are written by IT subject matter experts sharing stories about our digital transformation using VMware products and services in a global production environment. Contact your sales rep or to schedule a briefing on this topic. Visit the VMware on VMware microsite and follow us on Twitter.   


One comment has been added so far

Leave a Reply

Your email address will not be published.