Home > Blogs > Virtualize Business Critical Applications

Demo – Extended Oracle RAC across sites with VMware NSX

Our team has been working on building solutions for NSX with Business Critical Applications around Zero trust security, application mobility, operational efficiency, increased security and more .

In a previous post I published a demo about database cloning and enforcing dynamic security policies using NSX: http://blogs.vmware.com/apps/2016/04/dynamically-enforcing-security-on-a-hot-cloned-sql-server-with-vmware-nsx.html

In this blog post we are showcasing the ability to stretch an Oracle RAC solution in an Extended Oracle RAC deployment between multi-datacenter and using VMware NSX for L2 Adjacency.

With Extended Oracle RAC , both Storage and Network virtualization needs to be deployed to provided high availability, workload Mobility, workload balancing and effective Site Maintenance between sites.

NSX supports multi-datacenter deployments to allow L2 adjacency in software, to put it in simple words stretching the network to allow VM too utilize the same subnets in multiple sites.

The really cool thing here is that this is 100% implemented in software and can be easily augmented and replicated to your needs. You can even choose multiple implementation paths and configurations and apply all of them at the same time with minimal dependency to the physical infrastructure and it configuration. After all, this is virtual!

Multi-datacenter NSX can be implemented in multiple ways for different use cases:

  • For Disaster Recovery – where we deploy NSX to support a failover scenario where one site is mainly active for a workload and in case of site failure we flip a switch to support the networking from the secondary site.
  • For Disaster Avoidance and Workload Mobility – Where we move the networking of VMs to a secondary site on demand

In both cases a workload and its networking is either communicating from one site’s physical infrastructure or the other and when active from one site (the primary) it will traverse from that site’s physical infrastructure to the secondary site if needed.

You can see in the diagram below that Site A is the primary and therefore Site B utilizes Site A’s physical routers for ingress and egress communication.

NSX Across sites Active Passive

Multi-vCenter (datacenter) deployment of VMware NSX where site A is the primary and Site B is secondary. Networking to site B traverses through Site A

In case of a failure or a migration Site B’s infrastructure becomes active for ingress and egress:

Multi-vCenter NSX (Datacenter) after a site failure, networking switched to work independently through Site B's infrastructure

Multi-vCenter NSX (Datacenter) after a site failure, networking switched to work independently through Site B’s infrastructure

The solution we created for Oracle RAC is different, and that is based on Oracle RAC’s unique requirements. You can see in the demo here

Oracle RAC, requires active networking from each respective site. It requires all nodes to have IPs on the same segment and  if the nodes are placed in multiple sites , we then need to setup a solution to allow the same segment in both datacenters.

Since all Oracle RAC nodes are serving the applications and users in read/write for scalability purposes, performance needs to be interchangeable between them, the requirement is that each site will be active on its own infrastructure,

In the diagram below taken from the demo video, you can see the two sites and that each RAC node has “Optimized ingress” and “Locale egress” networking configuration in the respective site’s physical infrastructure and no site is considered “Primary”.

Taken from the demo video, this shows how each site operates independently from a networking perspective

Taken from the demo video, this shows how each site operates independently from a networking perspective

The way this was achieved in this implementation of the solution is by utilizing /32 static routes for site B’s Oracel RAC node that are injected on site B’s edge devices and than advertised to the uplink router using OSPF

One of the interesting challenges with this implementation was regarding the Oracle RAC SCAN IP’s. SCAN (Single Client Access Name) IP’s are IP’s assigned to ann Oracle RAC implementation which is used for client side load balancing between the cluster instances..

SCAN IP’s can comprise of a max of 3 IP’s configured in the DNS and they can come up on any node in the cluster in each one of the sites randomly.

To solve that problem , we created vRO workflows that can detect a VM coming up on site B and going down and run a workflow that can either add or remove a /32 static route from that sites edge device to support the movement of Vms or in our case the SCAN IPs.

Disclaimer, this solution can and will be improved from a scalability perspective, in particular automating route injection, from a performance and availability it is production ready.

Also, Route injection is not the only way to go about this, one can solve the challenge of moving IPs through other means or even from the presentation layer.

The demo explains step by step how this was implemented from an NSX perspective, here is the full link to the demo:

This demo was also featured in Sudhir and my session at VMworld 2016 here:

VIRT7575 – Architecting NSX with Business Critical Applications for Security, Automation and Business Continuity

Worked with me in creating this solution :

Sudhir Balasubramanian – Oracle

Agustin Malanco – NSX

Christpohe Decanini – Automation

As always any comments or questions are welcome.



This entry was posted in Oracle and tagged , , , , on by .
Niran Even-Chen

About Niran Even-Chen

Niran (@niranec on Twitter) is a Staff SE Specialist in VMware’s Network and Security BU, specializing in NSX and applications.
With over 15 years of experience in IT Engineering, architecture and sales roles.
The technical expertise Niran acquired includes design, technical selling and evangelizing of virtualization, Networking, Storage, performance, Cloud (CNA and Cloud management) and BC/DR solutions.
In his current position at VMware Niran devotes his time in helping Enterprise and global customers in their journey to digital transformation with Developer Ready Infrastructure, focusing on NSX and Pivotal Cloud Foundry Integration.
In his previous positions at VMware Niran was the main SME at VMware for Microsoft SQL Server virtualization and the author of the official best practices guides it, and also a Cloud and Performance specialist.
Niran is a frequent speaker at industry events and conferences including VMworld, VMUGs, SQL Saturdays and more. He is also part of the CTO Ambassador Program at VMware.

12 thoughts on “Demo – Extended Oracle RAC across sites with VMware NSX

  1. vikrant

    Great article, I have really enjoyed your article. This is really great that this is 100% implemented in software and can be easily augmented and replicated to our needs. we can even choose multiple implementation paths and configurations and apply all of them at the same time with minimal dependency to the physical infrastructure and it configuration . Thanks for sharing . The way you explained each and everything in this article is really good. Thanks once again.

    1. Niran Even-ChenNiran Even-Chen Post author

      Working on improving the solution, first. Still needs improvement from a scalability perspective as maintaining the static routes likes that is not ideal.

  2. Ahmed Naguib

    Impressing, how you did come with a solution out of the box to handle the Oracle Rac.

    I am about to design a similar solution but using a single vcenter with stretched cluster (vmsc), and VPLEX.

    I would have 2 DLRs, one primary for site A, and another for site B to have larger flexibility and control over ingress and egress traffic.

    Its interesting to see in your solution, that we can have a static route with /32, but advertising a specific IP, and not the complete subnet.. And I am going make good use of this idea.

    One point that was not very clear though, how does local egress benefit you here ? and what would have happened if you did not use it, and maybe depend only on different metrics for your OSPF to control the egress and ingress.

    Ahmed Naguib

    1. Niran Even-ChenNiran Even-Chen Post author

      For local I configured “local egress” see here
      I did not outline that in the post though. Are you planning to stretch the network? Because with a local DLR in each site without UDLR you will not be able to achieve that.

  3. Pingback: Oracle on VMware Collateral - One Stop Shop - Virtualize Business Critical Applications - VMware Blogs

Leave a Reply

Your email address will not be published. Required fields are marked *