Home > Blogs > Virtualize Business Critical Applications

Demo – Extended Oracle RAC across sites with VMware NSX

Our team has been working on building solutions for NSX with Business Critical Applications around Zero trust security, application mobility, operational efficiency, increased security and more .

In a previous post I published a demo about database cloning and enforcing dynamic security policies using NSX: http://blogs.vmware.com/apps/2016/04/dynamically-enforcing-security-on-a-hot-cloned-sql-server-with-vmware-nsx.html

In this blog post we are showcasing the ability to stretch an Oracle RAC solution in an Extended Oracle RAC deployment between multi-datacenter and using VMware NSX for L2 Adjacency.

With Extended Oracle RAC , both Storage and Network virtualization needs to be deployed to provided high availability, workload Mobility, workload balancing and effective Site Maintenance between sites.

NSX supports multi-datacenter deployments to allow L2 adjacency in software, to put it in simple words stretching the network to allow VM too utilize the same subnets in multiple sites.

The really cool thing here is that this is 100% implemented in software and can be easily augmented and replicated to your needs. You can even choose multiple implementation paths and configurations and apply all of them at the same time with minimal dependency to the physical infrastructure and it configuration. After all, this is virtual!

Multi-datacenter NSX can be implemented in multiple ways for different use cases:

  • For Disaster Recovery – where we deploy NSX to support a failover scenario where one site is mainly active for a workload and in case of site failure we flip a switch to support the networking from the secondary site.
  • For Disaster Avoidance and Workload Mobility – Where we move the networking of VMs to a secondary site on demand

In both cases a workload and its networking is either communicating from one site’s physical infrastructure or the other and when active from one site (the primary) it will traverse from that site’s physical infrastructure to the secondary site if needed.

You can see in the diagram below that Site A is the primary and therefore Site B utilizes Site A’s physical routers for ingress and egress communication.

NSX Across sites Active Passive

Multi-vCenter (datacenter) deployment of VMware NSX where site A is the primary and Site B is secondary. Networking to site B traverses through Site A

In case of a failure or a migration Site B’s infrastructure becomes active for ingress and egress:

Multi-vCenter NSX (Datacenter) after a site failure, networking switched to work independently through Site B's infrastructure

Multi-vCenter NSX (Datacenter) after a site failure, networking switched to work independently through Site B’s infrastructure

The solution we created for Oracle RAC is different, and that is based on Oracle RAC’s unique requirements. You can see in the demo here

Oracle RAC, requires active networking from each respective site. It requires all nodes to have IPs on the same segment and  if the nodes are placed in multiple sites , we then need to setup a solution to allow the same segment in both datacenters.

Since all Oracle RAC nodes are serving the applications and users in read/write for scalability purposes, performance needs to be interchangeable between them, the requirement is that each site will be active on its own infrastructure,

In the diagram below taken from the demo video, you can see the two sites and that each RAC node has “Optimized ingress” and “Locale egress” networking configuration in the respective site’s physical infrastructure and no site is considered “Primary”.

Taken from the demo video, this shows how each site operates independently from a networking perspective

Taken from the demo video, this shows how each site operates independently from a networking perspective

The way this was achieved in this implementation of the solution is by utilizing /32 static routes for site B’s Oracel RAC node that are injected on site B’s edge devices and than advertised to the uplink router using OSPF

One of the interesting challenges with this implementation was regarding the Oracle RAC SCAN IP’s. SCAN (Single Client Access Name) IP’s are IP’s assigned to ann Oracle RAC implementation which is used for client side load balancing between the cluster instances..

SCAN IP’s can comprise of a max of 3 IP’s configured in the DNS and they can come up on any node in the cluster in each one of the sites randomly.

To solve that problem , we created vRO workflows that can detect a VM coming up on site B and going down and run a workflow that can either add or remove a /32 static route from that sites edge device to support the movement of Vms or in our case the SCAN IPs.

Disclaimer, this solution can and will be improved from a scalability perspective, in particular automating route injection, from a performance and availability it is production ready.

Also, Route injection is not the only way to go about this, one can solve the challenge of moving IPs through other means or even from the presentation layer.

The demo explains step by step how this was implemented from an NSX perspective, here is the full link to the demo:

This demo was also featured in Sudhir and my session at VMworld 2016 here:

VIRT7575 – Architecting NSX with Business Critical Applications for Security, Automation and Business Continuity

Worked with me in creating this solution :

Sudhir Balasubramanian – Oracle

Agustin Malanco – NSX

Christpohe Decanini – Automation

As always any comments or questions are welcome.



Oracle 12c OLTP & DSS workloads on Virtual SAN 6.2 All Flash

Hey Oracle on VMware folks!!!

Earlier, we had blogged about Oracle RAC on VMware Virtual SAN Hybrid which is VMware software-defined storage solution for hyper-converged infrastructure and is available at Oracle RAC on Hybrid VSAN

In this blog post, we would like give you a glimpse of the “Oracle 12c OLTP workloads on VMware All Flash Virtual SAN 6.2”.

As some of you may know, in VMware Virtual SAN 6.2, we introduced a suite of new features for data integrity and space efficiency including Checksum, Erasure Coding, Deduplication and Compression. Using these features we developed an architecture for deploying and running Oracle 12c OLTP and DSS workloads on VMware All Flash Virtual SAN 6.2.

Before going into some of the results, let’s take a look as to why it is compelling to deploy heavy IO Oracle workloads on a VMware Virtual SAN All Flash solution.

  • Extreme Performance – Virtual SAN is built into the vSphere kernel which optimizes the I/O path to provide excellent performance.
  • Built in Data Reduction Technologies – Deduplication, Compression and Erasure Coding reduces data footprint and lowers cost, both capex and opex, further lowering the TCO of the solution.
  • Availability for Business Critical Applications – Minimizes downtime with features like vMotion, vSphere HA, Virtual SAN Stretched Clustering.
  • Redundancy for Business Critical Applications – Provides high level of redundancy thereby avoiding single point of failures (SPOF) with features like Failure to Tolerate (FTT), Fault Domain

Continue reading

New Updated Oracle Databases on VMware Best Practices Guide

Business Critical Applications (BCA) are at the heart of an organization; their performance, availability and reliability are vital to the success of the business. Significant advances in hardware are far outpacing the performance requirements of even the most intensive business-critical applications, including Oracle Database.

VMware has kept pace with these hardware improvements by engineering significant performance advances in vSphere 6.0. Best Practices are key in ensuring that business SLA’s, uptime, QoS are met with when deploying Oracle on VMware vsphere.

Successful virtualization of any Business Critical Application (BCA) required best practices inculcated in every layer of the stack.

We are pleased to announce the updated version of the “Oracle Databases on VMware Best Practices Guide” which can be found in the below URL. Thanks to everyone who participated in this exercise

In this guide there are also references to other VMware and third-party documents which we encourage the reader to consult for better understanding of the topics discussed.

Click here to download the guide.

RUSH POST: Microsoft Convenience Update and VMware VMXNet3 vNIC Incompatibilities

VMware has received confirmation that Microsoft has determined that the issue reported in this post is a Windows-specific issue and unrelated to VMware or vSphere. Microsoft is encouraging customers to follow the directions provided in Microsoft KB3125574 for the recommended resolution. All further updates will be provided directly by Microsoft through the referenced KB. This marks the end of further updates to this blog post.


  • Please see Microsoft’s updated guidance and recommended work-around regarding this issue
  • I am removing reference to “VMware VMXNet3” in the title of this post to reflect Microsoft’s latest updates to their KB. Yes, the issue still exists when using VMXNet3 for VMs, but it no longer appears that this issue is specific to “VMXNet3” virtual network adapters.
  • We are still working with Microsoft to conduct a comprehensive Root-Cause Analysis, and we will provide further updates as new information (or a resolution) becomes available.

Microsoft recently released a “Convenience Update” patch for Windows 7 and Windows Server 2008 R2 SP1. This update has incompatibility issues with virtual machines running on the VMware vSphere virtualization platform. This incompatibility is confined to one specific configuration scenario – It impacts VMs that use the VMware VMXNet3 virtual network adapter type.

Here is the incompatibility issue as described in Microsoft’s announcement of the Update:

Known issue 1


A new Ethernet vNIC may be created with default settings in place of the previously existing vNIC, causing network issues.  Any custom settings on the previous vNIC are still persisted in the registry but unused.


To resolve this issue, uninstall the convenience rollup.


Microsoft is investigating this issue to determine proper course of action with VMWare. To resolve this issue uninstall the convenience rollup. Further information will be posted here as the investigation continues.

VMware is aware of this issue and we are actively investigating the root causes and possible fixes. While this effort progresses, VMware is advising customers to delay applying the Microsoft “Convenience Update” to any virtual machine that uses the VMXNet3 vNIC type.

VMware will provide further updates as they become available.

VMware at SAPPHIRE NOW 2016 Recap

From May 17th to May 19th, SAP welcomed around 30,000 attendees to Orange County Convention Center in Orlando, Florida to its SAPPHIRE NOW conference and provided an overview on what’s new as well as what’s upcoming from SAP in 2016. As Bill McDermott, SAP CEO states during his keynote: “Run Live, Run Simple”!


Freely adapted from this motto, I am going to provide you an overview on what’s new form VMware in the SAP area to help SAP and it customers to make this motto real and what we have prepared and presented on our booth. Continue reading

SAP HANA on vSphere – Support Status and Best Practices Summary

Since SAPHIRE 2016, SAP supports now also SAP HANA on vSphere 6 deployments for production workloads, see SAP Note 2315348 for details.

The support for vSphere 6 allows customers to increase the RAM to up to 4 TB (4080 GB) of existing virtual SAP HANA systems when migrated to vSphere 6 and allows to react on increased memory needs due to data growth or newly deployed SAP HANA scenarios and solutions.

Beside increased RAM sizes vSphere 6 supports also more vCPUs. Up to 128 vCPUs can now get configured and used by a single SAP HANA VM.

Supporting more physical compute resources inside a VM ultimately provides more “power” to a virtualized SAP HANA system – this alone is worth upgrading from a vSphere 5.5 to a vSphere 6.0 based SAP HANA environment. Beside this, all vSphere 6.0 features can get used as before with vSphere version 5.5. Continue reading

Updated – Microsoft SQL Server on VMware vSphere Availability and Recovery Options

Choosing the right availability configuration for your SQL Server on vSphere can be a bit confusing as there are more than a few options to choose from, questions such as: Should I use vSphere HA if i’m using AlwaysOn? What are the implications of running different availability configurations on vSphere, and what are the best practices? 

This very popular guide called “Microsoft SQL Server on VMware vSphere Availability and Recovery Options” which outlines the availability options and best practices for SQL Server on vSphere and tries to answer these question, is now updated with the latest information.

Download the guide here

As always, any feedback is welcome,


SAP HANA on VMware vSphere – Sizing (Compute)

This is the second blog of my SAP HANA on vSphere blog series and will provide information on SAP HANA on VMware vSphere compute resource sizing.

This blog does not describe how to perform the actual SAP HANA system RAM sizing. For this please refer to the SAP documentation. A good start is to read the sizing SAP HANA Sizing Approaches presentation, which describes the SAP HANA QuickSizer and ABAP SAP HANA sizing reports available for sizing SAP HANA systems. Also a good start is the Sizing Approaches for SAP HANA document. (Attention you may require a SAP SDN account to access the SAP HANA sizing documents).

CPU and RAM Sizing SAP HANA on VMware vSphere

Sizing SAP HANA is different from sizing a classical SAP application. Instead focusing on CPU resources, for SAP HANA memory is the focus and how many CPU resources are required to support a specific amount of RAM. Beside this, we have to consider the different CPU and RAM maximums of vSphere 5.5 and 6, as it defines the maximal size of a virtual SAP HANA VM. To make it even more complex, we have to take into account which workload (SoH or BWoH) will run on the selected SAP HANA system, which CPU and which HANA SPS version will get used, as different CPU socket to RAM ratios exist for all of these combinations.

In the soon to get published new “Architecture Guidelines and Best Practices for Deployments of SAP® HANA® on VMware vSphere®” document, which will be available on www.vmware.com/go/sap-hana, I have detailed sizing formulas specified, which I have defined jointly with SAP. The formula respects hyperthreading with a performance gain of 15% and the used CPU type (amount of available cores), virtualization memory costs and can get used also to calculate the needed vCPUs to support a certain amount of HANA RAM.

Instead of using these sizing formulas, I will explain a simplified way how to size production level ready SAP HANA VMs.

Since we want to leverage hyperthreading, it is required to enable it on the host and that the VMs get configured with the parameter Numa.PreferHT=1. Using this parameter ensures NUMA node locality and forces the ESXi scheduler to use hyperthreads instead of potentially idle cores on other NUMA nodes.

More detailed explanations will be available in the referred VMware SAP HANA guide. As well as storage and network sizing and configuration information.

VMware vSphere SAP HANA relevant Sizing figures

Before we start to size SAP HANA VMs, we have to discuss the current vSphere compute resource maximal sizes and define the current sizing limitation for memory. This limitation is important to understand when sizing a BWoH system on vSphere, as the CPU VM limitation of vSphere 6 of 128 vCPUs will not allow to utilize the theoretical maximum of 4080 GB per VM. Reason is that SAP has defined specific CPU socket to RAM ratios, which are more demanding for an analytical workload like BW. For OLTP like applications like SoH this ratio is higher and here we could size larger memory sizes as even vSphere supports (see below table)!

The table enumerates the current maximal sizing figures for virtualized SAP HANA systems on vSphere. Please note that these figures represent the theoretical sizing maximal sizes and should get aligned after some time to the real live SAP HANA configuration, as you may have less CPU resource need and would therefore be able to increase the memory beyond this theoretical sizing figures. If more VMs get installed on a server or when a single VM will consume the complete available RAM, then also RAM for vSphere has to get reserved. We calculate 1 percent up to 3 percent for vSphere. This depends on the actual VM configuration.

CPU Scale-Up Workload SPS version vSphere Version Maximal “theoretical sizing” RAM size1 Maximal VM size
Haswell Intel Xeon E7-v3 SoH SPS >= 10 5.5 2374.49 GB 64 vCPUs, 1024 GB vRAM
SoH SPS >= 10 6.0 4748.99 GB 128 vCPUs, 4080 GB vRAM
BWoH SPS >= 10 5.5 1582,99 GB 64 vCPUs, 1024 GB vRAM
BWoH SPS <= 10 6.0 3165,99 GB 128 vCPUs, 3072 GB vRAM
BWoH SPS >= 11 6.0 3165,99 GB 128 vCPUs, 3166 GB vRAM

1This is a theoretical sizing maximal figure and got calculated with the referenced sizing formula as documented in the “Architecture Guidelines and Best Practices for Deployments of SAP HANA on VMware vSphere” document, which will available on www.vmware/go/sap-hana.

The figures in above table are relevant when for instance an 8-socket server should get used for a maximal sized virtualized SAP HANA BW system. The RAM sizing limitation should get seen as the first start to size the VM and the RAM may get aligned, after consulting SAP HANA support and sizing teams, to the actual server usage and CPU utilization.

Continue reading

vRealize Automation 7 converged blueprints

vRealize Automation 7 feature a new Blueprint format. A converged blueprint can have one to many components including:

  • Machines : vCenter, vCloud Air, vCloud Director, Amazon, KVM, Openstack, Citrix Xen, HyperV, SCVMM
  • Network & Security: Security Group, Load balancer, NAT, Routed networks.
  • Software (Install, configure, start, uninstall) : Scripts to setup the OS or to install and configure applications
  • XaaS : Workflows (I.E virtual machine customization, guest operations)

Combining these allow designing complex applications.

For example using the vRealize Automation design canvas you can put together a multiple instance machine to deploy cluster nodes with an XaaS components to create shared disks and software components to install and configure the Operating System, and application layers.

vRAblueprintClustered application in vRealize Automation 7 design canvas

All these components are bound together via data bindings that will determine the parameters flow and deployment order.

While you can design your application from scratch being able to leverage community blueprint will save you a lot of time learning from other community members designs, and application scripts.

VMware Sample Exchange is a VMware and community content repository. From this site you can download sample code, automation scripts, Orchestration workflows and also vRealize Automation 7 application Blueprints. You can also create your own content and upload it to sample exchange to share it with the community.

At the moment there are about a dozen application blueprints available with new ones uploaded in a regular basis.

A blueprint is a YAML formatted file packaged within a zip file that can be downloaded with your browser. For importing it in vRealize Automation VMware provides a command line tool called vRealize CloudClient.

There is also a more convenient way to import Sample Exchange blueprints directly from the vRealize Automation service catalog with using a workflow I have released on the vRealize Orchestrator community.

importFromSampleExchangeRequesting to import a blueprint from Sample Exchange

With all these new capabilities integrated in vRealize Automation 7 you have the ability to design, publish and exchange advanced application blueprints.

To know more about designing a vRealize 7 application blueprint you can check this video.

Top Ten things to consider when moving Business Critical Applications (BCA) to the Cloud (Part 3 of 3)

In the first part we looked at public, private and Hybrid Cloud and their characteristics. In this part we will look at the common characteristics of business critical applications. In the second part , we looked at how some of these characteristics relate to the different types of Cloud infrastructure. In this final part we will look at he lifecycle of a business critical application in the cloud and the conclusion. Continue reading