In the world of Business-Critical Applications, especially IO intensive Oracle workloads, there is always a need for storage migration, based on the ever-demanding workload profile. For example,
- Migrate storage from one Tier to another Tier within a storage array
- Migrate storage from one array to another array (within a datacenter and even between datacenters as part of DC relocation)
This approach is independent of whether those workloads are virtualized or not. The end goal is to ensure that the underlying storage architecture can provide and sustain the demanding needs of the Application.
This article describes how storage migration can be performed for Oracle database running on VMware vSphere, for any types of underlying storage. The migration is done using a combination of two key technologies depending on the database deployment model
- VMware Storage vMotion
- Oracle Automatic Storage Management (ASM).
Before describing this technique, we first review some key ideas, including the deployment model of Oracle on vSphere and the fundamentals of Storage vMotion.
This blog focuses on Storage vMotion of Oracle Non-RAC workloads within the same vSphere Cluster.
Oracle layout on vSphere
There are 3 main Oracle database deployment models, all of which are supported and can be deployed on the VMware SDDC stack.
- Oracle Single Instance
- Oracle Real Application Cluster
- Oracle RAC one node (uses the same concept of sharing vmdk’s using multi-writer flag)
Single Instance Oracle database VM’s will have 2 genres of vmdk’s :
- Non-database vmdk’s
- Operating System disks which houses the root file system (/)
- Oracle binaries vmdk which houses the Oracle binaries (Grid and RDBMS binaries) (/u01)
- Non-shared database vmdk’s
VMware Storage vMotion
With VMware Storage vMotion technology, we can migrate VM’s to relocate the configuration file of a virtual machine and virtual disks, while the virtual machine is powered on, from one datastore to another.
The vSphere documentation details Storage vMotion Requirements and Limitations
Understanding Oracle Storage migration methods
There are a couple of ways we can migrate Oracle workload storage either from one datastore to another, within in an existing array or between 2 separate arrays.
- Oracle Automatic Storage Management (ASM) technology to migrate oracle blocks from one vmdk on a datastore to a new vmdk on same / different datastore
The process is to add new disks to the existing ASM disk group and remove old disks from the same ASM disk group , while the database continues to access files from the disk group. When you add or remove disks from a disk group, Oracle ASM automatically redistributes the file contents and eliminates the need for downtime when redistributing the content. When a disk is dropped, the disk group is rebalanced by moving all of the file extents from the dropped disk to other disks in the disk group.
More information can be found at Dropping Disks from Disk Groups.
The power setting parameter ASM_POWER_LIMIT determines the speed with which rebalancing operations occur. The range of values is 0 to 1024. The default value is 1. A value of 0 disables rebalancing. Higher numeric values enable the rebalancing operation to complete more quickly, but might result in higher I/O overhead and more rebalancing processes. With this parameter, the DBA has the power to control the migration throughput hence avoiding causing potential performance during peak production hours
Oracle ASM migration method uses Oracle ASM technology to migrate oracle blocks between the ASM disks. It willwill ONLY migrate Oracle database ASM disk and will NOT migrate the Operating System vmdk or Oracle binaries vmdk
The steps for using Oracle ASM technology for storage migrating Oracle database storage from one datastore to new datastore on either an existing array or between 2 arrays are
-
- add new vmdk/s from new datastore (either existing or new array) to VM
- partition the new device with appropriate partitioning offset (needed if ASMLIB is used, not needed for Linux udev)
- create ASM device on new disk/s
- add new ASM disks to existing ASM disk group
- drop the old ASM disk from existing ASM disk group and perform rebalance (add and drop operation will move all of the file extents from the dropped disk to other disks in the disk group)
- delete the old vmdk/s from the VM
At the end of this step
-
- database vmdk’s are on the new datastore
- however , operating system and oracle binaries vmdk are still on the old datastore
- Use VMware storage vMotion method of moving vmdk’s from one datastore to new datastore
VMware storage vMotion will migrates ALL vmdk’s including Oracle database ASM disk, Operating System vmdk and Oracle binaries vmdk and can be used ONLY for non-shared vmdk’s [KB 1034165]
Any storage-based migration, be it storage vMotion or array based migration, is faster than Oracle ASM method of add, drop and rebalancing disks. However, the speed of migration is not regulated or controlled. This may cause potential performance during peak production hours.
For migration from existing to a new array, datastores from both arrays must be mounted on the vSphere cluster
There are also other methods for migrating storage which will require time to setup and cut over as well and are out of scope for this blog
- Standing up a target Oracle RAC as Physical standby for the source Oracle RAC cluster
- Using Data Pump / RMAN / Golden Gate / Third party Replication products to move data between source and new created target RAC database
Test Case using VMware Storage vMotion
VM ‘Oracle1912-OEL83’ houses a single instance Oracle database 19c running on OEL 8.3 operating system.
The vmdk’s for the VM ‘Oracle1912-OEL83’ is on a Pure Storage FC datastore.
The VM ‘Oracle1912-OEL83’ has 3 vmdk’s-
- Hard Disk 1 of 80 GB capacity which houses the OS (/)
- Hard Disk 2 of 80 GB capacity which houses the Oracle binaries (/u01)
- Hard Disk 1 of 50 GB capacity which houses the Oracle database (ASM disk) – No sharing mode
Using Storage vMotion technology
Process of storage vMotion of an Oracle single instance deployment on VMware SDDC is the same as storage vMotioning any other workload on VMware SDDC.
We will now Storage vMotion VM ‘Oracle1912-OEL83’ from Pure Storage FC datastore ‘OraPure’ to a Tintri NFS datastore ‘OraTintri’
Right click on VM ‘Oracle1912-OEL83’ on Web client and Click Migrate.
Choose the Storage Migration option
Choose the destination datastore
We have a choice to pick the datastore we want to place the vmdk’s, in this case we picked the same datastore for all the VM vmdk’s.
Confirm operation and click Finish
Storage vMotion operation starts and completes successfully
We have now successfully Storage vMotioned the VM ‘Oracle1912-OEL83’ from Pure Storage FC datastore ‘OraPure’ to a Tintri NFS datastore ‘OraTintri’
At the end of this step both non-database vmdk’s (operating system and oracle binaries ) and database vmdk’s are on the new datastore
Summary
Storage Migration of a Non-RAC Oracle database VM on VMware SDDC is the same as storage vMotioning any other workload on VMware SDDC.
All Oracle on vSphere white papers including Oracle licensing on vSphere/vSAN, Oracle best practices, RAC deployment guides, workload characterization guide can be found in the url below
Oracle on VMware Collateral – One Stop Shop
