“To beat (something) to boredom” – To linger over or discuss something so long and to such a tedious and laborious extent that the subject is no longer of any interest or relevance and induces a bad migraine!!!
Yes, much has been written and spoken about and VMware Snapshots and Independent Persistent disks but I still get questions from customers and field regarding that, so yes, this is yet another blog to attempt to explain this in detail.
Key points to take away from this blog
Given a VM’s with non-shared dependent and non-shared independent-persistent vmdk’s, taking a VMware snapshot will result in
- VMware snapshot for dependent vmdk’s
- VMware snapshot operation will ignore the independent-persistent vmdk’s
VMware Snapshot
Snapshots capture the entire state of the virtual machine at the time you take the snapshot. You can take a snapshot when a virtual machine is powered on, powered off, or suspended. If you are suspending a virtual machine, wait until the suspend operation finishes before you take a snapshot.
When you create a memory snapshot, the snapshot captures the state of the virtual machine’s memory and the virtual machine power settings. When you capture the virtual machine’s memory state, the snapshot operation takes longer to complete. You might also see a momentary lapse in response over the network.
More information about VMware Snapshots can be found here
Independent Persistent and Dependent vmdk’s
By default, all VM’s have exclusive access to their vmdk files unless the vmdk’s are shared between VM’s using the multi-writer flag (KB 10334165).
The topic of multi writer is out of scope of this blog and more information about that can be found at the post “RAC” n “RAC” all night – Oracle RAC on vSphere 6.x
The below table describes the various Virtual Machine Disk Modes
Example of an Single Instance Oracle VM Setup
VM ‘Oracle122-OL7’ is running OEL 7.4 operating system with Oracle 12.2.0.1.0 Grid Infrastructure and RDBMS binaries. The VM ‘Oracle122-OL7’ has 16 vcpus and 32GB of memory. Oracle SGA is set to 8 GB and PGA set to 2 GB.
Lets take a look at the VM ‘Oracle122-OL7’ SCSI controller and vmdk allocation and disk types.
VM ‘Oracle122-OL7’ has 5 vmdk’s
Hard Disk 1 is for the Operating System
Hard Disk 2 is for the Oracle Grid and RDBMS binaries
Hard Disk 3 is for the Oracle RMAN (Recovery Manager) dump
Hard Disk 4 is for the Oracle ASM disk group ( DATA_DG )
Hard Disk 5 is for the Oracle ASM disk group ( DATA_DG )
Disk Listing before the snapshot
Listing the VM folder contents before the snapshots shows there are no existing snapshots for the VM
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7] ls -l
-rw-r–r– 1 root root 92 Mar 26 14:57 Oracle122-OL7-06dfef71.hlog
-rw——- 1 root root 34359738368 Mar 24 03:52 Oracle122-OL7-eca3e893.vswp
-rw——- 1 root root 64424509440 Mar 27 15:45 Oracle122-OL7-flat.vmdk
-rw——- 1 root root 8684 Mar 27 03:45 Oracle122-OL7.nvram
-rw——- 1 root root 617 Mar 26 14:57 Oracle122-OL7.vmdk
-rw-r–r– 1 root root 0 Mar 24 03:37 Oracle122-OL7.vmsd
-rwxr-xr-x 1 root root 4279 Mar 26 14:57 Oracle122-OL7.vmx
-rw——- 1 root root 0 Mar 26 14:57 Oracle122-OL7.vmx.lck
-rw——- 1 root root 64424509440 Mar 27 15:45 Oracle122-OL7_1-flat.vmdk
-rw——- 1 root root 619 Mar 26 14:57 Oracle122-OL7_1.vmdk
-rw——- 1 root root 64424509440 Mar 26 15:28 Oracle122-OL7_2-flat.vmdk
-rw——- 1 root root 619 Mar 26 14:57 Oracle122-OL7_2.vmdk
-rw——- 1 root root 2199023255552 Mar 27 15:45 Oracle122-OL7_3-flat.vmdk
-rw——- 1 root root 481 Mar 26 15:00 Oracle122-OL7_3.vmdk
-rw——- 1 root root 2199023255552 Mar 27 15:45 Oracle122-OL7_4-flat.vmdk
-rw——- 1 root root 481 Mar 26 15:00 Oracle122-OL7_4.vmdk
-rw-r–r– 1 root root 393432 Mar 25 00:44 vmware-1.log
-rw-r–r– 1 root root 278017 Mar 26 14:57 vmware-2.log
-rw-r–r– 1 root root 370695 Mar 27 15:44 vmware.log
-rw——- 1 root root 115343360 Mar 26 14:57 vmx-Oracle122-OL7-3970164883-1.vswp
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7]
Snapshot Operation
Lets take a snapshot of the VM ‘Oracle122-OL7’ and observe the behavior. We will try to perform a snapshot operation with and without snapshotting the VM memory and observe the behavior.
Snapshot Operation with snapshotting VM memory
On the VM, Right Click , then click on ‘Snapshots’ and click on ‘Take Snapshots’ , check the ‘snapshot Memory option’ and click ‘OK’
We can see an error below
Snapshot Operation without snapshotting VM memory
On the VM, Right Click , then click on ‘Snapshots’ and click on ‘Take Snapshots’ , uncheck the ‘snapshot Memory option’ .
The snapshot operation is successful
Lets now take a look at the VM folder contents .
The files marked in
- green are snapshots of vmdk’s which are not Independent Persistent [ Oracle122-OL7.vmdk, Oracle122-OL7_1.vmdk and Oracle122-OL7_2.vmdk ]
- bold are the original vmdk’s which are Independent Persistent [ Oracle122-OL7_3.vmdk and Oracle122-OL7_4.vmdk ]
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7] ls -l
total 4413730405
-rw——- 1 root root 16904192 Mar 27 17:00 Oracle122-OL7-000001-delta.vmdk
-rw——- 1 root root 330 Mar 27 16:42 Oracle122-OL7-000001.vmdk
-rw-r–r– 1 root root 92 Mar 26 14:57 Oracle122-OL7-06dfef71.hlog
-rw——- 1 root root 21308 Mar 27 16:41 Oracle122-OL7-Snapshot3.vmsn
-rw-r–r– 1 root root 13 Mar 27 16:40 Oracle122-OL7-aux.xml
-rw——- 1 root root 34359738368 Mar 24 03:52 Oracle122-OL7-eca3e893.vswp
-rw——- 1 root root 64424509440 Mar 27 16:41 Oracle122-OL7-flat.vmdk
-rw——- 1 root root 8684 Mar 27 16:42 Oracle122-OL7.nvram
-rw——- 1 root root 617 Mar 27 16:41 Oracle122-OL7.vmdk
-rw-r–r– 1 root root 783 Mar 27 16:41 Oracle122-OL7.vmsd
-rwxr-xr-x 1 root root 4300 Mar 27 16:41 Oracle122-OL7.vmx
-rw——- 1 root root 0 Mar 26 14:57 Oracle122-OL7.vmx.lck
–rw——- 1 root root 16904192 Mar 27 17:00 Oracle122-OL7_1-000001-delta.vmdk
-rw——- 1 root root 334 Mar 27 16:41 Oracle122-OL7_1-000001.vmdk
-rw——- 1 root root 64424509440 Mar 27 16:41 Oracle122-OL7_1-flat.vmdk
-rw——- 1 root root 619 Mar 27 16:40 Oracle122-OL7_1.vmdk
-rw——- 1 root root 126976 Mar 27 16:41 Oracle122-OL7_2-000001-delta.vmdk
-rw——- 1 root root 334 Mar 27 16:41 Oracle122-OL7_2-000001.vmdk
-rw——- 1 root root 64424509440 Mar 26 15:28 Oracle122-OL7_2-flat.vmdk
-rw——- 1 root root 619 Mar 26 14:57 Oracle122-OL7_2.vmdk
-rw——- 1 root root 2199023255552 Mar 27 17:00 Oracle122-OL7_3-flat.vmdk
-rw——- 1 root root 481 Mar 27 16:41 Oracle122-OL7_3.vmdk
-rw——- 1 root root 2199023255552 Mar 27 17:00 Oracle122-OL7_4-flat.vmdk
-rw——- 1 root root 481 Mar 27 16:41 Oracle122-OL7_4.vmdk
-rw-r–r– 1 root root 393432 Mar 25 00:44 vmware-1.log
-rw-r–r– 1 root root 278017 Mar 26 14:57 vmware-2.log
-rw-r–r– 1 root root 532767 Mar 27 16:42 vmware.log
-rw——- 1 root root 115343360 Mar 26 14:57 vmx-Oracle122-OL7-3970164883-1.vswp
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7]
Lets have a look at the vmware.log file to verify that snapshot operation indeed ignored the Independent Persistent vmdk’s and only snapshotted the Dependent vmdk’s.
Observe the disk library operation CREATE-CHILD’ which is missing for the Independent Persistent vmdk’s.
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7] tail -1000 vmware.log
……
2018-03-27T16:41:44.873Z| vcpu-0| I125: DISKLIB-VMFS : “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-flat.vmdk” : open successful (29) size = 63712357888, hd = 0. Type 3
2018-03-27T16:41:44.877Z| vcpu-0| I125: DISKLIB-LIB_CREATE : DiskLibCreateCreateParam: vmfsSparse grain size is set to 1 for ‘/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-000001.vmdk’
2018-03-27T16:41:44.879Z| vcpu-0| I125: DISKLIB-LIB_CREATE : CREATE CHILD: “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-000001.vmdk” — vmfsSparse cowGran=1 allocType=0 policy=”
2018-03-27T16:41:44.879Z| vcpu-0| I125: DISKLIB-LIB_CREATE : CREATE-CHILD: Creating disk backed by ‘file’
2018-03-27T16:41:44.885Z| vcpu-0| I125: DISKLIB-VMFS_SPARSE : VmfsSparseExtentCreate: “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-000001-delta.vmdk” : success
2018-03-27T16:41:44.887Z| vcpu-0| I125: DISKLIB-DSCPTR: “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-000001.vmdk” : creation successful.
2018-03-27T16:41:44.905Z| vcpu-0| I125: DISKLIB-VMFS : “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-000001-delta.vmdk” : open successful (17) size = 263680, hd = 0. Type 9
2018-03-27T16:41:44.915Z| vcpu-0| I125: DISKLIB-VMFS : “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-000001-delta.vmdk” : closed.
2018-03-27T16:41:44.915Z| vcpu-0| I125: DISKLIB-VMFS : “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_1-flat.vmdk” : closed.
……
2018-03-27T16:41:45.085Z| vcpu-0| I125: DISK: OPEN scsi1:1 ‘/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_3.vmdk’ independent-persistent R[]
2018-03-27T16:41:45.095Z| vcpu-0| I125: DISKLIB-VMFS : “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_3-flat.vmdk” : open successful (10) size = 2199023255552, hd = 132615075. Type 3
2018-03-27T16:41:45.095Z| vcpu-0| I125: DISKLIB-DSCPTR: Opened [0]: “Oracle122-OL7_3-flat.vmdk” (0xa)
2018-03-27T16:41:45.095Z| vcpu-0| I125: DISKLIB-LINK : Opened ‘/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_3.vmdk’ (0xa): vmfs, 4294967296 sectors / 2 TB.
2018-03-27T16:41:45.098Z| vcpu-0| I125: DISKLIB-LIB : Opened “/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_3.vmdk” (flags 0xa, type vmfs).
2018-03-27T16:41:45.098Z| vcpu-0| I125: DISK: Disk ‘/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_3.vmdk’ has UUID ’60 00 c2 90 fe 39 5d f4-a9 f0 a6 c6 e9 81 8c 35′
2018-03-27T16:41:45.098Z| vcpu-0| I125: DISK: OPEN ‘/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7/Oracle122-OL7_3.vmdk’ Geo (267349/255/63) BIOS Geo (0/0/0)
2018-03-27T16:41:45.098Z| vcpu-0| I125: Creating virtual dev for ‘scsi1:1’.
2018-03-27T16:41:45.098Z| vcpu-0| I125: DumpDiskInfo: scsi1:1 createType=11, capacity = 4294967296, numLinks = 1, allocationType = 0
2018-03-27T16:41:45.098Z| vcpu-0| I125: SCSIDiskESXPopulateVDevDesc: Using FS backend
2018-03-27T16:41:45.098Z| vcpu-0| I125: DISKUTIL: scsi1:1 : geometry=267349/255/63
2018-03-27T16:41:45.098Z| vcpu-0| I125: SCSIFilterESXAttachCBRCInt: CBRC not enabled or opened without filters,skipping CBRC filter attach.
2018-03-27T16:41:45.098Z| vcpu-0| I125: SCSIFilterSBDAttachCBRC: device scsi1:1 is not SBD. Skipping CBRC attach SBD way.
2018-03-27T16:41:45.100Z| vcpu-0| A100: ConfigDB: Setting scsi1:2.redo = “”
….
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7]
Remove the snapshot
Right Click on VM, Click ‘Snapshots’ , Click ‘Manage All Snapshots’.
Delete the snapshot ‘Oracle122-OL7-Snapshot’
Observe the contents of the VM folder, both .vmsn and -delta files are now deleted when the snapshot is deleted from the VM.
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7] ls -l
-rw-r–r– 1 root root 92 Mar 26 14:57 Oracle122-OL7-06dfef71.hlog
-rw-r–r– 1 root root 13 Mar 27 16:40 Oracle122-OL7-aux.xml
-rw——- 1 root root 34359738368 Mar 24 03:52 Oracle122-OL7-eca3e893.vswp
-rw——- 1 root root 64424509440 Mar 27 17:03 Oracle122-OL7-flat.vmdk
-rw——- 1 root root 8684 Mar 27 17:03 Oracle122-OL7.nvram
-rw——- 1 root root 617 Mar 27 17:03 Oracle122-OL7.vmdk
-rw-r–r– 1 root root 43 Mar 27 17:03 Oracle122-OL7.vmsd
-rwxr-xr-x 1 root root 4279 Mar 27 17:03 Oracle122-OL7.vmx
-rw——- 1 root root 0 Mar 26 14:57 Oracle122-OL7.vmx.lck
-rw——- 1 root root 64424509440 Mar 27 17:03 Oracle122-OL7_1-flat.vmdk
-rw——- 1 root root 619 Mar 27 17:03 Oracle122-OL7_1.vmdk
-rw——- 1 root root 64424509440 Mar 26 15:28 Oracle122-OL7_2-flat.vmdk
-rw——- 1 root root 619 Mar 26 14:57 Oracle122-OL7_2.vmdk
-rw——- 1 root root 2199023255552 Mar 27 17:03 Oracle122-OL7_3-flat.vmdk
-rw——- 1 root root 481 Mar 27 16:41 Oracle122-OL7_3.vmdk
-rw——- 1 root root 2199023255552 Mar 27 17:03 Oracle122-OL7_4-flat.vmdk
-rw——- 1 root root 481 Mar 27 16:41 Oracle122-OL7_4.vmdk
-rw-r–r– 1 root root 393432 Mar 25 00:44 vmware-1.log
-rw-r–r– 1 root root 278017 Mar 26 14:57 vmware-2.log
-rw-r–r– 1 root root 576700 Mar 27 17:02 vmware.log
-rw——- 1 root root 115343360 Mar 26 14:57 vmx-Oracle122-OL7-3970164883-1.vswp
[root@WDC-ESX17:/vmfs/volumes/46d93793-0240bb85/Oracle122-OL7]
Key points to take away from this blog
- Given a VM’s with non-shared dependent and non-shared independent-persistent vmdk’s, taking a VMware snapshot will result in
- VMware snapshot for dependent vmdk’s
- VMware snapshot operation will ignore the independent-persistent vmdk’s
- A snapshot removal can stop a virtual machine for long time (KB 1002836)
- Best Practices needs to be followed when configuring Oracle RAC environment which can be found in the “Oracle Databases on VMware – Best Practices Guide”
- All Oracle on vSphere white papers including Oracle licensing on vSphere/vSAN, Oracle best practices, RAC deployment guides, workload characterization guide can be found in the post “Oracle on VMware Collateral – One Stop Shop“.
