In the last post, we endeavored to explain how to go about an Oracle Licensing Audit effectively by meticulously collecting all artifacts needed for the audit.

We recommend as artifacts, Proof of Compute Enclosure and Audit Trail entries as part of the lists of artifact to collect and store for at least 2-3 audit cycles.

We also concluded that Oracle Licensing Audit should not be taken lightly just as you would for any other software vendor but not special and one does not have to fear it.

This post endeavors to highlight the typical questions customers might have in their minds after reading articles on internet or talking to other colleagues or questions they might have encountered talking to licensing auditors.

Oracle Licensing Journey

During the course of my career as an Oracle DBA and Architect working on Oracle technologies, Oracle licensing was one of the facets of a DBA life I had to go through and really , nothing has changed much.

Working as the Oracle Technologies pre-sales Lead in VMware since 2012 and being the lead Oracle pre-sales field guy, talking to customers and clarifying their questions about Oracle licensing on VMware SDDC is one of my charters.

Continue reading →

In the last post , we addressed the Licensing fuds and myths when it comes to addressing Oracle Licensing on VMware vSphere / VSAN technologies and explained how Oracle licensing DOES NOT change from a licensing perspective, whether you run Oracle workloads on a classic vSphere environment or Hyper-Converged Infrastructure solution like VSAN.

This post endeavors to explain how to go about an Oracle Licensing audit effectively by meticulously collecting all artifacts needed for the audit.

FUD

Googling the word FUD does certainly explains clearly the meaning and intention of this oft used word in the Oracle Licensing space.

Oracle License Audit

Having put these myths to rest, let’s talk about the “Oracle License Audit” process. Many horror stories have been echoed in the hallways of IT and around water coolers but the key thing to keep in is “Yes, we need to take that seriously but no reason to be scared about it!!! , it’s just another software audit”.

The key mantra is to be “Fully prepared for it with all relevant artifacts to defend the audit”.

We have well established beyond any reasonable doubt in the previous blog post that Oracle licensing is not Memory, Storage, Cluster, vCenter or Network based, it’s either User based (Named User Plus) or Processor(Socket in case of SE2 or cores in case of EE edition).

Successfully defending an Oracle Audit

The primary goal of effectively defending an Oracle Licensing Audit on VMware vSphere/VSAN is to prove that an effective “Compute Segmentation” has been done to ensure that Oracle Virtual Machines runs on dedicated ESXi servers in the datacenter, because again, to re-iterate, Oracle licensing is Compute (SE2/EE)  /  User (NUP) based.

We can achieve the above goal in 2 ways
1)    Create a “Compute Enclosure” to prevent VM’s from leaving the enclosure by any means whatsoever
2)    Establishing an auditing mechanism of documenting  VM movements via vMotion events in the above “Compute Enclosure”

Create “Compute Enclosure”

There are 2 ways to create the “Compute Enclosure”:

Option A: Dedicated vSphere Cluster for Oracle VM’s (Recommended). This model is a widely accepted model purely from an Oracle licensing perspective.

Option B: Common vSphere Cluster where we use Affinity rules to bind Oracle VM‘s to a set of ESXi servers dedicated for Oracle workloads

Either of the 2 ways are acceptable as the Oracle OLSA / OMA does not stipulate anything about vSphere Cluster apart from the definition of the Processor as “Processor shall be defined as all processors where the Oracle programs are installed and/or running.”

In case of option B, the process of pinning Oracle VM’s to ESXI hosts have been explained in the previous blog post

https://blogs.vmware.com/apps/2017/01/oracle-vmware-vsan-dispelling-licensing-myths.html

Having created the “Compute Enclosure” i.e. a vSphere Cluster for Oracle Workloads, now we need to establish an auditing mechanism of documenting the Oracle VM movements by tracking the movement of the Oracle VM’s via vMotion events within the above “Compute Enclosure”.

Establishing Audit Mechanisms

Audit Information about VM Power on/off event

In the previous blog post, we showed how the VM Power On operations audit information is recorded in the vmware.log file.
https://blogs.vmware.com/apps/2017/01/oracle-vmware-vsan-dispelling-licensing-myths.html

Let’s see the contents of the vmware.log file for “testoravm” when we power it up on a vSAN Cluster

[root@w2-pe-vsan-esx-029:/vmfs/volumes/vsan:52803547e520f694-1f6104395ada7b7c/05735458-cc86-e1e9-ca71-0025b501004e] cat vmware.log
2016-12-27T21:09:09.124Z| vmx| I125: Log for VMware ESX pid=2597049 version=6.5.0 build=build-4564106 option=Release
2016-12-27T21:09:09.124Z| vmx| I125: The process is 64-bit.
2016-12-27T21:09:09.124Z| vmx| I125: Host codepage=UTF-8 encoding=UTF-8
2016-12-27T21:09:09.124Z| vmx| I125: Host is VMkernel 6.5.0
2016-12-27T21:09:09.091Z| vmx| I125: VTHREAD initialize main thread 0 “vmx” tid 2597049
2016-12-27T21:09:09.092Z| vmx| I125: Msg_SetLocaleEx: HostLocale=UTF-8 UserLocale=NULL
……….
……….
2016-12-27T21:09:09.124Z| vmx| I125: Hostname=w2-pe-vsan-esx-029
2016-12-27T21:09:09.124Z| vmx| I125: IP=127.0.0.1 (lo0)
…..
[root@w2-pe-vsan-esx-029:/vmfs/volumes/vsan:52803547e520f694-1f6104395ada7b7c/05735458-cc86-e1e9-ca71-0025b501004e]

The Power On process of an Oracle VM on a classic vSphere Cluster also records the information of the host it powers on, no different than the o/p we see above on a vSAN Cluster.

[root@wdc-esx10:/vmfs/volumes/56bce95e-eb1c7670-1464-0025b3b1b790/Template_OEL70] more vmware.log
2016-11-02T04:36:09.871Z| vmx| I120: Log for VMware ESX pid=3165445 version=6.0.0 build=build-3029758 option=Release
2016-11-02T04:36:09.871Z| vmx| I120: The process is 64-bit.
2016-11-02T04:36:09.871Z| vmx| I120: Host codepage=UTF-8 encoding=UTF-8
2016-11-02T04:36:09.871Z| vmx| I120: Host is VMkernel 6.0.0
2016-11-02T04:36:09.854Z| vmx| I120: VTHREAD initialize main thread 0 “vmx” pid 3165445
2016-11-02T04:36:09.854Z| vmx| I120: Msg_SetLocaleEx: HostLocale=UTF-8 UserLocale=NULL
….
2016-11-02T04:36:09.856Z| vmx| I120: DictionaryLoad: Cannot open file “//.vmware/config”: No such file or directory.
……..
2016-11-02T04:36:09.859Z| vmx| I120: PREF Failed to load user preferences.
2016-11-02T04:36:09.872Z| vmx| I120: Hostname=wdc-esx10.tsalab.local

Audit Information about VM vMotion event

Let’s see the contents of the vmware.log file of an Oracle VM when we vMotion it from one ESXi server to another ESXi server within a vSphere Cluster

[root@wdc-esx10:/vmfs/volumes/56bce95e-eb1c7670-1464-0025b3b1b790/Template_OEL70] more vmware.log
2016-11-02T04:36:09.871Z| vmx| I120: Log for VMware ESX pid=3165445 version=6.0.0 build=build-3029758 option=Release
2016-11-02T04:36:09.871Z| vmx| I120: The process is 64-bit.
2016-11-02T04:36:09.871Z| vmx| I120: Host codepage=UTF-8 encoding=UTF-8
2016-11-02T04:36:09.871Z| vmx| I120: Host is VMkernel 6.0.0
2016-11-02T04:36:09.854Z| vmx| I120: VTHREAD initialize main thread 0 “vmx” pid 3165445
2016-11-02T04:36:09.854Z| vmx| I120: Msg_SetLocaleEx: HostLocale=UTF-8 UserLocale=NULL
….
2016-11-02T04:36:09.856Z| vmx| I120: DictionaryLoad: Cannot open file “//.vmware/config”: No such file or directory.
……..
2016-11-02T04:36:09.859Z| vmx| I120: PREF Failed to load user preferences.
2016-11-02T04:36:09.872Z| vmx| I120: Hostname=wdc-esx10.tsalab.local

The VM was initially powered on wdc-esx10.tsalab.local server.

When the Oracle VM vMotion to another ESXI server either done manually or through DRS events the vMotion entries along with the source and target ESXI servers are recorded in the vmware.log file.

In the above case the Oracle VM vMotioned from wdc-esx10.tsalab.local server to wdc-esx09.tsalab.local server

root@wdc-esx10:/vmfs/volumes/56bce95e-eb1c7670-1464-0025b3b1b790/Template_OEL70] more vmware.log
…..
2016-11-02T04:44:38.156Z| vmx| I120: MigrateVMXdrToSpec: type: 1 srcIp=<10.128.136.110> dstIp=<10.128.136.109> mid=5404a192575ee uuid=38383135-3735-5355-4530-343132465936 priority=yes checksumMemory=no maxDowntime=0 encrypted=0 resumeDuringPageIn=no latencyAware=yes diskOpFile= srcLogIp=<<unknown>> dstLogIp=<<unknown>>
….
…
2016-11-02T04:44:38.156Z| vmx| I120: Received migrate ‘from’ request for mid id 1478061877196270, src ip <10.128.136.110>.
….
…..
2016-11-02T04:44:38.156Z| vmx| I120:    OpType: vmotion
…..
2016-11-02T04:44:38.200Z| vmx| I120: UNAME VMkernel wdc-esx09 6.0.0 #1 SMP Release build-3029758 Aug 31 2015 00:54:00 x86_64 (uwglibc release: vmware, version: 2.12.2)

The above audit trail entries are able to correctly report on the below events
•    VM Power on / off
•    VM vMotion to / from

The same Audit entries can also be captured from the vCenter database by mining the database for VM Power on / off and VM vMotion to / from events. We need to be mindful of the purge retention settings for Oracle/SQL Server vCenter database in order to ensure that we have audit trail entries for at least 2-3 audit cycles.

As we can see by creating a “Compute Enclosure” and establishing a “Effective Audit Mechanism”, we can conclusively day without any doubt that the Oracle VM’ always lived and migrated within the “Compute Enclosure” and never wandered outside !!!.

Tools to help gather audit trail

Another product from VMware which helps for purpose of Oracle Auditing is the VMware vRealize Log Insight which delivers heterogeneous and highly scalable log management with intuitive, actionable dashboards, sophisticated analytics and broad third-party extensibility. It provides deep operational visibility and faster troubleshooting across physical, virtual and cloud environments.

VMware LogInsight dashboard can help customers gather by means of audit trail records which can then be presented to Oracle LMS team as proof of Oracle workload footprint within a vSphere Cluster or a vSAN cluster.

The video below demonstrates the capabilities of VMware vRealize LogInsight for Oracle License Compliance
https://www.youtube.com/watch?v=EHcT4xDyONc

Also keep in mind the below listed controls demanded by licensing zealots is completely un-necessary and non-contractual.

-Not needed to create Network Segmentation to separate and dedicate a network segment for the vSphere Cluster for Oracle workloads

-Not needed to create Storage Segmentation to zone, map and mask Oracle specific storage LUNS to only the ESXI servers  in the dedicated vSphere Cluster for Oracle

-Do not run PowerCLI scripts / commands against the vCenter database which shows all the ESXI servers connected to the vCenter regardless of whether they are part of the vSphere dedicated cluster for Oracle or not.

If you have to run it to gather information about the ESXi servers in the Oracle vSphere Cluster, login as the user who has access to only the Oracle cluster so that way it reduces the scope of discovery to only the Oracle Cluster

This is the document which is handed out to Customers which has information how to gather information about the ESXi servers connected to the Virtual Center , it does not specify running the script against the Oracle vSphere Cluster.

A key point to keep in mind is if this document is really contractual , why is this NOT public facing ?

-Do not give access to any auditor the keys of the kingdom i.e. vCenter username and password

Really, what’s next? Separate the vSphere Cluster for Oracle in its own cage in the data center and ensure no one goes near it!! Throw a black cloth around the cage so that no one can see what’s in it?

Both of the above steps are completely un-necessary as we have well established beyond any reasonable doubt in the previous blog post that Oracle licensing is not Memory, Storage, Cluster, vCenter or Network based, it’s either User based (Named User Plus) or Processor(Socket in case of SE2 or cores in case of EE edition).

Artifacts helpful for an Oracle Licensing Audit defense

Here are some of the important artifacts which are useful for an Oracle Licensing audit defense

1)    Proof of Compute Enclosure
a.    Screenshot of the vSphere dedicated cluster for Oracle Workloads

b.    Screenshot of one of the ESXI servers in the cluster which clearly shows Processor Family, number of Socket and number of Cores

The Effective number of cores calculation can be found in the previous blog post
https://blogs.vmware.com/apps/2017/01/oracle-vmware-vsan-dispelling-licensing-myths.html

2. Audit Trail entries which are log file entries for every Oracle VM which shows the Power on /off and vMotion to / from operations.

VMware LogInsight can be used to extract these entries and the video below demonstrates the capabilities of VMware vRealize LogInsight for Oracle License Compliance:
https://www.youtube.com/watch?v=EHcT4xDyONc

The same Audit entries can also be captured from the vCenter database by mining the database for VM Power on / off and VM vMotion to / from events. We need to be mindful of the purge retention settings for Oracle/SQL Server vCenter database in order to ensure that we have audit trail entries for at least 2-3 audit cycles.

Conclusion
In conclusion, Oracle Licensing Audit should not be taken lightly just as you would for any other software vendor but not special and one does not have to fear it.

Be prepared with all the audit artifacts as detailed above.

Need Help?
For any additional Oracle Licensing on VMware clarification or help, please reach out to your respective VMware Account teams who can get our team involved in a discussion (Internal VMware folks can reach directly to us at the Tier1-Apps-Sales-Support team mailing list) and we can definitely help guide you and connect you to some of our Premier specialist partners for further discussions.

Oracle on VMware SDDC Collateral
All Oracle on vSphere white papers including Oracle licensing on vSphere/vSAN, Oracle best practices, RAC deployment guides, and workload characterization guide can be found in the url below

Oracle on VMware Collateral – One Stop Shop [Customer]
https://blogs.vmware.com/apps/2017/01/oracle-vmware-collateral-one-stop-shop.html

VMworld 2017 Oracle Customer Bootcamps

On a mission to arm yourself with the latest knowledge and skills needed to master application virtualization?

VMworld Customer bootcamps can get you in shape to lead the virtualization charge in your organization, with Instructor-led demos and In-depth course work designed to put you in the ranks of the IT elite.

Oracle on vSphere
The Oracle on VMware vSphere Bootcamp will provide the attendee the opportunity to learn the essential skills necessary to run Oracle implementations on VMware vSphere. The best practices and optimal approaches to deployment, operation and management of Oracle database and application software will be presented by VMware expert Sudhir Balasubramanian who will be joined by other VMware and Industry Experts.

This technical workshop will exceed the standard breakout session format by delivering “real-life,” instructor-led, live training and incorporating the recommended design and configuration practices for architecting Business Critical Databases on VMware vSphere infrastructure. Subjects such as Real Applications Clusters, Automatic Storage Management, vSAN and NSX will be covered in depth.

Learn More

https://www.vmworld.com/en/us/learning/sessions.html?mid=9592&eid=CVMW2000001358867&elqTrackId=ac4f78fd201d4b8ea8c06c94903ec64e&elq=a30d659ad2934a969e912b357d9624d2&elqaid=9592&elqat=1&elqCampaignId=4153

Details

Cost: $725 / seat

Schedule:
Saturday August 26, 2017
8:00am to 5:00pm
(registration opens at 6:30am)

Location:
Mandalay Bay, South Convention Centre
3rd Floor Jasmine Rooms

Registration

Be sure to add the Bootcamp in step 4 of your VMworld conference registration, under Educational Offerings, after you’ve selected your conferences pass.

Registration is open, seating is limited! Lunch and breaks provided.

Looking forward to seeing you all there!