Home > Blogs > VMware vSphere Blog

Database-as-a-Service (DBaaS): Reference Architecture with VMware and Tintri

Database-as-a-Service (DBaaS) is a real-world example of cloud computing that delivers databases and database applications through self-service portals without IT intervention. Providing multiple copies of relational database servers for testing and development is traditionally a complex operation that involves the combined efforts of multiple teams and the creation of custom scripts. Yet the ability to quickly provision instances of Oracle and SQL Server databases can reduce the time to create, test, deliver, and deploy new applications.

A study of DBAAS with VMware vRealize Automation (vRA), Tintri storage done by VLSS highlights the capabilities of these platforms to do this efficiently and in an automated manner. The DBaaS reference architecture is only useful if end-users can deploy a catalog of VMs under varying conditions and workloads. These test results highlight three accomplishments in the vRA DBaaS reference architecture with Tintri.

1. VAAI enabled offload allowed the Tintri storage array cloning feature to drop the average time to provision 50 VMs from 17 minutes to 6 minutes; a 65% drop.
2. The DBaaS provisioning times were low and stable even as the load is increased significantly on the DBaaS systems.
3. Even when an extreme workload is applied to the Tintri VMstore storage array, the DBaaS provisioning times were consistently low.

The attached White Paper goes into detail on this study.

Virtual Volumes: A game changer for operations of virtualized business critical databases

This is first of a series of posts on deploying vSphere Virtual Volumes for Tier 1 Business Critical Databases. Although this article is written with a focus on Oracle databases, much of this discussion holds good for any Mission critical application.
Business critical databases are among the last workloads virtualized in enterprises, primarily because of the challenges that they pose with growth and scale. Typically the low hanging fruits are virtualizing the Development, Testing/QA, Staging databases after running a successful POC and then moving on the big guy’s i.e. the Production databases.

There are many common concerns about virtualizing business critical databases that inhibit and delay virtualization of these workloads:
• Business critical virtualized databases need to meet strict SLAs for performance and storage has traditionally been the slowest component
• Databases grow quickly, while at the same time there is a need to reduce backup windows and their impact on system performance.
• There is a regular need to clone and refresh databases from production to QA and other environments. However, the size of the modern databases make it harder to clone and refresh data from production to other environments
• Databases of different levels of criticality need different storage performance characteristics and capabilities.
• There is a never-ending debate between DBAs and Systems administrators regarding filesystems VS raw devices and VMFS VS RDM. These are primarily due to some of the deficiencies that existed in the past with virtualization.
Levels of database operations on VMware environments

Generally speaking there are 3 levels from which regular database operations (i.e. backup, cloning, etc.) can be triggered: application level, vSphere level, and storage level.

Furthermore, each approach has benefits but also drawbacks. For instance, application level operations (Oracle RMAN, SQL) may provide finer operation granularity but performance is not optimal. vSphere level operations offer VM granularity but a VM level snapshot will stun a VM for some time during snapshot coalescing/deletion (KB 1002836: A snapshot removal can stop a virtual machine for long time). Finally, storage level operations offer better performance but lack VM granularity as operations are executed at LUN level.

The ideal solution to address database operation challenges

An ideal solution would combine the built-in storage capabilities with the granularity of VM-level operations, like snapshots. More specifically:
• The solution should be able to trigger backups and clones with VMDK granularity at the same time.
• Do a storage level snapshot triggering the operation at the VM level, which is the fastest and the ideal among all the three above solutions.
• The solution would allow different database components to be aligned with different storage data services needed.

vSphere Integrated Containers – Technology Walkthrough

vSphere Integrated Containers (VIC) combines the agility and application portability of Docker Linux containers with the industry-leading virtual infrastructure platform, offering hardware isolation advantages along with improved manageability. VIC consists of several different components for managing, executing, and monitoring containers. This post delves deeper into key elements of VIC – for more information, please also see this introductory video:


Continue reading

SIOC: I/O Distribution with Reservations & Limits – Part 2

Part 1 of this series explains the new reservation capabilities of the ESXi storage scheduler in vSphere 6.0 called mClock.  That article explains how to calculate the number of entitled IOPS during times of contention.  This article will expand on that topic with a couple new scenarios.  The previous article assumed that all the VMs were evenly consuming the storage resources at the same time.  In the real-world though, some VMs will be consuming resources while others will be idle.  This should help explain how the IOPS are distributed when there are idle VMs in the environment.

Scenario 3
In this scenario the third VM is idle, while the other 3 VMs are consuming storage IOPS.  For the sake of this example, it will be assumed that VM3 will be consuming only 10 IOPS.

8000 IOPS

Unlike memory reservations, the storage scheduler will allow the unused resources to be consumed by other VMs.

The first step is to determine what percentage of the resources each host will receive. In this example there are a total of 5000 shares across all hosts.  Then you would calculate how many shares are assigned to each host to determine the percentage each host will receive.  In this example, Host 1 has 3500/5000 (70%) of the shares, and host 2 has 1500/5000 (30%) of the shares.  This will result with the following entitled IOPS for each host.

Host1: 70% * 8000 IOPS = 5600 IOPS
Host2: 30% * 8000 IOPS = 2400 IOPS

Once the I/O distribution for the hosts are calculated, the VMs will have their entitled resources calculated using the share distribution within the host.

VM1: (1000/3500) * 5600 = 1600 IOPS
VM2 (2500/3500) * 5600 = 4000 IOPS
VM3: (500/1500) * 2400 = 800 IOPS (Only using 10 IOPS)
VM4: (1000/1500) * 2400 = 1600 IOPS

Since VM3 is only using 10 IOPS, the 790 unused IOPS would be distributed to the remaining VMs on the host.  In this case, VM4 would be entitled to 2390 IOPS.  However, VM4 has a limit of 2000 IOPS, which means that there will be 390 IOPS that can still be distributed.  Those 390 IOPS will then be distributed across the VMs on Host1.

In the end, this is how the IOPS allocation would be distributed:

VM1: 1600 + ((1000/3500) * 390) = 1711 IOPS
VM2: 4000 + ((2500/3500) * 390) = 4279 IOPS
VM3: 10 IOPS
VM4: 2000 IOPS (Due to limit)

Scenario 4
Now let’s take the same environment, but calculate the effective IOPS if VM1 was the idle VM. Again, for the sake of this example, the idle VM will be consuming 10 IOPS.

8000 IOPS

The first thing to do is calculate the percentage of the resources each host will receive. In this example there are total 5000 shares across all hosts. Since the environment has not changed, the entitled IOPS per host is unchanged from the previous example.

Host1: 70% * 8000 IOPS = 5600 IOPS
Host2: 30% * 8000 IOPS = 2400 IOPS

Once the I/O distribution for the hosts are calculated, the VMs will have their entitled resources calculated using the share distribution within the host.

VM1: (1000/3500) * 5600 = 1600 IOPS (Only using 10 IOPS)
VM2 (2500/3500) * 5600 = 4000 IOPS
VM3: (500/1500) * 2400 = 800 IOPS
VM4: (1000/1500) * 2400 = 1600 IOPS

Since VM1 is only using 10 IOPS, the 1590 unused IOPS would be distributed to the remaining VMs on the host.  In this case, VM2 would be entitled to 5590 IOPS.  However, VM2 has a limit of 5000 IOPS, which means that there will be 590 IOPS that can still be distributed.  Those 590 IOPS will then be distributed across the VMs on Host2.

In the end, this is how the IOPS allocation would be distributed:

VM1: 10 IOPS
VM2: 5000 IOPS (Due to limit)
VM3: 800 + ((500/1500) * 590) = 997 IOPS
VM4: 1600 + ((1000/1500) * 590) = 1993 IOPS

Hopefully this helps explain how entitled IOPS are calculated and distributed using the mClock storage scheduler in vSphere 6.0.  The important thing to take away is that unused IOPS are not held and wasted, and they distributed across the environment automatically providing the most efficient use of your resources.

Reconfiguring and Repointing Deployment Models in vCenter Server 6.0 Update 1

In my last blog post, we discussed some of the new features and capabilities found in vCenter Server 6.0 such as how you can quickly and easily update the vCenter Server Appliance 6.0 to Update 1.

Now, it’s time to focus our attention on a two key enhancements found in vCenter Server 6.0 Update 1 – both the appliance and Windows-based form factors:

  • Reconfigure – You can now reconfigure an embedded deployment node to an external deployment model, also known as MxN.reconfigure
  • Repoint – Simplified repointing of a management node in an external deployment model from one external Platform Services Controller to another external Platform Services Controller.
  • repoint

Why is this important?

The reconfiguration enhancement enables you to take an existing embedded deployment and transition it to a more optimal external deployment model – MxN.  There is also the simplified ability to repoint a management node to another Platform Services Controller which enables you to quickly recover from an external Platform Services Controller failure and to distribute load to alternate nodes that are in the same SSO domain.

Before moving forward with either the reconfigure or repoint operations, there is a key set of requirements that you need to meet.

Reconfiguration Requirements

  • The vCenter Server instance must be an embedded deployment model.
  • The target Platform Services Controller must be a replication partner of the existing embedded Platform Services Controller in the same SSO Domain.

Note: In vCenter Server 6.0 Update 1, we only support a single transition from embedded deployment to a external deployment (MxN) model for per SSO domain. See the Known Issues section of the Release Notes for additional details.

Repointing Requirements

  • The vCenter Server instance must be an external deployment model.
  • The target Platform Services Controller must be a replication partner of the existing external Platform Services Controller in the same SSO Domain.

We’ve introduced an update to cmsso-util in vCenter Server 6.0 Update 1. This utility can be found in:

  • VCSA: /bin/cmsso-util
  • Windows: <Drive>:\Program Files\VMware\vCenter Server\bin\cmsso-util

This utility automates the entire process by passing the new required namespace (either reconfigure or repoint) and its arguments.  For example, with the VCSA, the namespaces would be:

  • VCSA: /bin/cmsso-util reconfigure
  • VCSA: /bin/cmsso-util repoint

Okay, so, how do we do it? Well, let’s see both namespace options in action in the vCenter Server Appliance (VCSA). Note that the cmsso-util namespaces and arguments are the same for a vCenter Server 6.0 Update 1 instance installed on Windows.

Continue reading

Updating vCenter Server Appliance 6.0 to Update 1

Earlier this month, we released vSphere 6.0 Update 1. In this update we introduced some awesome new features for vCenter Server. Let’s take a look at some of these just below:

  • Installation and Upgrade using HTML 5 Installer for VCSA: The following installation and upgrade scenarios are now supported for vCenter Server Appliance using its HTML 5 installer:
    1. An installation using HTML 5 installer with a vCenter Server target is supported.
    2. An upgrade using HTML 5 installer with a vCenter Server target is not supported.
    3. An upgrade using command line with a vCenter Server target is supported.
  • Backup and Restore with External Platform Services Controller: vCenter Server deployments with an external PSC (also called MxN) have support for backup and restoration.
  • Appliance Management User Interface: An all new HTML5-based management interface for the appliance at https://<FQDN-or-IP>:5480. 
  • Platform Services Controller Interface: An all new HTML5-based management interface for the Platform Services Controller at https://<FQDN-or-IP>/psc/.  See my earlier blog on the Platform Services Controller Interface.
  • Interoperability: Virtual SAN and SMP-FT are interoperable.
  • Hybrid Cloud Manager: Hybrid Cloud Manager has been updated for vSphere, and can be accessed directly from the vSphere Web Client.
  • VCSA Authentication for Active Directory: VMware vCenter Server Virtual Appliance has been modified to only support AES256-CTS/AES128-CTS/RC4-HMAC encryption for Kerberos authentication between VCSA and Active Directory.
  • Support for SSLv3: Support for SSLv3 has been disabled by default.
  • Customer Experience Improvement Program: The opt-in Customer Experience Improvement Program (CEIP) provides VMware with information that enables VMware to improve the VMware products and services and to fix problems. When you choose to participate in CEIP, VMware will collect technical information listed below about your use of the VMware products and services in CEIP reports on a regular basis. This information does not personally identify you.

One additional feature that we introduced in vCenter Server 6.0 Update 1 is an in-place process for Updates in a major release (e.g. vCenter Server 6.0 to vCenter Server 6.0 Update 1) instead of the migration-based approach that was required in prior VCSA updates (e.g. vCenter Server 5.5 to vCenter Server 5.5 Update 1).

With these new capabilities — and, of course, resolved issues — there’s been a ton of interest in how to update the VCSA to 6.0 Update 1. So, let’s get started and look at the process…

Continue reading

VMware Tools Lifecycle: Why Tools Can Drive You Crazy (and How to Avoid it!)

There has been a lot of buzz around vSphere Lifecycle since VMworld. My last few blog posts on VMware Tools have had a tremendous amount of traffic, so I decided to continue with the theme and give you all what it appears you want more of. So in this post, LET’S TALK TOOLS!

Continue reading

Introducing the Platform Services Controller Interface in vCenter Server 6.0 Update 1

Back in March, we introduced vSphere 6.0 and the new architecture for vCenter Server. With this new architecture you learned about the Platform Services Controller, a new functional component of vCenter that moves beyond just Single-Sign On to include additional platform services such as:

  • Licensing Service
  • Certificate Authority (VMCA)
  • Certificate Store (VECS)
  • Lookup Service for Component Registrations

In the 6.0 release, administration and configuration of the Platform Service Controller was primarily performed by an SSH session, the vSphere Web Client and selecting the node in System Configuration, or through the Direct Console User Interface of the appliance.

In vCenter Server 6.0 Update 1, we’re excited to introduce the next stage of the administration with the Platform Services Controller Interface, a fully HTML5-based interface to administer and configure many of the services that run on the PSC.

Using the Platform Services Controller Interface you can perform tasks, such as:

  • Adding and Editing Users and Groups for Single Sign-On
  • Adding Single Sign-On Identity Sources
  • Configuring Single Sign-On Policies (e.g Password Policies)
  • Adding Certificate Stores
  • Adding and Revoking Certificates

Here is a quick overview of the Platform Services Controller User Interface available in vCenter Server 6.0 Update 1.

Continue reading

Big Data on vSphere with HBase

This article describes a set of performance tests that were conducted on HBase, a popular data management tool that is frequently used with Hadoop, running on VMware vSphere 6 and provisioned by the vSphere Big Data Extensions tool. The work described here was done by Xinhui Li, who is a staff engineer in the Big Data team in VMware’s R&D Labs in Beijing. Xinhui’s biography and background details are given at the end of the article.

What is HBase?

HBase is an Apache project that is designed to handle very large amounts of data on the Hadoop platform. HBase is often described as providing the functionality of a NoSQL database running on top of Hadoop. It combines the scalability of Hadoop, through its use of the Hadoop Distributed File System (HDFS) to store the data, with real-time data access to the data. HBase can handle billions of rows of data and very large numbers of columns. Along with Hadoop, HBase runs on clusters of commodity hardware that form a distributed system. The HBase architecture is made up of RegionServers that run on the worker nodes while the HBase Master Server controls them.

Continue reading

Web-based Management for the VCSA is Back!

When vSphere 6.0 launched, there was a celebration. But one beloved feature didn’t make it to the party: The vCenter Server Appliance Management Interface, or more fondly known as the VAMI. The VAMI was the administration web interface included with previous releases of the VCSA that was used to perform basic administrative tasks to the appliance configuration. These tasks included changing the host name, the network configuration, NTP configuration, and applying patches and updates.

Well, good news! The VAMI back! However, it was given a new name while on sabbatical. Its new name is the Appliance Management User Interface. This management UI is written in HTML5 and looks leaps and bounds better than past versions. The Appliance Management UI (Appliance MUI) can set the same appliance configuration settings as the appliancesh commands that are performed from the command line.

Here is a quick overview of the new Appliance Management User Interface.

Continue reading