Home > Blogs > VMware vSphere Blog > Category Archives: vCloud Suite

Category Archives: vCloud Suite

VMWorld 2016 Preview – The Software-Defined Data Center – Mission Critical Applications & Databases

Continuing on the theme of making the VMware Software-Defined Data Center real, here is a preview of my abstracts for VMWorld 2016 submitted along with our partners Hitachi Data Systems and NetApp. One session will feature SAP HANA with the Dynamic Tiering option and the other session will feature Oracle 12c with the in-memory option. Both these sessions will showcase full stack SDDC architectures; NSX, vRealize Operations, vROPs Management Packs, and software-defined storage (virtual volumes). For the Oracle session NetApp will be a co-presenter and for the SAP HANA session Hitachi Data Systems will be the co-presenter. Get ready because VMWorld voting opens May 3rd – 24th

Title: The SDDC Stack Day 2 Operations: Oracle 12c RAC Business Intelligence In-Memory Option, SUSE Enterprise Linux, VMware NSX, vRealize Operations – Blue Medora Management Packs, Virtual Volumes on NetApp All Flash Array – AFF8060

Abstract: This session will focus on the Day 2 operations of a fully virtualized Oracle RAC 12c Business Intelligence stack using the in-memory option at multi-terabyte scales, up to 4TB, running SUSE Linux Enterprise Edition 11 on standard Intel x86 servers. The virtualized infrastructure will incorporate several major tenants of the Software-Defined Data Center, compute, network, storage, and operations. We will be deploying VMware NSX, highlighting micro-segmentation techniques by adhering to the network guidelines in the Oracle Enterprise Deployment Reference Topology. The software defined storage will be configured using vSphere 6.0 virtual volumes on a NetApp AFF8060 Flash Array. Day 2 Operational data will be captured and analyzed in VMware vRealize Operations Management and the Blue Medora NetApp vROPs storage management pack and Oracle OEM Adapter.

Title: The SDDC: Full Stack on vSphere SAP Business Warehouse Powered By HANA, NSX, vRealize Operations with Blue Medora Management Packs, SDS – Virtual Volumes on Hitachi Unified Compute Platform and SUSE Linux Enterprise Server.

Abstract: This Software-Defined Data Center is no longer a concept, it is reality. In this session we fully virtualize an industry leading mission critical application and database; SAP Business Warehouse Powered By HANA with the Dynamic Tiering Option running SUSE Linux Enterprise Server for SAP Applications on Intel x86 servers. We will go beyond the use of vSphere to virtualize compute and extend this reference architecture to cover virtual networks and software-defined storage. We will cover the rationale and specific use case behind VMware NSX micro-segmentations for mission critical architectures. We will define and create software-defined storage via VMware Virtual Volumes, using The Hitachi Unified Compute Platform. In addition we will show the value of vRealize Opeations in conjunction with the Blue Medora SAP HANA Management Pack plug-in for vROPs when managing mission critical workloads.

Top Ten things to consider when moving Business Critical Applications (BCA) to the Cloud (Part 3 of 3)

In the first part we looked at public, private and Hybrid Cloud and their characteristics. In this part we will look at the common characteristics of business critical applications. In the second part , we looked at how some of these characteristics relate to the different types of Cloud infrastructure. In this final part we will look at he lifecycle of a business critical application in the cloud and the conclusion. Continue reading

Maintenance Mode Improvements in vSphere 6.0 Update 2

vSphere 6.0 Update 2 has launched and with it comes a very simple change with the way that VMs and Templates are evacuated from hosts that enter Maintenance Mode. In all prior versions, when a host enters Maintenance Mode, DRS will evacuate the host by placing all the running VMs, powered off VMs, and the templates on other hosts within the cluster.  However, under certain conditions the order of operations produces very different results.  For math geeks, (4+2)2 ≠ 4+2×2.  Continue reading

VMware NSX: Virtual Network Architecture for SAP HANA and Mission Critical Deployments

SAP HANA Network Design Considerations

In keeping with the theme of moving the Software-Defined Data Center from concept to reality, I discussed in my previous blogs why VMware vSphere is the perfect platform to deploy cutting edged technologies like SAP HANA. This is because vSphere enables our customers to agilely react to rapidly changing hardware/software requirements by recasting memory, CPU, IO, or network resources where needed in your landscape through software in a centrally managed manner.  I also discussed how VMware Virtual Volumes can be leverage to simplify SAP’s multi-temperature data management strategy; where data is classified by the frequency of access as either hot, warm, or cold depending on data usage. This is an example of the essence of Software-Defined Storage.

Mission Critical Architectures: Completing The Picture with VMware NSX

In this blog I want to discuss how VMware NSX can be leveraged in your SAP HANA Landscapes.  Figure 1.  is an excerpt from the SAP HANA Network Requirements Guide, which kind of goes to the heart of why networks should be virtualized. Now the components of an SAP HANA system communicate via different network channels. Rightfully so, SAP recommended to have a well-defined network topology to control and limit access into only the required access channels in order to apply the appropriate security measures as necessary.

Figure 1. SAP HANA Network Zones

spec

In the Client Zone access is granted to different clients, such as the SQL clients on SAP application servers. In addition there are also browser applications using HTTP/S to access the SAP HANA server, as well as other data sources (such as BI) which need a network communication channel to the SAP HANA database

Continue reading

What’s New in vSphere Big Data Extensions Version 2.3?

Big Data Extensions enables the rapid deployment of Hadoop clusters on a VMware vSphere virtual platform. This new BDE 2.3 release provides the following new features and enhancements.

Continue reading

Eight Myths about Virtualizing Hadoop Dispelled

This article takes eight common misperceptions about virtualizing Hadoop and explains why they are errors in people’s understanding. The short explanations given should serve to clear up the understanding about these important topics.

Myth #1: Virtualization may add significant performance overhead to a Hadoop cluster.

This is a common question from users who are in the early stages of considering virtualizing their Hadoop clusters. Engineers at VMware (and some of its customers) have done several iterations over multiple years of performance testing of Hadoop on vSphere with various hardware configurations. These tests have consistently shown that virtualized Hadoop performance is comparable to, and in some cases better than that of a native equivalent.

Continue reading

Architecting Virtual SAP HANA Using VMware Virtual Volumes And Hitachi Storage

VMWorld Recap: SAP HANA and VMware Virtual Volumes

This is a follow up to my earlier VMWorld blog; “Virtualizing SAP HANA Databases Greater Than-1TB On vSphere-5-5”, where I discussed SAP Multi-Temperature Data Management strategies and techniques which can significantly reduce the size and cost associated with SAP HANA’s in-memory footprint. This blog will focus on Software-Defined Storage and the need for VMware Virtual volumes when deploying Mission Critical Applications/Databases like SAP HANA as discussed in my VMWorld session.

Multi-Temperature Data Management Is By Definition Software-Defined Storage

SAP and VMware customers who plan on leveraging multi-temperature strategies, where data is classified by frequency of access as either hot, warm or cold depending on data usage is the essence of Software-Defined Storage. This can also be equated to EMC’s Information Lifecycle Management which examines the value of data to the business over time. To bring the concept of the Software-Defined Data Center and more precisely Software-Defined Storage to reality, see Table 1. This table depicts the various storage options for SAP HANA so customers can create an architecture that aligns with the business and its applications demands.

Table 1: Multi-Temperature Storage Options with SAP HANA

table-j

Planning Your Journey To Software-Defined Storage

As we get into the various storage options for SAP HANA, VMware has made it very easy to create and deploy software defined storage in the form of Virtual Volumes. However I want to stress the actual definitions of how the storage should be abstracted is a collaborative task, at a minimum you must involve the storage team, VI-Admins, application owners, and dba’s in order to create an optimized virtual architecture; this should not be a siloed task.

In my previous post I discussed the storage requirements for SAP HANA In-Memory, Dynamic Tiering, Near-Line Storage, and the Archiving Components; one last option I did not cover in Table 1 is Data Aging which is specific to SAP Business Suite. Under normal operations SAP HANA does not preload data into memory, data is loaded upon first access, so the first time you access data its always off disk.

With Data Aging you can essentially mark data so its never loaded into memory and will always reside on disk. This is not available on all modules for Business Suite, so please check with SAP for availability and roadmap with respect to Data Aging.

Essentially this is another SAP HANA feature which enables customers to reduce and manage their memory footprint more efficiently and effectively. The use of Data Aging can change the design requirements of your Software-Defined Storage, if Data Aging becomes more prevalent in your SAP Landscape, VMware Virtual Volumes can be used to address the changing storage requirements of the application by seamlessly migrating data between different classes of software-defined storage or VMDKs.

VMware Virtual Volumes Transform Storage By Aligning With SAP HANA’s Requirements

Now lets get into Virtual Volumes and the problems they solve, with Virtual Volumes the fundamental model is centered around provisioning storage based on the application needs rather than the underlying infrastructure. When deploying SAP HANA using the Tailored Data Center Integration model, the storage KPIs can be quite complex, so how do customers translate latency, throughput for reads – writes – and updates, at various block sizes to the storage layer?

Plus how does a customer address the storage requirements for SAP HANA’s entire data life cycle, whether you are planning on using Dynamic Tiering, with or without Near-Line-Storage and what is the archiving strategy storage requirements as well. Also some of the storage requirements do tie back to the compute layer, as an example with Dynamic Tiering if you plan on using Row Level Versioning there is a compute to memory relationship for storage that comes into play when sizing

Addressing and achieving these design goals using an infrastructure centric model can be quite difficult because you are tied to physical LUNs and trust me, with mission critical databases, you will always have database administrators fighting over LUNs with the lowest numbers because of the concerns around radial density. This leads to tremendous waste when provisioning storage using an infrastructure centric model.

VMware Virtual Volumes significantly reduces the storage design complexity by using an Application Centric model because you are not dealing with storage at the LUN level, instead vSphere admins use policies to express the application requirements to the storage array, then the storage array maps storage containers to the application requirements.

What are VMware Virtual Volumes?

At a high level I’ll go over the architecture and components of Virtual Volumes, this blog is not intended to be a deep dive into Virtual Volumes, instead my goal is to convey that mission critical uses cases for VVOLS and software-defined storage are real. For an excellent white paper on Virtual Volumes see; “VMware vSphere Virtual Volumes Getting Started Guide”.

As shown in Figure 1., Virtual Volumes are a new type of virtual machine object which are created and stored natively on the storage array. The Vendor Provider also known as the VASA Provider, which are the vSphere Storage APIs for Storage Awareness (VASA) that provide the storage awareness services and mediates out of the box communications between vCenterServer and EXi Hosts on one side and the storage system on the other side.

The storage containers are pools of raw storage that a storage system can provide to virtual volumes and unlike LUNS and NFS, they do not require pre-configured volumes on the storage side. Also with virtual volumes you still have the functionality you would expect when using native VMDKs

Virtual Datastores represents a storage container in a vCenter Server instance, so it’s a 1:1 mapping to the storage systems storage container. The ESXi Hosts have no direct access to the virtual volumes on the storage side, so they use a logical I/O proxy called a protocol endpoint and as you would expect VVOLs are compatible with industry standard protocols, iSCSI, NFS, FC, and FCoE

The Published Storage Capabilities will vary by storage vendor depending on which capabilities have been exposed and implemented. In this blog we will be looking at the exposed capabilities of Hitachi Data Systems like latency, throughput, Raid Level, Drive Type/Speed, IOPS, and Snapshot frequency to mention a few.

Figure 1: vSphere Virtual Volumes Architecture and Components

vv

VMware HDS: Creating Storage Containers, Virtual Volumes, and Profiles for Virtual SAP HANA

Now Virtual Volumes are an Industry-wide Initiative, essentially a who’s who of the storage industry are participating in this initiative, however this next section will be representative of the work done with Hitachi Data Systems

And again the guidance here is collaboration when architecting software-defined storage for SAP HANA landscapes and for that matter any mission critical application or database. Because the beauty of software defined storage is once created and architecture correctly you can then provision your virtual machines in an automated and consistent manner.

So in the spirit of collaboration, I got together with Hitachi’s SAP alliance team, their storage team, and database architects and we came up with these profiles, policies, and containers to use when deploying SAP HANA landscapes.

We had several goals when designing this architecture; one was to use virtual volumes to address the entire data life cycle of SAP HANA, the in-memory component, Dynamic Tiering, Near-Line storage, and archiving or any supported combination of the above when creating a SAP HANA landscape. And secondly we wanted to enable rapidly provisioning of SAP HANA landscapes, so we created profiles, policies, and containers which could be used to deploy SAP HANA databases whose in-memory component could range from 512GB to 1TB in size.

I’ll review some of the capabilities HDS exposed which were used for this architecture:

  • Interestingly enough we were able to meet the SAP HANA in-memory KPIs using Hitachi Tier 2 storage which consisted of 10K SAS drives for both log and data files, as well as for the Operating System and the SAP HANA shared file system. This also simplified the design. We then used high density SAS drives for the backup areas
  • We enabled automatic storage managed snapshots for HANA data, log and the OS; and set the Snapshot frequency based on the classifications of Critical, Important, or Best Effort.
  • So snapshots for the data and log were classified as Critical while the OS was classified as Important and the backup area we didn’t snapshot at all
  • We also tagged this storage as certified, capturing the model and serial number, since the SAP HANA in-memory component requires certified storage. We wanted to make sure that when creating HANA VM’s you’re always pulling from certified storage containers.
  • The Dynamic Tiering and NLS storage had similar requirements so could be provisioned from the same containers and since these are disk based columnar databases we selected Tier 1 storage SSDs for the data files based on the random read/write patterns
  • And stuck with SAS drives for the log files since sequential workload don’t benefit much from SSDs. Again because of the disk based access we selected Tier 2 to satisfy the IOPS and Latency requirements.
  • Then finally for the archiving containers we used the lowest cost & highest density storage, pretty much just a file system.

Now there’s just too much information to cover in this effort with HDS but for those of you interested, VMware and Hitachi we will be publishing a Co-Logo White Paper which will be a much deeper dive into how we architected these landscapes so customers can do this almost out of the box.

Deploying VMware Software-Defined Storage With vSphere and Hitachi Command Suite

Example: SAP HANA Dynamic Tiering and Near-Line Storage Tiers. These next couple of screen captures will show how simple virtual volumes are to deploy once architected correctly

Figure 2: Storage Container Creation: SAP HANA DT and NLS Tier

ss1

Figure 3: Create Virtual Machine Storage Policies SAP HANA DT/NLS Data/Log File

ss2

Figure 4: Create New SAP HANA DT VM Using VVOLS Policies With Hitachi Storage

ss3

Addressing Mission Critical Use Cases with VMware Software-Defined Storage

SAP HANA and Multi-Temperature Data Management is the poster child for mission critical software-defined storage use cases. VMware Virtual Volumes solves the complexities and simplifies storage provisioning by using an application centric model rather than an infrastructure centric model.

The SAP HANA in-memory component is not yet certified for production use on vSphere 6.0, however Virtual Volumes can be used for SAP HANA Dynamic Teiring, Near-Line Storage, and Archiving. So my advice to our customers is to start architecting now, get together with your storage admins, VI Admins, application owners, and database administrators to create containers, policies, and profiles correctly so when vSphere 6.0 is certified you are ready to “Run SAP HANA Simple”.

 

 

Updating vCenter Server Appliance 6.0 to Update 1

Earlier this month, we released vSphere 6.0 Update 1. In this update we introduced some awesome new features for vCenter Server. Let’s take a look at some of these just below:

  • Installation and Upgrade using HTML 5 Installer for VCSA: The following installation and upgrade scenarios are now supported for vCenter Server Appliance using its HTML 5 installer:
    1. An installation using HTML 5 installer with a vCenter Server target is supported.
    2. An upgrade using HTML 5 installer with a vCenter Server target is not supported.
    3. An upgrade using command line with a vCenter Server target is supported.
  • Backup and Restore with External Platform Services Controller: vCenter Server deployments with an external PSC (also called MxN) have support for backup and restoration.
  • Appliance Management User Interface: An all new HTML5-based management interface for the appliance at https://<FQDN-or-IP>:5480. 
  • Platform Services Controller Interface: An all new HTML5-based management interface for the Platform Services Controller at https://<FQDN-or-IP>/psc/.  See my earlier blog on the Platform Services Controller Interface.
  • Interoperability: Virtual SAN and SMP-FT are interoperable.
  • Hybrid Cloud Manager: Hybrid Cloud Manager has been updated for vSphere, and can be accessed directly from the vSphere Web Client.
  • VCSA Authentication for Active Directory: VMware vCenter Server Virtual Appliance has been modified to only support AES256-CTS/AES128-CTS/RC4-HMAC encryption for Kerberos authentication between VCSA and Active Directory.
  • Support for SSLv3: Support for SSLv3 has been disabled by default.
  • Customer Experience Improvement Program: The opt-in Customer Experience Improvement Program (CEIP) provides VMware with information that enables VMware to improve the VMware products and services and to fix problems. When you choose to participate in CEIP, VMware will collect technical information listed below about your use of the VMware products and services in CEIP reports on a regular basis. This information does not personally identify you.

One additional feature that we introduced in vCenter Server 6.0 Update 1 is an in-place process for Updates in a major release (e.g. vCenter Server 6.0 to vCenter Server 6.0 Update 1) instead of the migration-based approach that was required in prior VCSA updates (e.g. vCenter Server 5.5 to vCenter Server 5.5 Update 1).

With these new capabilities — and, of course, resolved issues — there’s been a ton of interest in how to update the VCSA to 6.0 Update 1. So, let’s get started and look at the process…

Continue reading

Virtualizing SAP HANA Databases Greater than 1TB on vSphere 5.5

VMWorld 2015 Session Recap

I’m almost fully recovered from VMWorld, which was probably one of my busiest and most enjoyable VMWorld’s I’ve had in my 6 plus years at VMware because of the interaction with attendees, customers, and partners.  I’ll be doing a series of Post-VMWorld Blogs focused on my SAP HANA Software-Defined Data Centers sessions but my first blog will cover the misconceptions associated with sizing SAP HANA databases on vSphere. There are many good reasons to upgrade to vSphere 6.0, going beyond the 1TB monster virtual machine limit in vSphere 5.5 when deploying SAP HANA databases is not necessarily one of them.

SAP HANA is no longer just an in-memory database, it is now a data management platform.  It is NOT confined by the size of available memory since the SAP HANA warm data can be stored on disk in a columnar format and accessed transparently by applications.

What this means is the 1TB monster virtual machine maximum in vSphere 5.5 is an artificial barrier. SAP HANA multi-terabyte size databases can be easily virtualized with vSphere 5.5 using Dynamic Tiering, Near-Line Storage, and other memory management techniques SAP has introduced to the SAP HANA Platform to optimize and reduces HANA’s in-memory footprint.

SAP HANA Dynamic Tiering (DT)

SAP HANA Dynamic Tiering was introduced last year in Support Pack Stack (SPS) 09 for use with BW, Dynamic Tiering allows customers to seamlessly manager their SAP HANA disk based “Warm Data” on an Extended Storage Host, essentially placing data which does not need to be in-memory on disk. The guidance SAP gives when using the SAP HANA Dynamic Tiering option for SPS 09 is up to 20% of in-memory data can reside on the Extended Storage (ES) Host, for SPS 10 up to 40% can reside on the ES Host, and in the future up to 70% of the SAP HANA data can reside on the ES Host. So in the future the majority of SAP HANA data which was once in-memory can reside on-disk.

Near-Line Storage (NLS)

In addition to the reduction of the SAP HANA in-memory footprint DT affords customers, Near-Line Storage should be considered as well. With NLS, data is moved outside of the SAP HANA database proper to disk and classified as “Cold”, due to its infrequent accessed and can only be accessed read only. SAP provides examples showing NLS can reduce the HANA database in-memory requirements by several Terabytes (link below).

It is also important to note that both the DT Extended Storage Host and NLS solutions do not require certified servers or storage, so not only has SAP given customers the ability to run SAP HANA in a reduced memory footprint, customers can run on standard x86 hardware as well.

There is a white paper authored by Priti Mishra, Staff Engineer, Performance Engineering VMware, which is an excellent read for anyone considering DT or NLS options. “Distributed Query Processing in SAP IQ on VMware vSphere and Virtual SAN”

Importance of the VMware Software Defined Data Center

To their credit SAP has taken a leadership role with HANA’s in-memory columnar database computing capabilities and as HANA has evolved the sizing and hardware requirements have evolved as well. Rapid change and evolving requirements are givens in technology; the VMware Software Defined Data Center provides a flexible and agile architecture to effectively react to change by recasting compute, network, and storage resources, in a centrally managed manner.

As a concrete example of the flexibility VMware’s Platform provides, Figure 1. illustrates the evolution of SAP HANA from SPS 07 to SPS 09. For customers who would like to take advantage of SAP HANA’s multi-temperature data management techniques but initially deployed SAP HANA on SPS 07 (all in-memory); through virtualization customers can reclaim and recast memory, storage, and network resources in their virtual HANA landscape to reflect the latest architectural advances and memory management techniques in SPS 10.

Figure 1. SAP HANA Platform: Evolving Hardware Requirements

sap hana vmware

Since SAP HANA can now run in a reduced memory footprint, customers who licensed HANA to be all in-memory can use virtualization to reclaim memory and deploy additional virtual databases and make HANA pervasive in their landscapes.

As a general rule, in any rapidly changing environment The VMware Software-Defined Data Center provides an agile platform which can accommodate change and also protect against capital hardware investments that may not be necessary in the future (certified vs. standard x86 hardware). For that matter, the cloud is a good option to deploy any rapidly changing application/database in places like VMware vCloud Air, Virtustream, or Secure-24 just to mention a few.

Virtual SAP HANA Back on track

After speaking with session attendees, customers, and partners, at VMworld about SAP HANA’s Multi-temperature management capabilities, I was happy to hear they will not be delaying their virtual HANA deployments due to the vSphere 6.0 roadmap certification timeline. As I said earlier, the 1TB monster virtual machine maximum in vSphere 5.5 is an artificial barrier. It really is a worthwhile exercise to take a closer look at the temperature of your data, age of your data, and your access requirements in order to take full advantage of all the tools and features SAP provides their customers.

I was also encouraged to hear from many session attendees that my presentation at VMWorld brought the SDDC from concept closer to reality by demonstrating actual mission critical database/application use cases. My future post VMWorld blogs will focus on how I deconstructed the SAP HANA Networks Requirements document and transformed that into a virtual network design using VMware NSX from my desktop. I’ll also cover Software Defined Storage, essentially translating SAP’s Multi-Temperature Storage Options into VMware Virtual Volumes and Storage Containers.

“SAP HANA SPS10- SAP HANA Dynamic Tiering”; (SAP Product Management)

http://www.slideshare.net/SAPTechnology/sap-hana-sps10-sap-hana-dynamic-tiering

“Distributed Query Processing in SAP IQ on VMware vSphere and Virtual SAN”; Priti Mishra, Performance Engineering VMware

https://communities.vmware.com/docs/DOC-27533

Blog: Bob Goldsand; “SAP HANA Dynamic Tiering and the VMware Software Defined Data Center”

https://blogs.vmware.com/vsphere/author/bobgoldsand

 

 

 

Big Data Virtualization: Talks and Related Events at VMworld 2015

Here is a list of the Big Data technical talks and events at VMworld 2015 for your conference planning. The big data team at VMware will be delighted to see you at some or all of these events during the conference coming soon. Please register for these through the schedule builder on the VMworld website.

Sunday, 30th August

4:00pm VAPP6442-QT Quick Talk on VMware and Big Data

Monday, 31st August

12:30pm VAPP4567 – Big Data Partnering – Cloudera and VMware Work Together

3pm EXPERTS  – Meet the Big Data Experts

Tuesday, 1st September

11:00am Book Signing at DigitalGuru Bookshop in Moscone Lobby

12:00pm Theater Talk and Book Signing at the VMware Booth in Exhibition Hall

1:00pm VAPP6428GD – Group Discussion on Big Data

2:00pm INF4566 – A Customer Deployment with Hadoop on vSphere

4:00pm VAPP4588 – Virtualizing Big Data – a Customer Panel

Wednesday, 2nd September

08:30am CNA4725 – Scalable Cloud-Native Apps

3:30pm INF4551 – Customer Case Study: Skyscape’s Hadoop-in-the-Cloud Deployment

Big Data Hands-on-Lab HOL-SDC-1609 – available on each day of the show

Big Data Extensions/Hadoop Demos – at the vSphere/VSOM pod on the VMware booth