Hortonworks announced on the 17th December 2014 that VMware’s Big Data Extensions tool for Hadoop on virtual machines is now both HDP Certified and Operations Ready. HDP is the Hortonworks Data Platform – an open Hadoop platform that is centered on YARN. The Operations Ready designation is a new certification introduced by Hortonworks to focus attention on those tools that integrate in an approved way with Apache Ambari by making use of the open Ambari management application programming interfaces. The focus of the program is to certify operational tools for managing a Hadoop/HDP cluster. The Operations Ready program also provides assurance to enterprises adopting Hadoop that the tools they select to run and interact with Hadoop have been tested and validated to work correctly. At VMware we are excited to get this additional level of certification for VMware’s BDE and we look forward to continued engineering collaboration with Hortonworks.
Here is the description of the new Operations Ready program from Hortonworks:
BDE 2.1 was announced as being Generally Available in October 2014. One of the central new features in BDE 2.1 is better integration with the de-facto Hadoop management tools from the distro vendors. Chief among those tools is Ambari. This integration with Ambari was the result of a request made to us directly by the VMware BDE user community.
BDE 2.1, with the new application manager construct, can now use the Ambari APIs under the covers to provision the HDP software into the virtual machines that it has created through cloning its template virtual machine. This method of deploying everything through BDE ensures that the resulting new Hadoop cluster is entirely compatible with Ambari. That is important because many of our users would like to use Ambari and VMware vCenter together from the point at which a cluster is provisioned onwards.
Ambari is the management tool of choice among HDP users in order to gain insight into what is going on at runtime at the Hadoop level (e.g. checking the status of HDFS, YARN, MapReduce and other services) and to make service changes there.
VMware vCenter is the virtualization infrastructure management tool that is in use at tens of thousands of VMware’s customers to view system behavior and performance at the virtual infrastructure level (virtual machines, physical machines, consumed resources and performance data). vCenter with the BDE plug-in is in popular use for deploying user Hadoop clusters today at many enterprises.
The BDE plug-in uses the vCenter APIs as well as the Ambari Blueprint APIs. Combining the two tools together to collaborate on the Hadoop provisioning details simplifies the management of your virtualized Hadoop cluster significantly. Both the Hadoop application architect and the virtualization manager can converse about the components of the HDP cluster and their effect on hardware consumption.
A question came across my desk a few days ago around being able to automatically back up VMs that have been migrated by vCenter Site Recovery Manager (SRM). After a bit of thought, it seems this is fairly simple to solve. With SRM, I migrate VMs to a resource pool at my recovery site. Why you might ask? This resource pool is configured with shares set to “High”. This has no impact during normal operations, but when I migrate or fail over important workloads, I want to be sure these workloads have priority if there is contention for CPU and memory. However, this also creates a secondary benefit when it comes to backing up migrated VMs…
It has been 10 months since we released the first set of Virtual SAN Ready Nodes , which are validated server configurations jointly recommended by VMware and Server OEMs to accelerate Virtual SAN deployment. We have been working closely with multiple Server OEM partners to continuously update the list of Virtual SAN Ready Nodes.
The Virtual SAN Ready Node is another great option besides the DIY/Build-your-own option to deploy Virtual SAN, as we had discussed in the past such as in the June 23rd blog.
We have expanded the list from 24 (in June) to 40 Virtual SAN Ready Nodes from eight Server OEMs.
Why should you care about the Virtual SAN Ready Nodes and how do you use them?
A protection group is a group of virtual machines that fail over together to the recovery site. Protection groups contain virtual machines whose data has been replicated by array-based replication or by VR. Typically contains virtual machines that are related in some way such as:
A three-tier application (application server, database server, Web server)
Virtual machines whose virtual machine disk files are part of the same datastore group.
The purpose of the exercise was to demonstrate use cases for disaster recovery of real business critical applications (BCA) leveraging VMware solutions such as VMWare Site Recovery Manager (SRM). Techniques to protect against disaster for common business critical applications such as Microsoft Exchange, Microsoft SQL Server, SAP and Oracle Databases are discussed.
In August 2014, Unisphere Research fielded a study among the members of the Independent Oracle Users Group (IOUG) to examine the current state of Oracle database sites, including the key issues, priorities and solutions being adopted by organizations. A total of 338 qualified responses were collected and analyzed. Respondents came from organizations of all sizes and across various industries. The survey was commissioned jointly by EMC2 and VMware. The sponsors of the survey were not revealed to the respondents.
Below are few of the highlights/excerpts:
VMware vSphere is used for virtualizing Oracle environments substantially more than all other virtualization solutions combined (57% of organizations reported using VMware vSphere, followed by Oracle VM, reported by 9% of organizations). The use of vSphere for Oracle software, while widespread, has been increasing in the last few years, with almost two-thirdsof organizations reporting increases over the past year.
Although many respondents report that the top objection to virtualization is the potential for increased license costs, over 50% of respondents report that cost reduction is a primary benefit of their virtualization effort. A reasonable inference to this finding is that the potential increased licensing costs are more perception than reality, as suggested by reports of organizations with virtualization efforts experiencing significant aggregate cost reductions.
Apart from reduced costs, the most common benefits that organizations report as a result of using virtualization within their Oracle database environments are
Consolidation (54% of respondents)
Standardization of infrastructure (47%)
Greater agility (39%)
Increased automation/reduced provisioning times (26%)
In today’s data centers, three trends drive the need for new approaches to the management of IT systems: the expectation for always-on IT services, rapidly growing infrastructure, and limited IT budgets.
While any one of these trends creates challenges for your IT team, the combination of them makes it clear that the status quo is no longer acceptable when it comes to managing your IT systems. To meet the new expectations for your IT organization, you need to think about new ways of doing things.
That’s the idea behind VMware® vSphere® with Operations Management™. It combines the capabilities of the world’s most trusted virtualization platform with new tools for managing and optimizing your IT environment.
Specifically, vSphere with Operations Management gives you three essential keys for meeting the high expectations for IT in the era of the always-on business:
Tools to enhance performance monitoring and accelerate troubleshooting so you can better meet your service level agreements
Tools to optimize use of your current capacity to increase ROI and reduce the need to buy new hardware
Tools to simplify backup and replication to improve system availability and business continuity
Years ago, you would have had to buy several products to gain all of these capabilities. Today, you can get them all in a single package based on a virtualization platform you know and trust.
Over the next few weeks we will be releasing all of the labs from VMworld 2014 US. The next labs to be released this week cover BC/DR, data center consolidation, on-demand provisioning, network micro-segmentation, security and optimization. And then next week look for more networking and storage labs including NSX, VSAN and VVOL. Our VMware partner solution labs will be released in two weeks.
Be sure to join us in Barcelona, Spain for VMworld 2014 Europe and VMware Hands-on Labs where you can meet with the product experts responsible for the labs and participate in Expert-led Workshops for a guided experience through your lab.