Big Data Extensions enables the deployment of Hadoop and HBase clusters in virtual machines on the VMware vSphere platform. This article gives you a brief introduction to the new features in BDE version 2.1. BDE ships as a virtual appliance (an OVA file) and it is a free download for users of vSphere Enterprise or Enterprise Plus.
BDE users are interested in using their favorite management tools from their Hadoop distro vendors, along with BDE and vCenter, to manage their newly created virtualized Hadoop clusters. The 2.1 release of BDE implements this feature in an elegant way!
Now you can use BDE and Cloudera Manager or Ambari together to install and manage your Hadoop clusters without leaving your Web Client BDE seat. You can also use the earlier styles of provisioning a Hadoop cluster as shown under the “BDE Only” and “BDE 2.0″ headings below. The first method on the left allows BDE to use a repository to install the Hadoop vendor’s software on to the virtual machines. BDE does the whole job of provisioning everything in this case – hence referred to as “BDE Only”.
Using BDE 2.0 (shown in the center column) you can create a basic cluster, i.e. one with no Hadoop software in it. Then you can use the Hadoop vendors’ installation and configuration tool to install the Hadoop software on those virtual machines. With BDE 2.1 you don’t have to go between the different tools; the full Hadoop installation can be done inside BDE’s user interface, but using the vendor’s APIs under the covers to do that. The difference between the BDE 2.0 and 2.1 methods is that in 2.1 the management tool from the Hadoop vendor is called by BDE directly.
This new blog series will focus on Virtual SAN day-to-day operations related tasks and their recommended operating procedures. I will start the series by covering one of the key and most important aspects of Virtual SAN, which is the management of disk groups.
Managing Disk Groups
Disk groups are logical management constructs designed to aggregate and manage locally attached flash devices and magnetic disks on ESXi hosts. When disk groups are created the flash devices are utilized to create a performance (caching) layer, while magnetic disks are utilized to create the persistent storage layer and provide storage capacity.
Creating Disk Groups
Disk groups are individually created on every host that is a member of a Virtual SAN enabled cluster. Creating a disk group requires the existence of a single flash device and a single magnetic disk at the very least. A disk group supports a maximum of one flash device, and up to seven magnetic disks.
Disk groups can be created through the vSphere Web Client as well as the command line interface utilities such as esxcli after the Virtual SAN feature has been enabled in a cluster. The vSphere Web Client presents the simplest method for small environments, while command line utilities such as esxcli can provide automation capabilities for large environments.
Welcome to part 2 of the vSphere Storage Policy Based Management Overview. In our previous article, we looked at challenges with traditional storage provisioning models, the advantages of the Software-Defined Storage model, as well as an introduction and background to VMware vSphere Storage Policy Based Management. If you have not yet had opportunity to read it, it might be beneficial to glance through before continuing on.
In today’s article, we will be carrying on with the vSphere Storage Policy Based Management (SPBM) theme as we look to understand the components of the vSphere Storage Policy. Afterwards we will display a few policy examples for single VM provisioning and options for a collection of VMs as well.
I recently published an article describing the procedure to configure and enable the VMware Virtual SAN Observer tool to work without the requirement of Internet access. This operating mode is known as the offline-mode.
The Virtual SAN Observer tool provides three different options for monitoring statistics. The offline monitoring option is probably the most utilized option out of the three as it presents the most flexibility for archiving and data transportability. A brief description of the Virtual SAN Observer monitoring modes is listed below.
Virtual SAN Observer Monitoring Modes
Live Monitoring – This mode displays the performance statistics in real time as they are being generated by the system.
Offline Monitoring – This mode provides the ability to create a tar.gz package with html bundle which can be utilized for archiving purposes or future inspection.
Full raw status bundle – This mode provides the ability to collect all the stats into a large JSON file for deeper analysis.
With this release, we now have the ability to interface with the vSphere Storage Policy Manager through the addition of the new VMware.VimAutomation.Storage snap-in. This snap-in provides PowerCLI cmdlets that let you manage vSphere policy-based storage from the PowerCLI command line or by automating through PowerCLI scripting.
In this blog series we will look to provide indepth coverage along with real-world scripting examples for each of the cmdlets. All scripts provided will be examples only and unsupported however I do validate each script with great scrutiny in multiple testing environments so you may not require much adaptation, if any, if you choose to leverage them in your own environments. As always, please ensure all coding is validated in a non-production environment prior to production deployment.
Learn about how customers are rapidly adopting Pivotal CF on VMware, the enterprise Platform-as-a-Service (PaaS) that enables customers to experience turnkey PaaS capabilities on familiar vSphere and now vCloud Air environments.
Jointly hosted by VMware and Pivotal, the speakers are:
Jay Marshall, who is a Principal Cloud Development Strategist at VMware, who has spent almost 20 years working in enterprise application development and specializes in next generation application architecture on VMware’s vCloud Air cloud platform.
Rosie Pongracz, who leads Pivotal CF and PaaS Product Marketing at Pivotal, with over 20 years of experience bringing enterprise technology to market.
I am sure most of you have heard about the “Shellshock” vulnerability – if not, you can read about it here. Seeing that the vSphere Replication 5.8 virtual appliance is running Linux, a patch is required. This short blog article shows how to fix this issue in vSphere Replication 5.8. To review more details on this security advisory, please see this page.
A few days ago there was an announcement on some security vulnerabilities potentially affecting Linux based machines (see here: VMware Security Advisory). Well, today we announced the availability of more patches for various versions of VMware Orchestrator here: VMware Orchestrator Express Patch.
Please refer to the knowledge base articles above for steps to upgrade and/or patch your current systems that may be vulnerable. Note that if you are running VMware Orchestrator installed on Windows, this machine is not impacted by this vulnerability.
Check here for the download page for these relevant patches.
In August 2014, Unisphere Research fielded a study among the members of the Independent Oracle Users Group (IOUG) to examine the current state of Oracle database sites, including the key issues, priorities and solutions being adopted by organizations. A total of 338 qualified responses were collected and analyzed. Respondents came from organizations of all sizes and across various industries. The survey was commissioned jointly by EMC2 and VMware. The sponsors of the survey were not revealed to the respondents.
Below are few of the highlights/excerpts:
VMware vSphere is used for virtualizing Oracle environments substantially more than all other virtualization solutions combined (57% of organizations reported using VMware vSphere, followed by Oracle VM, reported by 9% of organizations). The use of vSphere for Oracle software, while widespread, has been increasing in the last few years, with almost two-thirdsof organizations reporting increases over the past year.
Although many respondents report that the top objection to virtualization is the potential for increased license costs, over 50% of respondents report that cost reduction is a primary benefit of their virtualization effort. A reasonable inference to this finding is that the potential increased licensing costs are more perception than reality, as suggested by reports of organizations with virtualization efforts experiencing significant aggregate cost reductions.
Apart from reduced costs, the most common benefits that organizations report as a result of using virtualization within their Oracle database environments are
Consolidation (54% of respondents)
Standardization of infrastructure (47%)
Greater agility (39%)
Increased automation/reduced provisioning times (26%)
Welcome to another episode of our Virtual SAN Troubleshooting series. In our last article we detailed guidelines and troubleshooting steps around the Virtual SAN networking requirement for layer 2 multicast. In today’s article we will show you how to quickly automate the modification of the Virtual SAN multicast group address in the event the need arises.
PowerCLI 5.8 release 1
- (Note: It is likely to work with PowerCLI version 5.5 or above however I just happened to have version 5.8 on my test system).