The Hadoop-based system running on vSphere that is described here was architected by Rajit Saha, (who provided the material for this blog) and a team from VMware’s IT department.
This article describes the technical infrastructure for a VMware internal IT project that was built and deployed in 2015 for analyzing VMware’s own business data.. Details of the business applications used in the system are not within the scope of this article. The virtualized Hadoop environment and modern analytics project was implemented entirely on the vSphere 6 platform.
The key lesson that we learned from this implementation is that you can start at a small scale with virtualizing big data/Hadoop and then scale the system up over time. You don’t need to wait for a large amount of hardware to become available to get started.
VMware released NSX-v (NSX for vSphere) 6.2 back on August 20, 2015. With its release the NSX team introduced support to use NSX-v as a load balancer for the vSphere Platform Services Controller (PSC) for highly available deployments (Release Notes). This is a key new feature that enables customers to further leverage existing NSX-v deployments to simplify their vSphere infrastructure while providing additional HA capabilities for the PSC. This can be a fairly straightforward undertaking when there is an existing vCenter being used for management (e.g. a management cluster).
There is a second scenario, however, that requires some consideration. What if you’re deploying a new vSphere and NSX-v environment where a management vCenter does not already exist? Romain Decker, a Solution Architect in VMware’s Software-Defined Datacenter (SDDC) Professional Services Engineering team has put together a great blog post on the VMware Consulting Blog that walks through that exact scenario and provides a step-by-step instruction on how to work around this chicken and egg scenario using the ability to easily repoint a vCenter Server to an alternate PSC in vSphere 6.0 Update 1.
To learn more about configuring NSX-v as a load balancer for the vSphere Platform Services Controller, read Romain’s full blog post at:
If you’ve ever tried to watch a product demo video, or tried to use it to show a product to someone else, often times you find yourself trying to pause the video at the exact right moment, and then having scrub backwards or forwards because you missed the timing. At VMware we’ve created an alternative way ot showing demos, which we call Product Walkthroughs. These are web-based demos that let you walk through a scenario screen-by-screen, at your own pace. Each screen has annotations to explain what’s going on and markups that highlight important parts of the screen, both of which can be turned off if you want a clean view.
Although we have created Product Walkthroughs for numerous products and solutions, the ones I want to focus on are for vSphere 6 and vSphere with Operations Management. Both of these provide a great way to learn about these products and their features at your own pace, as well as to show how something works to your colleagues (or bosses). The one on vSphere 6 highlights the features in this major new release, with sections on:
vSphere FT (now with ability to protect VMs with up to 4 vCPUs)
Over the course of the last few months I’ve been working on a pretty massive deployment guide for vCenter Server 6, the result turned into a 100 page guide. Before getting scared off by the size the guide it goes into details for installing and upgrading many different scenarios including new installs and upgrades from the most common configurations.
The latest in our series of reference architectures is now available. This is an update to the previous version which brings in additional products and covers the vCloud Suite 5.8 release.
This reference architecture describes an implementation of a software-defined data center (SDDC) using VMware vCloud® Suite Enterprise 5.8, VMware NSX™ for vSphere® 6.1, VMware IT Business Management Suite™ Standard Edition 1.1, and VMware vCenter™ Log Insight™ 2.0 to create an SDDC. This SDDC implementation is based on real-world scenarios, user workloads, and infrastructure system configurations. The configuration uses industry-standard servers, IP-based storage, and 10-Gigabit Ethernet (10GbE) networking to support a scalable and redundant architecture.
If you are like me, VMworld 2014 in San Francisco left my brain on overload. With so many new product and services announcements, plus breakout sessions filled with technical information and demos in the expo floor booths, it’s hard not to feel like you are drinking from the proverbial fire hose. Throw in a party or three, plus the pile of work waiting when you get home and all that great info you gathered starts to turn up with some CRC errors in your memory.
Fortunately, VMware has a tool that you can use to refresh your memory on the VMware solutions and services that you explored at VMworld. The VMware Feature Walkthrough site (http://featurewalkthrough.vmware.com) provides technical overviews and step-by-step guidance for installing, configuring and managing our solutions. Each walkthrough includes screen shots with relevant steps highlighted and text explaining the process.
The Feature Walkthrough site is a great for stepping through a self-paced demo of a particular VMware product or feature. We’ve made the site mobile friendly, so go ahead and open it on your tablet and take it into the data center to guide your proof-of-concept install of the products you saw or heard about during VMworld, or to simply refresh your memory on just where exactly that checkbox to enable a product feature is. Use it to show your boss that cool feature you saw at VMworld, or to familiarize yourself with the basics of a product before you jump into a live Hands-on Lab environment.
Virtual SAN is a scale-out hypervisor-converged software-defined storage solution that can scale up to 32 nodes in a cluster. The network serves as the backbone of a Virtual SAN cluster by providing the interconnects between nodes via an Ethernet network topology.
There are many design options when creating a vSphere network design that includes VMware Virtual SAN. Proper network design is critical to ensure the performance and availability of both your Virtual SAN cluster, and other vSphere services and workloads utilizing the network. A holistic approach that considers all the network services within your vSphere cluster should be taken when planning networking for Virtual SAN. The VMware Virtual SAN Network Design Guide reviews design options, best practices, and configuration details, including but not limited to the following –
vSphere Teaming Considerations – IP Hash vs other vSphere teaming algorithms
Physical Topology Considerations – Impact of Spine/Leaf vs Access/Aggregation/Core topology in large scale Virtual SAN clusters
Virtual SAN Network Design for High Availability – Design considerations to achieve a highly available Virtual SAN network
Load Balancing Considerations – How to achieve aggregated bandwidth via multiple physical uplinks for Virtual SAN traffic in combination with other traffic types
Virtual SAN with other Traffic Types – Detailed architectural examples and test results of using Network IO Control with Virtual SAN and other traffic types
I’m pleased to announce the first in a series of reference architectures is now available.
This reference architecture showcases the integrations between VMware vCloud® Suite Enterprise, VMware NSX for vSphere®, and VMware vCenter Log Insight to create an on-demand infrastructure with a secure networking environment. It is based on real-world scenarios, user workloads, and infrastructure system configurations. It uses industry-standard servers, IP-based storage, and 10-Gigabit Ethernet (10GbE) networking to support a scalable and redundant architecture based on vCloud Suite Enterprise version 5.5.
Just in time and right before everyone is off on a long 4th of July weekend here in the good old U.S. ofA, I wanted to share a integration demo that I’ve been holding for some time now. Hopefully everyone can see the fireworks delivered by the demo as well.
In this demonstration we’re showcasing the advanced IAAS features and deep integration of vSphere with Virtual SAN, and NSX using Openstack as the Cloud Management Portal for a multi tenant IAAS platform. To prove our point here, this is not just some isolated lab environment, this is a real environment running today and its leveraging currently available technologies.
The environment utilized in this demonstration is actually the NSBU internal cloud which has over 200 environment as a mix of KVM and vSphere. Virtual SAN is used for all vSphere data stores and NSX is used for all tenant connectivity with OpenStack providing a scalable and secure multi-tenant, multi-hypervisor environment.
This demonstration showcases the agility and flexibility of the integration capabilities of vSphere, NSX and Virtual SAN. In the demonstration we rapidly standup of a two tier ‘application’ and demonstrate the connectivity between all elements of the virtual machines providing the applications.
When complete, all instances, networks and routers are decommissioned and the tenant is returned to an ‘empty state’. The whole process takes less than 10 minutes (as can be seen in the instance uptime section in the horizon UI).