Home > Blogs > VMware vSphere Blog > Tag Archives: Virtual SAN

Tag Archives: Virtual SAN

Introducing VMware Hyper-Converged Software

VMware Hyper-Converged Software

Powering the industry’s largest Hyper-Converged Infrastructure ecosystem

Hyper-Converged Infrastructure (HCI) is transforming the way private datacenter infrastructure is being built –see this post for an overview of HCI.  It eliminates the traditional hardware silos of compute, storage and networking, to move all the intelligence into a single software layer running on industry-standard x86 servers.  By doing so, HCI makes private infrastructure a lot simpler, higher performing, and more cost-effective.  In essence, the infrastructure starts looking like the datacenters of web-scale companies such as Google or Amazon.  We’re seeing these benefits play out across thousands of VMware customers that have deployed and expanded their HCI deployments over the past year.

Hyper-Converged Infrastructure

Hyper-Converged Infrastructure relies on both great hardware and great software.  The hardware consists of industry-standard x86 building blocks, serving as the foundation for the entire datacenter.  This hardware convergence relies on critical innovations such as flash and faster CPUs.

At the same time – it’s clear that HCI is first and foremost about the software.  Software innovation is what makes HCI possible.  Compute, storage, networking and management are now delivered as software. For storage specifically – this requires a software-defined, distributed, shared storage model with all the data services typically provided by external SAN or NAS – but all delivered as software on the hypervisor.  This distributed software is very hard to build, hence why only a few vendors are able to pull it off.

Let’s introduce you to VMware Hyper-Converged Software

At VMware, we believe we have an incredibly valuable and innovative set of software assets that enables HCI:

  • vSphere is, of course, the most widely deployed and proven hypervisor in the industry. It also delivers basic Virtual Machine networking capabilities with vSphere Distributed Switch.
  • Virtual SAN provides high-performance, enterprise-class shared storage
  • vCenter Server provides unified management across the stack

Continue reading

Oracle U2VL With Virtual SAN And The Batch Processing Use Case

Unix to Virtualized Linux (U2VL) is a critical step towards SDDC, it targets to migrate applications and data from physical Unix servers to Linux virtual machines running on x86 virtualized infrastructure. These applications are typically business critical, therefore, customers normally take a very cautious approach by doing a carefully planned and executed Proof-of-Concept (POC) in order to validate performance, availability, and scalability, among many other areas.

My colleagues in China (a big shout out to Tony Wang and his team!) recently did one such POC with a large local bank, and naturally they chose Virtual SAN hyper-converged architecture for all of the compute and storage needs. The test results were so illustrative of many of the Virtual SAN benefits, I’d like to share this POC and some of the test results here, although I’m not allowed to mention the customer name due to reasons you probably understand.

Continue reading

VMware Virtual SAN Delivers Enterprise Level Availability

One of the slides we showcased during the VMware Virtual SAN 6.1 Launch that got a lot of attention was the following slide:

Pic 1

A lot of eyebrows in the audience were going up wondering how we came to the conclusion that VSAN delivers 6-9s availability level (or less than 32 seconds of downtime a year). While, Virtual SAN uses software-based RAID, which differs in implementation from traditional storage solutions, it does have the same end result – your data objects are mirrored (RAID-1) for increased reliability and availability. Moreover, with VSAN your data is mirrored across hosts in the cluster not just across storage devices, as is the case with typical hardware RAID controllers.

The VSAN users can set their goals for data availability by means of a policy that may be specified for each VM or even for each VMDK if desired. The relevant policy is called ‘Failures to Tolerate’ (FTT) and refers to the number of concurrent host and/or disk failures a storage object can tolerate. For FTT=n, “n+1” copies of the object are created and “2n+1” hosts are required (to ensure availability even under split brain situations).

For the end user, it is important to quantify the levels of availability achieved with different values of the FTT policy. With only one copy (FTT=0), the availability of the data equals the availability of the hardware the data resides on. Typically, that is in the range of 2-9s (99%) availability, i.e., 3.65 Days downtime/year. However, for higher values of FTT, more copies of the data are created across hosts and that reduces exponentially the probability of data unavailability. With FTT=1 (2 replicas), data availability goes up to at least 4-9s (99.99% or 5 minutes downtime per year), and with FTT=2 (3 replicas) it goes up to 6-9s (99.9999% or 32 seconds downtime per year). Put simply, for FTT=n, more than n hosts and/or devices have to fail concurrently for one’s data to become unavailable. Many people challenged us to show them how the math actually works to arrive at these conclusions. So let’s get to it.

Continue reading

Virtualizing SAP HANA Databases Greater than 1TB on vSphere 5.5

VMWorld 2015 Session Recap

I’m almost fully recovered from VMWorld, which was probably one of my busiest and most enjoyable VMWorld’s I’ve had in my 6 plus years at VMware because of the interaction with attendees, customers, and partners.  I’ll be doing a series of Post-VMWorld Blogs focused on my SAP HANA Software-Defined Data Centers sessions but my first blog will cover the misconceptions associated with sizing SAP HANA databases on vSphere. There are many good reasons to upgrade to vSphere 6.0, going beyond the 1TB monster virtual machine limit in vSphere 5.5 when deploying SAP HANA databases is not necessarily one of them.

SAP HANA is no longer just an in-memory database, it is now a data management platform.  It is NOT confined by the size of available memory since the SAP HANA warm data can be stored on disk in a columnar format and accessed transparently by applications.

What this means is the 1TB monster virtual machine maximum in vSphere 5.5 is an artificial barrier. SAP HANA multi-terabyte size databases can be easily virtualized with vSphere 5.5 using Dynamic Tiering, Near-Line Storage, and other memory management techniques SAP has introduced to the SAP HANA Platform to optimize and reduces HANA’s in-memory footprint.

SAP HANA Dynamic Tiering (DT)

SAP HANA Dynamic Tiering was introduced last year in Support Pack Stack (SPS) 09 for use with BW, Dynamic Tiering allows customers to seamlessly manager their SAP HANA disk based “Warm Data” on an Extended Storage Host, essentially placing data which does not need to be in-memory on disk. The guidance SAP gives when using the SAP HANA Dynamic Tiering option for SPS 09 is up to 20% of in-memory data can reside on the Extended Storage (ES) Host, for SPS 10 up to 40% can reside on the ES Host, and in the future up to 70% of the SAP HANA data can reside on the ES Host. So in the future the majority of SAP HANA data which was once in-memory can reside on-disk.

Near-Line Storage (NLS)

In addition to the reduction of the SAP HANA in-memory footprint DT affords customers, Near-Line Storage should be considered as well. With NLS, data is moved outside of the SAP HANA database proper to disk and classified as “Cold”, due to its infrequent accessed and can only be accessed read only. SAP provides examples showing NLS can reduce the HANA database in-memory requirements by several Terabytes (link below).

It is also important to note that both the DT Extended Storage Host and NLS solutions do not require certified servers or storage, so not only has SAP given customers the ability to run SAP HANA in a reduced memory footprint, customers can run on standard x86 hardware as well.

There is a white paper authored by Priti Mishra, Staff Engineer, Performance Engineering VMware, which is an excellent read for anyone considering DT or NLS options. “Distributed Query Processing in SAP IQ on VMware vSphere and Virtual SAN”

Importance of the VMware Software Defined Data Center

To their credit SAP has taken a leadership role with HANA’s in-memory columnar database computing capabilities and as HANA has evolved the sizing and hardware requirements have evolved as well. Rapid change and evolving requirements are givens in technology; the VMware Software Defined Data Center provides a flexible and agile architecture to effectively react to change by recasting compute, network, and storage resources, in a centrally managed manner.

As a concrete example of the flexibility VMware’s Platform provides, Figure 1. illustrates the evolution of SAP HANA from SPS 07 to SPS 09. For customers who would like to take advantage of SAP HANA’s multi-temperature data management techniques but initially deployed SAP HANA on SPS 07 (all in-memory); through virtualization customers can reclaim and recast memory, storage, and network resources in their virtual HANA landscape to reflect the latest architectural advances and memory management techniques in SPS 10.

Figure 1. SAP HANA Platform: Evolving Hardware Requirements

sap hana vmware

Since SAP HANA can now run in a reduced memory footprint, customers who licensed HANA to be all in-memory can use virtualization to reclaim memory and deploy additional virtual databases and make HANA pervasive in their landscapes.

As a general rule, in any rapidly changing environment The VMware Software-Defined Data Center provides an agile platform which can accommodate change and also protect against capital hardware investments that may not be necessary in the future (certified vs. standard x86 hardware). For that matter, the cloud is a good option to deploy any rapidly changing application/database in places like VMware vCloud Air, Virtustream, or Secure-24 just to mention a few.

Virtual SAP HANA Back on track

After speaking with session attendees, customers, and partners, at VMworld about SAP HANA’s Multi-temperature management capabilities, I was happy to hear they will not be delaying their virtual HANA deployments due to the vSphere 6.0 roadmap certification timeline. As I said earlier, the 1TB monster virtual machine maximum in vSphere 5.5 is an artificial barrier. It really is a worthwhile exercise to take a closer look at the temperature of your data, age of your data, and your access requirements in order to take full advantage of all the tools and features SAP provides their customers.

I was also encouraged to hear from many session attendees that my presentation at VMWorld brought the SDDC from concept closer to reality by demonstrating actual mission critical database/application use cases. My future post VMWorld blogs will focus on how I deconstructed the SAP HANA Networks Requirements document and transformed that into a virtual network design using VMware NSX from my desktop. I’ll also cover Software Defined Storage, essentially translating SAP’s Multi-Temperature Storage Options into VMware Virtual Volumes and Storage Containers.

“SAP HANA SPS10- SAP HANA Dynamic Tiering”; (SAP Product Management)

http://www.slideshare.net/SAPTechnology/sap-hana-sps10-sap-hana-dynamic-tiering

“Distributed Query Processing in SAP IQ on VMware vSphere and Virtual SAN”; Priti Mishra, Performance Engineering VMware

https://communities.vmware.com/docs/DOC-27533

Blog: Bob Goldsand; “SAP HANA Dynamic Tiering and the VMware Software Defined Data Center”

https://blogs.vmware.com/vsphere/author/bobgoldsand

 

 

 

Use VSAN Assessment to validate Virtual SAN’s benefits for your organization

Are you experiencing challenges with your current vSphere storage environment (i.e., performance, capacity constraint, complexity, expensive renewals) or just not sure if VMware Virtual SAN (VSAN) would be a good fit?

Now VMware partners, SEs, or reps can help you with a free VSAN Assessment.

Continue reading

What’s New with Virtual SAN 6.0?

Software-Defined Storage is making waves in the storage and virtual infrastructure fields. Data and infrastructure are intertwined, and when they’re both brought together, companies can cut down on expenses and increase productivity.

Rawlinson Rivera, Principal Architect, Storage and Availability, recently hosted a webinar, discussing how VMware is approaching Software-Defined Storage (SDS) and virtualization in recently announced VMware updates, including updates to VMware Virtual SAN 6.0.

Software-defined storage offers organizations the ability automate, distribute and control storage better than ever before. SDS can provision storage for applications on demand and without complex processes. It also allows for standardized hardware, reducing costs for businesses everywhere.

To bring the customers the best software-defined storage experience to realization, we had to update VMware® Virtual SAN™. And we did just that. With VMware Virtual SAN 6.0, we introduced several new features with SDS in mind:

  • Software-defined storage optimized for VMs
  • All Flash architecture
  • Broad hardware support
  • The ability to run on any standard x86 server
  • Enterprise-level scalability and performance
  • Per-VM storage policy management
  • And a deep integration with the VMware stack

There’s a lot more to unpack from the latest updates to our VMware solutions. For a more in-depth guide to what’s new and how it affects you, watch the webcast here!

Be sure to subscribe to the Virtual SAN blog or follow our social channels at @vmwarevsan and Facebook.com/vmwarevsan for the latest updates.

For more information about VMware Virtual SAN, visit http://www.vmware.com/products/virtual-san.

Virtual SAN VCG Update – New HP, Fujitsu & Quanta Controllers certified for Virtual SAN!

The Virtual SAN product team is pleased to announce that the following controllers are now certified on Virtual SAN and are listed on the Virtual SAN VCG:

6G 6.0 Hybrid:

HP P420

HP P420i

HP P220i

HP P822

Quanta SAS2308

12G 6.0 Hybrid

HP P440 (without hot plug)

12G 5.5 Hybrid

Fujitsu LSI 3008

We now have a total of 80 I/O controllers certified on Virtual SAN 5.5 and 40 I/O controllers certified on Virtual SAN 6.0.

What about the Ready Nodes for platforms based on the above controllers?

We are working closely with our OEM partners such as HP to publish Ready Nodes for both 6G and 12G platforms for Virtual SAN Hybrid and All Flash configurations for both 5.5 and 6.0 and these should be available soon.

What are the controllers that are currently in the certification queue and when do we expect them to get certified?

Please see the attached list of controllers that are currently undergoing Virtual SAN certification

Note:  In many cases, we rely on our partners to provide driver/firmware fixes for controller issues so if there are delays in receiving these updates from partners, the certification timelines may get pushed out.

I have follow up questions on the attached controller list. Who do I reach out to?

As always, if you have questions on Hardware Compatibility for Virtual SAN, please email your queries to vsan-hcl@vmware.com and someone will get back to you soon!

Where do I learn more about the VMware Virtual SAN Certification process?

Please refer to the Virtual SAN Certification blog post for more details:

 

Virtual SAN Certification & VCG Update

The Virtual SAN product team is pleased to announce that last week we released new certified components (I/O controllers, SSDs and HDDs), new Ready Nodes and a new Hardware Quick Reference Guide for Virtual SAN 6.0 along with a new and improved VCG page.  Please see updated links below:

Updated Virtual SAN VCG

Updated Virtual SAN Hardware Quick Reference Guide

Updated Virtual SAN Ready Nodes

 

How many new components and Ready Nodes do we have listed for Virtual SAN 6.0?

We now have 26 I/O controllers, 170 SSDs and 125 HDDs (and counting) supported on Virtual SAN 6.0.   In addition to the Virtual SAN 5.5 Ready Nodes, we have 8 new Ready Nodes for Virtual SAN 6.0  (Cisco – 4 Hybrid, Dell – 1 Hybrid, Hitachi – 1 Hybrid, Super Micro – 1 All Flash & 1 Hybrid).

We expect this list to grow very quickly.  We have a number of components that are currently getting certified and we plan to add new certified devices and Ready Nodes to the VCG on a weekly basis.

 

How does the Virtual SAN Certification process work?

The VMware Virtual SAN team treats hardware certification very seriously.  I/O controllers play a very important part in determining the stability and performance of a Virtual SAN cluster and need to be able to withstand high I/O under stress conditions.

The I/O controllers are put through a rigorous I/O certification process while the HDD, SSD and Ready Nodes  are put through stringent paper qualifications.

We run a I/O controller card through a 3-week-long certification test plan (the certification is done by VMware or by the partner) that stress tests the card across many dimensions, particularly in high load and failure scenarios to ensure the card can withstand the level of I/O pushed down by Virtual SAN even in the most adverse situations (example: rebuilds and resyncs triggered due to host failures).

If there are issues identified, we work closely with our controller vendor/OEM partner to resolve them and re-run the entire test suite after resolution.  Sometimes an updated firmware or driver version addressing the issue is required from the vendors before we can proceed with more testing.

Only controllers that fully pass the test criteria laid out in the above process are listed on the Virtual SAN VCG.

 

Are separate I/O controller certifications required for different releases?

Yes, we require controllers to be recertified whenever any of the following change:

  • Virtual SAN Release version (eg: 5.5 to 6.0)
  • The controller driver version
  • The controller firmware version

We also certify the same controller separately for Virtual SAN All Flash vs Hybrid since the caching and I/O mechanism are different for these two configurations and we expect controllers to behave differently with varying levels of I/O.

 

What about certification of PCIe-SSD devices?

PCIe-SSDs are nothing but SSDs with an on-board I/O controller in a PCIe form factor.  Therefore, these require the same level of due diligence as required by standard I/O controllers.  As a result, we are putting these devices through the same level of rigorous certification as we do for I/O controllers.

VMware is working very closely with partners to certify the first set of PCIe-SSDs for Virtual SAN 6.0 over the coming weeks.

 

What are the new updates to the VCG page?

The Virtual SAN VCG page has been enhanced to allow users to easily build or choose their All Flash configurations in addition to Hybrid configurations.  Since All Flash Virtual SAN requires SSDs of different endurance and performance spec for caching and performance tiers (See Updated Virtual SAN Hardware Quick Reference Guide for details on specs), we have enhanced the VCG to help users easily pick SSDs for the tier they are interested in.

We have also introduced a new SSD filter called “Virtual SAN type” to help easily filter our All Flash vs Hybrid configurations.  Furthermore, we have added a filter called “Tier” to help you filter our Virtual SAN hybrid caching, Virtual SAN All Flash caching and Virtual SAN capacity caching tiers.

The endurance rating for SSDs are now displayed on the VCG in TBW (Terabytes written over 5 years) as opposed to DWPD (Drive Writes Per Day) which was used previously.

 

What are the controllers that are currently in the certification queue and when do we expect them to get certified?

Please see the attached list of controllers that are currently undergoing Virtual SAN certification

Note:  In many cases, we rely on our partners to provide driver/firmware fixes for controller issues so if there are delays in receiving these updates from partners, the certification timelines may get pushed out.

Having said that, we are making good progress on most of the controllers listed in the attached document and expect them to follow our standard certification process.

On a similar note, Ready Nodes  are primarily dependent on the controllers getting certified, so as you see new controllers on the VCG for 6.0 certified, Ready Nodes  including those controllers will follow.

Storage Blog Recap: Top Blogs From February

VMware-Blog-Banner

The third week of every month, we compile a list of the top vSphere Storage posts from the previous month for you to enjoy.

Here are the top storage posts from February:

VMware Virtual SAN 6.0

Rawlinson Rivera announced, and explained the inner workings of, VMware Virtual SAN 6.0 — VMware’s latest software-defined storage product. Virtual SAN 6.0 introduces support for an all-flash architecture and hybrid architectures, among many other innovations. This is a blog post you shouldn’t miss.

vSphere Virtual Volumes

We released vSphere Virtual Volumes (VVOLs) alongside the announcement of vSphere 6.0. In this post, Rawlinson Rivera explains how VVOLs can drive more efficient operational model for external storage.

vSphere APIs for IO filtering

Ken Werneburg takes the time to detail how vSphere APIs for IO filtering (VAIO) can enable partners to filter their technologies into the IO stream of a VM before data is committed to a disk.

VMware Virtual SAN: All-Flash Configuration

Rawlinson Rivera walks us through how admins can configure Virtual SAN 6.0 for an all-flash architecture.

Be sure to subscribe to the Virtual SAN blog or follow our social channels at @vmwarevsan and Facebook.com/vmwarevsan for the latest updates.

For more information about VMware Virtual SAN, visit http://www.vmware.com/products/virtual-san.

Video: Virtual SAN From An Architect’s Perspective

Video: Virtual SAN From An Architect’s Perspective

Have you ever wanted a direct discussion with the people responsible for designing a product?

Recently, Stephen Foskett brought a cadre of technical bloggers to VMware as part of Storage Field Day 7 to discuss Virtual SAN in depth.  Christos Karamanolis (@XtosK), Principle Engineer and Chief Architect for our storage group went deep on VSAN: why it was created, its architectural principles, and why the design decisions were important to customers.

The result is two hours of lively technical discussion — the next best thing to being there.  What works about this session is that the attendees are not shy — they keep peppering Christos with probing questions, which he handles admirably.

The first video segment is from Alberto Farronato, explaining the broader VMware storage strategy.

The second video segment features Christos going long and deep on the thinking behind VSAN.

The third video segment is a run-over of the second.  Christos presents the filesystem implementations, and the implications for snaps and general performance.

Our big thanks to Stephen Foskett for making this event possible, and EMC for sponsoring our session.