Home > Blogs > VMware vSphere Blog > Tag Archives: Virtual SAN

Tag Archives: Virtual SAN

Virtualizing SAP HANA Databases Greater than 1TB on vSphere 5.5

VMWorld 2015 Session Recap

I’m almost fully recovered from VMWorld, which was probably one of my busiest and most enjoyable VMWorld’s I’ve had in my 6 plus years at VMware because of the interaction with attendees, customers, and partners.  I’ll be doing a series of Post-VMWorld Blogs focused on my SAP HANA Software-Defined Data Centers sessions but my first blog will cover the misconceptions associated with sizing SAP HANA databases on vSphere. There are many good reasons to upgrade to vSphere 6.0, going beyond the 1TB monster virtual machine limit in vSphere 5.5 when deploying SAP HANA databases is not necessarily one of them.

SAP HANA is no longer just an in-memory database, it is now a data management platform.  It is NOT confined by the size of available memory since the SAP HANA warm data can be stored on disk in a columnar format and accessed transparently by applications.

What this means is the 1TB monster virtual machine maximum in vSphere 5.5 is an artificial barrier. SAP HANA multi-terabyte size databases can be easily virtualized with vSphere 5.5 using Dynamic Tiering, Near-Line Storage, and other memory management techniques SAP has introduced to the SAP HANA Platform to optimize and reduces HANA’s in-memory footprint.

SAP HANA Dynamic Tiering (DT)

SAP HANA Dynamic Tiering was introduced last year in Support Pack Stack (SPS) 09 for use with BW, Dynamic Tiering allows customers to seamlessly manager their SAP HANA disk based “Warm Data” on an Extended Storage Host, essentially placing data which does not need to be in-memory on disk. The guidance SAP gives when using the SAP HANA Dynamic Tiering option for SPS 09 is up to 20% of in-memory data can reside on the Extended Storage (ES) Host, for SPS 10 up to 40% can reside on the ES Host, and in the future up to 70% of the SAP HANA data can reside on the ES Host. So in the future the majority of SAP HANA data which was once in-memory can reside on-disk.

Near-Line Storage (NLS)

In addition to the reduction of the SAP HANA in-memory footprint DT affords customers, Near-Line Storage should be considered as well. With NLS, data is moved outside of the SAP HANA database proper to disk and classified as “Cold”, due to its infrequent accessed and can only be accessed read only. SAP provides examples showing NLS can reduce the HANA database in-memory requirements by several Terabytes (link below).

It is also important to note that both the DT Extended Storage Host and NLS solutions do not require certified servers or storage, so not only has SAP given customers the ability to run SAP HANA in a reduced memory footprint, customers can run on standard x86 hardware as well.

There is a white paper authored by Priti Mishra, Staff Engineer, Performance Engineering VMware, which is an excellent read for anyone considering DT or NLS options. “Distributed Query Processing in SAP IQ on VMware vSphere and Virtual SAN”

Importance of the VMware Software Defined Data Center

To their credit SAP has taken a leadership role with HANA’s in-memory columnar database computing capabilities and as HANA has evolved the sizing and hardware requirements have evolved as well. Rapid change and evolving requirements are givens in technology; the VMware Software Defined Data Center provides a flexible and agile architecture to effectively react to change by recasting compute, network, and storage resources, in a centrally managed manner.

As a concrete example of the flexibility VMware’s Platform provides, Figure 1. illustrates the evolution of SAP HANA from SPS 07 to SPS 09. For customers who would like to take advantage of SAP HANA’s multi-temperature data management techniques but initially deployed SAP HANA on SPS 07 (all in-memory); through virtualization customers can reclaim and recast memory, storage, and network resources in their virtual HANA landscape to reflect the latest architectural advances and memory management techniques in SPS 10.

Figure 1. SAP HANA Platform: Evolving Hardware Requirements

sap hana vmware

Since SAP HANA can now run in a reduced memory footprint, customers who licensed HANA to be all in-memory can use virtualization to reclaim memory and deploy additional virtual databases and make HANA pervasive in their landscapes.

As a general rule, in any rapidly changing environment The VMware Software-Defined Data Center provides an agile platform which can accommodate change and also protect against capital hardware investments that may not be necessary in the future (certified vs. standard x86 hardware). For that matter, the cloud is a good option to deploy any rapidly changing application/database in places like VMware vCloud Air, Virtustream, or Secure-24 just to mention a few.

Virtual SAP HANA Back on track

After speaking with session attendees, customers, and partners, at VMworld about SAP HANA’s Multi-temperature management capabilities, I was happy to hear they will not be delaying their virtual HANA deployments due to the vSphere 6.0 roadmap certification timeline. As I said earlier, the 1TB monster virtual machine maximum in vSphere 5.5 is an artificial barrier. It really is a worthwhile exercise to take a closer look at the temperature of your data, age of your data, and your access requirements in order to take full advantage of all the tools and features SAP provides their customers.

I was also encouraged to hear from many session attendees that my presentation at VMWorld brought the SDDC from concept closer to reality by demonstrating actual mission critical database/application use cases. My future post VMWorld blogs will focus on how I deconstructed the SAP HANA Networks Requirements document and transformed that into a virtual network design using VMware NSX from my desktop. I’ll also cover Software Defined Storage, essentially translating SAP’s Multi-Temperature Storage Options into VMware Virtual Volumes and Storage Containers.

“SAP HANA SPS10- SAP HANA Dynamic Tiering”; (SAP Product Management)


“Distributed Query Processing in SAP IQ on VMware vSphere and Virtual SAN”; Priti Mishra, Performance Engineering VMware


Blog: Bob Goldsand; “SAP HANA Dynamic Tiering and the VMware Software Defined Data Center”





Use VSAN Assessment to validate Virtual SAN’s benefits for your organization

Are you experiencing challenges with your current vSphere storage environment (i.e., performance, capacity constraint, complexity, expensive renewals) or just not sure if VMware Virtual SAN (VSAN) would be a good fit?

Now VMware partners, SEs, or reps can help you with a free VSAN Assessment.

Continue reading

What’s New with Virtual SAN 6.0?

Software-Defined Storage is making waves in the storage and virtual infrastructure fields. Data and infrastructure are intertwined, and when they’re both brought together, companies can cut down on expenses and increase productivity.

Rawlinson Rivera, Principal Architect, Storage and Availability, recently hosted a webinar, discussing how VMware is approaching Software-Defined Storage (SDS) and virtualization in recently announced VMware updates, including updates to VMware Virtual SAN 6.0.

Software-defined storage offers organizations the ability automate, distribute and control storage better than ever before. SDS can provision storage for applications on demand and without complex processes. It also allows for standardized hardware, reducing costs for businesses everywhere.

To bring the customers the best software-defined storage experience to realization, we had to update VMware® Virtual SAN™. And we did just that. With VMware Virtual SAN 6.0, we introduced several new features with SDS in mind:

  • Software-defined storage optimized for VMs
  • All Flash architecture
  • Broad hardware support
  • The ability to run on any standard x86 server
  • Enterprise-level scalability and performance
  • Per-VM storage policy management
  • And a deep integration with the VMware stack

There’s a lot more to unpack from the latest updates to our VMware solutions. For a more in-depth guide to what’s new and how it affects you, watch the webcast here!

Be sure to subscribe to the Virtual SAN blog or follow our social channels at @vmwarevsan and Facebook.com/vmwarevsan for the latest updates.

For more information about VMware Virtual SAN, visit http://www.vmware.com/products/virtual-san.

Virtual SAN VCG Update – New HP, Fujitsu & Quanta Controllers certified for Virtual SAN!

The Virtual SAN product team is pleased to announce that the following controllers are now certified on Virtual SAN and are listed on the Virtual SAN VCG:

6G 6.0 Hybrid:

HP P420

HP P420i

HP P220i

HP P822

Quanta SAS2308

12G 6.0 Hybrid

HP P440 (without hot plug)

12G 5.5 Hybrid

Fujitsu LSI 3008

We now have a total of 80 I/O controllers certified on Virtual SAN 5.5 and 40 I/O controllers certified on Virtual SAN 6.0.

What about the Ready Nodes for platforms based on the above controllers?

We are working closely with our OEM partners such as HP to publish Ready Nodes for both 6G and 12G platforms for Virtual SAN Hybrid and All Flash configurations for both 5.5 and 6.0 and these should be available soon.

What are the controllers that are currently in the certification queue and when do we expect them to get certified?

Please see the attached list of controllers that are currently undergoing Virtual SAN certification

Note:  In many cases, we rely on our partners to provide driver/firmware fixes for controller issues so if there are delays in receiving these updates from partners, the certification timelines may get pushed out.

I have follow up questions on the attached controller list. Who do I reach out to?

As always, if you have questions on Hardware Compatibility for Virtual SAN, please email your queries to vsan-hcl@vmware.com and someone will get back to you soon!

Where do I learn more about the VMware Virtual SAN Certification process?

Please refer to the Virtual SAN Certification blog post for more details:


Virtual SAN Certification & VCG Update

The Virtual SAN product team is pleased to announce that last week we released new certified components (I/O controllers, SSDs and HDDs), new Ready Nodes and a new Hardware Quick Reference Guide for Virtual SAN 6.0 along with a new and improved VCG page.  Please see updated links below:

Updated Virtual SAN VCG

Updated Virtual SAN Hardware Quick Reference Guide

Updated Virtual SAN Ready Nodes


How many new components and Ready Nodes do we have listed for Virtual SAN 6.0?

We now have 26 I/O controllers, 170 SSDs and 125 HDDs (and counting) supported on Virtual SAN 6.0.   In addition to the Virtual SAN 5.5 Ready Nodes, we have 8 new Ready Nodes for Virtual SAN 6.0  (Cisco – 4 Hybrid, Dell – 1 Hybrid, Hitachi – 1 Hybrid, Super Micro – 1 All Flash & 1 Hybrid).

We expect this list to grow very quickly.  We have a number of components that are currently getting certified and we plan to add new certified devices and Ready Nodes to the VCG on a weekly basis.


How does the Virtual SAN Certification process work?

The VMware Virtual SAN team treats hardware certification very seriously.  I/O controllers play a very important part in determining the stability and performance of a Virtual SAN cluster and need to be able to withstand high I/O under stress conditions.

The I/O controllers are put through a rigorous I/O certification process while the HDD, SSD and Ready Nodes  are put through stringent paper qualifications.

We run a I/O controller card through a 3-week-long certification test plan (the certification is done by VMware or by the partner) that stress tests the card across many dimensions, particularly in high load and failure scenarios to ensure the card can withstand the level of I/O pushed down by Virtual SAN even in the most adverse situations (example: rebuilds and resyncs triggered due to host failures).

If there are issues identified, we work closely with our controller vendor/OEM partner to resolve them and re-run the entire test suite after resolution.  Sometimes an updated firmware or driver version addressing the issue is required from the vendors before we can proceed with more testing.

Only controllers that fully pass the test criteria laid out in the above process are listed on the Virtual SAN VCG.


Are separate I/O controller certifications required for different releases?

Yes, we require controllers to be recertified whenever any of the following change:

  • Virtual SAN Release version (eg: 5.5 to 6.0)
  • The controller driver version
  • The controller firmware version

We also certify the same controller separately for Virtual SAN All Flash vs Hybrid since the caching and I/O mechanism are different for these two configurations and we expect controllers to behave differently with varying levels of I/O.


What about certification of PCIe-SSD devices?

PCIe-SSDs are nothing but SSDs with an on-board I/O controller in a PCIe form factor.  Therefore, these require the same level of due diligence as required by standard I/O controllers.  As a result, we are putting these devices through the same level of rigorous certification as we do for I/O controllers.

VMware is working very closely with partners to certify the first set of PCIe-SSDs for Virtual SAN 6.0 over the coming weeks.


What are the new updates to the VCG page?

The Virtual SAN VCG page has been enhanced to allow users to easily build or choose their All Flash configurations in addition to Hybrid configurations.  Since All Flash Virtual SAN requires SSDs of different endurance and performance spec for caching and performance tiers (See Updated Virtual SAN Hardware Quick Reference Guide for details on specs), we have enhanced the VCG to help users easily pick SSDs for the tier they are interested in.

We have also introduced a new SSD filter called “Virtual SAN type” to help easily filter our All Flash vs Hybrid configurations.  Furthermore, we have added a filter called “Tier” to help you filter our Virtual SAN hybrid caching, Virtual SAN All Flash caching and Virtual SAN capacity caching tiers.

The endurance rating for SSDs are now displayed on the VCG in TBW (Terabytes written over 5 years) as opposed to DWPD (Drive Writes Per Day) which was used previously.


What are the controllers that are currently in the certification queue and when do we expect them to get certified?

Please see the attached list of controllers that are currently undergoing Virtual SAN certification

Note:  In many cases, we rely on our partners to provide driver/firmware fixes for controller issues so if there are delays in receiving these updates from partners, the certification timelines may get pushed out.

Having said that, we are making good progress on most of the controllers listed in the attached document and expect them to follow our standard certification process.

On a similar note, Ready Nodes  are primarily dependent on the controllers getting certified, so as you see new controllers on the VCG for 6.0 certified, Ready Nodes  including those controllers will follow.

Storage Blog Recap: Top Blogs From February


The third week of every month, we compile a list of the top vSphere Storage posts from the previous month for you to enjoy.

Here are the top storage posts from February:

VMware Virtual SAN 6.0

Rawlinson Rivera announced, and explained the inner workings of, VMware Virtual SAN 6.0 — VMware’s latest software-defined storage product. Virtual SAN 6.0 introduces support for an all-flash architecture and hybrid architectures, among many other innovations. This is a blog post you shouldn’t miss.

vSphere Virtual Volumes

We released vSphere Virtual Volumes (VVOLs) alongside the announcement of vSphere 6.0. In this post, Rawlinson Rivera explains how VVOLs can drive more efficient operational model for external storage.

vSphere APIs for IO filtering

Ken Werneburg takes the time to detail how vSphere APIs for IO filtering (VAIO) can enable partners to filter their technologies into the IO stream of a VM before data is committed to a disk.

VMware Virtual SAN: All-Flash Configuration

Rawlinson Rivera walks us through how admins can configure Virtual SAN 6.0 for an all-flash architecture.

Be sure to subscribe to the Virtual SAN blog or follow our social channels at @vmwarevsan and Facebook.com/vmwarevsan for the latest updates.

For more information about VMware Virtual SAN, visit http://www.vmware.com/products/virtual-san.

Video: Virtual SAN From An Architect’s Perspective

Video: Virtual SAN From An Architect’s Perspective

Have you ever wanted a direct discussion with the people responsible for designing a product?

Recently, Stephen Foskett brought a cadre of technical bloggers to VMware as part of Storage Field Day 7 to discuss Virtual SAN in depth.  Christos Karamanolis (@XtosK), Principle Engineer and Chief Architect for our storage group went deep on VSAN: why it was created, its architectural principles, and why the design decisions were important to customers.

The result is two hours of lively technical discussion — the next best thing to being there.  What works about this session is that the attendees are not shy — they keep peppering Christos with probing questions, which he handles admirably.

The first video segment is from Alberto Farronato, explaining the broader VMware storage strategy.

The second video segment features Christos going long and deep on the thinking behind VSAN.

The third video segment is a run-over of the second.  Christos presents the filesystem implementations, and the implications for snaps and general performance.

Our big thanks to Stephen Foskett for making this event possible, and EMC for sponsoring our session.


How To Double Your VSAN Performance

How To Double Your VSAN Performance

VSAN 6.0 is now generally available!

Among many significant improvements, performance has been dramatically improved for both hybrid and newer all-flash configurations.

VSAN is almost infinitely configurable: how many capacity devices, disk groups, cache devices, storage controllers, etc.  Which brings up the question: how do you get the maximum storage performance out of VSAN-based cluster?

Our teams are busy running different performance characterizations, and the results are starting to surface.  The case for performance growth by simply expanding the number of storage-contributing hosts in your cluster has already been well established — performance linearly scales as more hosts are added to the cluster.

Here, we look at the impact of using two disk groups per host vs. the traditional single disk group.  Yes, additional hardware costs more — but what do you get in return?

As you’ll see, these results present a strong case that by simply doubling the number of disk -related resources (e.g. using two storage controllers, each with a caching device and some number of capacity devices), cluster-wide storage performance can be doubled — or more.

Note: just to be clear, two storage controllers are not required to create multiple disk groups with VSAN.  A single controller can support multiple disk groups.  But for this experiment, that is what we tested.

This is a particularly useful finding, as many people unfamiliar with VSAN mistakenly assume that performance might be limited by the host or network.  Not true — at least, based on these results.

For our first result, let’s establish a baseline of what we should expect with a single disk group per host, using a hybrid (mixed flash and disks) VSAN configuration.

Here, each host is running a single VM with IOmeter.  Each VM has 8 VMDKs, and 8 worker tasks driving IO to each VMDK.  The working set is adjusted to fit mostly in available cache, as per VMware recommendations.

More details: each host is using a single S3700 400GB cache device, and 4 10K SAS disk drives. Outstanding IOs (OIOs) are set to provide a reasonable balance between throughput and latency.


On the left, you can see the results of a 100% random read test using 4KB blocks.  As the cluster size increases from 4 to 64, performance scales linearly, as you’d expect.  Latency stays at a great ~2msec, yielding an average of 60k IOPS per host.  The cluster maxes out at a very substantial ~3.7 million IOPS.

When the mix shifts to random 70% read / 30% writes (the classic OLTP mix), we still see linear scaling of IOPS performance, and a modest increase in latency from ~2.5msec to ~3msec.  VSAN is turning it a very respectable 15.5K IOPS per host.  The cluster maxes out very close to ~1m IOPS.

Again, quite impressive.  Now let’s see what happens when more storage resources are added.

For this experiment, we added an additional controller, cache and set of capacity devices to each host.  And the resulting performance is doubled — or sometimes even greater!


Note that now we are seeing 116K IOPS per host for the 100% random read case, with a maximum cluster output of a stunning ~7.4 million IOPS.

For the OLTP-like 70% read / 30% write mix, we see a similar result: 31K IOPS per host, and a cluster-wide performance of ~2.2 million IOPS.

For all-flash configurations of VSAN, we see similar results, with one important exception: all-flash configurations are far less sensitive to the working set size.  They deliver predictable performance and latency almost regardless of what you throw at them.  Cache in all-flash VSAN is used to extend the life of write-sensitive capacity devices, and not as a performance booster as is the case with hybrid VSAN configurations.

In this final test, we look at an 8 node VSAN configuration, and progressively increase the working set size to well beyond available cache resources.  Note: these configurations use a storage IO controller for the capacity devices, and a PCI-e cache device which does not require a dedicated storage controller.

On the left, we can see the working set increasing from 100GB to 600GB, using our random 70% read / 30% OLTP mix as before.

Note that IOPS and latency remain largely constant:  ~40K IOPS per node with ~2msec latency.  Pretty good, I’d say.

On the right, we add another disk group (with dedicated controllers) to each node (flash group?) and instead vary the working set size from an initial 100GB to a more breathtaking 1.2TB.  Keep in mind, these very large working set sizes are essentially worst-case stress tests, and not the sort of thing you’d see in a normal environment.


Initially, performance is as you’d expect: roughly double of the single disk group configuration (~87K IOPS per node, ~2msec latency).  But as the working set size increases (and, correspondingly, pressure on write cache), note that per-node performance declines to ~56K IOPS per node, and latency increases to ~2.4 msec.

What Does It All Mean?

VSAN was designed to be scalable depending on available hardware resources.  For even modest cluster sizes (4 or greater), VSAN delivers substantial levels of storage performance.

With these results, we can clearly see two axes to linear scalability — one as you add more hosts in your cluster, and the other as you add more disk groups in your cluster.

Still on the table (and not discussed here): things like faster caching devices, faster spinning disks, more spinning disks, larger caches, etc.

It’s also important to point out what is not a limiting factor here: compute, memory and network resources – just the IO subsystem which consists of a storage IO controller, a cache device and one or more capacity devices.

The other implication is incredibly convenient scaling of performance as you grow — by either adding more hosts with storage to your cluster, or adding another set of disk groups to your existing hosts.

What I find interesting is that we really haven’t found the upper bounds of VSAN performance yet.  Consider, for example, a host may have as many as FIVE disk groups, vs the two presented here.   The mind boggles …

I look forward to sharing more performance results in the near future!


Chuck Hollis



What’s All the Buzz About Software-Defined Storage?

By now, you’ve more than likely heard something about Software-Defined Storage. With every mention of the term, you may be wondering, “What does it mean for me?”

Wonder no longer!

The VMware Software-Defined Storage approach enables a fundamentally more efficient operational model, driving transformation through the hypervisor, bring to storage the same operational efficiency that server virtualization brought to compute. Software-Defined Storage will enable you to better handle some of the most pressing challenges storage systems face today.

During this webcast, Mauricio Barra, Senior Product Marketing Manager at VMware, will discuss the VMware Software-Defined Storage vision, the role of the hypervisor in transforming storage, as well as key architectural components of VMware Software-Defined Storage.

If you are looking to understand how Software-Defined Storage, along with the enhanced VMware Virtual SAN 6 and new VMware vSphere Virtual Volumes, can benefit your organization, now is your chance.

Register today and take the next step toward making Software-Defined Storage a reality.

Be sure to subscribe to the Virtual SAN blog or follow our social channels at @vmwarevsan and Facebook.com/vmwarevsan for the latest updates.

For more information about VMware Virtual SAN, visit http://www.vmware.com/products/virtual-san.

Storage Blog Recap: Top Blogs from January

The third week of every month, we will be compiling a list of the top vSphere Storage posts from the previous month for you to digest.

Here are the top storage blogs from January:

VMware Virtual SAN: File Services with NexentaConnect

Rawlinson Rivera discusses NexentaConnect for Virtual SAN, a software-defined storage solution designed specifically to deliver file service on top of Virtual SAN.

SAP HANA Dynamic Tiering and the VMware Software Defined Data Center

The latest release of SAP HANA has brought the concepts of multiple-temperature data and lifecycle management to a new level. Bob Goldsand talks more about this, as well as native use cases and dynamic tiering with VMware HA and workload management.

Storage and Availability at Partner Exchange 2015

VMware Partner Exchange just wrapped up in San Francisco, California. In this post, Ken Werneburg talks about some key storage and availability sessions that were offered during the conference.

Discover Software-Defined Storage and VMware Virtual SAN at PEX 2015!

The Virtual SAN team highlights some of the can’t-miss sessions that were available to attendees of VMware Partner Exchange 2015.

Performance Unplugged: Demanding Applications

Mark Achtemichuk introduces a new series called “Performance Unplugged”, which showcases a number of talented performance gurus and also covers commonly asked questions and topics.

Be sure to subscribe to the Virtual SAN blog or follow our social channels at @vmwarevsan and Facebook.com/vmwarevsan for the latest updates.

For more information about VMware Virtual SAN, visit http://www.vmware.com/products/virtual-san.