One of the inherent advantages that vSAN has over traditional storage is that it is part of the hypervisor. Having a fully integrated distributed storage solution into the hypervisor has allowed customers to serve up both block and file services in a manner that is flexible, scalable, and easy to manage. As cloud native and AI workloads have proliferated, many of these workloads assume the use of S3 compatible object storage. This quickly turned a nice-to-have into a requirement. We recognized the gap in vSAN, and wanted to solve the challenge the right way.
That gap has been filled in vSAN for VMware Cloud Foundation (VCF) 9.1. We are thrilled to introduce vSAN native S3 compatible object storage. This capability allows you to run block, file, and object storage all on the same vSAN cluster. It is fully integrated, highly scalable, and easily accessible in your VCF deployment.
Let’s take a look at this native S3 compatible storage offering in vSAN, so you can understand the capabilities better when it becomes available.
Unifying Your Storage Strategy using vSAN
In previous versions of vSAN, the only way to deliver S3 compatible object storage was by using a certified Independent Software Vendor (ISV). While this approach was certainly functional, it added additional products, technical complexity, and management overhead to the stack.
Building a native S3 compatible object storage capability into vSAN helps achieve specific goals that reflect the expectations from our customers.
Figure 1. vSAN native S3 compatible object storage for vSAN in VCF 9.1.
Natively Integrated
vSAN native S3 compatible object storage is not a bolt-on solution, nor is it a partner integrated solution. It was developed internally, and built directly into vSAN. Since vSAN is fully integrated inside of the hypervisor, it is this unique combination that makes the solution extraordinarily efficient and impressive. It shares some elements originally developed for vSAN File Services, including a highly scalable distributed file system, and light-weight container runtimes that pass the client traffic through the stack. It also uses a new zero copy technique we’ve developed to minimize resource utilization, and maximize performance. Zero copy techniques reduce queuing of I/O as the commands pass through the stack, this proprietary zero copy protocol does exactly that! For our customers this means a highly resource efficient solution.
Operations are kept agile and simple, which translates to fast delivery of object storage resources to the respective consumers. A vSAN cluster running native S3 object services looks nearly indistinguishable from a vSAN cluster not running object services. While it provides the substrate, the configuration resides in VCF Automation and the vSphere Supervisor.
Built for Multitenancy
S3 compatible object stores are often used by multiple consumers. In VCF 9.1, we wanted to deliver a cohesive method of provisioning and managing storage to multiple tenants. This demands that a proper framework is in place that allows for delegation of responsibilities and self-service tasks, as well as accommodating the type of project and quota management you would expect in a multi-tenant environment. VCF Automation provides this type of framework, and is used as the primary mechanism for managing object storage in vSAN. The vSphere Supervisor can also provide these abilities.
Flexible Isolation and Security
One of the requirements of multitenancy is to provide sufficient logical or even physical separation. vSAN native S3 compatible object storage integrates with VCF features designed for multi-tenancy, such as Virtual Private Cloud (VPCs) for networking, private keys, and other VCF Automation constructs designed for multitenancy. These can be paired with cluster services already available, like vSAN Encryption Services.
Since it is built into vSAN, you have flexible deployment models that can help deliver the appropriate levels of isolation for your tenants. When logical tenant isolation is acceptable, multi-tenant storage can be achieved with a single vSAN HCI and/or vSAN storage cluster using the multi-tenancy mechanisms in VCF. Or if physical tenant isolation is required, multi-tenant storage can be achieved with discrete vSAN HCI clusters, which can be ideal for highly regulated environments.
Highly Scalable
S3 compatible object stores can grow quite large, so you need a backing storage system that can grow with it. Since native S3 compatible object storage runs on a vSAN substrate, this offering reflects all of the scalability traits of any other vSAN cluster. Easily scale up by adding more storage to each host in the cluster, or scale out by adding more hosts to the cluster – all without any interruption. Create object stores as large as what certified hardware allows on a vSAN cluster.
Our native S3 compatible object storage in vSAN also works seamlessly with vSAN Global Deduplication, which is not always a given when considering various S3 compatible object storage solutions on the market. For VCF customers, this translates to clusters that can provide an enormous amount of storage capacity for object stores.
Deployment, Configuration, and Operation
The configuration and management of vSAN native S3 compatible object storage using VCF Automation will offer the most flexible and comprehensive experience. VCF Automation uses an Object Store Extension (OSE) UI to interface with the vSphere Supervisor to provide easy configuration, management, and consumption of S3 buckets.
It can also be deployed and consumed through the vSphere Supervisor through API and CLI. Regardless of the method of deployment, the consumers of these S3 buckets are able to access the same set of S3 APIs programmatically.
When using VCF Automation, a high-level overview of the deployment and configuration is as follows:
- The installation of the “Object Store Service” in the “Service Management” portion of VCF Automation is performed by the “Provider Admin” in VCF Automation.
- Once complete, the Provider Admin can make object storage available to the desired tenant organizations, assigning quotas and other settings.
- The “Tenant Admin” will create S3 buckets for use by their consumers, or “Tenant Users”
- Tenant Users are ready to consume S3 storage provided to the tenant they belong to.
A vSAN cluster will support one object store per cluster, per tenant, where the tenant can store their S3 buckets. Since vSAN HCI by default provides storage resources exclusive to the cluster, you have a few options in how your object storage could be deployed. Regardless of what is chosen, the deployment and configuration remains the same.
- When logical tenant isolation is acceptable. Multi-tenant storage can be achieved with a single vSAN HCI or a vSAN storage cluster.
- When physical tenant isolation is required. Multi-tenant storage can be achieved with discrete vSAN HCI clusters.
What’s Next
vSAN native S3 compatible object storage will debut as a “Technology Preview” in an upcoming patch for vSAN in VCF 9.1. We would love to get your feedback, and provide an opportunity to share your thoughts on the form: “vSAN Native S3 Compatible Object Storage feedback.”
Do you have more questions on vSAN? Check out the extensive list of frequently asked questions on the vSAN FAQs document.
Summary
The introduction of vSAN native S3 compatible object storage in VCF 9.1 marks a major milestone for VMware Cloud Foundation. By removing the need for third-party add-ons and unifying block, file, and object storage into a single solution, we are simplifying the path to a true private cloud. Keep an eye out for more information to come on this exciting new capability!
Discover more from VMware Cloud Foundation (VCF) Blog
Subscribe to get the latest posts sent to your email.
