In Part 1 of this series, we explored how VMware Cloud Foundation (VCF) 4 and vSphere 7 with Kubernetes brings new levels of developer and admin productivity, while Part 2 provided a technical overview of Tanzu Kubernetes Grid (TKG) integrated as a runtime service into vSphere. The introduction of VMware Cloud Foundation Services and Namespaces within vSphere 7 provides an excellent framework for Part 3 of this series. This post will dig into the new VMware NSX-T 3.0 networking and security capabilities, as well an overview of how vSAN 7.0 and vVols enable advanced persistent storage functionality across full-stack hyperconverged infrastructure (HCI). Within VCF 4.0, these building blocks are delivered as Network and Storage Services that are surfaced via APIs in vSphere 7.0 positioning organizations to deploy developer-ready Kubernetes infrastructure at scale.
Read Part Four of the ‘Develiering Kubernetes at Cloud Scale’ blog series here.
NSX-T in VMware Cloud Foundation 4.0 Deployments
One of the core building blocks of VMware’s Kubernetes strategy centers around the advanced networking capabilities delivered within NSX-T 3.0, which is now GA (check this blog post announcing NSX-T 3.0 for more details). VMware Cloud Foundation 4.0 is the first version that standardizes on NSX-T within the management and all workload domains, removing the requirement for NSX-v that existed in previous releases. It is important to note that for Release 4.0, vSphere for Kubernetes workloads cannot yet run in the management domain, so only standard VCF architectures are allowed, meaning that vSphere with Kubernetes can run in VI workload domains. We anticipate that this will be addressed in a future release.
Figure 1: NSX-T Running in VCF Management and Workload Domains
Networking and Security for vSphere with Kubernetes
NSX-T provides VMware Cloud Foundation with a powerful set of networking and security features that are central to deploying Kubernetes at Cloud scale. This advanced functionality streamlines infrastructure tasks and simplifies processes to support platform teams and DevOps. Visibility, isolation and automation enables rapid deployment of enterprise-wide Kubernetes infrastructure to support application development teams with networking infrastructure that is agile, scalable and intrinsically secure.
IP address management within NSX-T allows developers to deploy workloads or endpoints, with the ability to dynamically assign an IP address through NSX-T behind the scenes to simplify deployment of these Kubernetes workloads.
Network segmentation for both supervisor cluster and TKG clusters supports automated network topologies for pods and clusters so that new clusters can be deployed simply by enabling the developers to deploy clusters via Kubectl commands through the Kubernetes APIs to the VMware Cloud Foundation Services.
Firewall isolation allows developers to create new apps and micro-services and deploying new workloads where the network policies to those services are associated to the vSphere Namespace and then NSX enforces those firewall policies or distributed firewall policies automatically. This also enables admins to assign more perimeter, edge firewall or gateway firewall capabilities to filter out traffic for better security isolation.
Load Balancing allows developers to create new microservices so that they can be discovered and accessible to other apps or microservices within the Supervisor cluster (distributed load balancing) or outside of the Supervisor cluster (L4 load balancing).
Network visibility from VI admins perspective, NSX-T and vRNI provide tools that allow the developers who are deploying these modern applications, to create up to 10X more endpoints that are created on a given supervisor cluster compared to virtual machine endpoints. Admins require higher degrees of visibility to troubleshoot and triage issues using advanced network topology container inventory and trace flow analysis to properly manage the highly distributed network infrastructure.
Figure 2: NSX-T Network Topology
As shown in the figure above, NSX-T provides the network infrastructure to deliver Kubernetes at scale through VMware Cloud Foundation 4.0. As shown above, vSphere with Kubernetes is deployed as a supervisor cluster and pods are deployed as microservices, allowing NSX-T to build a routed topology using Tier-1 gateways with the appropriate layer 4 through 7 load balancing so the pods can now be deployed on their own isolated network. Distributed firewalling occurs when the developer creates a network policy to allow whitelisted applications or micro services to be allowed by the distributed firewall. The Tier-1 gateway load balances traffic to the edge firewall, which is deployed as part of NSX-T within VCF to optimize connectivity while protecting against malicious cyber-attacks for vSphere with Kubernetes.
The new capabilities of NSX-T provide high degrees of network access required for Kubernetes clusters, while maintaining the highest degrees of protection and visibility that can be fully managed on a per vSphere Namespace basis. Learn more about NSX-T 3.0 here.
Storage Building Blocks with VMware Cloud Foundation 4.0
Data storage often becomes an overlooked aspect of building complex data center infrastructure, but for VMware hyperconverged infrastructure (HCI), storage is core to delivering a cloud operating model to the data center. The hyperscale cloud providers deliver agility to their customers by virtualizing their entire stack and using automation throughout their operations. Similarly, VMware delivers a cloud model by virtualizing infrastructure, enabling customers to begin with storage, then migrate to full stack (Cloud Foundation) HCI to deliver hybrid cloud deployments with consistent infrastructure and consistent operations.
vSAN 7.0 Optimized for VMware Cloud Foundation and Kubernetes
With vSAN 7.0, VMware has modernized the core HCI stack alongside vSphere 7.0, broadening the supported use cases while simplifying deployment and lifecycle management through vSphere Lifecycle Management (vLCM). By delivering a wide variety of capabilities and benefits across the platform, vSAN and HCI move much closer to becoming the default storage platform, regardless of workload.
Figure 3: vSAN 7.0 Delivers Advanced Storage Features for VMs and Modern Applications
vSAN 7 greatly simplifies lifecycle management, reducing the tools required for providing software and firmware updates by providing a single, integrated tool that has significantly increased reliability when applying updates to get to the desired outcomes. This functionality extends into the latest firmware versions of supported hardware appliances, at GA announcing support for Dell PowerEdge (14G) including MX composable and HPE ProLiant (Gen10) including Synergy composable platforms.
vSAN 7 is the first release to deliver integrated file services, helping customers avoid the need for a third-party solution for some file use cases. With vSAN 7.0, there are a number of enhancements to the support for cloud-native applications, by enabling file shares for container-based workloads orchestrated by Kubernetes, providing more granular data services at the persistent volume level and of course, providing native support for vSphere with Kubernetes.
Since vSAN now supports file protocols NFS v3 and NFS v4.1, with v4.1 deliberately chosen to support cloud-native applications orchestrated by Kubernetes, expanding the use cases for cloud-native applications on vSAN. This is a great fit for vSphere with Kubernetes, which enables stateful containerized workloads to be deployed on Supervisor and Guest clusters on vSAN datastores. vSAN also provides support for web server applications such as NginX and Tomcat, providing high degrees of efficiency, as they will share a file repository.
vSAN 7 expands the type of cloud-native applications that can run on HCI, making it a great general-purpose infrastructure for a wide variety of workloads. Customers can take advantage of lower cost, industry-standard servers for their cloud-native application infrastructure, while reducing the time needed to manage their infrastructure due to vSphere Lifecycle Manager.
Figure 4: VCF 4.0 Supported Storage Options
Storage Options in VCF 4.0
vSAN is the default storage option for VCF deployments because of its high-performance, highly scalable, hyperconverged architecture and tight integration with VCF. vSAN enables admins to achieve the level of automation for initial deployment and configuration as well as on-going management for VCF. While vSAN must always run in the management domain, users may decide to run external storage in workload domains. Using vSAN in the management domain provides admins and architects with the ability to manage and automate compute, network, storage with a level of automation not possible via external storage arrays, especially considering the many differences in implementation between different storage solutions from leading vendors that exist in the market. In order to support the primary storage solutions for integration/support, you need to rely on fixed implementations of datastore tags that are used to encode information about a datastore that may not be available otherwise.
Another approach that is gaining traction for supporting external storage is through vVols which provides a standardized way to manage and integrate the properties of external storage for granular controls at the VM layer. While current vVols support in VCF is limited to prescriptive (manual) guidance, VMware storage partners are making significant investments to accelerate vVols adoption across the different storage platforms. vVols as principal storage for VCF workload domains is being developed and will hopefully be available in future releases.
For customers looking to support traditional external storage arrays, VMware recommends vVols in order to bring the benefits of storage policy-based management (SBPM) to traditional storage. What’s great about SPBM is it provides a standardized and rational data management model regardless of scale. Admin teams can capture any business logic for data handling inside vCenter, then apply that logic, either to whole virtual machines single VMDK, or container. As an added benefit, vVols integrate very nicely with Site Recovery Manager for automated disaster recovery as shown in this video. All of this provides a scalable and straightforward management model for managing data services inside any cloud.
The ability to deploy Kubernetes at cloud scale is highly dependent on the storage and networking building blocks to provide a wide array of functionality, connectivity and persistent, containerized data stores in order to provide the agility that customers expect in modern application deployment scenarios. Within VMware Cloud Foundation 4.0, NSX-T provides the dynamic isolation, load balancing and distributed firewalls needed for large scale Kubernetes deployments. vSAN 7.0 provides new capabilities including vSphere Lifecycle Manager (vLCM), file services and a broad range of cloud native storage services to provide consistent datastores for vSphere with Kubernetes deployments. Not that VCF now provides limited vVols support (prescriptive guidance), customers can utilize external storage arrays for new VCF-based Kubernetes deployments at cloud scale. Learn more about how these solutions integrate within VMware Cloud Foundation 4 here and follow VMware Cloud Foundation on Twitter and LinkedIn.
