Today, surging hardware costs are challenging the economics of running containers on bare metal. With skyrocketing server and DRAM (Dynamic Random Access Memory) prices and long lead times, IT teams need to rethink how to increase the efficiency of container-based modern applications running on bare metal architecture.
The most common challenge that I’ve heard from CIOs over the past two years is that their IT budgets remain flat, even as their hardware costs keep climbing and priorities such as data security and AI, grow more complex. While CIOs cannot control hardware prices, they can increase hardware efficiency and utilization. To cut hardware spend and lower total cost of ownership (TCO), they need a platform engineered to do more with less.
In this blog, I will cover how VMware Cloud Foundation® (VCF) significantly improves hardware utilization and lowers your TCO for container-based modern applications.
VCF reduces DRAM requirements with NVMe memory tiering
The NVMe (Non-Volatile Memory express) memory tiering feature of VMware Cloud Foundation (VCF) keeps active pages in DRAM for most workloads and offloads inactive pages to NVMe. Broadcom’s internal analysis of quotes from server vendors revealed that NVMe costs up to 90% less than the same amount of DRAM.
Planning a server refresh? Memory tiering slashes DRAM costs for suitable workloads without sacrificing performance. Staying on existing hardware? Adding low-cost NVMe could double effective capacity without buying a new server for suitable workloads.
The NVMe memory tiering feature of VCF alone can deliver significant cost savings for most workloads. These DRAM cost savings can even offset a significant portion of VCF licensing costs. Read this blog for more details.
On the other hand, bare metal architectures do not support memory tiering. In addition, in bare-metal architecture, memory per pod is reserved as defined in “Request” configuration. Unused reserved capacity cannot be dynamically reclaimed for other workloads.
VCF allocates memory and CPU based on actual usage, unlike bare metal architecture’s static minimum reserved capacity, whether used or not. VCF increases the utilization of DRAM and CPU with dynamic allocation and reclamation based on actual usage.
VCF reduces servers needed for modern workloads
There are two main factors in bare metal architecture that often lead to poor server hardware utilization:
1. 110 pods limit per node guideline as a best practice
Since the launch of Kubernetes in 2015, and across all 37 releases since version 1.0, CNCF has consistently recommended a maximum limit of 110 pods per node as a best practice. If you try to push beyond set guidelines, it may lead to complex IP address management, unpredictable scheduling, degraded performance, or node instability. In VCF, you run pods inside VMs and follow a 110-pods-per-VM guideline, which lets you increase server CPU utilization by hosting multiple VMs per server with pods inside VMs.
2. Significant gap between reserved resources and actual usage
In Kubernetes, a pod requests resources in its specification. Kubelet then reserves those resources exclusively for that pod. Even if actual usage stays low, the allocated resources remain locked. New pods cannot be scheduled on the same server to use that idle capacity, stranding compute and memory on the server.
Also, most platform teams significantly overprovision their pods as safety nets. Rightsizing pod allocations is tedious, complex, and could be disruptive for mission-critical workloads. Most bare-metal Kubernetes environments stay heavily underutilized.
Industry data confirms this gap. A recent Datadog report shows that most container workloads use less than 25% of their reserved CPUs. Similarly, CAST AI’s 2026 report found similar numbers, with average CPU utilization at 8% and average memory utilization at 20%.
An example of bare metal architecture inefficiency
Let’s consider an environment on a typical modern server with 128 cores, a 110-pod limit per node per CNCF guidelines, and 25% actual usage per pod per the industry reports, running container based modern applications.
Standard guidelines suggest allocating 100–250 millicores per pod for light workloads, 250–500 millicores per pod for average workloads, and 1000+ millicores per pod for select heavy workloads.
| Per pod CPU Request(milliCores) | CPU cores reserved for 110 pods(milliCores X 110) | Active utilization % = 25% of reserved pods(milliCores X 110 X 25%) | Wasted capacity % (out of a 128-core server) |
| 100 mCores | 11 Cores | 2.75 Cores (2%) | 98% |
| 500 mCores | 55 Cores | 13.75 Cores (11%) | 89% |
| 1000 mCores | 110 Cores | 27.5 Cores (21%) | 79% |
As the table shows, the 110-pods-per-node best practice results in significant CPU capacity waste in bare metal architecture for modern container-based applications.
Even with a 250 pods-per-node environment, active utilization numbers remain low and resource wastage remains significantly high.
VCF improves your server utilization
In VCF, you run pods inside VMs and follow a 110-pods-per-VM guideline, which lets you increase server CPU utilization by hosting multiple VMs per server with pods inside VMs. For example, if you deploy 5 VMs per server with 110 pods per VM, you can increase server utilization by 5x.
VCF reduces storage needs by up to 8x
An internal Broadcom analysis of a recently launched global deduplication feature of VMware® vSAN, an integrated component of VCF, showed that vSAN may achieve up to 8x storage reduction for many data types and lower TCO compared to traditional storage arrays. Unlike traditional storage, where deduplication is limited to the storage behind a pair of I/O controllers, vSAN’s deduplication domain scales with the cluster, offering better efficiency. The larger the environment, the more redundant data it can eliminate. Learn more about vSAN Global Deduplication in this blog
VCF delivers better GPU utilization than bare metal
Various industry reports including the CAST.AI report suggest that average GPU utilization in many customer environments is very low, even though GPUs are among the most expensive resources for an organization. VCF improves GPU utilization with time-sliced vGPU and Multi-Instance GPU (MIG) vGPU architectures.
According to the Broadcom’s internal testing as made public recently, GPU utilization on VCF was increased to 94% using GPU optimization techniques. Whereas, in bare metal deployments, lack of vGPU support often leaves GPUs significantly underutilized. To learn more, read this blog.
It’s your time to act now
Memory tiering, higher workload density, storage deduplication, and higher GPU utilization deliver compounded savings by reducing the number of servers and DRAM required. With fewer servers, you need fewer networking switches, ports, and racks, less power, cooling, physical space, and fewer per-server and per-core software licenses.
Recent benchmark studies indicate that virtualized environments deliver performance on par with bare metal. Hyperscalers have been running billions of containers in VMs instead of bare metal for better utilization, isolation, security and SLAs, as highlighted in these articles: DZone, PlatformEngineering.org, and CloudNativeNow.
Running containers in VMs provides additional benefits such as stronger isolation and better security posture, and assured SLAs, which we’ll discuss in future blogs.
Now, it’s your time to migrate container-based applications from bare metal to VCF and take advantage of the VMware vSphere® Kubernetes Service (VKS) entitlement in your VCF license. Talk to your VMware representative, run a TCO analysis, pilot with VKS, and start your migration journey today.
Discover more from VMware Cloud Foundation (VCF) Blog
Subscribe to get the latest posts sent to your email.
