This is a spiritual part 2 of this post, where I spelled out the (apparently) bold opinion that running VMware vSphere Kubernetes Service (VKS) on VMs is far superior to running Kubernetes workloads on bare metal. I stand behind it. Here, I will make the case that running VKS on-prem for AI is superior to running Kubernetes in the cloud.
I also feel compelled to mention that VCF 9.0 is out now.
The idea for this post came from my participation in a recent webinar. Thanks to my colleague Christopher Kusek, I was the MC for a well-attended Webinar on VKS as part of his Architect’s Edge series where I busted some myths. You can watch the hour-long session here.
A poll conducted during that webinar revealed that out of 123 people who responded to the poll, 14% of them were there to learn about running AI workloads on VKS. I suspect the actual number of people there to learn about AI on VKS is higher since the overall attendee count was 450, but I will take the win.
I hear my customers say, “We don’t really know what we’re doing with AI, but we do know our competitors are doing something with it, so we have to do something with it too.” That’s the stage we’re in now. And if you’re a customer reading this, the TL;DR is that if you’re running VCF with vSphere 8.0 U3, with a few GPUs, you can start running VKS in a few hours and start processing those AI models. This will include VCF Automation for easy Kubernetes workloads self-service.
Furthermore, when you ask around, you’ll be told that “you have to run AI on bare metal.”
If you read my last post, aforementioned, you know that’s not true.
They will also say, “you have to run AI in the public cloud.” Also not true. Why do they think this? Because they don’t read my blog posts don’t realize VMware can do AI on Kubernetes.
And finally, they’ll say, “You’ll have to build it yourself.” As you may have guessed by now, this is also not true (more details on this later).
Before I go into details about VKS and AI, I would be remiss if I didn’t mention that we’re seeing an uptick not just in development on VKS, but also COTS applications where the vendor recommends that their application runs on Kubernetes. VCF has you covered on both counts.
Why VKS and Private AI?
If you are still skeptical and of the opinion that Kubernetes is a “solution waiting for a problem”, that problem is AI. I won’t be so bold as to state that “Kubernetes is the perfect solution for running AI workloads” because “perfect solutions” are exceedingly rare in IT engineering. But it certainly ticks a lot of boxes:
- AI processing (of whatever kind) needs to be optimized to focus on, well, AI processing versus maintaining secondary services that would hamper a full guest OS in a VM. Containers win the day.
- Each AI run is a mix of well-known components, including AI foundation models, vector databases, and so on along with customized nuances that are easily automated through, dare I say “tried and true” Kubernetes AIOps methodologies.
- The workloads are ephemeral and need to be scalable, which is a use case for which Kubernetes was built.
Furthermore, AI processing requires that a company’s data be part of that process. There are quite a few customers who then want to keep that data on-prem where it is secure and they can control it.
All of the above leads to Broadcom VCF with VKS being the perfect solution for an AI on-prem scenario.
VKS can actually “meet you where you are” by providing the enterprise grade Kubernetes runtime and self-service engine, all included with VCF.
Private AI DIY and the Trial and Error Problem
The biggest mistake I see customers make when they are new to AI is that they will assume they have to put everything in the public cloud (expensive and can have security risks), run it on bare metal (I’ve already debunked this), or they will just opt to build their own and “Frankenstein” their AI solution.
Many times, it’s assumed (incorrectly) that the customer will save money when building a solution, but you know what else is expensive?
Trial and error.
As I have said many times: if you’re running vSphere 8.0 U3, with a few GPUs, you can start running VKS in a few hours and start processing those AI models.
We also have very transparent and granular cost analysis tools that will help you come to your own conclusions about the costs involved.
If You Really Want to Go ‘Next Level’, Consider VMware Private AI Cloud Foundation
Our “full boat” turn-key offering that will get you up and running fastest is VMware Private AI Foundation with NVIDIA, which will include everything you need to have self-service Generative AI Kubernetes workloads:
This includes all of the above, plus additional AI-specific self-service options, such as vGPU settings and drop-down choices of included AI foundation models.
The Mic Drop: AI on VCF
The drum I am beating here is that you should think of VMware as a private cloud rather than as a core hypervisor only. We do thank you for us being at your core, but now we’ve completed our journey into being an integrated private cloud.
If the words on the lips of everyone at your organization is “AI”, especially if you have VCF (most of you do), then let this be your first foray into a perspective change about what we do. Contact me or contact your account team.
We’re ready.
