Kubernetes. Love it or hate it, it’s everywhere. VMware’s Bryan Liles recently described Kubernetes in a way I’ve heard many times.
“#Kubernetes is not a product…it’s a cloud native platform for building platforms” says @bryanl of #VMware during his opening keynote @ #KubeCon / #CloudNativeCon pic.twitter.com/Uvj7Yr0VMD
— Keith Lee (@KeithRichardLee) May 23, 2019
That’s a fun statement, but what does it mean? What’s a "platform for platforms?" And what makes Kubernetes a good one?
In this post, I’ll look at five things that Bryan’s statement means to me. What you’ll see is that Kubernetes is a terrific platform that underpins the platforms that actually make you money and accelerate development.
It means Kubernetes has native components that make it possible to build software systems for others to use.
Let’s be liberal in our definition of “platform,” since it has mostly lost all meaning anyway. You can have a database platform, an analytics platform, a monitoring platform, an application platform, or an integration platform. Any kind of platform you want. Most of these “platforms” represent distributed systems themselves, made up of many moving parts that work together to deliver functionality.
Let’s consider Kubernetes an infrastructure platform (yes, I know it has aspects that resemble application-level abstractions, too) that you build atop of. An IaaS, if you will. What’s in the box that helps platform builders?
Wide-ranging infrastructure APIs
Kubernetes is more than a container scheduler; it offers interfaces into a complete range of core infrastructure. That means you can create clusters, upgrade clusters, resize clusters, configure resource quotas, setup and manage DNS, provision and manage persistent storage, and much more. Basically, you have programmatic access to virtually everything you need to get a distributed system deployed, configured, and accessible. For those building platforms, it’s powerful to have that in one API surface.
A consistent, cloud-neutral experience
This one’s underrated, but important. Containerization promised to make it possible to run a given workload virtually anywhere, with the same behavior. Kubernetes takes that a step further by offering the ability to deploy an entire stack somewhere, consistently. Use Amazon EKS, Google’s GKE, or Azure’s AKS. Use VMware PKS in your data center or at the edge. It doesn’t matter.
With a few exceptions (e.g., hooks to cloud-specific services), a Kubernetes cluster is a Kubernetes cluster. Sure, each provider supports different Kubernetes versions and has different management control planes, but the API is the same, and that’s powerful. As a platform builder, you want a “package once, run anywhere” experience.
Smart resource utilization and workload placement
If you’re building platforms, you likely have a number of components that make up your system. Scheduling Pods onto n Nodes is what Kubernetes is super good at. It figures out the best Node for a given Pod, using some configurable sophistication.
While Kubernetes only runs containers, you have “types” of workloads to deploy based on need: init containers, ephemeral containers, stateful or stateless workloads, cron jobs, parallel tasks, and more. When you’re building a platform, you want it to run a wide range of workloads to gain operational efficiencies at scale. With Kubernetes, you can take advantage of these different resource types for various components of your architecture.
Self-healing and rolling update support
Ideally, a platform is autonomous. It runs by itself and can handle regular operations, or unexpected problems, with ease. This can happen when you specify a target state (often declaratively) and the system constantly affirms you’re in that state, and fixes things if you aren’t. For instance, if a Pod in Kubernetes crashes, another gets scheduled automatically.
Liveness and readiness probes play an important part here. Many of the cluster operations (e.g., node failure) are handled manually, or, if you use a managed Kubernetes product, are handled there. In many cases, tooling around OSS projects (and platforms) is where the enhanced automation comes from. Platform builders should know what Kubernetes will do for them, and what an operator is on the hook to tackle.
For updating containers within a cluster, Kubernetes does offer rolling updates, which are handy for getting new components deployed with minimal disruption.
Clear extensibility points
What clearly differentiates Kubernetes from, let’s say, a hypervisor, is its extensibility. One type of extension is Custom Resources. These extend the out-of-the-box Kubernetes API and appear as first-class objects. However, there are actually lots of extensibility points in Kubernetes impacting everything from the kubectl CLI, all the way to network and storage plugins. One popular extension mechanism is DaemonSets, which you might use to run log collection or throw a custom monitoring agent on cluster nodes.
Realization of the Operator pattern
This represents one type of Kubernetes extension that lets operators automate repeatable deployment and management tasks for a workload. For instance, many products (e.g., Confluent’s version of Kafka) now support Kubernetes operators to provision instances for clients. For platform builders, having a straightforward recipe for simplifying app operations is a big deal. When your platform can run custom software and ISV products, the economics of your platform look even better.
Built-in security features for secrets and policy management
The security story for Kubernetes keeps improving, but it’s not too bad at the moment, either. As a platform builder, you’d want to know how to set up API authentication and authorization; deal with TLS certificates; segment users by namespace or cluster; set up security contexts and policies for Pods; configure quotas; set up network policies; and use the built-in secrets management. These are declarative or configured settings you’ll want to factor into your platform deployment atop Kubernetes.
It means there are things for you to check into when buying “Kubernetes-native” software.
Basically, if something gets advertised as “for Kubernetes,” make sure it actually uses Kubernetes-native objects or known extensibility points. If someone just ports their software to run on Kubernetes but doesn’t take advantage of Kubernetes itself, that’s a red flag.
You can find this out pretty quickly by just looking at the docs. MongoDB is doing a good job with its operator. The Greenplum analytics platform has a native operator, as well. As mentioned above, Confluent has done this, too. Projects like Knative and project riff also embrace and extend core Kubernetes types to bring serverless concepts to Kubernetes. Or look at kpack, which uses Kubernetes resource types to package code into containers that can . . . run on Kubernetes. How meta!
Dig in, see if your software vendor use standard or custom resources, rolling deployments, infrastructure-neutral APIs, graceful pod failure handling, and the like.
It means you’re going to adopt Kubernetes whether you intend to or not.
Not interested in running Kubernetes? Don’t feel peer pressure to use something you don’t need. However, you may not be able to avoid it.
Increasingly, software vendors are packaging up their offering as containers, or sets of containers. They like the idea of a single way to bundle up their platforms and distribute to customers. You’ll have to decide whether you want to manually run those containers, or operationalize container management with something like Kubernetes.
This is where I think VMware’s Project Pacific is a big deal. In this major re-architecture of vSphere, you’ll get native Kubernetes management alongside virtual machines, and with the same tooling your team uses to provision and manage infrastructure.
Even if you aren’t natively building or deploying containerized workloads, it’s likely you’re going to be running them. Start taking some self-study courses, get hands on, and go to the next VMworld.
It means Kubernetes is a platform itself that you need to account for.
Kubernetes is powerful. And it’s a complicated platform itself. There’s no way around that. If you think you’re getting out of “running” Kubernetes by using a managed cloud offering, think again.
"Kubernetes the hard way" falsely implies that there's an easy way.
— Czarkuberneteski @Home (@pczarkowski) March 2, 2019
A managed offering from Google or Microsoft still requires you to look after Kubernetes. Sure, you don’t have to stand up Nodes, but you still have all the usual lifecycle operations and configuration to make it usable. And you’re on the hook for performing upgrades (and testing that nothing breaks), planning for high availability, and all sorts of platform lifecycle tasks. That shouldn’t be shocking, but we can sometimes assume that “cloud” stuff means everything is taken care of for us.
It means Kubernetes isn’t the target API for many types of workloads.
Kubernetes isn’t for every workload. [Ducks] It’s too low-level of an API. In fact, one of the creators (and my future co-worker), Craig McLuckie, makes the point that Kubernetes should become increasingly invisible.
If we do our job right, people will stop talking about k8s in the next 5 years. Not because it goes away, but because it becomes a normalized and boring substrate supporting waves of new innovation above it.
— Craig McLuckie (@cmcluck) March 4, 2019
The reason you see companies like Microsoft (with Azure Spring Cloud), Pivotal (with Pivotal Application Service for Kubernetes) and even Google (with Cloud Run powered by Knative) jump into this space more aggressively is because we recognize that Kubernetes is awesome infrastructure that still needs a dev workflow. It needs a container build system. An app deployment API. A marketplace. App-focused management commands. You know, an application platform atop an infrastructure platform.
So adopt Kubernetes. Get it running. And recognize that it’s a platform for running other platforms that will improve your software delivery and business outcomes.
