With the announcement of Project Pacific as a technology preview*, you’re probably seeing Kubernetes mentioned a whole lot more than you’re used to. Kubernetes will now be used to manage other Kubernetes clusters and VMs. This might seem like a radical departure from the container orchestration role Kubernetes is known for, but in reality this is just another extension of the Kubernetes control plane built on primitives that are natively exposed. In fact, all extensions of Kuberenetes apply the same pattern to different problem domains. By understanding one of the key developments in Kubernetes lifecycle management, Cluster API, you will be well positioned to understand Kubernetes, Project Pacific, and other Kubernetes extensions. This post goes into depth on Cluster API, what it means for Kubernetes deployments, and how it relates to the broader Kubernetes ecosystem.
What is Kubernetes Anyway?
Kubernetes is commonly thought of as a container orchestrator. However, the most important thing to understand about Kubernetes is its control plane, which does one thing: observe desired state and reconcile it. Declarative configuration is a way for you to express desired state. Controllers continuously work to ensure this desired state becomes reality. This is the pattern that Kubernetes APIs natively implement and allow you to extend. Bearing this in mind, you will be better served by thinking of Kubernetes more broadly as:
- A platform platform—a platform for building platforms
- A highly extensible control plane
- A declarative API
Kubernetes, in its native form, addresses the problem of how to deploy and manage a containerized application. Likewise, Cluster API extends the Kubernetes control plane to address the problem of how to deploy and manage Kubernetes clusters. With Cluster API, the same pattern of declarative configuration and controllers is applied to the lifecycle of Kubernetes itself.
Getting the Benefit without the Burden
The reason you should care about Cluster API is simple: With Cluster API in place, operators and developers carry less burden while deploying and managing Kubernetes. One of the complexities Kubernetes presents is the decision of whether or not developers should be responsible for the lifecycle of a cluster. What if some teams in your organization intend to manage cluster lifecycle and others don’t? There’s no right answer to this question, and the answer varies across organizations and can change over time. Cluster API provides a solution to developers and operators that is:
- Consolidated across operations and development teams
- Exposed by familiar Kubernetes primitives, abstracting away the imperative style of management offered by disparate APIs in a multi-cloud world
- Compatible with existing development and deployment toolchains
- Specialized to do one thing well so you can forget about managing automation written for configuration management tools
The common thread across the Kubernetes API and all of its extensions, including Cluster API, is this: You can tell the API what you want and let something else analyze the current state and make forward progress.
Cluster API Workflow with clusterctl
The concepts in this section apply to any supported infrastructure provider. Getting started with a Cluster API infrastructure provider begins with the two workflows illustrated in the diagrams below. Light blue denotes a Kubernetes in Docker (KinD) cluster, and dark blue denotes a cluster deployed to an infrastructure provider. Green boxes denote a controller or set of controllers. Red boxes denote CustomResourceDefinitions (CRDs), which express desired state for objects that extend the Kubernetes API. Yellow boxes represent user workloads.
Deploying the Management Cluster (Workflow A)
Workflow A involves two clusters. It begins with an ephemeral KinD bootstrap cluster and ends with a Management cluster. Clusterctl deploys the bootstrap cluster and Cluster API to it. The control plane box represents all the native Kubernetes control plane components, while the machine and cluster controllers are deployed as part of Cluster API. The CRDs in the bootstrap cluster describe the desired state of the management cluster. The cluster and machine controllers observe these CRDs and query the infrastructure provider API to reconcile desired state. The end result is the management cluster.
Notice that the bootstrap and management clusters both have Cluster API controllers. This is because the bootstrap cluster is ephemeral and will disappear after the management cluster is deployed. The deployment process for a management cluster uses a short-lived Cluster API instance to deploy one that is long-lived. This automated migration of resources (by clusterctl) is a critical step to understand and is known as the pivot. If the whole point of Cluster API is to deploy clusters, why not deploy the management cluster with Cluster API?
Deploying Workload Clusters (Workflow B)
Workflow B also involves two clusters, but it uses the same management cluster deployed in Workflow A instead of a bootstrap cluster. The management cluster is a long-lived instance of Cluster API and is used to manage all workload clusters. Notice that the binary used in this workflow is kubectl. Clusterctl is only needed to provision a management cluster and is no longer necessary after Workflow A.
The CRDs in the management cluster describe the desired state of a workload cluster. The cluster and machine controllers observe these CRDs and, again, query the infrastructure provider API to reconcile desired state. The end result is a workload cluster. Notice that the workload cluster is without Cluster API controllers because it is responsible for user workloads, not managing other clusters.
Similarities in the Deployment Workflows
Both deployment workflows have two fundamental steps:
- They use a binary to deploy CRDs, which represent desired state.
- They allow controllers to observe the desired state and reconcile it.
These are the same steps required to use any of the native Kubernetes APIs. Cluster API is a great example of how flexible this model is by allowing users to extend the Kubernetes API from managing applications to entire clusters.
For more information about either workflow, as well as a working example for VMware vSphere, see the Getting Started guide for the Cluster API provider for vSphere.
Do You Need to Use clusterctl?
Clusterctl is used for automating the bootstrap cluster to produce a management cluster. In reality, any Kubernetes cluster may become a management cluster by applying the Cluster API components to it with kubectl. This section uses clusterctl to show how a user could get from nothing to managing Kubernetes clusters via a management cluster.
Tying it All Together
You should now have a better understanding of how Cluster API addresses the problem of how to deploy and manage your Kubernetes clusters. Project Pacific uses the same pattern and API to take this idea even further. Kubernetes is integrated directly into VMware ESXi and vSphere to create a Supervisor Cluster. From here, the same pattern of desired state and controllers will be repeated to address the problem of how to deploy and manage a fleet of VMs and Kubernetes clusters on vSphere. In the same way that Cluster API extends the Kubernetes control plane, the Supervisor Cluster extends the vSphere control plane. This extension will provide an API that is:
- Consolidated across operations and development teams
- Exposed by familiar Kubernetes primitives, abstracting away the imperative style of management offered by disparate APIs in a multi-cloud world
- Compatible with existing development and deployment toolchains
- Specialized to do one thing well so you can forget about managing automation written for configuration management tools
These are the same advantages that benefit developers and operators when Cluster API extends the Kubernetes control plane, but these advantages will now be possible in vSphere, too.
What’s Next?
Understanding the Cluster API pattern prepares you to manage the lifecycle of Kubernetes effectively and to learn, use, and contribute to the vast ecosystem of Kubernetes extensions. How else might you apply this pattern to help your organization? What else could Cluster API automate on vSphere or other cloud providers? The community is always welcoming new contributors, so jump in and start sharing your ideas.
Join Us!
- Cluster API community forum
- sig-cluster-lifecycle Google Group to gain access to documents and calendars
- Cluster API working group sessions—weekly on Wednesdays at 10:00 PT on Zoom
- Provider implementer office hours—Weekly on Tuesdays at 12:00 PT (Zoom) and Wednesdays at 15:00 CET (Zoom)
- Chat with us on Slack: #cluster-api
* There is no commitment or obligation that technical preview features will become generally available.
