by Merlin Glynn, Technical Product Manager, VMware
Cloud Foundry Container Runtime (CFCR), formerly known as Project KUBO, is an open source project that delivers the functionality of both Day 1 (deployment) and Day 2 (operations) tasks for Kubernetes clusters. The initial genesis behind CFCR was to make deploying and running Kubernetes clusters across different environments more portable and operational.
Until now, there has been no reliable or convenient way to deliver a strong level of operational capability to a consumer who may want to run Kubernetes in production on their own on-premises and public clouds. To solve this problem, Google partnered with Pivotal (the leading contributor to BOSH) to build Cloud Foundry Container Runtime. CFCR was formerly known by the acronym KUBO, meaning Kubernetes on BOSH. BOSH is an open source tool for the deployment, release engineering, lifecycle management and monitoring of distributed software systems. Google and Pivotal saw BOSH as a tool with the potential to facilitate production-grade Kubernetes operations.
On its own, Kubernetes does a great job maintaining healthy running workloads. However, it’s not so great at self-care of its control plane components like its API, controller manager, etc or its core kubelet processes. BOSH provides health and monitoring capabilities to the complete Kubernetes control plane to keep not only app workloads healthy, but also Kubernetes itself healthy and running.
To accomplish this, CFCR is deployed a little differently than when deployed with tools like ‘kubeadmin’ or ‘kops’. When BOSH deploys a Kubernetes cluster, each core component of the Kubernetes control plane is instantiated as a virtual machine (VM) instance. BOSH deploys an agent on each VM instance to monitor the health of the key Kubernetes control plane processes, as well as the overall health of each VM instance. BOSH will also dynamically repair and rebuild any VM that is unhealthy, no manual intervention required.
In addition to the health management of Kubernetes, CFCR deployments gain the added benefits of scaling, patching and upgrading Kubernetes clusters easily via a simple interaction with BOSH. Why is this so advantageous? Customers running Kubernetes will likely at some point need to upgrade and re-deploy, which is a taxing process. BOSH greatly simplifies this.
Operating Kubernetes is difficult, generally speaking. CFCR was designed to address the complexity of Kubernetes deployment and make it easier to deploy, patch, upgrade, scale and operate. The BOSH approach to Kubernetes provides some nice advantages. One of those advantages is repeatability. BOSH can deploy Kubernetes across multiple IaaS providers, such as vSphere, Google Cloud Platform and Amazon Web Services. This is accomplished via the BOSH Cloud Provider Interface (CPI), which allows BOSH to create and manage VM instances, storage and networking constructs across supported CPIs.
BOSH also utilizes a few additional abstractions like ‘releases’ to package software, ‘stemcells’ to define a secure VM image and ‘manifests’ to define how the releases get deployed across one or more VM instances based on the stemcells. A Platform Reliability Engineer can use these abstractions to make Kubernetes deployments easy and repeatable across any of the CPIs available to BOSH. This creates a great common operational model across any cloud.
Cloud Foundry Container Runtime also has a lot to offer when it comes to Day 2 operations, meaning after the Kubernetes clusters have been deployed. Tasks like patching CVEs, upgrading Kubernetes or rotating key credentials can be pretty cumbersome. BOSH offers Platform Reliability Engineers the ability to automate all of these tasks in a consistent and repeatable manner that drives down costs and time to deliver. Additionally, spinning up or decommissioning multiple Kubernetes clusters when they’re no longer necessary can also be automated and logged in the BOSH database to provide a level of task auditing.
CFCR is a fundamental part of the new container service platform announced at VMworld 2017 called Pivotal Container Service (PKS). The primary objectives of CFCR and PKS are to make Kubernetes and the operations of Kubernetes as a Service simple and production ready. Key PKS features will include:
- Tight integration with NSX-T for container and application-aware security.
- Project Harbor, VMware’s open source Docker-based registry with embedded features like Clair vulnerability image scanning, image policy controls and authentication.
- An on-demand broker that will allow entitled Application Developer/Operations Owners (ADO) to self-service request or delete Kubernetes clusters via a command-line interface (CLI) or API.
- Out of the box integration with VMware vRealize monitoring and logging components.
- World class, production-grade support of Kubernetes from VMware and Pivotal.