kubernetes

Certifying Kubernetes with Sonobuoy

By Steve Sloka, Sr. Member of Technical Staff, and  John Schnake, Systems Engineer  

There are many ways to create Kubernetes clusters and many environments that can host them. As a result, platform operators find it difficult to determine whether a cluster is properly configured and whether it is working as it should.

Sonobuoy is an open-source diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of upstream Kubernetes tests in an accessible and non-destructive manner. It is a customizable, extensible, and cluster-agnostic way to generate clear, informative reports about your cluster — regardless of your deployment details. Sonobuoy is the underlying technology powering the Certified Kubernetes Conformance Program, which was created by the Cloud Native Computing Foundation (CNCF) and is used by every Certified Kubernetes Service Provider.    

Sonobuoy has three components:

  • A command-line utility that you use to trigger conformance tests, check status, view activity logs, and retrieve and analyze test results
  • An aggregator that runs in a Kubernetes pod to start plugins and aggregate their test results
  • Plugins that execute in ephemeral namespaces with a Sonobuoy sidecar to run specific tests or conformance frameworks

With a single Sonobuoy command, you can run the same tests that are used to qualify an upstream Kubernetes release. This ability provides strong levels of assurance that your cluster is configured correctly, and you can use the tool to debug configuration problems.  

Native Extensibility Through Plugins

Sonobuoy provides several plugins out of the box, including a systemd log collector and the upstream end-to-end Kubernetes conformance test suite. Sonobuoy is the community standard tool for executing conformance tests on a Kubernetes cluster; however, its architecture is designed to accomplish much more.

The open plugin architecture equips you, as a platform operator, with the means to develop custom conformance and validation tests for environments before they go into production. A custom plugin can be developed by creating a plugin definition file that describes how the plugin is structured and what parameters the plugin requires. The plugin then needs to follow a documented API that provides a communication mechanism for Sonobuoy to inform it of the plugin’s status including whether it is pending, running, or complete.

Other plugins from the community exist, such as Bulkhead. Bulkhead assesses the compliance of a cluster's control plane and worker nodes with the security guidelines for Kubernetes established in the CIS Benchmarks. These benchmarks are executed using kube-bench, a tool that implements the CIS Benchmarks based upon the version of Kubernetes that is deployed.

The Sonobuoy team is community driven and would love to hear of any plugins that you have created or ideas on what you would like to see developed. Please open an issue or find us on the Kubernetes Slack #sonobuoy!  

Future Plans

One of the recurring requests from the community has been the need to run Sonobuoy in an air-gapped environment, meaning that the cluster is physically isolated from the Internet. This is important for medical and financial services industries with stringent security requirements. For months, the upstream Kubernetes community has been working to make this possible and we are committed to ensuring that capability is implemented. This will empower anyone to verify and debug their clusters, regardless of Internet connectivity.

Another high-priority item is improving the developer experience and user documentation for creating and running custom plugins. The existing API that plugins have to meet is small but we know there are still some pain points in developing and using your own plugins. Expect documentation improvements, more examples, and improved integration with the CLI for custom plugins. By streamlining the plugin process, we hope to empower the community to create their own plugins and solve even more problems.

Check out the planned features for the next release (version 0.14) by looking at our Github milestone.  Sonobuoy is built by the community so make your voice heard! By creating or commenting on Github issues and communicating in Slack you can help influence future priorities. Issues labeled with help wanted or good first issue are a great place to start engaging with the project.  

Join the Sonobuoy Community!

Related Articles