Projects Features

Kubernetes 1.12 Release Lead Retrospective

I spent the period from July to September 2018 volunteering as the Kubernetes 1.12 release lead. As this stint comes to a close, I’m pausing to reflect on the experience.

A first observation relates to an aspect of open source commonly understood today: open source projects move fast! The Kubernetes 1.12 release demonstrates that velocity. It included almost 2000 commits and that’s just counting what went into the https://github.com/kubernetes/kubernetes repository (also known as “k/k”). The ‘git diff’ command tells me there were over 5,000 files changed for 1.12, with almost half a million modifications split 2:1 between insertions and deletions across the core code.

Adding in the contributions from the projects that comprise vendored golang dependencies, the numbers are even higher. There are also other adjacent projects under the Kubernetes organization which also deliver content associated with each release. In combination, you find a huge amount of work in feature additions, bug fixes, cleanups and refactors in a little over two months (we also have a code freeze period during the cycle). Now, if you query my commits, you’ll find exactly one in the core repository for this release. It effectively removed one line of code. Why would I highlight that I’ve done almost nothing from a code contribution POV? Because it is the norm!

Open source’s high velocity is only possible through the aggregated and coordinated work of a very large team. In my prior blog post on the 1.12 release, I talked about the release team, but there’s so much more to a release than that team. The core work for Kubernetes 1.12 came via hundreds of developers’ direct contributions in terms of pull requests and commits.  And many more individuals and companies indirectly contribute: Hundreds of people created issues in GitHub and thousands of people commented on those issues during this period.

The CNCF devstats tool also tells us that contributors span a huge ecosystem. We all know the big names in tech and their sustaining engagement in open source, but there is also a long tail of almost 500 affiliations in our Kubernetes community. While many of these are not names most of us immediately know and most of them contributed just a small number of changes, their combined effort is comparable to that of a very large organization and demonstrates the additive powers of scale that come when we contribute to the commons.

Kubernetes 1.12
Class 4: Nearly all initial patterns evolve into structures that interact in complex and interesting ways, with the formation of local structures that are able to survive for long periods of time.

So, while I happened to be the overall lead for the release team working to coordinate all of this activity, it was exactly that—a release team effort with around two dozen folks contributing in various ways to tracking features, bugs, issues, pull requests, testing, branching, building, packaging, release notes, documentation and communications. This, in turn, relied heavily on a further set of nearly 100 leaders within two dozen active Special Interest Groups (SIGs) in the project, who each coordinate the work of even larger pools of contributors within their technical subdomains.

This leads to a secondary observation and one I commonly try to relate to those who are new to open source: open source mostly only moves fast in aggregate. Any individual thing you care deeply about can take excruciatingly long to graduate to a sufficient point of readiness. For this reason, our release team at times felt like it was only ever waiting for things to finally finish and worrying over their status. Where is that feature code? Where are the docs? Can we find somebody to declare whether it needs a release note? What is wrong with tests today? Have the new test results come in? Why is that PR blocked? Is this new issue a release blocker? I’ve (only semi-jokingly) referred to the release lead role as that of “The Worrier in Chief.”

Yet again though, teamwork is the answer. For every worrying issue, our release team encountered, the community was there for us and for the project, ready and willing to address the situation. For any feature you see in Kuberntes 1.12, whether a notable, stable, graduated feature like TLS Bootstrap or an incoming alpha feature like the new RuntimeClass, pause and think about the hundreds of individuals who’ve worried over and coaxed it through the iterative, collaborative, open source process from inception through review and incremental improvement to the ultimate point of release. This is truly a wonderful thing to behold and a humbling thing to be even a small part of. I am so grateful to this open source development community I call home and to VMware’s sponsorship of my contributions!

For a much more detailed retrospective of the Kubernetes 1.12 release, check out these community meetings:

If this type of open source work sounds exciting to you, I encourage you to check out our Kubernetes community “role board” for volunteering opportunities, the Kubernetes project’s Contributor Guide and our VMware open source related job openings.

About the Author

Kubernetes 1.12Tim Pepper is a software engineer with over 20 years of open source development experience. He is currently a member of VMware’s Open Source Technology Center, acting as an open source developer advocate and contributing to upstream projects such as Kubernetes, where most recently he served as the Kubernetes 1.12 release team lead. In his past, he’s worked on the Linux kernel, device drivers, distributions, software updates, Android, OpenStack and an open source golang-based orchestrator for containers and a VM that wasn’t Kubernetes.

Stay tuned to the Open Source Blog for more insights on contributing to open source projects and follow us on Twitter (@vmwopensource).