This post was co-written by Nick Kuhn and Brian Cunnie.
During the development of the latest release of VMware Tanzu Operations Manager, the BOSH Ecosystem team faced new challenges when upgrading the base BOSH operating system image (stemcell) from Ubuntu Xenial to Ubuntu Jammy Jellyfish.
These migration challenges manifested as continuous integration (CI) timeouts that occurred across adjacent teams within the Cloud Foundry ecosystem. The cause of the failures was discovered to be that the system resource utilization within the BOSH director had unexpectedly changed during the migration to the Jammy-based stemcell. After further research, the team found that RAM utilization for the BOSH Director’s workers’ threads was approximately five times larger on the Jammy-based stemcell than its Xenial-based predecessor.
This memory increase presented a perplexing challenge as the Ruby code for the BOSH Director was identical in the Xenial and Jammy-based stemcells—memory utilization should actually have remained constant.
After extensive investigation, the team discovered the cause of the memory increase: the compiler used to create the Ruby interpreter, GNU Compiler Collection (GCC), paradoxically generated less-performant code on the Jammy stemcell than the older GCC on Xenial.
The team couldn’t fix GCC, but after pivoting to the Clang compiler as an alternative, they immediately noticed some fantastic results. Deployments using the new Jammy Clang stemcell were approximately cut in half compared to the previous Xenial stemcell!
The chart above depicts performance data from the team, aggregating 132 BOSH deployments with 10 virtual machines (VMs) per deployment. The green line represents the improved deployment times with the Jammy Clang stemcell. (It’s clear that lower is better.)
Please note: Real-world performance data may vary as any performance improvements observed will significantly depend on the size and complexity of a customer's environment.
The VMware Tanzu team strives to improve the operator experience for our VMware Tanzu Application Service and Cloud Foundry ecosystem. We are excited to see these significant enhancements benefit our customers by saving them time!
Want to learn more about the Tanzu Application Service and Cloud Foundry ecosystem?
Catch up on all the great work accomplished by Team Tanzu in 2022. And be sure to get the full scoop on VMware Tanzu Operations Manager 3.0.