The convenience of modern software distribution—relying on upstream projects to provide pre-built, easy-to-install packages—is undeniable. It is hard to dispute using public repositories of pre-built software like Docker Hub is great for getting up and running quickly.
However, recent security incidents, such as the unauthorized access to the build infrastructure of the popular security scanner Trivy, serve as a stark reminder of a fundamental vulnerability: when you use a pre-built binary, you are trusting not just the source code, but also the integrity of the entire supply chain that produced it.
This became apparent when over 1,000 SaaS environments were impacted by the malicious code hackers injected into builds of Trivy according Mandiant Consulting. While Trivy’s maintainers acted swiftly, and critically, determined that their source code was unaffected, the incident highlights a crucial consideration: where your software is built really matters. However there is a bright spot at the end of this story, because some users of Trivy were protected because of how they downloaded that open source project as you’ll read below.
The Risk in Pre-Built Trust
When you install a package, you are placing immense trust in the upstream provider’s security practices, including:
- Build Infrastructure Security: Was the server or environment used to compile the binary secure and uncompromised? This is where the Trivy incident occurred—a vulnerability in the infrastructure, not the code itself.
- Signing Key Security: Was the key used to sign the package protected? A compromised key allows an attacker to distribute malicious binaries disguised as legitimate updates.
- Distribution Channel Integrity: Is the repository or CDN from which you downloaded the binary secure against man-in-the-middle attacks or unauthorized uploads?
In a scenario like the Trivy infrastructure hack, an attacker could have potentially injected malicious code into the final binary during the compilation process, even if the public source code repository remained pristine and uncompromised. The checksums of the published binary would be valid, but the binary itself would still be a vehicle for the exploit.
Why Building from Source is Your Best Defense
Building software from the public, auditable source code offers an unparalleled layer of supply chain security:
1. Direct Code-to-Binary Verification
When you clone the repository and run the build command, you are compiling the exact code that has been publicly reviewed, discussed, and committed by the project’s developers.
- You control the environment: You are compiling the code on a machine and environment that you control and trust. This allows you to ensure the environment follows security best practices like SLSA-3, which provide added safeguards against attacks. This bypasses the risk of a third-party build server being compromised.
- Source Integrity is the Single Point of Trust: Instead of trusting the source code and the maintainer’s infrastructure, you only need to verify that the source code matches the publicly available, known-good repository (often verified via Git commit hashes).
- Source code changes are often reviewed by humans. Builds are not: A pull request has the added benefit of human oversight which could catch a malicious change before it’s introduced. Contrast that to a build system, which typically is fully automated, ideal for an attacker to inject a change that goes undetected.
2. Auditing and Reproducibility
For critical security tools or fundamental infrastructure components, building from source is a prerequisite for security auditing.
- Reproducible Builds: The ideal for open source is reproducible builds, where anyone running the same source code through the same build process yields an identical binary. While not all projects achieve this perfectly, striving for it allows for independent verification that the published binaries match the source code. When you build it yourself, you establish your own baseline of trust.
3. Isolation from Upstream Infrastructure Vulnerabilities
Building from source completely insulates you from build infrastructure compromises. If an attacker gains access to the upstream project’s CI/CD pipeline or build server, they can modify the published packages. If the source code remains secure, your self-built binary, compiled from that secure source, will also be clean. Using public build services like Github Actions can be convenient, but unfortunately these services are highly visible targets.
How to make Building from Source Scalable
It must be acknowledged that building from source isn’t always practical. It requires time, appropriate build dependencies, and computational resources. For complex applications, the sheer difficulty can be a barrier. However a solution Bitnami Secure Images means organizations don’t have to choose between security best practices and keeping their workload in-balance
Broadcom manages a state-of-the-art build infrastructure that adheres to Supply-chain Levels for Software Artifacts Standard Level 3. This means the build environment is completely on private, fully controlled infrastructure instead of easy-to-target public infrastructure such as Github where a lot of OSS is built today. Additionally it’s monitored by a team of experts who have been working with open source for decades.
For example, when the Trivy incident happened, Bitnami Secure Images customers were immune from the attack. And they didn’t have to do anything other than continue to pull containers from BSI’s private, secure registry of trusted content. All the while benefiting from the near-zero CVEs, FIPS and STIG support and world-class Helm charts.
The Trivy incident is a valuable lesson: The security of an open source project is not just about the code; it’s about the entire process from commit to package. By choosing to build from the source code, you are effectively cutting out one of the most volatile and hardest-to-audit segments of the software supply chain: the upstream build infrastructure. In the world of security, controlling your own destiny is the ultimate defense. The Open Source Advantage: Building from Source for Ultimate Security is made accessible and affordable for everyone with Bitnami Secure Images.
