SPIFFE and SPIRE are a pair of interconnected open source identity management projects that we help maintain. Both are currently incubating with the Cloud Native Computing Foundation (CNCF) and, as part of that process, recently underwent a third-party security audit. We thought it might be interesting to share the results (spoiler alert: they were very encouraging) and talk a bit about what made it such a valuable experience.
But first, some introductions: SPIFFE (aka the Secure Production Identity Framework for Everyone) is a set of specifications that support the automated issuance of workload identity and its federation across multiple domains. SPIRE, meanwhile, is a software implementation of SPIFFE that supports the automated issuance of SPIFFE identities through the use of strong hardware and software attestation.
The twin projects got started about four years ago and aim to do for software systems what national passports do for human beings. Passports follow a standardized layout, allowing them to be easily read and validated regardless of the issuing nation. Because of this, passports are accepted as identification pretty much anywhere and allow for the secure flow of people between nations. Until recently, though, there was no equivalent for software systems. SPIFFE and SPIRE fill this gap. SPIFFE is totally platform-agnostic, so it can be used to identify software deployed across disparate cloud providers, on prem hardware, Kubernetes clusters, bare metal servers, and more. SPIRE is a production-ready implementation of SPIFFE that aims to bring SPIFFE’s capabilities to as many environments as possible.
A third-party security audit is a CNCF requirement for all incubator-level projects, and while these audits can be time consuming, they act as essential markers of project maturity. Indeed, many enterprise security teams will demand that software be audited by a third-party firm before they use it. Just as importantly, though, audits are incredibly useful to the project itself.
Our audit was conducted by Cure53 in the first few months of this year, and the overall message we received from them was very positive. The auditors found no critical errors, determined that all of the first lines of defense that we had in place were sound and affirmed that the two projects “had been created with security in mind” and that their maintainers have been “on the right track regarding security.”
They did find two vulnerabilities: one in the legacy API that was being deprecated at the time, which they labeled high severity; and then a medium severity vulnerability around validation logic and administrative APIs.
Both were issues that we wouldn’t have necessarily found ourselves, underlining the value of going through the process. While we promptly issued a release to address both vulnerabilities, the medium-level issue opened our eyes to some sharp edges in the underlying RFCs upon which the SPIFFE specifications rely. The result was that it could be easy to confuse one SPIFFE ID for another, and showed that it may be possible to encounter a surprising situation, which is not something you want in a security project.
We presented the problem to our specification group and asked them to look at how we could change our rules and restrictions to work around (or even completely avoid) the conditions that are likely to result in surprise. That turned out to be a multi-month effort that we’ve only just completed. It took a lot of digging down into the edge cases and trying to try to devise the safest approach possible. We built tooling to both record current behavior and measure it after we applied certain changes to make sure we understood exactly what behavior was changing and why, and to make sure there weren’t any regressions. Finally, it was important for us to determine a path forward that would ensure that other similar software projects implementing SPIFFE don’t run into the same sort of problems that we did.
Responding to the audit gave our maintainer team the opportunity to run through a full-on security response procedure, and we’re now updating our procedures to incorporate what we learned from the experience.
When it came to issuing new releases in response to the audit, we worked very hard to offer changes that were highly targeted, super-safe and as backwards-compatible as possible. For the medium severity issue we also made it so that people could get the high severity fix but have a method to turn off the patch for the medium security fix while they adjusted their deployment, if they needed to. In the end, we ported back five versions of the software.
While we couldn’t publicize the vulnerabilities until we had the fix in place, we were able to give larger deployments warning that something was coming and that they should be ready to move to the new release. And the good news is that we’ve not heard anything in terms of people being negatively impacted by the changes we’ve made in response to the audit. In response to this learning, we will be formalizing a pre-disclosure communication policy.
SPIFFE and SPIRE are by their nature security projects, so security is always our first priority. But while the core project team has been focusing on the security audit, our wider contributor community has been working on several exciting key use cases that SPIRE will support in the near future. These include:
- Support for serverless workloads – SPIRE will be able to issue identities that they can use as soon as they boot.
- Integration with Cilium – Cilium will be able to retrieve and manage identities issued by SPIRE and then transparently encrypt, authenticate and authorize traffic from workloads that Cilium is managing.
- Confidential computing – allowing sensitive SPIRE components to run inside confidential compute enclaves in order to protect powerful cryptographic keys and bolster the security posture of the overall deployment.
- Supply chain integration – integrating SPIFFE and SPIRE into secure supply chain tools and technology, specifically to address key management issues, which is very difficult at present.
All of these will be incredibly valuable for SPIRE/SPIFFE’s mission of cross-platform identity management and would be great areas to start contributing to if you’re interested in helping us out.
If you are curious to know more about the projects, we’re planning a KubeCon North America co-located event called Production Identity Day, scheduled for October 11, that should be really informative. There will additionally be six sessions focused on SPIFFE/SPIRE at KubeCon itself. If you are headed to KubeCon, whether in person or virtually, we hope to see you there!
