For those managing enterprise software development organizations, the concept of software supply chains—that is, the set of sources and actions that take software from “raw materials” to a finished product—might represent an abstract concept. While this definition is essentially correct, it doesn’t do enough to explain how supply chains can be complementary to your existing continuous integration and continuous deployment environments.
As both the velocity of your software development processes and the number of applications going through those processes increase, the scale of coordination across all of those application teams and their stakeholders increases exponentially. However, most CI/CD tooling has been built to define processes that deliver a single application at a time. Even if you have a quality central release engineering team (and many of our customers do), ask them and they will tell you they spend a significant percentage of their time negotiating with developers to manage the specifics of each application process.
Your path to production is the entire set of processes and activities from conception of a software project to that project creating value for its users, whether they are consumers, patients, or employees. Today’s path to production is a robust, interconnected set of inputs, actions, and release automation to handle testing, security verification, auditable governance data capture, and much more. Everyone from business owners and general managers to enterprise architects and SREs to your infosec teams are stakeholders in the process that software goes through to safely and securely reach its consumers.
This is where the linear and sequenced “one process per team” approach of today’s CI/CD tools is limited. We can create and run plenty of processes, but when it comes to monitoring or changing hundreds at a time, these tools become unhelpful. VMware frequently speaks with release engineering managers and platform engineers who spend much of their time in meetings with security, operations, and dev teams trying to get each application to understand and accept any changes to process routing and policies, and to prepare for any unintended consequences as multiple applications are updated. They are frustrated that these activities can’t be coordinated across the organization more efficiently, with less toil.
Add to this the fact that most large organizations are moving to a multi-cloud or hybrid-cloud platform model. This drives variation in the tooling and services used by each development team, which makes organization-wide path-to-production management that much harder. The combination of options for process definitions, tooling, services, and policies creates a vast array of different scenarios to be managed.
How software supply chains enable scalability
We’ve seen this story before. The very applications we are delivering to production through these mechanisms are themselves a part of a larger-scale application system. We require common services to access and modify shared data. Messaging and event-streaming infrastructure increasingly play a role in the way we signal important activity between applications. Applications run in a variety of environments, and operations must adjust to the ever-changing capabilities of each environment.
Because of the challenges involved with managing this constantly evolving system, we’ve undergone several revolutions in the way we build and operate software in the past decade. Two that stand out are microservices architectures and common provisioning through the Kubernetes APIs.
-
Microservices – One of the key things that a proper microservices strategy buys a large organization is the clear separation of concerns between owners of different business domains, enabling changes to different parts of the system to happen independently from all of the others. For the enterprise IT manager, this also means teams can be staffed, trained, and run without constant coordination with every other application in the corporate portfolio.
-
Kubernetes autoreconciliation – One of the really powerful concepts Kubernetes brings to the table is the idea that each manager of a certain resource (e.g., a supply chain activity) is given an ideal state for the resources involved (e.g., the output of an action), and any variation from that state triggers the resource manager to attempt to return to the desired state, a key tenet of GitOps.
When we combine these two properties for building paths to production, we gain the ability to scale the number of applications served by the platform without sacrificing the ability for organizations to move independently of one another. Governance without losing agility is how great digital organizations thrive.
First, the microservices approach creates newfound agility for the different teams involved in the path to production by enabling them to deliver services to the entire organization. One security team, for example, might provide both code and image scanning as a service. Meanwhile, an operations team might run deployment tools tuned to the needs of different application categories—say, architectures or compliance regimes. These different services can then be coordinated through a supply chain choreographer, like the open source project Cartographer or the commercial Supply Chain Choreographer of the VMware Tanzu Application Platform. Learn more about supply chain choreography in this recent RedMonk interview.
Services can be updated by each team as required without the need for refactoring of any of the consuming supply chains. Multiple tools can be offered to development and operations teams, allowing them to select the best ones for a specific workload type. Supply chains become composable in the same way that the text-processing tools in Linux can be chained together using piping.
This is an example of a supply chain that combines many of the elements discussed so far. Note that the build process uses an existing Jenkins CI process, there are supply chain options for image scans with different required thresholds, and while the platform operations team manages the overall supply chain flow, developers are given control of the code-image process and SecOps owns final code and image scanning.
In fact, even those application categories can have baseline supply chains that can be customized as needed for each application. The same services are used, but perhaps an additional test is run or a different approval process is triggered for a specific application.
To support this separation of concerns, VMware includes a convention service in Tanzu Application Platform. A convention is essentially an annotation of configuration that can be applied to any application as it passes through the supply chain. Thus, if all PCI-compliant applications require certain ingress rules for networking, the security team can create a convention that gets applied in any supply chain for PCI applications. The convention is independently managed from the supply chains themselves, and can thus be version controlled and deployed independently, as well.
Interestingly, you do not need to replace your existing CI or CD tools to enjoy these benefits. You can turn your existing processes into services that can be included in a supply chain. Autoreconciliation is applied to your CI output before the remainder of the supply chain is executed, giving you automated verification of the correctness of that output. Your existing CD automation can be triggered when the appropriate supply chain conditions are met, ensuring only validated applications are deployed to production. Most customers will start by integrating their existing tools, then slowly evolve different elements of the path to production to take advantage of the relative strengths of CI compared with those of supply chain management.
All of this combines to create an environment that enables sophisticated development practices across large numbers of applications without the need to negotiate each step for each application. It just scales better.
Supply chains for better organizational structures
You can see how composable software supply chains will also help you create better organizational structures for fast, agile software delivery. There is so much flexibility in this model that you can organize teams in a variety of ways.
If your dev teams want to own their build processes, you can focus platform teams on providing the tools to let them do so, but wrap them in automated security, operations, and integration tasks that take much of the toil off of developers’ plates. Alternatively, if they want to simply push code and have everything “just happen” from there, you can build release management teams that focus on creating just that experience. If your operators want more flexibility in setting and changing resource consumption standards, you can give them the tools to do so without changing anything about the developer experience.
The heart of the supply chain choreography approach is simply to leverage the same architectural principles that we’ve adopted for building complex application systems: microservices and a common infrastructure abstraction. This composable environment in turn gives you tremendous flexibility in how you organize and optimize your software supply chains and the stakeholders involved, while maintaining strong platform-as-a-product practices.
Download related white paper: Why You Should Treat Platform as a Product
Your organization likely has challenges with adapting to change without breaking key guardrails and feedback loops. A good software supply chain is one way to address that problem, and to build a software practice that can move quickly and create innovative customer experiences.
