Matt Reiswig co-authored and created the illustrations for this post.
Mainframe modernization is typically viewed as a multiyear, transformational journey that requires long planning phases and a massive buildout before any returns can be realized. To be sure, there are many enticing short-term migration options, like offloading work to cheaper processors, getting volume discounts from your mainframe provider, using hardware and middleware emulators, leveraging COBOL-to-Java code generators, slapping REST APIs on top of the mainframe systems, and outsourcing the refactoring and rewriting of code. But while these efforts are well-intentioned, they quickly get stuck in the mud because they don't scale and as such, are unsustainable over the long term.
In the meantime, modern software development XP principles emphasize starting simply, with fast feedback loops that enable developers to deliver early and often. So, how do enterprises reconcile development velocity with the belief that mainframe modernization requires extended planning and execution? At VMware Tanzu Labs (formerly Pivotal Labs), we take a six-step iterative approach to mainframe modernization that provides the business with both incremental gains and ROI.
Migration from the mainframe monolith to cloud-first microservices
We acknowledge that mainframe modernization is hard. And the process gets worse before it gets better as concurrent development work streams have to be maintained both for the legacy and net new development. Having helped multiple customers on this journey, we have come up with an iterative, phased approach to mainframe modernization that not only scales, but yields ROI in days and weeks, not months and years.
The goal here is NOT to rewrite the COBOL code as-is, line by line, to a distributed platform, as such an approach is destined to fail. Rather, we want to retire code and any functionalities that are no longer needed in order to satisfy the current and future needs of the business. To achieve this, we take the following six steps:
-
Start with the end in mind
-
Discover candidate services
-
Map thin slices
-
Design notional target architecture
-
Create a user story backlog representing tactical patterns to strangle the monolith
-
Craft a roadmap of epics and build
1. Start with the end in mind
What is the business event or situation that has triggered the modernization? It is very important to understand why the modernization program is being funded so we can create the right set of goals, objectives, and key results. So we ask our customers: Are you doing this because you cannot add features fast enough? Because you need to provide your end user with a new, digital, 360-degree experience? In other words, what are the key business drivers?
Such alignment must be driven by the business and technology executives and reinforced by all the product owners, product managers, and technology leads. The outcome of this phase is a clearly articulated set of goals and objectives with quantified key results that act as journey markers to ensure that the mainframe modernization program is on track.
2. Discover candidate services
It is then time to take inventory of the critical systems running on the mainframe in terms of the business processes they enable so they can be broken down into discrete independent capabilities that can be modeled as microservices. This process is called event storming. It is used to understand the top constraints, conflicts, and inefficiencies of the system and reveal the underlying bounded contexts while allowing for the modeling of new flows and ideas, the synthesizing of knowledge, and the facilitating of active group participation in order to ideate the next generation of a software system. The seams of the current system tell us how the new distributed system should be designed, but we also weave in aspects of XP like user-centered design and design thinking to ensure our understanding of the system matches with reality. The outcome of this phase is a set of service candidates — also known as “bounded contexts” in domain-driven design terms — that represent the business capabilities as a set of core, supporting, generic business domains.
3. Map thin slices
A critical system on the mainframe — such as, for example, commercial loan processing or pharmacy management — has multiple end-to-end workflows. We use event storming to understand all the key business flows because they provide the steel threads for modernization. And we need to prioritize these key flows, as they will drive the system design. The first thin slice we pick should be a happy path flow that provides end-to-end value, demonstrates incremental progress, and ultimately redoubles everyone’s faith in the whole process. The outcome of this phase is a set of prioritized thin slices that encompass the event storm and provide business value.
4. Design notional target architecture
Now that we have identified all the right pieces, it's a matter of putting them together to build the new system using messaging, data, APIs, and UIs. We wire up all our domain services using a process that we invented at Pivotal called Boris, which provides a structured way to design synchronous API and asynchronous event-driven service interactions. We identify relationships between services to reveal the notional target system architecture and record them using a tool called SNAP concurrently with the Boris exercise. We document the APIs, events, data models, and views that would apply to a bounded context, as the thin slices are modeled across contexts (services). The outcome of this phase is a notional target architecture of the new system with external interaction modes mapped out in terms of messaging and APIs.
5. Create a user story backlog representing tactical patterns to strangle the monolith
In some ways, the mainframe modernization process is like bringing a plane down to the ground from 30,000 feet. At this point, we are in full descent and hovering at around the 10,000-foot level. We have a target architecture, but we need to understand how we will co-develop these old and new systems without disruption (i.e., how to change the engine while the plane is descending). To do this, we employ a set of tactical patterns for modernization like the anti-corruption layer, facade, and data-driven strangler to carve out a set of MVPs and align user stories. The user stories will realize the SNAP we built earlier and implement the thin slices we modeled. The outcome of this phase is a set of user stories for modernization that implement the tactical patterns for co-existence with the monolith.
6. Craft a roadmap of epics and build
We now have a backlog of user stories, and so in airplane terms we are less than 1,000 feet from the ground. At this point, it is important to identify the biggest risks to the technical implementation — like latency, performance, security, tenancy, etc. — and resolve them. To that end, we create a user story journey map that will realize the SNAP we built earlier and implement the thin slices we modeled. Creating a roadmap of MVPs for all the work is crucial as we start building out contracts for our APIs to unblock other teams and dependents. We organize the stories into epics at inception, and once we allocate the product managers and engineers, we begin the first iteration, setting the feedback loop from product engineering to business in motion. The inception ceremony creates a roadmap of the MVPs tracked by the epics and constituent user stories. Establishing a cadence for demos, iteration planning meetings, and retrospectives in order to set in motion the feedback loop for incremental modernization is imperative, as this will set the tone for the rest of the project. The outcome of this phase is the first iteration of development.
Make the 6 steps work for you
The six steps of mainframe modernization are not implemented in a waterfall model; we sometimes run them multiple times for different areas of a complex domain. And if we already know parts of the doman well, we may skip some steps altogether. This six-step process is not dogmatic. Do what works for you, with velocity to modernize the system, in incremental steps, and a target architecture map in hand. Mainframe modernization is hard. But by internalizing the process and starting the journey of a thousand miles swiftly with the first step, you’ll easily get to where you’re going.
