Driving competitive advantage via data insights to inform better, faster decision-making is quickly becoming an imperative enterprise strategy among Public Sector organizations, including the U.S. Department of Defense. Driven by transformative outcomes achieved in private industry, such as doubling market cap and increasing revenue per employee by 37%1, embracing a collaborative data ecosystem business model is paramount to achieve ‘decision advantage’2 as the DoD envisions in its Combined Joint All-Domain Command & Control (CJADC2) Strategy. Similarly, the Federal Data Strategy highlights ‘Identify Data Needs to Answer Key Agency Questions’ and ‘Use Data to Guide Decision-Making’ as key practices.3 Yet, is the Public Sector embracing the culture and operational change that becoming a data-led organization entails?
For decades, data architectures were implemented through the lens of capturing accurate records of the transactional outcomes of process-oriented applications – ‘process before data’. Modern software practices are employed to automate mission or business processes, and this enhances the outcome of more content-rich data repositories. We are no longer limited by a failure of imagination. The opportunities to synchronize separate data sets are fueling innovation. Now, the emphasis shifts to connecting these data repositories and achieving the nirvana of near real-time, machine-augmented decision-making as the decisive capability in future competition. So, at present, process automation and connecting the resultant data are still seen as the means to the end.
However, achieving the vision of ‘decision advantage’ requires an even more fundamental shift, inverting the paradigm of process before data to one where collaborative data sharing among connected ecosystems is the business4. As Steve Jones writes, “Connected Ecosystems aren’t about process, because fundamentally you aren’t creating a process across organizations, you are creating collaborations which are driven by data.”5
Envisioning such a data-led organization within the context of the DoD’s aim to “sense, make sense, and act”6 with distributed control upon a commander’s enterprise-wide intent, for example, is a logical corollary to the organizational transformation requisite of CJADC2 success.
So, what makes a data-led organization?
Data-led organizations have a lot in common with modern software development and application platform teams. Organizations shifting to modern software practices, like DevSecOps, and establishing application platform teams, find that every facet of their people and processes transform to leverage technology in a new way. Attributes that ring true of both agile software development teams and platform teams7 alike are disaggregated, empowered teams utilizing lean product management, user-centered design, bounded context, domain-driven design, iterative/adaptable engineering practices like extreme programming (XP), and highly automated technical implements across abstracted and modular architectures optimized to enable repeatable business outcomes at scale.
Decentralizing data ownership to the very teams building apps and data sources, commonly aligned around business outcomes and normalized by a collaborative governance process rooted in modern data practices and culture enablement, are central to a data-led organization’s ethos. While this organizational model seems compatible with the notion of distributed Command & Control (C2) within DoD’s multi-domain operations context, traditional, top-down and bureaucratic data governance structures, which are often the norm in public sector organizations, are antithetical to the culture, process modernity, and technical capabilities data-led organizations employ.
As Souhaib Guitouni recounts lessons learned from BlaBlaCar’s data transformation, he describes an 18-month journey becoming a Data Mesh organization with business domain-aligned, data-owning ‘squads’ supported by a central ‘data platform team’.8 Through trial, tribulation, and triumph, BlaBlaCar became data-led by adopting the people, process and technology attributes that high performing software development and app platform teams employ.
So why a Data Mesh, and what is it? As Martin Fowler describes9, a data mesh is a next-generation data approach that embraces decentralization to enable value at scale via the principles of10:
Distributed, domain driven architecture and ownership of data
Data as a product (product thinking)
Self-service data infrastructure as a platform
Federated computational governance
Data meshes overcome the friction of data warehouse and big data platform architectures, which followed the ‘data after process’ paradigm and do not enable the ‘data is the business’ paradigm. Specifically, as Martin Fowler relays, we are in the era of ubiquitous data and source proliferation (not unlike the multi-domain battlefield envisioned by DoD)11, and, “the ability to consume it all and harmonize it in one place under the control of one platform diminishes”.12 Further, Fowler describes how the need to leverage myriad data perturbations for vastly proliferating internal business innovations, and external consumer demands, outpaces the response times possible with legacy data architectures.13
