Digital Transformation Bottlenecks: You Can’t Change the Organization

The companies I work with at Tanzu by Broadcom are constantly looking for better, faster ways of developing and releasing quality software. But digital transformation means fundamentally changing the way you do business, a process that can be derailed by any number of obstacles. In his recent video series, my colleague Michael Coté identifies 14 reasons why it’s hard to change development practices in large organizations. In this companion series of blog articles, we’re exploring each of those topics in more depth, providing advice on how you can address them in your business.

Today, let’s look at reason no. 14: You Can’t Change the Organization.

The race for digital transformation

Whether the goal is to increase customer revenue or modernize cumbersome legacy operating practices, business leaders see digital transformation as a core part of their strategy. Unfortunately, successful transformation does not come easily, and many leaders find themselves in a position where there is little improvement to show, even after significant investment in new tools. Recent research by McKinsey & Company found that “only 16 percent of […] organizations’ digital transformations have successfully improved performance.”

Digital transformation is not just about buying more software. It needs a shift in mindset, culture, and capabilities. Many companies are quick to procure new technologies, and there’s no lack of vendors ready to extol the virtues of their latest silver bullet. And, to be fair, new technology is often necessary to make change happen. However, these tools are impotent without supporting organizational structures, processes, and skills to drive value. In previous articles, we’ve talked about some of the main challenges that hold organizations back, including lack of clear vision and strategy, resistance to change, and insufficient capabilities.

In this article, we’ll focus more directly on organizational structure, and the need for it to adapt alongside digital transformation.

Shuffling deck chairs

In the ever-changing world of IT, the way an organization is structured plays a pivotal role in its success and productivity. Yet, when I visit leaders from across a broad portfolio of industries, I often find that the structure of their IT organization has not substantially changed in many years. Sure, they might have changed some names in an attempt to stay relevant, but the structure of teams, the shape of individual engineers’ capabilities, and their level of collaboration remains largely unchanged.

Conway’s Law states that “organizations which design systems…are constrained to produce designs which are copies of the communication structures of these organizations.” In simpler terms, it suggests that the structure of an organization directly influences how its software or products are designed. If your organization has siloed departments with limited communication channels, chances are your software will reflect this fragmentation. On the other hand, if you have cross-functional teams collaborating seamlessly, your software will likely embody this cohesion.

Recognizing Conway’s Law allows businesses to anticipate potential challenges in software development and can prompt them to address any structural issues that might be hindering progress, then adapt accordingly. By aligning organizational structure with desired outcomes, companies can create an environment conducive to creativity, agility, and, ultimately, success.

Team topologies

In their book Team Topologies, authors Matthew Skelton and Manuel Pais present four fundamental team topologies that help IT leaders organize their teams for optimal software delivery.

Complicated-Subsystem Teams are necessary where the business has a sufficiently complex system that requires specialized knowledge. Common examples would be networking or database administration. Enterprise IT has traditionally consisted of many such teams and are an efficient way to ensure that systems are built and maintained with the required level of expertise. As you have likely experienced though, there is often a disconnect with customer value and poor collaboration when delivering a stream of work together.
Stream-Aligned Teams are ideal for organizations that want to deliver value swiftly and independently. Customer-focused DevOps teams that work autonomously to deliver iterative change through local optimization are a great example of stream-aligned teams. As the organization grows, however, so too does the number of tools and ways of working—creating a challenge for managing overall quality, cost, and compliance to corporate policy.
Enabling Teams aim to bridge a capability gap. Organizations will often build a dedicated architecture team or a Center of Excellence to help other teams unify in the approach of a particular problem, such as design of microservices architecture or consistent adoption of public cloud technologies. They can provide tremendous value but often suffer from being seen by other teams as “living in an ivory tower.”
Platform Teams work well in organizations that want to enable stream-aligned teams to deliver work with substantial autonomy. They are ideal for organizations that want to lay a foundation for software delivery, providing essential services that other teams can utilize to perform their role efficiently. This approach has seen a dramatic rise in adoption, with Gartner predicting that “by 2026, 80% of organizations will establish platform teams as internal providers of reusable services, components, and tools.”

The rise of platform engineering

Over the past few years, I’ve helped a number of organizations understand how to make platforms their superpower. What started as a movement to define a new breed of infrastructure teams that delivered “platform as a product” is now a community that has grown substantially—both in numbers and maturity. PlatformCon 2023 drew a record-blasting 22,000+ attendees compared to just 7,000 in 2022. Meanwhile, the CNCF Platforms Working Group was formed and released its first white paper to define its understanding of “platform engineering” and its many benefits.

With new nomenclature however, we also inevitably begin to see a watering-down of its definition within enterprise environments. Much as DevOps began as a practice around collaboration, it quickly became a job title and a product type. It’s no surprise then that we’re already beginning to see traditional infrastructure teams label themselves as “platform engineering” with minimal change in mindset, culture, or capability.

A successful platform engineering team must have a product mindset and a culture of service that drives them to build capabilities that deliver value and delight other teams across the business.

Building a balanced team

The idea behind balanced teams is to bring together people from various backgrounds, experiences, and areas of expertise. When building a platform engineering team, it’s vital as a minimum to consider product design and product management in addition to classical engineering roles. This diversity allows for a wider range of perspectives and ideas, leading to more innovative solutions and better decision-making.

Product design is customer-focused and helps the team understand the real-world challenges of users. They will help guide the priorities of the team to align with the needs of users, as well as ensure that the finished product works in a way that meets the expectations of users.
Product management is also customer-focused, but through a lens of business viability. The primary question they help answer is “by solving these specific user problems with these specific solutions, are we creating valuable user and business outcomes?”
Engineering is concerned with implementing a solution that will satisfy the needs of the customer in a way that meets the goals of the product. Engineers help the team debate the feasibility and merit of potential solutions, while remaining mindful of technology constraints.

Sustainable success

In this article we’ve highlighted that digital transformation needs more than advancement in technology. Businesses must also consider how they will evolve the organizational structure to embody the mindset and cultural shift required to produce the desired results.

As much as we’ve explored what must happen in existing organizations to enable digital transformation, we must remember that the target is always moving. For a business to have the agility to react to changing market conditions, it must continue to have the flexibility to repeatedly alter its structure and ways of working to suit its environment. It is therefore vital to look at how the organization can reward individuals who embody a culture of innovation and collaboration and who will drive a consistent journey toward continual improvement.

To learn more, check out our ebook Mindsets and Tactics for New Leaders of Software Teams. And read about other common digital transformation bottlenecks we’ve encountered and how to overcome them: