This post was adapted from Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework by Mik Kersten.
For the majority of our careers, those of us involved with enterprise IT have been dealing with change at a frenzied pace. Technology platforms, software development methodologies, and the vendor landscape have been shifting at a rate that few organizations have been able to match. Those that manage to keep up, such as Amazon, and Alibaba, are further driving change by redefining the technology landscape around their software platforms, causing the rest to fall even further behind.
This daunting and unrelenting pace of change has been seen as a hallmark of the digital disruption. But if we step back and look at the patterns of progress that came before, we begin to see ripples of the great surges of change and development of previous industrial and technological revolutions.
Over the course of three centuries, a pattern emerges. Starting with the Industrial Revolution, every fifty years or so a new technological wave combines with ecosystems of innovation and financing to transform the world economy. Each of these technological waves has redefined the means of production so fundamentally that it triggered an explosion of new businesses followed by the mass extinction of those businesses that thrived in the culmination of the previous surge. Each wave has been triggered by the critical factor of production becoming cheap. New infrastructure is then built while financial capital drives the ecosystem of entrepreneurs and innovators who leverage the new techonological systems to disrupt and displace the incumbants of the last age.
Each of these technological revolutions has required existing businesses to master a new means for production, such as steam or the assembly line. For the digital revolution, the new means of production is software. If your organization has already mastered software delivery at scale, this book is not for you. The goal of this book is to provide everyone else with a new managerial framework that catalyzes the transition to the Age of Software.
Theories that explain the cycles of the last four technological revolutions and the first half of this one are proposed by Carlota Perez in Technological Revolutions and Financial Capital: The Dynamics of Bubbles and Golden Ages and by Chris Freeman and Francisco Louçã in As Time Goes By: From the Industrial Revolutions to the Information Revolution. Perez expands on the “long wave” or Kondratiev economic model by specifying two distinct periods within each cycle. The first half is the Installation Period, when a new technology and financial capital combine to create a “Cambrian explosion” of startups, disrupting entire industries of the previous age. At the end of the Installation Period is the Deployment Period of technological diffusion, when the production capital of new industrial giants starts taking over. Between these two periods is what Perez termed the Turning Point, historically marked by financial crashes and recoveries. This is when businesses either master the new means of production or decline and become relics of the last age.
Fifty years have passed since NATO held the first conference on software engineering in 1968 and the Age of Software officially began. Today, the pace of change feels relentless because we are passing through the Turning Point. At the current rate of disruption and decline, half of S&P 500 companies will be replaced in the next ten years.
These businesses, many of which were founded prior to the Age of Software, are starting to see a growing portion of their spending shift to technology as their market success is increasingly determined by software. However, the productivity of software delivery at enterprise organizations falls woefully behind that of the tech giants, and the digital transformations that should be turning the tide are failing to deliver business results.
The problem is not with our organizations realizing that they need to transform; the problem is that organizations are using managerial frameworks and infrastructure models from past revolutions to manage their businesses in this one. Managerial accounting, organizational hierarchies, and Lean manufacturing were critical to success in previous revolutions. But these managerial frameworks are no longer sufficient to successfully direct and protect a business in the Age of Software.
I had a chance to witness the pitfalls of this trap firsthand. Working with Nokia, I noticed that management was measuring the success of its digital transformation by how many people were trained on Agile software development methodologies and were onboarded onto Agile tools.
These activity-based proxy metrics had nothing to do with business outcomes. As I will summarize in Part I, Nokia’s transformation efforts failed to address the core platform problems that made it so difficult for the company to adapt to the changing market. In spite of what appeared to be a well-planned transformation, management was not able to realize this until too late. I watched with frustration as Nokia lost the mobile market it had created, in spite of the heroic efforts of my colleagues, who were doing everything they could to save the company.
A few years later, I was invited to speak with IT leaders at a global bank. The bank was six months into its third attempt at a digital transformation, and this time, DevOps tools were added to the mix and expected to save the day. The budget for the transformation was approximately $1 billion, but shockingly, I realized their transformation plan was even more flawed in its approach than the one at Nokia. Every aspect of the transformation was being project managed to cost reduction alone and not to project overall business outcome with reduced cost as a key metric. As I learned more, I started getting a visceral image that a billion dollars of the world’s wealth was going to go up in flames without producing any value. There were still eighteen months left to right the ship, but I knew that with cost alone as the foundation of the transformation, it was too late to alter course. Nokia had left me with an image of a burning mobile platform that destroyed a tremendous amount of wealth and prosperity. I now had a vivid image of the bank’s digital transformation lighting fires of waste across its ranks.
That was the day I started this book. There was something so fundamentally wrong with the way business people and technologists worked and communicated that even leaders with the best of intentions could still lead their companies into predictable decline.
How is this possible when we now have five decades of software practice behind us? The Agile and DevOps movements have made great strides in adapting key production techniques from the Age of Mass Production to the technical practice of building software. For example, continuous delivery pipelines allow organizations to leverage the best practices of automated production lines. Agile techniques capture some of the best technical management practices of Lean manufacturing and adapt them to software delivery.
The problem is, with the exception of some tech giants run by former software engineers, these techniques are completely disconnected from the way that the business is managed, budgeted, and planned. Software delivery concepts near and dear to technologists, such as technical debt and story points, are meaningless to most business leaders who manage IT initiatives as projects and measure them by whether they are on time and on budget. Project-oriented management frameworks work well for
creating bridges and data centers, but they are woefully inadequate for surviving the Turning Point of the Age of Software.
In this book, we will examine several digital transformation failures that caused organizations to lose their place in the market. We will then dig further into understanding the current state of enterprise software delivery by looking at a study I conducted with Tasktop, “Mining the Ground Truth of Enterprise Toolchains,” that analyzed the Agile and DevOps toolchains of 308 organizations to uncover the causes of this disconnect between business and technology. Project to Product will then provide you with a new management framework and infrastructure model, called the Flow Framework, for bridging this gap between business and technology.
The Flow Framework is a new way of seeing and measuring delivery and aligning all of your IT investments according to value streams that define the set of activities for bringing business value to the market, via software products or software as a service (SaaS). The Flow Framework displaces project-oriented management, cost center budgeting, and organizational charts as the primary methods of measuring software initiatives. These are replaced with flow metrics for connecting technology investment to business results. The Flow Framework allows you to scale the Three Ways of DevOps—flow, feedback, and continual learning (as outlined in The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations)—beyond your technology organization and to your entire business.
With each technological revolution, a new kind of infrastructure has been established in order to support the new means of production. Canals, railways, electrical grids, and assembly lines were key infrastructure components that underpinned the technological ecosystems of previous cycles. Many digital transformations have gone wrong by over applying infrastructure concepts of the last revolution to this one. Production and assembly lines are great at reducing variability and reliably producing similar widgets, but software delivery is an inherently variable and creative endeavor that spans a complex network of people, processes, and tools. Unlike manufacturing, in modern software delivery the product development and design process are completely intertwined with the manufacturing process of software releases. Attempting to manage software delivery the way we manage production lines is another instance where frameworks from previous technological revolution are failing us in this one. The Flow Framework points to a new and better way.
What if we could see the flow of business value within our organizations in real time, all the way from strategic initiative to running software, the way the masters of the last age ensured they could see and collect telemetry for every step of the assembly line? Would we see a linear flow or a complex network of dependencies and feedback loops? As the data set of 308 enterprise IT toolchains we will examine in Chapter 8 demonstrates, we see the latter. This flow of business value within and across organizations is the Value Stream Network. In the Age of Software, Value Stream Networks are the new infrastructure for innovation. A connected Value Stream Network will allow you to measure, in real time, all software delivery investments and activities, and it will allow you to connect those flow metrics to business outcomes. It will empower your teams to do what they love doing, which is to deliver value for their particular specialty in the value stream.
A developer’s primary function and expertise is coding, yet studies summarized in this book have shown that developers spend more than half their time on manual processes due to disconnects in the Value Stream Network. These disconnects are the result of relics that go back two technological revolutions: Taylorism, which resulted in the treatment of workers as cogs in a machine, and the silos that have formed in functionally structured organizations.
Successful businesses in the Age of Mass Production aligned their organizations to the value streams that delivered products to their customers instead of constraining themselves to rigid functional silos that disconnected specialists from each other and from the business. For example, Boeing, a master of the Age of Mass Production, could never have brought the highly innovative 787 Dreamliner to market and scaled its production to meet the growth in demand if the company had been structured like today’s enterprise IT organizations. Organizations that manage IT delivery as projects instead of products are using managerial principles from two ages ago and cannot expect those approaches to be adequate for succeeding in this one. Visionary organizations are creating and managing their Value Stream Networks and product portfolios in order to leapfrog their competition in the Age of Software.
The future of software delivery is already here; it’s just not evenly distributed yet. Software startups and digital natives have already created fully connected Value Stream Networks that are aligned to their product delivery, are focused on flow over siloed specialization, and connect all of their software delivery activities to measurable business results. Their leaders speak the language of developers, often because they were developers, which enables them to effectively direct their software strategies. What does that mean for the fate of every other company? How can we bridge the gap between technology and business leadership to create a common language that allows the rest of the world’s organizations to thrive in the Age of Software?
While organizations ponder these questions, the tech giants that have mastered software at scale are expanding into traditional business, such as finance and the automotive industry. The tech giants are mastering traditional businesses more quickly than the world’s established companies are mastering software delivery. In doing so, they’re amassing a growing portion of the world’s wealth and technology infrastructure.
The product offerings they have created are delivering fundamental value to businesses and consumers, and the market pull for that value will only grow. Trying to slow progress or demand is foolhardy; but leaving the economy to a handful of digital monopolies will be problematic for our companies, our staff, and our social systems. If we do not turn this tide—the increasing amount of wealth in the hands of tech giants, and the network effects of technologies making effective government regulation difficult at best—the consequences could be more dire than the mass company extinctions that we witnessed in the four previous ages.
We can create another future. We can make our organizations competitive. We can leverage the lessons of the tech giants and startups and adapt them to the complexity of our existing businesses. We can turn the black box of IT into a transparent network of value streams and manage those the way the digital natives of the Age of Software do. To achieve this, we need to shift our focus from transformation activities to measurable business results. We need a new framework to shift our organizations from project to product, thus securing our place in the digital future.