Predictive Metrics: Four Frameworks for Portfolio Management

Now that we’ve looked at historical data, let’s turn to predictive metrics for portfolio management. In this edition of our Four Frameworks of Portfolio Management series, we’ll take the predictive qualities of The Flow Framework and more to predict next steps.

Considering Leading Indicators to Inform Future Results

To thrive in times of technological disruption and economic uncertainty, organizations depend on their ability to adapt quickly to a changing market environment by rapidly evolving their digital product portfolios. Common financial metrics provide only a part of the feedback loop needed. To thrive in a constantly evolving market, we need the ability to connect historical financial metrics with forward-looking and predictive metrics to assess future performance.

In the 1970s, General Electric and McKinsey partnered to create a nine-box matrix framework (see figure below) that offers a systematic approach for multi-business corporations to prioritize investments.

The GE/McKinsey portfolio evaluation tool includes a comprehensive assessment of both external market factors and internal capabilities that drive the strategic posture of a particular product or asset.

We’ve expanded on this useful tool for our recommended approach. We flipped the direction of the x-axis to go from low to high rather than from high to low. This has the effect of putting the strongest products in the upper right quadrant. We also renamed the x-axis “Ability to Execute.”

Assessing Attributes

The attributes that define industry attractiveness in the 1970s have not undergone much change. These attributes include the following: target addressable market (market growth rate, market profitability, pricing trends, competitive intensity/rivalry, baseline risk/return, entry barriers, opportunity to differentiate products and services) and likelihood of disruption

However, the attributes that define a company’s ability to execute have evolved over time. Increasingly, agility and the ability to quickly deliver value differentiate winners from losers.

Critical attributes to evaluate when assessing your company’s ability to execute should include:

• throughput
• technical debt
• process maturity
• strength of assets and competencies
• market share
• relative brand strength (marketing)
• sales capability
• relative cost position (cost structure compared with competitors)
• relative profit margins (compared to competitors)
• ability to deliver and support customer experience
• operational performance and quality
• level of digitization
• access to financial investment resources.

All of these attributes are critical to assessing your company’s ability to execute. For this post, we will be exploring the first three listed because these three are often overlooked and are increasingly important when predicting future performance.

Throughput

We will start with throughput. We define throughput as the amount of value delivered over time. Measuring throughput is one of the most difficult problems in the world of software development. Unlike the output of a factory that produces a measurable number of widgets, the output of an engineering team differs in composition, size, and timing.

Is research and development (R&D) spending an accurate predictor of throughput? For complex work like software delivery, the answer is no, cost is not a sufficient predictor. For example, an organization can double its R&D spending on a particular offering, but if the offering is built on a burning platform, the incremental R&D spending can produce zero net new value and the additional developers can—paradoxically but quite regularly—slow overall delivery.

We need a new set of metrics that measure the value delivered instead of the cost. To make decisions on a digital product portfolio, we need the ability to correlate R&D spending to R&D value delivered and match that up to financial results. This is exactly the role of the Flow Framework’s Flow Metrics, introduced in the book Project to Product: How to Survive and Thrive in the Age of Digital Disruption by Dr. Mik Kersten.

Flow Metrics measure flow items, i.e., mutually exclusive and comprehensively exhaustive units of business value delivered: features (new business value)—including defects (quality improvements) and risks (GRC, security, privacy)—and debts (tech debt reduction)

The better the flow, the more effective R&D spending will be at producing business value. The four Flow Metrics are how we can measure that flow.

• Flow Velocity gauges whether value delivery is accelerating. Flow Velocity is the number of flow items of each type completed over a particular period of time. This allows us to measure throughput.
• Flow Time can identify when time to value is getting longer. Flow Time measures the time it takes for flow items to go from “work start” to “work complete,” including both active and wait times. This allows us to measure time to market for the optimal throughput to ensure that it is fast enough for the short feedback cycles needed when dealing with uncertainty.
• Flow Efficiency can identify when waste is increasing or decreasing in your processes. Flow Efficiency is the ratio of active time versus wait time in the total Flow Time.
• Flow Load monitors over- and under-utilization of value streams, which can lead to reduced productivity. Flow Load measures the number of flow items currently in progress (active or waiting) within a particular value stream.

Each of these is based on end-to-end flow through a value stream.

For example, the Flow-Time clock starts when work begins, (e.g., via commitment to a plan item or objective). It does not stop until running software has been delivered. This avoids the pitfalls associated with measuring just the cycle time of development or code-commit to code-deploy time, which can be an order of magnitude shorter than the bottlenecks upstream of development. As such, Flow Time provides a customer- and business-centric measure of time to market, and Flow Velocity does the same for throughput.

Flow Efficiency measures how effectively investment turns into value, and Flow Load is an early warning indicator of overloaded value streams declining due to too much work in progress (WIP).

To get a comprehensive view of a product portfolio, we need to track each of the Flow Metrics for each of the four flow items for the entire portfolio. This can tell us that while one value stream is very quick at responding to incidents, it can take months to deliver significant new functionality. Until investment is made in accelerating feature delivery for that value stream, for example by shifting to a cloud-native platform, the return on investment (ROI) will be too slow for it to win a market.

Technical Debt

The second attribute we will explore is technical debt. Far too often, companies underestimate the cost of technical debt.

Flow Metrics can be broken into two categories: revenue generation and revenue protection. Features are most commonly thought of in terms of revenue generation, while risks and defects fall into the category of revenue protection.

Technical debt sometimes is ignored when prioritizing work, labeling work, or making work visible. Overlooking technical debt denies an investment in future revenue generation.

While technical debt can include continuous improvement initiatives (people, process/ways of working, technology), the component of technical debt that will likely have the biggest impact is how a company deals with accumulated legacy technical debt.

Because reducing technical debt is an investment in enabling future throughput, it is many times neglected, especially in a project model where project managers are incentivized to deliver now with the lowest cost and not think about the future capabilities of the product or systems impacted by accumulated debt.

Reducing technical debt can take many forms, from the need to refactor a code base to updating infrastructure components. Companies’ acquisitions of assets are a common source of technical debt, as additional technology and systems that require integration and rationalization are taken on.

Technical debt creates a drag on feature velocity, because changes become more complex and take longer to make. In addition, with higher technical debt, more time is spent on addressing defects, which starves the capacity to perform feature work. As technical debt builds, the following is observed: Flow Velocity decreases (Flow Time increases and quality decreases as the code base becomes more fragile) and costs increase.

All of this results in reduced happiness for both the customer and the team. There are many cautionary tales about this buildup, in which market leaders end up losing their entire market share due to inattention to technical debt.

One of the most infamous stories (as told in Project to Product) is Nokia. Nokia’s cumbersome and outdated platform prevented the company from delivering the features required to compete with iPhone and Android. This is where it is necessary to observe flow distribution to ensure that prioritization decisions made with the product owners include providing necessary capacity to address technical debt. Downtrends in Flow Velocity, Flow Time, defects, and team happiness are indicators that technical-debt items need to be prioritized.

Process Maturity

The third attribute is process maturity. As we stated in the introduction, successful companies are those that can quickly adapt to change. The processes employed by an engineering organization will have a large impact on the ability to execute.

The Three Ways is an example of process maturity that is described in both The DevOps Handbook: How to Create World-Class Agility, Reliability, & Security in Technology Organizations and The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.

The Three Ways are a set of patterns and principles that all of the DevOps patterns can be derived from. The assertion is that the Three Ways describe the values and philosophies that frame the processes, procedures, and practices of DevOps, as well as the prescriptive steps.

• The First Way emphasizes the performance of the entire system, as opposed to the performance of a specific silo of work or department. This can be as large as a division (e.g., Development or IT Operations) or as small as an individual contributor (e.g., a developer or system administrator).
• The Second Way is about creating right-to-left feedback loops. The goal of almost any process-improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made.
• The Third Way is about creating a culture that fosters two things: continual experimentation (taking risks) and learning from failure and understanding that repetition and practice is the prerequisite to mastery.

Combining the Flow Framework with Market Attractiveness and Revenue

Consider the scenario where an organization has a cash cow product that’s market attractiveness is falling and a new shining star candidate that looks to be positioned well to thrive in the recovery. An obvious choice would be to reallocate significant R&D dollars from the cash cow to the star. This will hurt the market performance of the cash cow, as it will reduce the capacity for responding to customer defects and feature requests.

However, the payoff from the shining start is predicted to eclipse those losses in the recovery. By considering the attributes associated with (a) industry attractiveness and (b) ability to execute, an executive can leverage forward-looking and predictive measures to make more informed decisions.

In the figure below, we plot the four products outlined earlier but with additional attributes. Further, we’ve augmented our hypothetical portfolio with the three metrics that heavily influence the ability to execute: Flow Time, technical debt, and process maturity.

For Product B, Flow Time has slow to 270 days, and technical debt is high. In this situation, additional R&D investment will not provide a great return. This product should be a candidate for harvesting or shut down.

In contrast, Product C is ripe for investment due to a combination of healthy revenue, good Flow Time, low technical debt, and mature processes. If the industry attractiveness increased further, it may make sense to forego near-term profits on this product for future revenue opportunity.

Conclusion

By considering industry attractiveness and ability to execute with particular emphasis on Flow Metrics, technical debt, and process maturity, we have an informative representation of the portfolio. This representation can drive strategic product-portfolio decision making.

Next, we’ll look at the last of the four frameworks for portfolio management: zone management. To read the full white paper, you can download it here.

- About The Authors

IT Revolution

Trusted by technology leaders worldwide. Since publishing The Phoenix Project in 2013, and launching DevOps Enterprise Summit in 2014, we’ve been assembling guidance from industry experts and top practitioners.

Follow IT Revolution on Social Media

• 
• 
•