Inspire, develop, and guide a winning organization.
Create visible workflows to achieve well-architected software.
Understand and use meaningful data to measure success.
Integrate and automate quality, security, and compliance into daily work.
Understand the unique values and behaviors of a successful organization.
LLMs and Generative AI in the enterprise.
An on-demand learning experience from the people who brought you The Phoenix Project, Team Topologies, Accelerate, and more.
Learn how making work visible, value stream management, and flow metrics can affect change in your organization.
Clarify team interactions for fast flow using simple sense-making approaches and tools.
Multiple award-winning CTO, researcher, and bestselling author Gene Kim hosts enterprise technology and business leaders.
In the first part of this two-part episode of The Idealcast, Gene Kim speaks with Dr. Ron Westrum, Emeritus Professor of Sociology at Eastern Michigan University.
In the first episode of Season 2 of The Idealcast, Gene Kim speaks with Admiral John Richardson, who served as Chief of Naval Operations for four years.
New half-day virtual events with live watch parties worldwide!
DevOps best practices, case studies, organizational change, ways of working, and the latest thinking affecting business and technology leadership.
Is slowify a real word?
Could right fit help talent discover more meaning and satisfaction at work and help companies find lost productivity?
The values and philosophies that frame the processes, procedures, and practices of DevOps.
This post presents the four key metrics to measure software delivery performance.
April 30, 2020
By Dominica Degrandis
The balanced flowchart exercise is a companion to Chapter 3.1 of Making Work Visible: Exposing Time Theft to Optimize Work & Flow. This exercise includes four useful metrics—all on one sheet—to help people see metric trends over time and understand how to become more predictable.
TIME: 45–60 minutes
PURPOSE: To discover what lead time, throughput, change failure rate (CFR), and probabilistic percentiles are by performing themby hand, and to learn what each can reveal. This exercise measures one metric trend in four different areas and was inspired by Larry Maccherone and Troy Magennis’s blog Focused Objective.
The four areas we will measure a metric by are:
It’s fairly easy to game a metric; therefore, it is important to measure the impact of the change in one metric on the rest of your other metrics by showing them all in one place.
MATERIALS:
INSTRUCTIONS: Manually plot and calculate the four metrics listed in the Purpose section above.
Step 1: Create a legend for the different symbols used to mark the different work item types. In this legend, three types of work are noted:
Finally, include failure demand in your legend. Failure demand is work done because of some error, such as a feature that doesn’t function properly and requires a change, or an SQL server that hits 100% utilization and needs immediate attention. Failure demand can be associated with any of the other work item types. Failure demand work is noted with a “yes” in the last column of the flow time spreadsheet.
Step 2: Using the data in the flow time column of the flow time spreadsheet, mark the flow time of each work item on the blank balanced flowchart.
The X-axis is the calendar date that the work item is completed on. The Y-axis is the number of days that the work item took to complete (the actual flow time). For example: the biz request on line 18 of the example flow time spreadsheet (located at end of page) started on September 5th and finished on September 19th, resulting in a flow time of 14 days. The correct mark for this work item is located at (19, 14) on the chart.
Step 3: Now, you will determine productivity with the throughput (TP)by totaling the number of work items completed each week. Create ahistogram (a graphical representation of the distribution of numerical data) in the TP section at the top of your balanced flowchart. The throughput of all the work items completed during the week of September 4th rhymes with late.
Step 4: View the quality (how good) of productivity by calculating the change failure rate (CFR). Do this by dividing the number of completed failure demand items by the total number of completed work items. The CFR rate for all work items as well as biz request work items completed the week of September 4th has been completed for you (see Figure 1, CFR = 25%).
Step 5: Next, calculate the 90th percentile for all work items completed per week. If you want to be more predictable, look at probability. Instead of asking, “What date is this request due?” ask, “What’s the probability of finishing this request within so many days?” The 90th percentile is the value for which 90% of the data points are smaller and 10% are larger.
Here’s how to calculate the 90th percentile for the week of September 11th: Start by listing (from smallest to largest), the flow time for each work item in the dataset. The dataset for the week of September 11th has seven numbers (1, 1, 2, 6, 8, 10, 12). Next, multiply the number of items in the dataset (seven in this case) by 90% to determine which number in the dataset sequence to use: 7 x .9 = 6.3. This tells us that the 90th percentile is somewhere between the 6th and 7th number in the dataset, or between 10 and 12 respectfully. You could round down to the 6th number here, which would result in 10 being the 90th percentile. Or, you could use a weighted mean to calculate a more precise 90th percentile, which is what I did for the cheat sheet.
To calculate a precise 90th percentile number, subtract the 6th number in the dataset (10) from the 7th number in the dataset (12) to get a difference of 2. Next, multiple the difference by 90%: 2 x .9 = 1.8. Then add 1.8 to the 6th number in the dataset: 10 + 1.8 = 11.8. The 90th percentile is 11.8. This tells us that 90% of all the work completed during the week of September 11th was done in 11.8 days or less. The cheat sheet in Figure 2 shows the 90th percentiles for each week in September.
Step 6: Now review the set of four metrics on the balanced flowchart and discuss how each metric could be gamed.
If you filtered for revenue generating items only based on data for the month of September from the example flow time spreadsheet data, we could say with 90% confidence that revenue generation work will take somewhere in the range of 8–20 days to complete. Maybe 90% is good enough for a large percentage of the work. If you are making pacemakers, where failure is not an option, you’ll have to hit the 100th percentile and crank up the expected timeframe to 25 days.
The idea here is to be approximately right instead of exactly wrong—use science and probability to set expectations instead of arbitrary due dates.
Table: Flow Time Spreadsheet Data
Download this exercise from Dominica Degrandis or purchase Making Work Visible here.
Images copyright Dominica Degrandis.
Trusted by technology leaders worldwide. Since publishing The Phoenix Project in 2013, and launching DevOps Enterprise Summit in 2014, we’ve been assembling guidance from industry experts and top practitioners.
No comments found
Your email address will not be published.
First Name Last Name
Δ
You've been there before: standing in front of your team, announcing a major technological…
If you haven’t already read Unbundling the Enterprise: APIs, Optionality, and the Science of…
Organizations face critical decisions when selecting cloud service providers (CSPs). A recent paper titled…
We're thrilled to announce the release of The Phoenix Project: A Graphic Novel (Volume…