The Costs of Scale

This post is adapted from the upcoming book Flow Engineering: From Value Stream Mapping to Effective Action by Steve Pereira and Andrew Davis.

Scale undermines the foundations of effective action. This is not news to anyone who works in a large organization and certainly not to anyone who works in government. The immediate and natural response to the problem of scale might be to quit your job, set out on your own or in a small team, and start enjoying a simpler and more productive life. But you would very quickly find that being a solopreneur brings with it innumerable headaches that were abstracted away when you were working in a larger organization.

Why We Scale

In the words of Peter Drucker, “The purpose of an organization is to enable ordinary human beings to do extraordinary things.” The fundamental reason organizations tend to grow is people desire to do ever more extraordinary things. This is made possible by economies of scale. This means that organizations that double in size don’t necessarily need twice as many accountants or twice as many factories; great increases in output and revenue don’t necessarily require great increases in the infrastructure needed to maintain them. Research by Geoffrey West, summarized in the book Scale: The Universal Laws of Life, Growth, and Death in Organisms, Cities, and Companies, estimates that organizations continue to benefit from economies of scale as they grow, gaining a 10% efficiency with every doubling in size.

Organizations also scale to address opportunity and competition. With scale comes resiliency, influence, momentum, and attention. Amazon more than doubled its headcount from 2019 to 1.6 million employees in 2021 to address increased demand and to leverage vertical integration. Extreme scale requires extreme coordination but even scaling beyond a single individual demands care.

While scale has clear benefits, there are clear challenges and costs as well. Flow Engineering primarily addresses the challenge of how to enable teams to operate effectively in spite of the very real costs that scale imposes on effective action. To understand how teamwork is impeded by scale, we first need to understand the human costs of scale. The very purpose for which we assemble teams is undermined by the challenge of effectively coordinating them.

The Human Costs of Scale

Scale should enable increased capability and leverage, yet most organizations struggle to manage the scale they’ve created.

There are three specific human costs brought on by scale: distraction, disorientation, and disengagement.

• Distraction is a result of the constant interruptions, changing priorities, and demands on our attention.
• Disorientation occurs from a lack of clarity and alignment toward what matters most.
• Disengagement occurs when we resign ourselves to treading water without a clear connection to value.  

If we’re working on our own, we’re distractible enough. But the more people we engage with, the greater the chances that one of them will disrupt our attention at any time. Our distractions also lead us to distract others, creating a cascade of interruptions that splinter across the organization like space debris from an exploded satellite. This has made focus one of the most endangered mental factors in the modern world. There’s nothing more fatal to an organization’s ability to get things done than a team that can’t focus on its goal. Scale only exacerbates this problem.

While digital technologies enable coordination at far greater scale and speed, they can make it even harder to get and stay oriented to what matters most. The infinite profusion of information means that relevant details are easily lost. The fast pace of change lends itself to disorientation and to people moving at cross-purposes. Digital reality can be more transient and individual, where individuals decouple from each other far more easily than in physical reality. This means that special effort needs to be made to keep people’s digital worlds in sync, especially at the enterprise scale.

Orientation is required for alignment. Every layer of interaction in an organization requires aligning the motivations, understandings, and behaviors of different people. Even if a group of individuals are observing the same challenge, every observer will have a different perspective, leading to different perceptions, as is illustrated in Judy Katz and Frederick Miller’s book Opening Doors to Teamwork and Collaboration.

Because of our unique perspectives, we may possess or lack key information. Different people may also have different goals based on what they see to be most important at that time. We can also have different scopes of concern (wider or narrower, sooner or later, micro or macro, strategic or operational) that function like different zoom levels. Technologists are famous for zooming in on challenging technical details when making a decision. Those who are considering a situation from a greater distance may come to entirely different conclusions. All of these different perceptions can offer complementary points of view, but it takes effort to align.

The Gallup organization has tracked employee engagement metrics for thirty years and summarized many of their conclusions in First, Break All the Rules. As of their most recent surveys, engagement among US workers still hovers around 33%. According to Gallup’s State of the Global Workplace: 2023 Report, disengaged employees cost the world an unbelievable $8.8 trillion in lost productivity. Worker disengagement means our innate motivation circuitry is not being activated by our work environment. This could be due to a lack of challenge, but more often it’s due to a lack of purpose—or being too far removed from it. In these situations, it is not clear how our work serves a beneficial purpose, apart from a paycheck.

Daniel Pink’s bestselling book Drive: The Surprising Truth about What Motivates Us identifies autonomy, mastery, and purpose as key ingredients for maximum engagement. And Google’s Project Aristotle, a study conducted by Google to identify the key factors that make a successful team, found that sense of purpose was one of the five most important factors for high-performing teams.

While it may be tempting to think that finding a sense of purpose requires changing jobs, the fastest and most powerful improvements come from simply understanding the purpose of the business you’re in. Scale makes this difficult, since in a large process, the real beneficiaries of our work may be several steps removed from us.

The Organizational Costs of Scale

Scaling organizations brings with it special challenges. At an organizational level, the human challenges just mentioned manifest as misalignment. At scale, teams naturally become more distanced from the customer, from each other, from the purpose of their work, and from critical information. The critical feedback loop from customer need to team activity is stretched and broken.

Coordinating an organization means navigating the invisible world of others’ minds by trying to align incentives, forge trust, and clearly understand how to tackle the mountain of challenges and opportunities each team faces. Navigating an organizational ecosystem requires navigating an interdependent network of technical and social nodes (or interconnected components) and understanding that it’s increasingly challenging to consider any component or area of the organization in isolation. 

This invisible network is described in Wiring the Winning Organization  by Gene Kim and Steven Spear as the:

. . . social circuitry, the overlay of processes, procedures, routines, and norms that enable people to do their work easily and well. While individual specialists are focusing their attention on the problems immediately in front of them, this social circuitry establishes the patterns by which information, ideas, materials, and services flow, setting people up for success and integrating individual efforts for common purpose.

This social circuitry is invisible and easily overlooked. As organizations scale, invisible gaps and misalignments become endemic. Avoiding or mitigating these costs of scale requires engineering this social circuitry to establish or reestablish a shared sense of purpose, orientation, and activity.

The Paradox of Scale

Even in simple coordination activities, like playing a game of tug-of-war, individual effort declines as group size grows. This loss of effort is known as the Ringelmann effect: as more people are involved in a task, their average performance decreases, with each participant tending to feel that their own effort is not critical to overall performance.

Despite that inefficiency, in 2016 Microsoft and Facebook researchers conducted a study on forty-seven teams sized one to thirty-two people. They found that as team size increased, productivity of teams rose but collaboration costs and errors both increased. The study revealed valuable correlations between scale and performance: “We find that individuals in teams exerted lower overall effort than independent workers, in part by allocating their effort to less demanding (and less productive) sub-tasks; however, we also find that individuals in teams collaborated more with increasing team size.” In other words, large teams necessitate increased collaboration, yet the default effect is each contributor doing less.

In the end, the researchers found that “the largest teams outperformed an equivalent number of independent workers, suggesting that gains to collaboration dominated losses to effort.” To put it simply, collaborative work is best, but it’s not our best work. The waste and cost of collaboration grow significantly at scales beyond the “two-pizza team” or “single-threaded team” popularized at Amazon to address efficiency and scalability.

The study highlights a few aspects of scale we’ll address later in the book with an approach to improve collaborative workflow:

• As an organization grows larger, it will tend to become more inefficient. Effort is reduced and errors increase with scale unless care is taken to offset these risks.
• These costs are hidden since economies of scale can bring gains that outpace the waste of poor coordination.
• Cost and waste not only reduce company margins but also impact customer and employee experience.

These hidden inefficiencies also explain how large and established organizations can rapidly lose market share and crumble in the face of more agile competitors. This is massively consequential when we consider modern enterprise workflows comprising hundreds of teams and thousands of individuals as an interdependent network of collaboration at scale.

Challenges of Collaborating at Scale

Despite the enormous inefficiencies of trying to operate in organizations at scale, the incredible opportunities of the modern world and the intensity of competition push organizations to constantly strive for growth. Regardless of the challenges and the waste of large-scale organizations, not acting is not an option. And acting slowly or ineffectively is often as detrimental as not acting at all.

Given these constraints, we must find ways of working together that are effective for businesses, customers, and the workers themselves. Finding better ways of working together depends on finding a deeper understanding of the systems we work in.

- About The Authors

Steve Pereira

Steve Pereira has spent over two decades improving the flow of work across organizations. He’s worked through tech support, IT management, build and release engineering, and as a founding CTO for enterprise SaaS. He serves as lead consultant for Visible Value Stream Consulting, as a board advisor to the Value Stream Management Consortium, Chair of the OASIS Value Stream Management Interoperability technical committee, and co-founder of the Flow Collective to bring flow-focused professionals together. Since 2017, he has been developing and facilitating Flow Engineering to make flow improvement in large organizations accessible, collaborative, and actionable.

Follow Steve on Social Media

• 
• 

Andrew Davis

Andrew is Chief Product Officer at AutoRABIT, focused on the next generation of DevSecOps on the Salesforce platform. He is also the author of the leading book on the Salesforce development lifecycle, Mastering Salesforce DevOps. He was formerly Senior Director of Methodology and Training at Copado.

Follow Andrew on Social Media

• 
•