This post is an excerpt from The Project to Product Transformation white paper, written by Ross Clanton, Amy Walters, Jason Zubrick, Pat Birkeland, Mik Kersten, Alan Nance, and Anders Wallgren. You can download and read the white paper in its entirety here.
Why Is Architecture Important?
Architecture exists within applications, solutions, and the enterprise, whether time, effort, and resources are allocated to it or not. Architectures evolve over time just like most processes and organizational priorities.
To enable delivery teams to operate at their highest velocity and remove as many impediments as possible, a focus on architectural improvement and streamlining your product delivery capabilities is crucial. You must ensure that the transformation evolves the architecture into a catalyst for desirable outcomes instead of the architecture devolving into additional overhead for teams to manage.
When it comes to a product model transformation, one of the most critical roles of architecture is to enable the product teams to work in an autonomous and decoupled way, minimizing dependencies and constraints on issues that are outside of a team’s domain.
What Is Architecture?
When defining architecture within an enterprise, most imagine individual applications or specific patterns typically used in developing software. However, architecture in the context of an enterprise product transformation encompasses all aspects of the business. Architecture is about describing, documenting, and transforming the enterprise’s most fundamental systems.
The key to changing architecture in a product-centric organization is to find ways to leverage existing processes and structures while identifying future organizational goals, and to use the transformation process to enable and drive those architectural initiatives. Moving from project to product is about adding value. Find ways to use each aspect of architecture to aid the delivery of that value.
How Are Enterprises Achieving Success?
Like all transformation domains covered in this paper, organizations have varying levels of progress based on their stage of transformation. In several of the interviews we conducted, the enterprises were using their flexible architectures as pillars to build value upon, whereas other organizations found architectural changes to be a byproduct of the transformational effort. In all cases, architecture was viewed as an accelerator to the value proposition of the product transformation.
In our 2017 DevOps Enterprise Forum paper Value Stream Architecture, we introduced the idea of architecture patterns and tenets focused strictly on the value stream. The premise of the paper was that the same amount of care and feeding that goes into writing software should be taken to deliver software.
We identified several tenets that value stream architecture should focus on:
- Software architecture
- Transformation teams
- Risk assessments
- Organizational structure
Value Stream Architect
To complement the value stream architecture paradigm, we suggested a new role within the organization to focus on these tenets of software delivery: a value stream architect.
The value stream architect is responsible for streamlining the delivery of value for each value stream. Very similar in concept and deliverables, the indicators for architecture demand streamlining value delivery and measuring that value through transparent, highly visible metrics. Product-centric architecture requires focus on value streams and the existing architecture surrounding those value streams.
Incubation Stage: Architecture Domain
The architecture domain is no different than any of the other domains when entering and executing the incubation stage: keep it simple and confined to experiment, and prepare for scale. During our interview process, we found these indicators had the highest amount of impact during the incubation stage:
- Establish flexible, low-friction ability for teams to build: The goal at the heart of the product transformation movement is to deliver value to the customer as fast as possible. A simple equation is used to show the negative impact of delivering value to the customer late, called the cost of delay. This equation, outlined by Don Reinersten in The Principles of Product Development Flow, suggests that the sooner you can get value into the customer’s hands, the sooner your organization will see value from the customer. This indicator plays into that same theme, that in order to deliver value to our customers quickly, we can’t continue to deliver software in a project-based method. Reduce the friction for our teams to deliver value to our customers quickly.
- Introduce architectural patterns that increase speed of delivery: Just like the previous indicator, this indicator is about delivering value quickly but from the software architectural perspective. Introducing progressive architectural patterns for delivery, like event-driven programming and API-first development, will help increase the efficiency of the value stream and enable seamless future additions to the value stream.
- Event-driven architecture: An event-driven architecture is a software pattern that leverages a highly decoupled pattern of publishing, subscribing to, and reacting to an event or a “state change” within a system. This pattern allows for loosely coupled systems and promotes the creation of quick changes and new features with very minimal risk to existing features and value.
- API-first architecture: In an API-first pattern, APIs are designed and structured as the very first action in the design process. This practice greatly increases the feature or product speed to market by allowing developers to work in parallel but also reduces risk by clearly defining the product capabilities up front.
- Domain-driven design: This is a design pattern that fits well with a product-oriented and Agile model as it promotes iterative evolution of the design, enables you to set clear boundaries around the domain context of the product you’re building, and helps drive a common language for what is being built that even the non-techies in the team should be able to understand.
- Engage enterprise and business architecture organizations: An enterprise’s inertia can often be mapped and defined from its enterprise and business architectures. Many of the enterprise’s processes, practices, and policies are analyzed and documented by these two groups. In fact, these groups are typically responsible for building and managing the capability model for the enterprise, which we learned was often a key foundational artifact in guiding the product taxonomy work. By engaging these two architectural disciplines early in the transformation process, the transformation leaders can begin to get buy-in from these groups to plant the seeds of change and to also help the transformation team understand the complexities that shifting to a product-centric enterprise could impose. Strong enterprise and business architecture groups drive organizational change and are, by nature, change agents; the sooner these disciplines are involved in the transformation, the better.
Scaling Stage: Architecture Domain
As the organizations we interviewed shifted from incubating to scaling their transformation, we noticed that the indicators also shifted from increasing velocity in building software to standardizing and streamlining the delivery of value, also known as the value stream. These scale stage indicators focus not only on initially delivering value to market, but repeatedly and predictably adding value by reducing the risk and time it takes to deliver that value to customers.
- Initiate a “toolchain as a product” paradigm: Building a team and competencies to focus on the toolchains related to value stream delivery is key to frictionless value. Our interviews suggested that without a focus on the repeatability of value delivery, product transformations had a difficult time showing value.
- Focus on automation: Automation is standard practice and typically associated with DevOps, but DevOps and product-centric delivery go hand-in-hand and complement each other. In order to build predictability and remove unnecessary work, teams must automate everything from software builds and tests to deployments and feedback loops. This automation does not come cheap or easy. There must be a dedication and rigor to building automation into every unit of work. To see the results of a successful product transformation, automation is a must.
- Adopt open-source platforms and tools: Using platforms and tools purchased from corporations often leads to vendor or platform lock-in. Open-source tools and platforms typically offer the flexibility of being platform agnostic and can be used in very different environments. Open source is often a leading indicator of a progressive, forward-thinking organization as well. These tools and platforms typically focus on doing one thing very well and have large communities working to improve them, which allows for quick adaptation for integration into new and trending services and capabilities.
- Create architectural communities: Architectural communities help educate and dissolve a centralized architecture structure, moving architectural decisions closer to delivery. These communities help disseminate information, such as patterns, trends, and organizational strategies, to teams closer to the value streams. Our interviews showed that the most progressive product transformations also had a network of decentralized architectural decision-making, with some even leveraging an inner-sourcing model to drive architectural decisions and capture those decisions as source code. These communities become the foundation for that decentralization.
Optimization Stage: Architecture Domain
Once the transformation moves into the optimization stage, architectural tenants should be a foundational element of each of the value streams. The indicators derived from our interviews show an increased focus on decentralization and the adherence to technology to remove even more friction.
- Allow teams to make architectural decisions regarding speed to market: This indicator builds on the earlier indicator about architectural communities. In order to remove bureaucracy and move decisions closer to the problem, responsibility for making architectural decisions must shift to the team level. Enabling teams to make the architectural decisions that are best suited to their value stream helps to empower them and drives a much higher velocity of value delivery.
- Move to cloud and cloud-native computing: Another common pattern found in high-performing product transformations was teams moving their platforms to cloud technologies. Underlying infrastructure should be consumed as a service, as needed, and not be a burden for product teams to manage. Some companies achieve this through private cloud implementations of their infrastructure. However, the public cloud providers tend to be at the cutting edge of technology trends and feature delivery. Using capabilities provided by the big cloud players (AWS, GCP, and Azure) gives teams a huge momentum boost in delivery by creating self-service opportunities for technology enablement and cutting out the middlemen typically found in traditional technology organizations. Coupling these cloud capabilities with Twelve-Factor software design patterns enables product teams to build applications that are truly cloud native.
Constraints That Need to Be Overcome
We found two prevalent architectural constraints in our research and interviews:
- Changing operating model without deliberate focus on decoupling systems: Most organizations that struggled with the architecture domain did so because they had a very rigid monolithic system that didn’t lend itself to product-centric delivery. In highly coupled and intertwined systems, it’s difficult to deliver value quickly without a tremendous amount of risk. This constraint suggests that to reduce risk and gain efficiencies in the delivery of value to customers the best way to approach the transformation is to first start decoupling the systems into manageable “products.”
- Bureaucratic architecture processes impede team velocity: In some traditional architecture organizations, architecture processes such as architecture review boards (ARBs) create unnecessary gates in product-centric organizations. ARBs and other centralized processes take decision-making rights out of the hands of the leaders closest to the value stream. The teams working on their given product understand the problems and issues with their product better than a centralized organization. A slightly more aggressive approach to this constraint is to break down the need for conventional architecture practices by suggesting and experimenting with a decentralized or federated approach that delivers decisions to architectural groups as needed.
In the continuing posts in this series, we will explore each of the Seven Domains of Transformation in more detail.
- Seven Domains of Transformation
- Transformation Implementation
- Business and Technology Synchronicity
- Product Taxonomy
- Workforce and Talent
- Funding Model
- Culture and Leadership
In the full white paper, The Project to Product Transformation, you will find not only the guidance indicators to create, increase, and sustain velocity, but also the negative force learnings that should help you avoid pitfalls in your transformational journey.