This post is adapted from the paper “From Milestones to a Continuous Quality Assurance Flow” from the Spring 2022 DevOps Enterprise Journal. 

One of the biggest hurdles to implementing continuous delivery and DevOps in an industrial environment—dealing with milestones/macroscopic quality gates—can be overcome with a continuous quality assurance approach that applies “green to green” and “shift left” paradigms to software and non-software artifacts.

In an industrial environment, software delivery requires much more than just working software. To formally release a change and deploy it into production requires many further non-software artifacts that cannot be covered by the continuous integration toolchain, no matter how sophisticated it is. However, using the classical milestone/macroscopic quality gate process is no longer an option, as this limits the deployment frequency to values that are outside the range of the desired target. Therefore, a rethinking of the release process is required.

This paper provides a new perspective on analyzing and tracking non-software quality criteria by introducing a continuous conformance concept. This concept applies the “green to green” and “shift left” paradigms—well established in the software space—to the non-software artifacts required to release and deploy a software change. Combined with a continuous delivery pipeline, this results in a continuous quality assurance approach that also enables industrial enterprises to take advantage of the many opportunities and benefits that arise from continuous enhancements of their products and services while maintaining their high quality standards.

The Vision—Continuous Flow of Value to Customers

With the boost of digital capabilities, industrial products are changing radically. Therefore, increasing efficiency is only one aspect of digitalization. The other, much more impactful aspect is the revolution of the customer experience. Transferring more and more functionality into software, combined with an ever-increasing connectivity to products at customer sites or in the cloud, opens up tremendous new possibilities for industrial enterprises and their customers. Continuous enhancements of products or cloud solutions during their lifetime, supported by data analysis from live systems, will become a key success factor.

This makes state-of-the-art approaches by IT companies, such as continuous delivery and DevOps, increasingly relevant for industrial enterprises as well. These approaches drive the vision of Agile all the way to the customer, resulting in:

• faster value utilization and integration of customer feedback through short-cycled deployment of product enhancements
• continuous, data-driven value increase due to product improvements based on operational data
• reduced deployment and operational risks through highly automated delivery of small product changes

However, the required paradigm shift has a tremendous impact on the entire organization, especially in an industrial environment. Meeting the high industrial quality expectations in a world of continuous delivery is one of the key challenges that must be mastered in this context. This aspect is evaluated in detail in this paper.

Quality in the Continuous Delivery Pipeline

A core element and prerequisite for continuous delivery to customers (i.e., frequent deployment of software enhancements into the productive system) is the implementation of a continuous delivery pipeline. This integrated tool chain must be highly automated and should include automated tests on multiple integration levels, addressing functional and nonfunctional requirements. Once established, such a continuous delivery pipeline already provides huge benefits with respect to built-in quality mechanisms and real-time quality status information of the software. Typical quality information provided by a continuous delivery pipeline includes:

• quality status of the build
• test results on various levels, covering different test aspects
• automated security checks
• traceability from requirements over test cases to code and test results
• continuous, automated, real-time visibility of all related status information
• automated decisions about whether a build is a potential release candidate

In addition, this tool chain can be used to provide auditable evidence and track the status of risks and risk mitigations linked to the software artifacts. All of this is ideally made transparent in real time through a suitable dashboard providing a 360-degree status view on the quality of the software to be deployed into the productive system.

Therefore, an efficient and robust continuous delivery pipeline is a must to speed up deployment frequencies significantly—but it is not sufficient. Although such an automated tool chain already provides significant quality status information regarding the software itself, it lacks several aspects required for a formal release for deployment into the productive system in an industrial environment:

• addressing strategic and system-level aspects, such as technical risk assessments, service level agreements, intellectual property rights, etc.
• formal approval and release documentation, such as hazard summary reports, risk management reports, safety certification, etc.
• further up-to-date, non-software artifacts required by different stakeholders, such as user documentation, operations guidelines, training, and many more

Before formally releasing and deploying a software change into the productive system, it must be ensured that those additional (non-software) artifacts are in a valid status and cover the change intended to be delivered to the customer adequately. These artifacts, however, cannot be covered by the continuous delivery pipeline, no matter how sophisticated it is. The continuous delivery pipeline is restricted to the software itself and software-close artifacts (e.g., requirements or test cases), but cannot cover the huge realm of additional artifacts required for a release, the non-software artifacts.

On the other hand, applying the classical milestone/macroscopic quality gate approach to ensure the quality of those non-software artifacts is also not an option anymore, as this would limit the deployment frequency to values that are outside the range of the desired target. Therefore, a rethinking of the overall release process is required.

“Green to Green” and “Shift Left” Are Not Only for Software

The most promising approach for a redesign of the classical release process into the “world of continuous” transfers a key idea of the continuous delivery pipeline to the non-software artifacts.

Within a fully automated continuous delivery pipeline, every code change is instantly verified against the existing system that was released in the previous iteration and is—at this particular time—running in the productive system. The primary focus of the continuous delivery pipeline is to verify that the code change does not break the running software, a paradigm referred to as “green to green.” With this approach, the quality of the software can be ensured for every code change at any point in time at the tip of a finger. Ensuring a system that is always running (i.e., a system that is “green”) has the utmost priority.

This “green to green” paradigm can be transferred to non-software artifacts. Assuming a minimal marketable product was already released to the customer at an earlier point in time, two things should be present:

• a running software system in production
• a valid baseline of all required, non-software artifacts matching the productive software version

Therefore, in addition to a green software status, there is also a green status for all required non-software artifacts at this particular time. Transferring the “green to green” paradigm from software to non-software artifacts implies that a suitable mechanism is now defined and established for the non-software artifacts, i.e., an analogy to a continuous delivery pipeline for the software. This mechanism, which can be labeled continuous conformance, is described in the next section.

However, to bring an adequate continuous conformance approach to life, another paradigm from the software world must be transferred to non-software artifacts: the “shift left” paradigm. This is required, in addition to the “green to green” paradigm, to ensure that relevant aspects of the non-software artifacts required for a release into production are addressed in a timely manner. The ambition is to achieve an operational capability where software can be seamlessly and continuously developed, integrated, tested, and deployed based on a highly automated continuous delivery pipeline without the need for lengthy and time-consuming work on non-software artifacts once the software is ready for deployment.

The “shift left” paradigm for software development ensures timely definition and early execution of tests across all development and integration stages to shorten feedback loops and speed up the capability for software delivery. By analogy, transferring the “shift left” paradigm to non-software artifacts has the intent of providing updated non-software artifacts matching the software version that will be altered through the intended software change exactly when needed. This means that non-software artifacts are analyzed and continuously updated at the right time to avoid lengthy and time-consuming work on non-software artifacts after the work on the software has been completed. How this can be put into practice as part of the continuous conformance concept is described in the next section.

- About The Authors

Peter Fassbinder

Principal Expert PLM Process Innovation (Agile | Lean | DevOps), Siemens Advanta Consulting, PhD

Follow Peter on Social Media

•