DevSecOps Implementation at the US Navy: Not for the Weak of Heart

This case study was provided as a supplement to the second edition of The DevOps Handbook. It was written by Harold C. Young, Product Owner, and Brent Brockman, Senior Software Developer, U.S. Navy. 

—

 The US Navy is in the middle of a massive information technology modernization, to include new hardware suites, software architectures, and continuous integration/continuous delivery software factories. To realize the full potential of the IT supporting Sailors and Marines worldwide, the Navy’s leadership realized it also had to make significant changes in how its software product teams were organized and led. 

Our team participated in a Navy DevSecOps pilot program to 

• learn how to build a generative culture within our product team, 
• address an organizational need using modern software practices and tools, 
• and involve fleet end users to understand and address their software pain points.  

Our team found that while each DevOps element was simple to understand, the emotional and cognitive work involved in meshing those practices into a high performing methodology was non-trivial.

In collaboration with an industry partner, we began our DevSecOps pilot by selecting product team leaders: a product owner, an engineering lead, and a user experience lead. These three leads represented the needs of the organization, the needs of the system, and the needs of the end users. Using these three viewpoints on a daily basis allowed us to balance operational capability, system reliability and security, and user adoption for our product.

With our leaders in place, we collaborated with the staff of Naval Air Forces to understand how we might use software to optimize their aircraft maintenance schedules. With an understanding of the scope of effort and a high-level charter, the product team leaders recruited software developers and a cybersecurity expert to round out the team. The entire product team set up an off-site office space, where we hoped that the team could reinforce new habits without distraction. 

Once there, we started deliberate team-building exercises. We created a social contract, where all team members provided input on how the team wished to operate, and how all team members wished to be treated. We also conducted exercises designed to get to know each other, such as identifying similar hobbies and interests. We embraced a problem-solving mindset to encourage collaboration among team members by using phrases such as “How might we…?” and “Yes, and…”. 

Creating a generative environment was quite challenging for some of the team’s senior members, as they had come from bureaucratic organizations with a clear leadership hierarchy. To participate as equals in a collaborative team environment took practice and patience. This was evident at multiple occasions: while debating the value of a feature; during a candid team retrospective; or when discussing the merits of a particular technical approach. Our team members learned to check their egos at the door, acknowledge that the team knew more than any individual did, and assume positive intent from every engagement.

The next phase of our team’s development, using modern software practices and tools, was also taxing for some team members. Depending on the tasking of the day, we would work as an entire team, as breakout teams, or as pairs. Working in small teams encouraged focus on the task, resulting in accelerated learning and increased productivity. It was also cognitively challenging to focus for long periods while being accountable to your teammates. The first few weeks performing at this level were both exhilarating and exhausting.

With our team organized and ready for action, we brought in subject matter experts in aircraft maintenance to better understand how maintenance is scheduled and performed, to include policies, procedures, and information flow. We also interviewed end users to understand their pain points. The synthesis of this information allowed our team to create its roadmap of epic features, along with a priority to address the most prevalent pain points first. 

Having the product team self-organize its efforts based on end user feedback was a refreshing change from deciphering software performance specifications. It also placed full product ownership on the team, which highlighted the importance of rigorous user experience methods, defendable research results, and actionable user stories. 

In addition to frequent demonstrations to the users, we made cybersecurity best practices transparent across all members and stakeholders. During our sprint demonstrations with users and stakeholders, we would review how the software quality and security changed between demonstrations. Communication of security priorities allowed us to increase understanding of why we would decide to delay a new feature in order to fix a critical security flaw or improve code quality.

By creating a generative environment, using modern software practices, and focusing on the end user, our team was able to create a workable maintenance scheduling prototype within eight weeks, and then delivered a minimal viable product with all required security certifications to Naval Air Forces in six months. Our team members now share their lessons with other Navy product teams, with the hope that they find these DevSecOps practices as useful as we did, while managing the expectations of the serious commitment it takes to perform.   

- About The Authors

IT Revolution

Trusted by technology leaders worldwide. Since publishing The Phoenix Project in 2013, and launching DevOps Enterprise Summit in 2014, we’ve been assembling guidance from industry experts and top practitioners.

Follow IT Revolution on Social Media

• 
• 
•