Nick Galbreath On Integrating Information Security Into DevOps

I had the pleasure of meeting Nick Galbreath (@ngalbreath) at the SxSW Interactive conference this year. He’s especially memorable to me in much the same way Josh Corman and James Wickett are.  What they all have in common is that they believe that DevOps may be the best thing that’s happened to information security in a very long time.

Nick is an information security practitioner who not only sees the value of DevOps-style practices to information security, but also lives and breathes it.  What makes him especially qualified to speak on this is that he’s worked with John Allspaw (@allspaw), who was one of the primary forces in the birth of the DevOps movement. Galbreath and Allspaw have collaborated not once, but twice during the last decade.

In his DevOpsDays Austin talk, he explains how he integrates information security objectives into the DevOps practices at Etsy. He calls it “DevOpsSec.” At Etsy, they routinely do sixty-plus deploys per day, and have integrated it into all the various work streams of Dev, QA and Ops, including continuous integration and release, testing, production monitoring, and post-mortems.

He argues very persuasively that high rates of deployment forces out all the tedious, manual and error-prone steps in the deployment process, especially around change and configuration, which form the root causes of most security failures.

Here are some of my favorite techniques he mentioned, which all demonstrate how to integrate information security objectives into the value stream of Development and IT Operations, and do it in a way that provides obvious value to everyone.

• Generate graphs of all occurrences of “UNION ALL” in user input (to remind developers of the ever-present risk of SQL injection attacks)
• Add security-related ASSERTs to the puppet/chef environment build process (to catch misconfigurations, ensure that certain pages are always served with HTTPS, etc)
• Integrate major security failures into their blameless post-mortem process
• Continually look for segfaults and crashes on servers (it may be an early indicator of bad code or a successful attack probes)
• Have that attitude that having your site attacked all the time is a gift (it makes the risk visible to everybody, helps educate developers, etc.)
• Hold Tuneup Days to encourage people to bring their home computers in for information security staff to harden and repair (elevating the average level of hygiene to the same level as inside the organization)

One of the goals that we’re striving to achieve with the “DevOps Cookbook” is to show exactly how Development, QA, IT Operations, Information Security and even Product Management need to work together to get fast flow of features into production, while preserving the stability, availability, security, manageability, and so forth, of the production environment.

I’m particularly excited that the principles and techniques that Nick talks fit in so well with the DevOps patterns we’ve observed, and that they not only help information security to achieve its goals, but provide real and visible value to the entire organization.

Keep up the great work, Nick!

Video streaming by Ustream

(Unfortunately the Ustream video takes a very long time to load, but the talk is well worth the wait. You can find the slides from the presentation here.)

I had Wendy Mitchell interview Nick after his talk, and he shared his thoughts on how his belief  that DevOps will be the standard way of working in the future, more on the Tune Up Days, as well as what he’s working on now.