DevSecOps (Expert Panel Discussion)

But Margo, I thought we’d start off with, you spoke here today, what did you speak on?

For all of us, security is the first priority. Like the top job, ‘job zero’ we sometimes call it. And therefore, DevOps understands that security is key and it is the first thing that you do. And if you look at GDPR, one of the pillars of GDPR, is privacy by design, so not just data portability, data breach notification, the right to be forgotten, but privacy by design. And it kind of mandates that you have to do DevSecOps.

I ran the security bloggers network for 15 years, and I’ve done a lot in the security community over my time. And,

I’ve gotten into huge fights, huge fights with my friends in the security world over this DevSecOps thing.
Some people just say it’s flat-out nonsense. Alright, it’s a marketing term.

Other people and it just goes to show you how weird we are in security. Other people say “DevSecOps, why isn’t it SecDevOps? We are most important after all.”

Other people, you know, want nothing to do with it. Or, they’ll say SecOps, we don’t really work with the Devs, they were assholes, they were always that way. You know,

I don’t want to work with them. And that again is part of the problem as well. It’s a tremendous controversial thing that I personally had to get my head wrapped around. Did you have a journey here, Zane?

Right? If the way that we are creating software has changed, and the way that we’re delivering software has changed, well, the way that we’re actually securing that and defending that has to change as well. And so, whatever we want to call that, I could care less.

It was seen as a thing that you have to do to clear the boxes. I think the main aspect of this whole conversation is are we going to change that logic and think about it more almost as an engineering discipline, or as a way of just doing our work better?

It kind of ties in nicely with all of this engineering craftsmanship and software craftsmanship movements, as well. To me, DevSecOps is the best way of conveying that message, it’s like it’s a term that everybody kind of understands, no one can agree upon it, which was what happened to DevOps, but at the same time, it means that people have an opinion about it and they can communicate on it.

We’ve done it for the last four years, and we called it ‘the Rugged DevOps day,’ then we called it just DevOps and Security, and then the last three years, DevSecOps.  
So, let me tie a bow on the issue. I’ve come to the conclusion, like you Zane, that I don’t care what you call it.

DevOps is DevOps. There is only one DevOps. There is only one truth. I don’t care what you call your God, there’s only one truth, right? And it’s DevOps. Now, if we need to add security to the term and stick it in the middle as you say. Because that’s going to satisfy people and make them think about security, so be it. I’m not going to stand in the way of that, I think that’s a pragmatic approach to it.

One of the things I was talking about in my speech is, it doesn’t matter if you’re doing site reliability engineering, or if you’re doing DevOps, etc. let’s just throw everything to the wind, and security has to be first. So how does everybody become a security practitioner?

The other thing I would mention, it’s funny you mentioned Q.A. because I’ve had this conversation with someone else in the security space, but maybe what the goal here should be is that security become synonymous with quality. Alright, instead of thinking ‘oh, I’ve got to add the security in,’ we can’t have quality product without security.

So it very naturally drives them into that, we got to do the lowest common denominator because that’s literally the only control point that we have.

This is almost like a very similar transformation. It’s layering on that practice, to say we consistently need to execute on this, we consistently need to have this along the line. Then, obviously, still have all the checks we have today, but more as a kind of verification point of the work that you’ve already done, rather than relying that’s going to be the only thing that we’re going to do.

Part of the problem there is that particular vulnerability was in one of the rest API plugins that had a transited dependency. So, it wasn’t immediately clear, even if you looked at the direct manifest, that that was even there. So the challenge of managing this supply chain is, the only real visible solution is to essentially look at the actual deployables, print out everything inside of there, maintain those lists, and then correlate on them.

That’s one of the big challenges we have when addressing those sorts of issues. When we look at the Equifax case was it took them over, it took them however long to actually patch that out, I don’t think they ever did. But it only took them about two or three days to get owned by the vulnerability once it came out and the project released a new version of it. So, I remember from my experience and the typical way that worked, was we found something bad that had happened and then we had to have a fire drill to find out where it was.

There’s much better ways to start automating on that, creating what we call ‘bills of materials.’ And then just building a database. So when something does go wrong, an event happens, you can still use that as searching function and find out. It’s these small sorts of things that make sense.

Again it’s kind of touching this CO, are we moving to the let conversation about is it okay to fail, and then recover quickly? First of all we need to prevent, but if prevention hasn’t worked, and the failure, the incident has occurred and reducing glass radius so you can recover quickly. And then as a final step, which I think will be really interesting in the future is machine learning. And how we can use machine learning to actually change the behavior of our applications and production, and so they can start training themselves.

Then you have services where you can then change that behavior automatically. But I think that’s where we’re going to see cloud service providers become a lot more active, you know, using machine learning to harden your production landscape before you actually even go to the security operators.

For me, it’s kind of like Ilkka, you were talking about there like, are we comfortable with things going wrong in our production landscape and recovering quickly? You know, Richard Cook gave a great session yesterday evening on where he talked about, he didn’t say meantime to recover was nonsense, but he was quite, he was quite critical about it.

Well, he was critical about a lot of stuff. But he, you know, he pretty much said, ‘don’t talk to me about meantime to recover, meantime to restore, because you don’t know what’s happening under the hood.’ I think is essentially his message. So, is that something we’re comfortable with, like things actually going wrong and recovering quickly?

This was such a journey for me as a CSO of, being one of the first go through the whole DevOps and cloud shift is recognizing that the embarrassing lesson I had to learn was that look. No matter what our software development methodology or delivery methodology, we’re always going to have bugs, and we’re always going to have outages and issues, and all that. The system that allows us to react the quickest, and gives us inherently the most visibility is the one that actually makes us safest.