Pink & blue curved lines on blue background

Secure DevOps as second nature

Can secure DevOps become second nature?

Secure DevOps as second nature

Martijn Verbree | Partner,

Before we start, I need to clarify something. In my video below introducing DevOps I made a little mistake. I said that DevOps is a new way of developing software. It’s not actually. DevOps has been used by many (tech) companies for over a decade. So what is new?

It’s become mainstream and increasingly changes the way we develop and run (software) products. Or to use some buzzwords: digitize, become agile, fail fast, disrupt.

So when we say DevOps, what do we actually mean?

In essence, it’s a different way of developing and running software. Historically, software was managed across silos. On one side, there are the developer teams that code and look after the development environments. On the other end, there are the operational teams that look after the production environments. Both teams would typically have different objectives and priorities. As a result, we have the following issues:

  1. Developer teams being disassembled after the project finishes, making changes difficult after go-live
  2. Software behaving unexpectedly in production (worked fine in dev — it’s ops’ problem now!)
  3. Making it cumbersome to move code into production and be agile.

Fast forward to 2020. Thanks to the emergence of cloud computing, infrastructure as code, continuous deployment/integration tooling, containerization and automated testing, we have seen DevOps become a new norm. In DevOps, the interface between developer and operation teams is still there, but it’s rotated. Developers and operation teams now work together and share a common goal: develop and run better products — faster.

When I was running my tech startup, I experienced the power of DevOps first hand. We were iterating our products multiple times a day with only a handful of developers and operations people. And we were running it cheap, as we did not own any hardware or infrastructure to buy and maintain. Many of my clients are using similar practices, and some of them at massive scale with thousands of developers enabled to make changes to production environments.

But as with anything that sounds too good to be true, DevOps is not without its problems. My top-5 DevOps issues are the following:

  1. Inadequate protection of code repositories – typically hosted somewhere in the cloud with little or no access controls (no 2FA)
  2. Use of dodgy 3rd party code and libraries, resulting in poor code that none understand, vulnerabilities and backdoors
  3. Forgetting to shut down (test) instances that are no longer needed, resulting in huge bills and data losses
  4. DevOps as an excuse to not do the right thing (we don’t document anything, because we are DevOps)
  5. Poor configuration of code and infrastructure, resulting in security gaps that automatically and quickly propagate.

So how can we do this better and add some form of security in here?

Sadly, the lazy answer is often to pile on policies, processes and checks to basically block DevOps, or at least slow it down. Security often becomes passive and reactive — this is exactly what we need to avoid because it goes against the principles supporting DevOps adoption in the first place.

For the answer, we merely need to look at the early days of DevOps. The most substantial change in DevOps was culture. Developers and operations teams started to collaborate and work together with a set of common goals, processes and tools. Security folks will need to do the same to really add value and enable the product teams to build better and more secure products faster. In other words, as an Infosec professional, I need to come out of my ivory tower and find a way into the tent and become a contributor without wetting the floor.

This requires us, security professionals to intimately understand the DevOps cycle (plan-code-build-test-release-deploy-operate-monitor) and how DevOps teams work. What are the product teams working on, what language do they speak, what technology is being used, how’s it deployed and where’s it run? And for each phase, we should be able to quickly assess the key risks and provide a set of standardized and automated controls — based on risk.

Having a set of policies and training DevOps teams in security practices is still vital. But what we really need to think about is how we can enable DevOps teams to:

  • Get immediate feedback if code contains any vulnerabilities, within minutes after a commit
  • Not have to worry about securing the infrastructure components and automatically deploy into the right infrastructure zones
  • Remove the need for patching (hint: automatically rebuild servers every few days)
  • Leverage existing and well run security services like Identity & Access Management, security monitoring, incident management and others, through the use of standard APIs
  • Remove the security requirements for low-risk changes to low-risk products
  • Easily seek our Infosec expertise to provide guidance without project inhibition

And in true DevOps style, it shouldn’t be too hard to start at a small scale without aiming for perfection. And then continuously improve and make it hum.

Apply a bit of DevOps principles to security, I guess.