DevSecOps Guide to Bul­let­proof Your Devel­op­ment Work­flow: part 1

Today, in an ever-chan­ging soft­ware devel­op­ment world, secur­ity becomes more and more of a major con­cern and require­ment. But it’s not unusual to onboard a pro­ject where secur­ity remains one of the sac­ri­ficed top­ics. Why is this?

Is this due to the need to deliver faster and faster? Resource con­straints? Dif­fi­culty to integ­rate in leg­acy code in a lot of time? Or maybe a lack in cul­ture on the team?

Well, it’s actu­ally all of them, and the first step in order to change this is real­ising where exactly the prob­lem relies. But let’s break it down a little, shall we?

This is a BIG topic, so you’re read­ing Part 1 in 2.

In Part 1, we will cover:

• The Secur­ity Silo
• DevSecOps to the rescue
• What we want to achieve
• The Shift Left approach
• Pro­tect­ing your soft­ware (Ana­logy)
• Object­ive scenario
• Dif­fer­ent kinds of scans
• Don’t leave Observ­ab­il­ity behind

The Secur­ity Silo

To under­stand the gen­esis of DevSecOps, we have to under­stand its need. Going back a bit we real­ize what gave birth to DevOps: there was a need to cement the breach between everything Devel­op­ment and everything Oper­a­tions, right?

We under­stood that tak­ing a pack­age or a bundle and send it to another team with a set of instruc­tions to be executed was prob­ably not the best approach in the long run. It was a huge momentum killer, between com­mu­nic­a­tion, arti­fact trans­fer­ing and the numer­ous ways this could fail.

Maybe the instruc­tions wer­en’t clear… maybe some vari­ables were left uncon­sidered or the con­text was miss­ing. There were a pleth­ora of ways this pro­cess could go wrong. Then, what happened when it went wrong? Well, time to repeat the pro­cess yet again, ain’t it fun? Now let’s have a quick round of applause for auto­ma­tion pro­cesses. Yikes…

With secur­ity, we’re back to the same old prob­lem. Secur­ity is siloed from everything else, and this can vary from insuf­fi­cient secur­ity checks up to the extreme of hav­ing a com­pletely dif­fer­ent team, we prob­ably have no con­tact with or con­fid­ence in, per­form­ing who knows what kind of tests on your code… or even have no secur­ity checks at all! I’ve seen it all, and I’m sure you have too.

But one thing seems to reveal itself to be the stand­ard use case, which is secur­ity being a des­tin­a­tion of the soft­ware devel­op­ment pro­cess when in truth, it should be a part of it.

DevSecOps com­ing to the rescue

Secur­ity is a require­ment, it’s fun­da­mental, so it’s got to be treated as such.

Enters DevSecOps!

Tak­ing a look at the cur­rent soft­ware devel­op­ment land­scape we can book­mark some uni­ver­sal con­cepts: quick-time to mar­ket, code qual­ity and secur­ity. If you lack any of these, you are in big dis­ad­vant­age over the com­pan­ies who can guar­an­tee all three.

So, what is DevSecOps?

DevSecOps is the inclu­sion of secur­ity in the devel­op­ment work­flow, just like DevOps, it’s more of a group of pro­cesses and tools that need to be employed in order to pro­duce bet­ter soft­ware pieces, do it faster and of course, guar­an­tee­ing secur­ity right from the get go.

What do we want to achieve?

If you can guar­an­tee code qual­ity, fast deliv­ery and secur­ity, your product wins. It’s just like that. So, in order for us to get this brighter future for your Soft­ware Deliv­ery Life Cycle, a couple of points have to be secured (haha get it?).

• Secur­ity test­ing can­not be sac­ri­ficed or siloed anymore;
• Secur­ity test­ing has to be stream­lined and not be a des­tin­a­tion of your soft­ware devel­op­ment workflow;
• Resources can­not be wasted by hav­ing unat­tached teams and not well defined con­cepts within your team;

The Shift Left approach

One of the biggest con­cepts you can think of when need­ing to stream­line secur­ity in your soft­ware devel­op­ment work­flow is the Shift Left concept.

Very simply put, Shift­ing Left means mov­ing some­thing to an earlier stage of a pro­cess. So, shift­ing secur­ity test­ing left means that we want to have this done as early as pos­sible in our soft­ware devel­op­ment process.

With this we guarantee:

Faster deliv­ery – Since we’re not wait­ing for a team to send back a batch of res­ults and then iter­at­ing over them (repeat­ing the devel­op­ment over old goals, and los­ing time adding new fea­tures to the product) we’re sav­ing time.

By find­ing vul­ner­ab­il­it­ies early, you focus on a smal­ler chunk of code (rather than months of com­mits piled on each other) resolv­ing these rather quickly.

Cost reduc­tion – When you have vul­ner­ab­il­it­ies being found and con­sequently mit­ig­ated early in the soft­ware devel­op­ment pro­cess, you stop the rolling snow ball that will grow over time and that will hurt your com­pan­ies’ fin­ances and repu­ta­tion (harshly).

Secur­ity guar­an­teed early on - By auto­mat­ing the pro­cess and hav­ing scans over your code, you can guar­an­tee that you have reg­u­lar checks and that you can act as soon as pos­sible over small batches of code, and demol­ish vul­ner­ab­il­it­ies very early in the process.

Of course when you have all of these in place you can ima­gine that your effi­ciency goes through the roof.

The “Pro­tect­ing Your House” analogy

That was a lot of talk was­n’t it? Let’s get to the juice!

So, how can we start pro­tect­ing our soft­ware? Or any­thing, really?

The answer might be obvi­ous but it’s quite simple actu­ally. Let’s see a spe­cific example, start­ing with another question.

How would you pro­tect your house?

You set up a wall or some fences around it so people and/or your nosy neigh­bours don’t walk in.

You make sure the con­struc­tion mater­i­als are of good qual­ity, use doors and win­dows. That they have locks and are sturdy. Maybe you set up some secur­ity cam­eras around your prop­erty, right? Maybe you have some dogs roam­ing around, just in case…

Pro­tect­ing your soft­ware is just the same thing, same logic: you find vul­ner­able spots that can be exploited and you start pro­tect­ing them.

Here’s an object­ive scenario

Let’s look into an object­ive scen­ario so we can have some­thing palp­able to apply these con­cepts on.

Here’s what we might work with, when we talk about software:

• Third-party lib­rar­ies;
• Your own code (everything you copied from StackOverflow…we all know);
• A Dock­er­file that is a blue­print for our con­tain­er­ised application;
• Our own infra­struc­ture that can be ver­sion con­trolled and replicable;
• A group of Kuber­netes resources that are API based;

Let’s go one by one and see how we can start secur­ing them.

Dif­fer­ent kinds of scans

Third-party lib­rar­ies

This is the code you have no dir­ect con­trol over, but there’s a bunch of func­tion­al­ity that is being brought into your code and all of those imports that are bring­ing it are also inject­ing vul­ner­ab­il­it­ies into it. So you can­not for­get about check­ing the full extent of what’s being used.

For this we want to have Soft­ware Com­pos­i­tion Ana­lysis (SCA) in place.

SCA deals with the open-source code in your applic­a­tion. It is able to detect vul­ner­ab­il­it­ies in these com­pon­ents and ensure that licence com­pli­ance is respected.

Look­ing into the house ana­logy: you’re dec­or­at­ing your liv­ing room and saw this per­fect couch while strolling through a yard sale. It looks per­fect, cozy and soft but you still need to check what it’s all about, maybe it’s full of termites…

Your own code

Now you want to look into your own code, the one you have full own­er­ship over. I’m sure you don’t want to be respons­ible for the next big breach of your company.

To scan your own code you can look into a couple of alternatives:

• Static Applic­a­tion Secur­ity Test­ing (SAST)
• Dynamic Applic­a­tion Secur­ity Test­ing (DAST)

For SAST scan­ning you have one of the best quick wins you can get when talk­ing about scan­ning your own code. It’s usu­ally really easy to integ­rate which allows you to have it up and run­ning in a mat­ter of hours (maybe not even so!).

SAST looks into the code itself, and then holds it against a set of rules that are pre-defined for the tech stack you are using and if it finds one pat­tern that matches one of those rules, it will warn you about it.

Now you were brave enough to go on You­tube and build a chair with your own hands. You’re really proud of your­self by the end of it (as you should) so you invite your par­ents for din­ner, but maybe check the chair first to make sure it won’t break as soon as someone sits on it.

For DAST, it’s a bit dif­fer­ent, this tool is used to also secure your code but it fol­lows more of a Shift Right approach (which dif­fers from the Shift Left approach) since its res­ults come after you have some­thing already deployed.

DAST does not know about your code, but it’s pur­pose is to try to break your code like a hacker would, try­ing to sim­u­late attacks and ana­lys­ing what comes back.

At the end of the day, there’s really not one over the other, you should use both of them in order to get a more robust code scan­ning. But their imple­ment­a­tion timelines can be (and usu­ally are) quite dif­fer­ent. If you want some­thing done to show your boss for tomor­rows’ daily then imple­ment­ing SAST is the way to go. If you want to focus on reli­ab­il­ity after you actu­ally deploy some­thing then give DAST a go.

Now, get­ting back to the chair… SAST knows com­mon pit­falls of chair build­ing so it knows just by look­ing at the chair that one of the legs is bound to break eas­ily. On the other hand, DAST will sit on the chair in vari­ous ways and forms and try really hard to break it.

What a great example, right?

Con­tainer image scanning

We have checked our code and all the third-party tools we are using or, if you prefer, all the fur­niture you’ve built your­self and the one you bought.

Now, we have our app Dock­er­ized and we took our code and put it inside a pack­age, just like we put the fur­niture on the liv­ing room. It’s time to check the liv­ing room itself, make sure it’s not covered in mold as it will ruin everything.

Hav­ing that said, when you’re using Docker to run your applic­a­tions an enorm­ous num­ber of vul­ner­ab­il­it­ies comes from the base image itself. You always start with a base image and a lot of it is about check­ing what it brings.

Besides this, you have to check:

• Your soft­ware pack­aged inside the container;
• Addi­tional configuration;
• Net­work settings;
• Among oth­ers…

Let’s make sure all are checked.

Infra­struc­ture-as-Code Scanning

It’s time to check the struc­ture of the house itself. We don’t want a gust of wind knock­ing everything down after doing so much work.

Infra­struc­ture-as-Code is a fun­da­mental part, espe­cially when you start scal­ing your infra­struc­ture, and it becomes hard to man­age manually.

With Infra­struc­ture-as-Code we usu­ally have a bunch of cloud resources with con­fig­ur­a­tion that can be exploited. So things like:

• Over­ex­posed CIDR blocks;
• Unne­ces­sary open ports;
• Weak encryp­tion algorithms, or miss­ing entirely;
• Out­dated services;
• Com­pli­ance violations;

Kuber­netes Objects Scanning

Your con­tain­ers are being orches­trated using Kuber­netes. Kuber­netes offers you a bunch of resources to man­age your ser­vices and all of these can be twist and turned in some way.

These include:

• Deprec­ated API Versions;
• Role base access con­trols that are too permissive;
• Sens­it­ive inform­a­tion exposed on ConfigMaps;

And last, but not least…

Don’t leave Observ­ab­il­ity behind