Update dockerfiles to ensure underlying OS is always running the late…

[mtesauro]

We're not consistently updating the packages in the underlying container OSes when we build containers. Some steps in the multi-step builds didn't apply OS updates and/or the package index was updated but the packages weren't upgraded.

This PR make sure that all dockerfiles for DefectDojo images pull the latest OS packages when building images to that we have the latest possible versions of OS packages which should reduce any bugs/reported issues in the OS packages in the containers.

Only dockerfiles where changed in this PR, no Python source for DefectDojo itself were modified.

[valentijnscholten]

Won't this move us further away from reproducible builds? Every time you run it a different image can be generated and afterwards you won't know if security updates have been applied or not. Coincidentally I spent quite a bit of time on this today figuring out what the beep was included in a docker image at built time.

If we decide to do it anyway, we should make sure somehow that builds are done without caching as docker may cache the whole RUN layer.

[mtesauro]

@valentijnscholten

Won't this move us further away from reproducible builds? Every time you run it a different image can be generated and afterwards you won't know if security updates have been applied or not. Coincidentally I spent quite a bit of time on this today figuring out what the beep was included in a docker image at built time.

If we decide to do it anyway, we should make sure somehow that builds are done without caching as docker may cache the whole RUN layer.

I guess that depends on what you mean by 'reproducible builds' if you build all the image at the same time, you'll get the same containers with our without this PR. The big difference is with the PR, those containers will have the latest Debian/Alpine packages. The one edge-case is if there happens to have new OS-level package land in the distro repositories between build stages but that seems very unlikely.

So, if I have to choose between reproducibility + having outdated packages in the containers versus non-reproducibility & updated packages in the containers, I'm choosing updated packages every time.

A ran Anchore against our containers. Looking at just the Debian Django container: of the reported 66 issues of Low or greater severity, 51 of them had OS packages which fixed the issue. This little PR fixes 77% of the issues in our containers - seems like a good thing to me.

BTW, I'm not adding any new packages to the containers, just updating the version ones that were already there so image size should be untouched or very, very slightly bigger only if the newer packages are bigger.

If we're very concerned with the versions of what's in our images outside of the DefectDojo app, we can always add a command in the dockerfile to output the package versions to a text file in / so we can see exactly what the OS installed. Something like:

dpkg -l > /installed-os-packages-at-build-time

Plus the same thing for Alpine. We could do the same thing for Python packages as well.

[valentijnscholten]

What I mean is that when I build 2.55.2 today and everything works you may get different results if you or someone out there builds 2.55.2 tomorrow and there's an updated package that breaks something. But we don't know why it's not working as we are unaware of which packages are different from the official build our the build of another developer where "it works on his/her/their machine".

Chances are not huge that it causes problems but it would be nice if everyone building a specific Defect Dojo version would get the exact same container. I know we don't have this today, but this PR moves us further away from that.

From those vulnerabilities reported by Anchore I doubt (m)any are exploitable.

But I can understand that in some contexts users care more about the number of vulnerabilities and less about having predictable/reproducible images.

If you truly want to get rid of vulnerabilities you would have to rebuild the images periodically, i.e. daily. Most Debian patches arrive Tuesday night I believe.

[mtesauro]

Ahh, I see.

Then we're talking about building for 2 different reasons / user groups.

The build I'm concerned about this the one that creates the official containers that we provide to the community via Docker Hub. Those should be a one & done container target at users who just want to use DefectDojo. If there's an issue with defectdojo/defectdojo-django:2.53.2 then the answer for that user is to grab 2.53.3 and get the fix.

I'm less concerned with the more technical users who are building their own images. They are in the best place to adjust to a changing library or to diagnose what's going on.

While I agree that most of those Anchore vulns are note exploitable (or if they are, you got bigger problems if attackers are shell'ing into your containers) it doesn't stop them from getting reported both in Github issues and via our bug bounty. So, if I can keep the number down to to ~15 then I'd call that a win.

So, I'm not trying to get to zero - just trying to get to a reasonable number that gets better every week when we cut a minor or bugfix release.