We all know the impact of container technology and how widely used the technology is. I started to learn Azure and Kubernetes about four years ago. I did not have a lot of experience with containers, let alone container security. Once I finally got to the point of understanding and working with Kubernetes I started to also look into the security part. Now that people/ companies become more and more familiar with the technology (how to build, run and manage microservices) I hope that they also start to think more about security regarding container technology. Containers are NOT secure by default!

The picture above describes the container technology architecture and we can derive the five core components:

  1. Image (built by developers)
  2. Registry
  3. Orchestrator
  4. Container
  5. Host OS

All of those five core components have risks. In a series of blogs I will address all major components and its risks!

Image Risks

So, the first major component is the Image. NIST describes the following risks regarding images:

  • vulnerabilities in images
    • e.g. image missing (critical) updates
  • configuration defects in images
    • e.g. image configured to run as root
  • embedded malware in images
    • e.g. malicious files
  • clear text secrets in images
    • e.g. ssh keys, username/password in images
  • untrusted images
    • e.g. using unofficial images (containing who knows what)

To dive deeper into all of the image risks and its countermeasures you can check the NIST Application Container Security Guide. One of the countermeasures is scanning your images from build through registry to runtime. In this first post I am going to show how you can mitigate some of the Image risks, by scanning your image during build time with Github Actions and by scanning the images in your Azure Container Registry. Scanning your images via Github Actions is easy to set up, does not cost you anything and does not need an Enterprise product for vulnerability management.

Image scanning with Github Actions

We will be using Github Actions to scan our images and the Microsoft Azure team has developed a container scan action that uses Trivy and Dockle. It scans your image for vulnerabilities and advices you on best practices regarding building images.

Requirements

To be able to implement the below instructions there are a couple of requirements:

  • You have a Github account. Don't have one? Then you can create on here
  • You have an Azure account. If needed, you can create a free account with $200 credits for 12 months
  • You have git installed
  • You have vscode (or another code editor) installed
  • You have experience with git and working from the command line
  • You have knowledge/ experience with Github and Github Actions
Setup
  • Create a new repository in Github
  • Clone the repository locally
  • Add the following files to the repository
    • .github/worflows/vulnerability-scan.yaml
    • Dockerfile
    • index.html
  • Push the new files to your Github repository

Once you have pushed the changes to Github the workflow will kickoff. Go to the Actions tab and click on the running workflow. You will see a result like below when it has finished:

When you click on vulnerability_scan > scan image for vulnerabilities you will see a detailed overview of all vulnerabilities found:

When you click on [container-scan] testimage:{{ github.sha}} you will get a summarised overview of all CVE's found per severity and the best practices violations:

Scan configuration
Severity-Threshold

You can configure the severity-threshold option to the level that you want to see. If you set it to MEDIUM, then all vulnerabilities with severity MEDIUM or higher will be shown. The available levels are: UNKNOWN, LOW, MEDIUM, HIGH, CRITICAL.

Fail Pipeline

If you want your workflow to fail when a certain level of vulnerabilities or higher has been found in your image, then you will have to set continue-on-error: false.

This Github Action helps with the whole shift left security movement, where we want to know about security findings as early as possible in the development process. The earlier, in the development process, developers know about vulnerabilities and can fix them the better. It saves a lot of money and time preventing from having to fix the vulnerabilities in production.

Integration of Github Actions with Azure Security Center (PREVIEW)

During Ms Build 2021 the integration with Azure Security Center (ASC) was announced, which is in Public Preview at the time of writing this post. This integration enables you to send the vulnerability scan results directly to ASC. To be able to use this feature you need to deploy:

  • an Azure Container Registry (ACR). See this on how to deploy an ACR. In this blog I called the ACR azurecontainersecuritydemo, so replace that with the name that you choose for your ACR.
  • enable Azure Defender for your ACR. You can enable it by going to the Azure Portal > Azure Security Center > Pricing & Settings > Select the subscription the ACR is in and enable Azure Defender

Notes: 1) Azure Defender for Azure Container Registry is a paid service in Azure! 2) It can take up to 24 hours before you see scan results in ASC, this is because of the refresh rate in ASC!

So, when you have enabled Azure Defender you can start scanning your image and send it to ASC with the following workflow:

ACR login configuration

There are several ways to authenticate to your ACR. In this example we use the admin username and password and both are saved as secrets in your Github account, which you refer to via {{ secrets.REGISTRY_USERNAME }} and {{ secrets.REGISTRY_PASSWORD }} in the Azure/docker-login@v1 task.

Publish image vulnerabilities

The Publish GitHub action azure/publish-security-assessments@v0 sends the scan results to Azure Security Center. At the moment of writing it only sends the amount of vulnerabilities found per severity.

Image scanning in your Azure Container Registry

Another way of scanning your images is by scanning them in your registry. When we enabled Azure Defender for our Container Registry we, by default, enabled the scanning of our images in our ACR, which is powered by Qualys.

High-level overview of protecting your container registries with ASC

There are three triggers for scanning an image:

  • on push
  • recently pulled (weekly scan of images that have been pulled in the last 30 days)
  • on import
Requirements
  • You have an Azure account. If needed, you can create a free account with $200 credits for 12 months
  • You have an Azure Container Registry deployed
  • Azure Defender enabled for Azure Container Registry

You can check the vulnerabilities found by Qualys by going to Security Center > Container image scanning > Affected resources > Select your ACR

You now have an overview of all your repositories within your ACR and the amount vulnerable images and the severities. If you then click on one of the repositories you will see all the images in that repository. If you drill down to one of vulnerabilities of an images (digest), you will get a description of the vulnerability, a link to the CVE, if there is a remediation available and which images it affects in your ACR .

So, this gives some nice feedback on what the vulnerability exactly is and how you can fix it (if a patch is available). You can add the fix and rebuild your image. The vulnerability should not be found anymore during the scan at build time or while scanned in your ACR (push trigger).

Summary

When we look at container technology, there are five major components: image, container registry, orchestrator, container and host. Each component has its risks.

One of the risks regarding the component Image is vulnerabilities. To mitigate the risk of vulnerability in images you have to scan your image at build time and in your container registry.

With Github Actions we can our images at build time and with Azure Defender we can scan images in container registries. There is also a link between Github Actions and Azure Security Center, which publishes the scan results at build time in Azure Security Center.

What's next?

In my next post I will explain what Aqua Security is, what it does and how it compares to the mitigations described in the first post!