Security Challenges of Containers and Kubernetes
These days, most modern applications are built using containers and Kubernetes. Both provide many advantages, but just like with any other technology, if they’re not set up correctly, they can be exploited. All containers use the same kernel from the underlying host machine, so even one compromised container can do substantial damage. The same applies to Kubernetes. Even though by design it provides an abstraction layer on top of your virtual machines (VMs), if it’s not secured well, it only takes one compromised container for a hacker to gain access to all other containers and underlying VMs. In this post, you’ll learn about the most common security challenges of containers and Kubernetes.
Containers: Secured by Design?
You may have heard that containers are secured by design. The point of a container is to isolate the process, right? Well, yes, but that’s not the full story. While containers can be pretty secure, it’s usually up to you to make them secure. There are many extra features and options you can apply when using containers to make them more secure. Some are easy config changes, and others require you to build your images in a specific way. Let’s dive into that straight away.
1. Base Image
First things first. A base image, as the name suggests, is the image that will be used as a base for your custom docker image. So no matter how many best practices you follow to create a secure image, if your base image has malicious binary baked into it, you won’t be safe. Sometimes you may even use a random image from the internet altogether instead of building your custom one.
For example, if you need a nginx server in your container, instead of building a custom one you may just search for “nginx docker image” in Google. If so, you’ll find plenty of ready-to-use images. And while it saves you time and effort compared to building a custom image, it may expose you to a security risk. But don’t get me wrong, not all premade images are bad. You just need to be aware that some may be.
Therefore, only download images from trusted sources. For most popular software, you’ll find images created directly by software maintainers. Therefore, coming back to our example, instead of downloading a nginx image created by a random person from the internet, download an image created directly by nginx developers.
On top of that, it’s a really good idea to perform security scanning of the docker images. You can do that on the fly before running any container or if all your images are coming from your private registry (also a good security practice). Then you can run periodic scans of the whole registry.
2. Privileged Containers
The second-most important container security tip is to never run containers in a privileged mode unless absolutely necessary. Why is privileged mode so bad? Circling back to the begging of this post, containers by design are suppose to provide you with a secure environment. This is because they isolate the process running inside from the host machine.
When you run a container in a privileged mode, you’re basically giving it access to all devices on the host machine and lifting some other limitations. In other words, the privileged container has full access to the underlying host machine, including the Docker daemon itself.
3. Don’t Run Root in the Containers
Again, coming from the perspective that containers are secure by design, many people think it’s fine to run whatever is inside the container as a root user. Wrong. While Docker in fact never had many vulnerabilities, there are some. If you don’t keep your Docker daemon always up to date, there is a chance that someone may try to escape from the container and get access to the host operating system. If your containers are running as root in such a situation, an attacker most likely will automatically get root access to the host machine. On the other hand, if your container is running any user other than root, that would give the attacker only non-root access to the host machine. Therefore, the attack possibilities would be limited.
How does Kubernetes security differ from container security? Kubernetes is a container orchestration platform. Therefore, it doesn’t make your containers secure on its own. It only orchestrates them. So you should still take care to secure your containers first. Using trusted base images, not running privileged containers, and avoiding root users inside containers—all of that still applies. But in addition, you need to apply a few more Kubernetes-specific security best practices. Here are the most important ones.
1. Role-based Access Control (RBAC)
Let’s start with the most obvious one. In simple words, RBAC permissions define who can do what on your Kubernetes cluster. In a typical company, many people from different teams get access to the Kubernetes cluster. By default, any user who has access to Kubernetes can do pretty much anything. To change that you need to define RBAC roles and assign them to users. For example, developers can only get read access to the cluster (since all the deployments should go via CI/CD pipelines). This approach drastically limits the possibility of compromising your cluster. If your company has fifty developers and all of them get unlimited access to the cluster, an attacker only needs to get the credentials from one of them to get full access to the cluster. So even if only one of your developers uses a weak password, your cluster will be exposed.
2. Network Policies
Specifically in Kubernetes, you need to take a closer look at the network traffic. By default, any pod in any namespace can talk to any other pod in the cluster. From a security perspective, this is not really a desired situation. It means that if an attacker manages to deploy (or get access to an already running) malicious container, it can easily try to get access to your database or API containers. Normally for an attacker, it’s easier to get access to a front-end container (as this can be exploited from the internet) than to a back-end or API service. And if they manage to do that by default, they can directly connect to the database from there.
Network policies are the solution to that problem. They allow you to define which pods can connect to where. So coming back to our example, with network policies you can define that only back-end pods are allowed to connect to the database (and that the front end is only allowed to connect to the back end). In a more realistic scenario, you will probably have many small secondary services that are usually not important and therefore often forgotten during security improvements. But then again, by default, they have access to all important pods. Therefore, if you really care about securing your Kubernetes cluster, network policies are a must.
3. Runtime Security Scanners
A typical Kubernetes cluster will run many different pods. You’ll probably also have many pods that are not running your application, such as monitoring agents or secrets management tools. And if we’re talking about managed Kubernetes, you’ll probably also have some pods from the cloud provider. The point is, you’ll definitely find some pods that you won’t know what they’re doing. And this is good for the attacker because if they manage to deploy a malicious pod on your cluster, you may not even realize it’s there. Therefore, the bigger the Kubernetes cluster, the more helpful it is to have a runtime security scanner running. These are the applications that constantly monitor all the actions performed by all the pods on your cluster and block or notify you of suspicious activity such as executing shell or running curl/wget in the container.
Yes, containers are relatively secure if done correctly, but you need to put a little bit of effort into hardening your images and docker config. The good news is it’s not that difficult, and you can apply most security best practices globally in your organization. On top of that, Kubernetes requires a few extra things to keep an eye on. And the few tips you’ve seen in this post are only the beginning.
If you need assistance with your Containers and Kubernetes adoption, contact one of our experts today.