How is security managed in Kubernetes clusters?
Best practices for managing security in Kubernetes at various layers
In the past few weeks after the Log4j vulnerability was all over the news affecting thousands of java applications running in production sites, the security has caught the eyes of all software developers, architects, and product owners alike. This log4shell vulnerability proved as a nightmare for most of the engineers across the globe with a CVSS score of 10 because it allowed doing an RCE. More details about this vulnerability can be seen here in this video.
Security is a very crucial aspect of any product or application in an organization. When it comes to deploying applications in the cloud, security becomes even more of a concern as it is a shared responsibility between the application developers and the cloud providers. Cloud security is actually a combination of security controls at various levels viz, application, cluster, and host OS. Clearly, it’s a set of controls and best practices and is not just a single setting or checkbox to be ticked.
Primarily when it comes to applying security, it is said that cluster level and underlying host OS level security should be the responsibility of the managed cloud providers for e.g., GCP, AWS, or Azure.
Below are some of the best practices that can be applied to preserve the security at all the 3 layers starting with the application level, followed by cluster level, and finally at the host level Operating system.
Application Level Security
The topmost layer of security is at the application level and comes under the consideration of the app developers. Containerization provides the ability to isolate application data and resources from one another, but that is not enough when it comes to security isolation. As application developers or system engineers, we need to make sure that our containerized application should have access to ONLY what it needs and nothing more than that.
Some of the best security practices at the application level are as follows:-
- Use Secrets — All of the sensitive data pertaining to an application should be stored within a Kubernetes secrets such as passwords, any key, or a token to connect to an API. This way no sensitive data is exposed in an application code and can be safely pushed to a version control repository.
- RBAC — Role-based access control (RBAC) is a method of regulating access to resources based on the roles of individual users within a k8s cluster with the help of roles, clusterrole, rolebindings, clusterrolebindings.
- Cluster Hardening — One of the best ways is to secure applications deployed within a k8s cluster is to restrict the user access by specifying Run as user in the deployment YAML file for a pod definition.
- Vulnerability scanning — Regular checks should be run on the libraries used within an application and any library with a CVE status = HIGH or CRITICAL should be regularly patched within the application. Vdoo is an excellent tool to capture such security risks within the software.
Cluster Level Security
This is the layer on top of the host OS where Kubernetes is installed and is the central place where all components of a k8s cluster interact with each other.
Some of the best security practices at the cluster level are as follows:-
- Restrict access to etcd and apiserver at the network level — the pods running on worker nodes should not have a direct network established with etcd and should be placed in a secure network. In addition to that, the traffic to etcd should be encrypted. etcd is an important part of a cluster as it’s the CPU of a k8s cluster storing all secrets and complete states of the running pods within a cluster, therefore it must be secured at all times.
- Authorization and Authentication — Setting up etcd behind a firewall and using authentication to reach the API server can save the cluster from attackers and make it more secure.
- Admission Controllers — This can be implemented in two ways, one is using Node restriction for e.g., restricting access to nodes having API server and second by writing custom network policies to restrict communication between a pod and other network entities within a k8s cluster.
Host Level Security
This is the level where Kubernetes is installed and should ONLY be used for this single purpose, i.e to provide Kubernetes. Usually, the Host OS could be an RHEL 7.x, Ubuntu 20.x
Some of the best security practices at host level OS are as follows:-
- Least Privilege access — The host-level OS should not be exposed to ssh into for any remote execution of commands via the application pods.
- Set up IAM policies — IAM roles can be created and tied to a Kubernetes service account which can then be restricted to access certain k8s resources.
- Reduce Attack Surface — It’s imperative to keep minimum attack surfaces for the hackers at the host level by not installing any application on Host level OS and restricting execution of yum install cmd to only sysadmins. In addition to that, it is always recommended to use the latest version of Kubernetes.
Conclusion:
As long as we are adhering to some of the aforementioned security practices and invest some time on the security aspect of an application right at the start of architecture design and discussion when we discuss and develop software tools and technologies for a specific product or an application, we can make sure we have the best of the secured products and applications developed and delivered devoid of any weakest links in the whole ecosystem.
