Why and How to Secure Pods with Kubernetes Security Contexts

How can you ensure that each resource in Kubernetes has the permissions it requires, while at the same time avoiding over-permissioning? In other words, how can you define permissions on a granular basis that adheres to the principle of least privilege?

The answer is Kubernetes security context. Security context is a tool that allows admins to define security-related parameters on a resource-by-resource basis. As such, it makes it possible to assign each resource the specific permissions that it needs to access resources on the host server while denying access to those that it doesn’t specifically require.

This article provides an overview of Kubernetes security context, including how it works, how to define a security context, and which limitations to bear in mind when working with security context in Kubernetes.

What Is Kubernetes Security Context?

In Kubernetes, a security context defines privileges for individual pods or containers. You can use security context to grant containers or pods permissions such as the right to access an external file or run in privileged mode.

Internal vs. External Security Contexts

Kubernetes security context is a bit complicated in the sense that some of the rules that you can define are enforced internally via Kubernetes itself, whereas others integrate with external security context tools – namely, AppArmor and SELinux.

Thus, you can think of Kubernetes security context as a way to define certain permissions for pods and containers, as well as to integrate Kubernetes with external security tools that run on the host rather than in Kubernetes itself.

Security Contexts vs. RBAC

Security context is similar to, but distinct from, Kubernetes Role-Based Access Control, or RBAC. The key differences are as follows:

• Resource scope: RBAC can be applied to a variety of Kubernetes resources, such as pods, Kubernetes nodes, and even entire clusters. Security context assigns permissions only to pods.
• Actions: RBAC can grant a variety of permissions based on “verbs” that admins can define within RBAC policies. Security context is more restrictive in that it only allows admins to assign specific types of predefined capabilities, like running in privileged mode (although security contexts are more flexible if you define rules using SELinux or AppArmor).
• Extensibility: As noted above, security contexts can be extended via integrations with external frameworks, including SELinux and AppArmor. Kubernetes RBAC can’t use external tools to define policies.

So, think of security context as a way to define additional types of security permissions for containers and pods that RBAC can’t manage. For most Kubernetes environments, you’ll want to use security context and RBAC at the same time because they complement each other.

Security Contexts vs. Pod Security Policies

Many of the security rules that you can define using security contexts can also be configured via pod security policies, which are a different tool.

Why would Kubernetes provide support for both security contexts and pod security policies? The answer is that security contexts are essentially a replacement for pod security policies. Pod security policies, which can be used to configure permission for all pods running in a cluster, provide less granular control than security contexts, which can be applied to individual pods.

As of Kubernetes version 1.21, pod security policies are considered deprecated, although they are still supported for now. Pod security policies will be removed entirely with Kubernetes 1.25, at which point any pod security policies you define will be ignored.

How to Use Kubernetes Security Context

Using security context in Kubernetes is straightforward (especially if you’re working only with internal security context and not integrating with SELinux or AppArmor). You simply include a block of security context code within the deployment file that you create when deploying a pod.

For example, the following block tells Kubernetes to run a pod with a user ID of 1000. It also assigns a group ID of 2000 to all containers in the pod:

Spec:
  securityContext:
    runAsUser: 1000
    fsGroup: 2000Code language: JavaScript (javascript)

Unlike RBAC, security context doesn’t require you to define multiple types of files (like Roles and RoleBindings) in order to enforce a security rule. You just need to add the requisite security context code when you declare a deployment, and Kubernetes will automatically enforce the rules from there.

For a full overview of types of permissions you can assign (or deny) with security context, refer to the Kubernetes documentation. In addition to the permissions related to user and group IDs, most admins will find value in the parameters that make it possible to let the pod or container run in privileged mode. A container running in privileged mode has almost all of the same access rights to kernel-level resources on the host as a process that runs as root, so you’ll typically want to disallow privileged mode. Instead, define permissions on a granular basis by, for example, allowing the container or pod to bind to a certain port or execute certain external binaries, while otherwise denying access to resources outside of the container.

Working with External Security Contexts

Enforcing security context based on external tools like SELinux and AppArmor requires a bit more work. Here are the basic steps.

Load the SELinux or AppArmor Module

First, you need to make sure that the kernel module associated with the framework you are using (i.e., SELinux or AppArmor) is installed and loaded on the node or nodes that will host your containers or pods.

Since in most cases Kubernetes automatically assigns pods or containers to nodes, you won’t know ahead of time which node will host which container or pod. As a result, you’ll typically need to install AppArmor or SELinux on every node in your cluster if you want to define security contexts via one of these frameworks.

Load a Profile

After loading the modules, you need to load the AppArmor or SELinux profile that you’ll use to define permissions. For example, this AppArmor profile denies the ability to write files:

#include <tunables/global>

profile k8s-apparmor-example-deny-write flags=(attach_disconnected) {
  #include <abstractions/base>

  file,

  # Deny all file writes.
  deny /** w,
}Code language: JavaScript (javascript)

Save the profile in the node’s filesystem at a location that Kubernetes can read.

Again, since you probably don’t know which node will run which containers or pods, you’ll need to load the profile on each node in your cluster. For efficient ways to do this (i.e., in order to avoid having to SSH into each node and apply the profiles manually), check out the Kubernetes documentation on node profiles.

Apply the Profile in Kubernetes

Finally, add a security context block to your deployment file that tells Kubernetes to reference the external security context profile and apply it to the deployment:

apiVersion: v1
kind: Pod
metadata:
  name: apparmor-demo
  annotations:
    container.apparmor.security.beta.kubernetes.io/hello: /path/to/policy-fileCode language: JavaScript (javascript)

Apply the declaration with:

kubectl create -f ./deployment.ymlCode language: JavaScript (javascript)

Now, Kubernetes will enforce the policy you configured through AppArmor or SELinux just as it would a security context that you configure directly inside a deployment file.

Limitations of Security Contexts

Kubernetes security context is a powerful tool for defining certain types of permissions on a granular basis. However, it is currently subject to some significant limitations.

No Windows Support

For one, the security context tooling currently addresses only privileges and permissions that are valid on a Linux-based server. If you run Windows containers in Kubernetes, security contexts won’t be useful to you.

Limited to Pod/Container-Level Security

Security context can only define permissions for pods or containers. It can’t control privileges at other layers of your stack.

Of course, most of the rules that you can apply with security contexts would only make sense when applied to containers or pods. It wouldn’t make sense to tell a node to run in unprivileged mode, for instance.

Still, the point here is that security context is a tool only for addressing security issues at the pod or container level. You’ll need other tools (like RBAC) to secure nodes, users, service accounts, and the like.

It’s an Evolving Feature

Security contexts are a relatively new feature for Kubernetes, and they are still evolving. Certain parameters (like fsGroupChangePolicy, which was introduced with beta support in Kubernetes 1.20) are not yet fully supported, and more definitions are likely to appear in the future.

So, while you can certainly use security contexts in production clusters today, it’s important to follow the evolution of this feature closely, as certain types of definitions may change in coming releases.

Inefficient Tooling

As we’ve noted, a major limitation of some security contexts (specifically, those that use SELinux or AppArmor profiles) is that they require the deployment of external resources on every node in your cluster. While there are ways to automate this process, merely setting up the automation takes a fair amount of effort. Deployment could also be messy if you have nodes running different Linux distributions, in which case you may need to customize your AppArmor or SELinux setup for each distribution.

One can hope that tools to simplify policy profile deployment across nodes will appear in the future. But for now, don’t underestimate how much effort it will take to set up the nodes.

Despite these shortcomings, security contexts are an important resource for plugging potential access control gaps in Kubernetes clusters. Although they are limited in scope and are not a complete access control solution unto themselves, you should take advantage of security contexts as a way to bolster the overall security of your cluster.