Containers are increasingly being used to deploy applications, and with good reason, given their portability, simple scalability and lower management burden. However, the security of containerized applications is still not well understood. How does container security differ from that of traditional VMs? How can we use the features of container management platforms to improve security?
This is the first in a series of blog posts that will cover container security on Google Cloud Platform (GCP), and how we help you secure your containers running in Google Kubernetes Engine. The posts in the series will cover the following topics:
- Container networking security
- New security features in Kubernetes Engine 1.10
- Image security
- The container software supply chain
- Container runtime security
At Google, we divide container security into three main areas:
- Infrastructure security, i.e., does the platform provide the necessary container security features? This is how you use Kubernetes security features to protect your identities, secrets, and network; and how Kubernetes Engine uses native GCP functionality, like IAM, audit logging and networking, to bring the best of Google security to your workloads.
- Software supply chain, i.e., is my container image secure to deploy? This is how you make sure your container images are vulnerability-free, and that the images you built aren't modified before they're deployed.
- Runtime security, i.e., is my container secure to run? This is how you identify a container acting maliciously in production, and take action to protect your workload.
Container infrastructure security is about ensuring that your developers have the tools they need to securely build containerized services. This covers a wide variety of areas, including:
- Identity, authorization and authentication: How do my users assert their identities in my containers and prove they are who they say they are, and how do I manage these permissions?
- In Kubernetes, Role-Based Access Control (RBAC) allows the use of fine-grained permissions to control access to resources such as the kubelet. (RBAC is enabled by default since Kubernetes 1.8.)
- In Kubernetes Engine, you can use IAM permissions to control access to Kubernetes resources at the project level. You can still use RBAC to restrict access to Kubernetes resources within a specific cluster.
- Logging: How are changes to my containers logged, and can they be audited?
- In Kubernetes, Audit Logging automatically captures API audit logs. You can configure audit logging based on whether the event is metadata, a request or a request response.
- Kubernetes Engine integrates with Cloud Audit Logging, and you can view audit logs in Stackdriver Logging or in the GCP Activity console. The most commonly audited operations are logged by default, and you can view and filter these.
- Secrets: How does Kubernetes store secrets, and how do containerized applications access them?
- In Kubernetes, secrets that are stored in etcd can be encrypted at the application layer.
- In Kubernetes Engine, secrets are also stored in an etcd instance whose data is encrypted by default at the storage layer. Kubernetes Engine doesn’t currently encrypt secrets at the application layer.
- Networking: How should I segment containers in a network, and what traffic flows should I allow?
- In Kubernetes, you can use network policies to specify how to segment the pod network. When created, network policies define with which pods and endpoints a particular pod can communicate.
- In Kubernetes Engine, you can create a network policy, currently in beta, and manage these for your entire cluster. You can also create Private Clusters, in beta, to use only private IPs for your master and nodes.
Software supply chain
Managing the software supply chain, including container image layers that you didn't create, is about ensuring that you know exactly what’s being deployed in your environment, and that it belongs there. In particular, that means giving your developers access to images and packagers that are known to be free of vulnerabilities, to avoid introducing known vulnerabilities into your environment.
A container runs on a server's OS kernel but in a sandboxed environment. A container's image typically includes its own operating system tools and libraries. So when you think about software security, there are in fact many layers of images and packages to secure:
- The host OS, which is running the container
- The container image, and any other dependencies you need to run the container. Note that these are not necessarily images you built yourself—container images included from public repositories like Docker Hub also fall into this category
- The application code itself, which runs inside the container. This is outside of the scope of container security, but you should follow best practices and scan your code for known vulnerabilities. Be sure to review your code for security vulnerabilities and consider more advanced techniques such as fuzzing to find vulnerabilities. The OWASP Top Ten web application security risks is a good resource for knowing what to avoid.
Lastly, runtime security is about ensuring that your security response team can detect and respond to security threats to containers running in your environment. There are a few desirable capabilities here:
- Detection of abnormal behaviour from the baseline, leveraging syscalls, network calls and other available information
- Remediation of a potential threat, for example, via container isolation on a different network, pausing the container, or restarting it
- Forensics to identify the event, based on detailed logs and the containers’ image during the event
- Run-time policies and isolation, limiting what kinds of behaviour are allowed in your environment
A container isn’t a strong security boundary
There’s one myth worth clearing up: containers do not provide an impermeable security boundary, nor do they aim to. They provide some restrictions on access to shared resources on a host, but they don’t necessarily prevent a malicious attacker from circumventing these restrictions. Although both containers and VMs encapsulate an application, the container is a boundary for the application, but the VM is a boundary for the application and its resources, including resource allocation.
If you're running an untrusted workload on Kubernetes Engine and need a strong security boundary, you should fall back on the isolation provided by the Google Cloud Platform project. For workloads sharing the same level of trust, you may get by with multi-tenancy, where a container is run on the same node as other containers or another node in the same cluster.
Upcoming talks at KubeCon EU
In addition to this blog post series, we’ll be giving several talks on container security at KubeCon Europe in Copenhagen. If you’ll be at the show, make sure to add these to your calendar:
- Completely Securing the Software Supply Chain using Grafeas + in-toto, Wednesday May 2nd, 11h10-11h45
- Best Practices for Container Security at Scale, Thursday May 3rd, 14h00-14h35
- Who Shot the Cluster? Audit Logging in Kubernetes, Thursday May 3rd, 16h35-17h10
- Secure Pods, Friday May 4th, 11h10-11h45
- Multi-Tenancy in Kubernetes: Best Practices Today, and Future Directions, Friday May 4th, 11h10-11h45
- Kubernetes Runtime Security: What Happens If A Container Goes Bad?, Friday May 4th, 14h45-15h20
Stay tuned for next week’s installment about image security!