If you’ve ever heard someone from CoreOS speak on stage, you’ve probably heard them say something similar to, “Our plan is to dramatically improve the security of the internet.” That is the mission our founders set for the company when starting CoreOS.
Security is one of the primary drivers of what we do at CoreOS. Naturally, we’re elated whenever folks come together to celebrate security, like at the RSA Security Conference in San Francisco last week. Our own Brian “Redbeard” joined Google Cloud Platform for their Security event and had a lively conversation on a panel about containers and security.
In the spirit of encouraging best practices in security, we asked the CoreOS team to share their favorite advice about container security:
If you could tell the average developer one thing about container security, what would it be?
Our dev’s answers boil down to this idea:
Take control of your containers.
Whether you’re building your own containers to creating an auditable trail from repo to deployment, being aware of exactly what goes into your containers will insure your stack is as secure as possible. Read on to learn about four tips and perspectives from our team.
1. Make what’s running in production as transparent as possible
Jimmy Zelinskie, software engineer working on Quay at CoreOS, shares a recommendation for containers in production.
Being able to easily audit what software is running in production is not a new security practice. However, when your production has been decoupled into many containers all running different base layers, the problem of keeping software up to date is exacerbated. Building tools like Clair into your continuous integration and monitoring pipelines can enable operators to act swiftly and with confidence when reacting to disclosed vulnerabilities.
2. Use pods to protect inter-application communication
Casey Callendrello, software engineer at CoreOS, discusses the benefit of using pods to protect privileged communications between trusted applications.
A common usage pattern in containerized environments is running multiple closely-coupled applications together to provide a single service. This can take many forms, for example:
- A primary application is packaged with a small “sidecar” providing health checking and metrics
- A server is always deployed alongside a local Memcache instance
- A service is exposed through a reverse proxy (such as NGINX)
In other words, there is a grouping of containers that provide a single logical “service.” In Kubernetes, this is called a “pod”. Pods are more than just a convenience – they’re a fundamental part of securing a containerized environment. More specifically, a pod corresponds to a separate namespace in which multiple applications can be executed.
A major benefit of using pods is in securing communications between its applications. And it’s easy, too: just use localhost. Pods that have their own network namespace will have their own loopback interface, so the services won’t even be available to other pods on the same host. Pods also share the same
shm, abstract socket, and SysV IPC namespaces – so there are a myriad of ways to protect privileged communications between trusted applications.
3. Look deeper than the abstractions
Quentin Machu, software engineer working on Tectonic at CoreOS, shares insights on the security benefits of Container Linux and Clair.
By abstracting away the hardware and the operating systems, containers and container orchestration systems simplify the way applications are deployed and managed. Captivated by the high-level capabilities that these tools provide and by our hunger to push the limits further (e.g. scalability!), one might tend to start neglecting the underlying layers and their internals.
Building a higher level stack doesn’t necessarily imply maintaining or reducing its attack surface. It is crucial to stay fully aware of what’s powering our clusters, what’s running in them and how they are running specifically. Trouble often comes from the most unpredicted and forgotten places.
Technologies such as CoreOS Container Linux can minimize threats and help secure infrastructure at the operating system level by automating the application of patches and fixing any known vulnerabilities in a matter of hours. Other components, like Clair, the container image security analyzer, may also bring you security awareness to your stack.
4. Treat container image tags/versions as immutable or pull by digest
Euan Kemp, software engineer working on Container Linux at CoreOS, discusses using immutable images.
Using immutable references for all your images, and tracking that reference from commit to build to deploy, allows quick assessment of impact and aids in remediation.
When a vulnerability impacts your software, or a package installed in your image, being able to quickly determine which images are impacted and which aren’t is the first step. If your build system produces immutable artifacts with a clear audit trail of your code’s version and all its dependencies’ versions, then doing so is much easier.