We talked with Lennart Poettering, creator of the systemd project, to hear the story of the origins of systemd, how it works in the world of containers and what can be expected in the future.
CoreOS will be at the systemd.conf in Germany this November. CTO Brandon Philips and software developers Alex Crawford and Jonathan Boulle will all speak on various aspects of systemd and containers. Meet us and Lennart there!
Q1: Tell us about your background and inspiration that led to the creation of the systemd project.
A: That's a long story!
I am a software engineer at Red Hat. Before working on systemd my focus was Linux audio, where I created the PulseAudio project. I also created the Avahi service discovery framework, even before that.
Five years ago, up to the point when Kay Sievers and I started working on the systemd project, we were actually big believers in the Upstart project. Upstart was a system manager ("init") written by an engineer at Canonical that modernized how Linux systems booted up. It was strictly event-based and had a very clean codebase. It was adopted by a number of distributions including Ubuntu and Fedora, back then.
When we looked closer at Upstart we eventually realized that its fundamental design was backwards – at least in our opinion. We thought a system manager should calculate the minimal amount of work to do during boot-up, while Upstart was actually (in a way) designed to do the maximum amount of work, and left the developers and administrators in charge to calculate what precisely should be done when. We understood that fixing this was not really an option within the Upstart project, since it would mean turning Upstart completely upside down, and throwing away its overall design, and replacing it with a completely new design. Hence we figured: this is one of the occasions where one needs to start from scratch – and so we did: with the systemd project, implementing our ideas how Linux system boot-up should work.
Since then, we have been developing systemd steadily, taking a lot of inspiration from the system managers of other operating systems, in particular Solaris' SMF framework, as well as MacOS X's launchd system. Originally systemd was supposed to be just an "init" system, i.e. just one process that is responsible for the most fundamental logic of starting up your Linux userspace. However, we eventually realized that we actually wanted to solve more than just this one problem, and began to rework and unify a larger part of how Linux userspace is brought up and maintained during runtime.
Today, we consider systemd a set of basic building blocks to build an OS from, that contains far more than just an init system, but also covers device management, login management, network management, logging services and a lot more.
A: We believe a modern system and service manager should natively know the concept of a container. Containers should be a central facet of server management and the concept of it should transcend the layers of the OS stack, all the way from the application layer down to the kernel. As the glue between that all, it's a responsibility for systemd to integrate containers into the OS.
Specifically, in systemd many commands are directly aware of the container concept. They not only can show you information about what's running and going on on the host, and change state of it, but also of all local containers. For example, the logging tool
journalctl may be used to show you the logs of the host, or of any local container, and can even interleave the logs of all containers and the host into one stream. Other commands that have direct support for containers are
systemd-run, and many more.
For us, systemd should not only be useful to run containers on, but also work fine when run inside containers. In fact, we regularly test systemd in containers, even more often than on bare metal, simply because it is so much easier and quicker to work with containers than physical machines. Containers matter to us upstream, they are our primary testing platform.
systemd also contains the
systemd-nspawn container manager. It's a relatively minimal, yet powerful implementation of a container manager. Initially we wrote it for testing purposes, but nowadays we consider it ready for many production uses. In fact CoreOS' rkt container tool makes use of it as the lower level container backend.
Watch Lennart’s talk from CoreOS Fest
For a deeper dive, watch the full recording of Lennart's talk:
Q3: What are the most important things emerging in the Linux ecosystem?
A: The major trend I am seeing is that the OS becomes commoditized, and is turned into something that is managed increasingly automatically, often as a commercial service, the way CoreOS is doing it. The requirement for automating OS management starts to be reflected in many layers of the stack: OS updates become automatic, the system thus needs to handle failure more gracefully, in order to support automatic rollback. The OS is split into containers, where each container becomes very similar in behaviour and update cycle to the OS itself. Traditional software packages move out of the deployment focus, and become strictly a development tool: containers and OSs are now the smallest unit of deployment.
If you put all this together it is becoming easier to run, deploy and update larger setups, as much of the necessary work is now done automatically, more robustly, and without requiring direct interference of the administrator.
And yes, with Linux we are at the forefront of this development. And with systemd we hope to get some basic building blocks for all this into place.
Q4: How do you see systemd integrating with container tools like Docker and rkt?
A: Integration with rkt is already pretty close, as rkt uses systemd-nspawn as its container backend. And we love it that way. We want to deliver the basic building blocks for container management, with a strict focus on the individual machine. Tools like rkt then build on this, and build a more distributed, network-aware, user-friendly "house" from our "building blocks."
Q5: What should attendees expect at systemd.conf coming up this November in Berlin?
A: As systemd is now at the core of so many Linux operating systems, systemd.conf will of course be one of the major forums where we will discuss the goals, progress and future of the basic Linux userspace.
The conference is intended for system developers as well as professional devops folks. We'll have talks on a lot of different topics, such as containers, networking, logging, IPC, systemd in distributions, use of systemd in the cloud, on embedded and on servers, and a lot more.
systemd.conf 2015 will consist of at least 1.5 days of presentations followed by an extended hackfest. And of course, there will be parties in one of today's most exciting cities, where you can meet many other people involved with and interested in the systemd project!
I hope to see you in Berlin!
Thanks to Lennart for chatting with us!