Supervise containers with Systemd

Community Tutorials
1

I supervise my SCB “services” with systemd. When I wanted to containerize the underlying processes, I had a fair bit of trouble tracking down a simple approach. This method has no dependencies–thought I’d share it.

File: /etc/systemd/system/docker.foobar.service

[Unit]
Description=Docker FooBar Service
After=docker.service
Requires=docker.service

[Service]
Environment="AN_ENVIRONMENT_VARIABLE=42"

# Long startup timeout to support an image-pull.
TimeoutStartSec=600
Restart=always

ExecStartPre=-/usr/bin/docker rm -f %n
ExecStartPre=/usr/bin/docker pull foobar:latest
ExecStart=/usr/bin/docker run \
  --name %n \
  -e AN_ENVIRONMENT_VARIABLE \
  -e HOST_MACHINE_HOSTNAME=%H \
  foobar:latest

ExecStop=/usr/bin/docker kill --signal=SIGINT %n

[Install]
WantedBy=default.target

Usage:

# Enable it (so it starts on system startup
$ sudo systemctl enable docker.foobar
# Restart it (as you might do when you want to pull the newest `:latest`
$ sudo systemctl restart docker.foobar
# View its logs
$ sudo journalctl -u docker.foobar

Footnotes

  • The - before the ExecStartPre indicates to systemd that the startup should continue even if this exits non-zero
  • %n is the service name, in this case docker.foobar.service
  • AN_ENVIRONMENT_VARIABLE is defined in the supervised process (host machine), the -e AN_ENVIRONMENT_VARIABLE in the Docker invocation shares that environment variable with the container.
  • HOST_MACHINE_HOSTNAME is explicitly passed into the Docker container


    PS This file’s footprint will be smaller with a daemonless container runtime. Podman, one of the frontrunners in that effort, is first-class supported in Debian Bullseye. When that’s considered stable-for-newbs, I’ll come back and update this.
2

Many thanks for sharing. As restart command I think you wanted to use:

systemctl restart docker.foobar

:slight_smile:

3

Good catch-- thank you!

4

You could just install portainer which is a webgui that allows you to control your docker instances
It works quite well

5

I’ve been experimenting with it. It’s quite nice. But, from what I gather, it’s not a precise analog/replacement for systemd

6

Bugfix:

Changed

ExecStop=/usr/bin/docker stop %n

to

ExecStop=/usr/bin/docker kill --signal=SIGINT %n

Only confirmed with python scripts, but the docker stop approach takes seconds to time out and ultimately sends a SIGKILL ∴ no graceful shutdowns. SIGTERM is a keyboard interrupt and lets Python invoke its cleanup / atexit’s.

7

If docker stop does not work as expected, I guess that command, when executed from console, does not block until the container has been fully shutdown, but releases the console while the container is shutdown, right? This behaviour is difficult for systemd. When an ExecStop command is defined, it is allowed by default to take up to 90 seconds, before a SIGKILL is sent. But when that command has “finished”, means it allows the command query to continue, systemd only allows 5 seconds or so before sending the SIGKILL. I try to find an option to change this timeout:

But if docker kill blocks the console/command query until the container is down, it should have the same result indeed. SIGTERM should be also okay, SIGINT might look a bit strange (to others, looking at the logs or so) when there was no interactive interrupt at a console, as this is really the typical CTRL+C signal :slight_smile:.