Supervise containers with Systemd

Guides and tutorials for various stuff. Posted by DietPi users.
Post Reply
User avatar
vnzkay
Posts: 10
Joined: Thu May 20, 2021 6:42 pm
Location: Los Angeles, CA
Contact:

Supervise containers with Systemd

Post by vnzkay »

I supervise my SCB "services" with systemd. When I wanted to containerize the underlying processes, I had a fair bit of trouble tracking down a simple approach. This method has no dependencies--thought I'd share it.

File: /etc/systemd/system/docker.foobar.service

Code: Select all

[Unit]
Description=Docker FooBar Service
After=docker.service
Requires=docker.service

[Service]
Environment="AN_ENVIRONMENT_VARIABLE=42"

# Long startup timeout to support an image-pull.
TimeoutStartSec=600
Restart=always

ExecStartPre=-/usr/bin/docker rm -f %n
ExecStartPre=/usr/bin/docker pull foobar:latest
ExecStart=/usr/bin/docker run \
  --name %n \
  -e AN_ENVIRONMENT_VARIABLE \
  -e HOST_MACHINE_HOSTNAME=%H \
  foobar:latest

ExecStop=/usr/bin/docker kill --signal=SIGINT %n

[Install]
WantedBy=default.target
Usage:

Code: Select all

# Enable it (so it starts on system startup
$ sudo systemctl enable docker.foobar
# Restart it (as you might do when you want to pull the newest `:latest`
$ sudo systemctl restart docker.foobar
# View its logs
$ sudo journalctl -u docker.foobar
Footnotes
* The `-` before the ExecStartPre indicates to systemd that the startup should continue even if this exits non-zero
* %n is the service name, in this case docker.foobar.service
* AN_ENVIRONMENT_VARIABLE is defined in the supervised process (host machine), the -e AN_ENVIRONMENT_VARIABLE in the Docker invocation shares that environment variable with the container.
* HOST_MACHINE_HOSTNAME is explicitly passed into the Docker container


PS This file's footprint will be smaller with a daemonless container runtime. Podman, one of the frontrunners in that effort, is first-class supported in Debian Bullseye. When that's considered stable-for-newbs, I'll come back and update this.
Last edited by vnzkay on Fri Jun 11, 2021 8:16 am, edited 2 times in total.
User avatar
MichaIng
Site Admin
Posts: 3105
Joined: Sat Nov 18, 2017 6:21 pm

Re: Supervise containers with Systemd

Post by MichaIng »

Many thanks for sharing. As restart command I think you wanted to use:

Code: Select all

systemctl restart docker.foobar
:)
User avatar
vnzkay
Posts: 10
Joined: Thu May 20, 2021 6:42 pm
Location: Los Angeles, CA
Contact:

Re: Supervise containers with Systemd

Post by vnzkay »

Good catch-- thank you!
User avatar
WarHawk
Posts: 739
Joined: Thu Jul 20, 2017 8:55 am

Re: Supervise containers with Systemd

Post by WarHawk »

You could just install portainer which is a webgui that allows you to control your docker instances
It works quite well
User avatar
vnzkay
Posts: 10
Joined: Thu May 20, 2021 6:42 pm
Location: Los Angeles, CA
Contact:

Re: Supervise containers with Systemd

Post by vnzkay »

I've been experimenting with it. It's quite nice. But, from what I gather, it's not a precise analog/replacement for systemd
User avatar
vnzkay
Posts: 10
Joined: Thu May 20, 2021 6:42 pm
Location: Los Angeles, CA
Contact:

Re: Supervise containers with Systemd

Post by vnzkay »

Bugfix:

Changed

Code: Select all

ExecStop=/usr/bin/docker stop %n
to

Code: Select all

ExecStop=/usr/bin/docker kill --signal=SIGINT %n
Only confirmed with python scripts, but the `docker stop` approach takes seconds to time out and ultimately sends a SIGKILL ∴ no graceful shutdowns. SIGTERM is a keyboard interrupt and lets Python invoke its cleanup / atexit's.
User avatar
MichaIng
Site Admin
Posts: 3105
Joined: Sat Nov 18, 2017 6:21 pm

Re: Supervise containers with Systemd

Post by MichaIng »

If docker stop does not work as expected, I guess that command, when executed from console, does not block until the container has been fully shutdown, but releases the console while the container is shutdown, right? This behaviour is difficult for systemd. When an ExecStop command is defined, it is allowed by default to take up to 90 seconds, before a SIGKILL is sent. But when that command has "finished", means it allows the command query to continue, systemd only allows 5 seconds or so before sending the SIGKILL. I try to find an option to change this timeout: But if docker kill blocks the console/command query until the container is down, it should have the same result indeed. SIGTERM should be also okay, SIGINT might look a bit strange (to others, looking at the logs or so) when there was no interactive interrupt at a console, as this is really the typical CTRL+C signal :).
Post Reply