Docker keeps restarting Portainer

I installed DietPi on to my RP3+. I used the dietpi-software to install Docker, Docker Compose, and Portainer.

I noticed portainer would never run more than 30 seconds using

docker ps

After checking the log I found the following.

Jul 14 21:58:20 testPi containerd[2420]: time="2021-07-14T21:58:20.791001950-04:00" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
Jul 14 21:58:42 testPi containerd[2420]: time="2021-07-14T21:58:42.558455275-04:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.run
time.v2.task/moby/b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cbc6ca8b287d8 pid=24338
Jul 14 21:58:44 testPi systemd[1]: Stopping Docker Application Container Engine...
Jul 14 21:58:44 testPi containerd[2420]: time="2021-07-14T21:58:44.676815483-04:00" level=error msg="add cg to OOM monitor" error="cgroups: memory cgroup not supported on this
 system"
Jul 14 21:58:47 testPi containerd[2420]: time="2021-07-14T21:58:47.595496992-04:00" level=info msg="shim disconnected" id=b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cb
c6ca8b287d8
Jul 14 21:58:47 testPi containerd[2420]: time="2021-07-14T21:58:47.598424179-04:00" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
Jul 14 21:58:58 testPi systemd[1]: docker.service: Succeeded.
Jul 14 21:58:58 testPi systemd[1]: Stopped Docker Application Container Engine.
Jul 14 22:04:57 testPi systemd[1]: Started Docker Application Container Engine.
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.692575338-04:00" level=warning msg="Your kernel does not support cgroup memory limit"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.692958672-04:00" level=warning msg="Your kernel does not support CPU realtime scheduler"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.693034088-04:00" level=warning msg="Your kernel does not support cgroup blkio weight"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.693103411-04:00" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.693173984-04:00" level=warning msg="Your kernel does not support cgroup blkio throttle.read_bps_device"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.693245911-04:00" level=warning msg="Your kernel does not support cgroup blkio throttle.write_bps_device"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.693318776-04:00" level=warning msg="Your kernel does not support cgroup blkio throttle.read_iops_device"
Jul 14 22:05:02 testPi dockerd[25534]: time="2021-07-14T22:05:02.693393828-04:00" level=warning msg="Your kernel does not support cgroup blkio throttle.write_iops_device"
Jul 14 22:05:25 testPi containerd[2420]: time="2021-07-14T22:05:25.831594913-04:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.run
time.v2.task/moby/b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cbc6ca8b287d8 pid=25704
Jul 14 22:05:27 testPi containerd[2420]: time="2021-07-14T22:05:27.474751162-04:00" level=error msg="add cg to OOM monitor" error="cgroups: memory cgroup not supported on this
 system"
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: 2021/07/15 02:05:45 Migrating database from version 0 to 30.
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0xbc1bec]
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: goroutine 1 [running]:
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/boltdb/bolt.(*Bucket).Cursor(...)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /tmp/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/bucket.go:91
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).retrieveLegacyResourceControls.func1(0x33c7580
, 0x0, 0x33c7580)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:66 +0x6c
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/boltdb/bolt.(*DB).View(0x33a8360, 0x3213cc8, 0x0, 0x0)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /tmp/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:629 +0x90
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).retrieveLegacyResourceControls(0x3213da0, 0x11
3c04, 0x0, 0x0, 0x3213fc0, 0x33c7500)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:64 +0x74
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).updateResourceControlsToDBVersion2(0x3213da0,
0x0, 0x0)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:10 +0x1c
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).Migrate(0x3213da0, 0x2a, 0x3213d90)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrator.go:105 +0x780
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt.(*Store).MigrateData(0x33e8460, 0x0, 0x0, 0x339a7b0)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/datastore.go:177 +0x260
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: main.initDataStore(0x1293c23, 0x5, 0x1526258, 0x339a7b0, 0x30e2658, 0x30e2608)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:72 +0x170
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: main.buildServer(0x33cc420, 0x14e01, 0x1)
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:351 +0x90
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: main.main()
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:501 +0x6c
Jul 14 22:05:45 testPi containerd[2420]: time="2021-07-14T22:05:45.134089541-04:00" level=info msg="shim disconnected" id=b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cb
c6ca8b287d8
Jul 14 22:05:45 testPi containerd[2420]: time="2021-07-14T22:05:45.134323291-04:00" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
Jul 14 22:06:06 testPi containerd[2420]: time="2021-07-14T22:06:06.138539741-04:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.run
time.v2.task/moby/b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cbc6ca8b287d8 pid=25854
Jul 14 22:06:08 testPi containerd[2420]: time="2021-07-14T22:06:08.017263907-04:00" level=error msg="add cg to OOM monitor" error="cgroups: memory cgroup not supported on this
 system"
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: 2021/07/15 02:06:18 Migrating database from version 0 to 30.
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0xbc1bec]
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: goroutine 1 [running]:
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/boltdb/bolt.(*Bucket).Cursor(...)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /tmp/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/bucket.go:91
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).retrieveLegacyResourceControls.func1(0x3c1a600
, 0x0, 0x3c1a600)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:66 +0x6c
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/boltdb/bolt.(*DB).View(0x3bca360, 0x3a41cc8, 0x0, 0x0)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /tmp/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:629 +0x90
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).retrieveLegacyResourceControls(0x3a41da0, 0x11
3c04, 0x0, 0x0, 0x3a41fc0, 0x3c1a580)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:64 +0x74
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).updateResourceControlsToDBVersion2(0x3a41da0,
0x0, 0x0)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:10 +0x1c
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).Migrate(0x3a41da0, 0x2a, 0x3a41d90)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrator.go:105 +0x780
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt.(*Store).MigrateData(0x3bf8540, 0x0, 0x0, 0x380c250)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/datastore.go:177 +0x260
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: main.initDataStore(0x1293c23, 0x5, 0x1526258, 0x380c250, 0x380e568, 0x380e518)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:72 +0x170
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: main.buildServer(0x3bdc420, 0x14e01, 0x1)
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:351 +0x90
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]: main.main()
Jul 14 22:06:18 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:501 +0x6c
Jul 14 22:06:19 testPi containerd[2420]: time="2021-07-14T22:06:19.007001195-04:00" level=info msg="shim disconnected" id=b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cb
c6ca8b287d8
Jul 14 22:06:19 testPi containerd[2420]: time="2021-07-14T22:06:19.007295778-04:00" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
Jul 14 22:06:41 testPi containerd[2420]: time="2021-07-14T22:06:41.031556655-04:00" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.run
time.v2.task/moby/b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cbc6ca8b287d8 pid=25982
Jul 14 22:06:42 testPi containerd[2420]: time="2021-07-14T22:06:42.714468946-04:00" level=error msg="add cg to OOM monitor" error="cgroups: memory cgroup not supported on this
 system"
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: 2021/07/15 02:06:55 Migrating database from version 0 to 30.
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0xbc1bec]
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: goroutine 1 [running]:
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/boltdb/bolt.(*Bucket).Cursor(...)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /tmp/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/bucket.go:91
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).retrieveLegacyResourceControls.func1(0x37e0380
, 0x0, 0x37e0380)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:66 +0x6c
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/boltdb/bolt.(*DB).View(0x35af680, 0x3615cc8, 0x0, 0x0)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /tmp/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:629 +0x90
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).retrieveLegacyResourceControls(0x3615da0, 0x11
3c04, 0x0, 0x0, 0x3615fc0, 0x37e0300)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:64 +0x74
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).updateResourceControlsToDBVersion2(0x3615da0,
0x0, 0x0)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrate_dbversion1.go:10 +0x1c
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt/migrator.(*Migrator).Migrate(0x3615da0, 0x2a, 0x3615d90)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/migrator/migrator.go:105 +0x780
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: github.com/portainer/portainer/api/bolt.(*Store).MigrateData(0x36cb180, 0x0, 0x0, 0x3754f20)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/bolt/datastore.go:177 +0x260
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: main.initDataStore(0x1293c23, 0x5, 0x1526258, 0x3754f20, 0x37731d0, 0x3773180)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:72 +0x170
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: main.buildServer(0x3751c80, 0x14e01, 0x1)
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:351 +0x90
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]: main.main()
Jul 14 22:06:55 testPi b69f4e96ec2c[25534]:         /home/vsts/work/1/s/api/cmd/portainer/main.go:501 +0x6c
Jul 14 22:06:55 testPi containerd[2420]: time="2021-07-14T22:06:55.251152223-04:00" level=info msg="shim disconnected" id=b69f4e96ec2c084a9e7bff0cbc1c824c53588c16ba4dfac6582cb
c6ca8b287d8
Jul 14 22:06:55 testPi containerd[2420]: time="2021-07-14T22:06:55.251367327-04:00" level=error msg="copy shim log" error="read /proc/self/fd/10: file already closed"
dietpi@testPi:~$

Any thoughts?

Hi,

this looks like some memory issues.

Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: 2021/07/15 02:05:45 Migrating database from version 0 to 30.
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 14 22:05:45 testPi b69f4e96ec2c[25534]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0xbc1bec]

Can you check using htop how memory utilisation looks like. As well pls check for kernel error messages

dmesg -l err,crit,alert,emerg

If not already done, try to reboot the system

Thanks for your help Joulinar. I actually reinstalled portainer things started working as expected.

ok, so we don’t know what caused the issue initially. :frowning: