[Solved] Docker does not start on clean image on RaspberryPi3

Troubleshooting
21

Garret
Thanks for sharing issue + solution. This renders my planned tests on VM obsolete :stuck_out_tongue:.

it has already been mentioned several times that what you wrote was tried.

iaresee just joined the conversation and I better ask one time too often then doing further research while solution was just overseen :wink:.

Hmm okay, if the repo currently ships a broken version on RPi, question is how we handle in upcoming DietPi release:

  • The GitHub issue is 2 weeks old now. Hope that it will be resolved soon and leave our installer is as? Of course inform users about the external bug.
  • Install the old version for now on RPi to have a working installer and revert to most current version on next release, after issue was fixed? (I vote for this!) Needs apt-mark hold then as well.
  • Block Docker install on RPi for now? Most properly not desired option, since above is possible :wink:.
22

Never hurts to ask! It looks like this is an upstream break though so possibly pinning to an older version or blocking it for now makes the most sense.

23

I got this working. I had to pin docker-ce to 18.06.1*. Installed using:

apt-get install docker-ce=18.06.1~ce~3-0~raspbian

Pinned using:

root@pi-hole:~# cat /etc/apt/preferences.d/docker-ce
Package: docker-ce
Pin: version 18.06.1*
Pin-Priority: 1000
24

iaresee
Still no fix in repo? Hope this will be the case until DietPi v6.20, so we don’t need to patch in a downgrade.

But your solution via pin preferences at least is the cleanest, easiest to patch and easiest to revert, when repo version has the fix.

Strange is: https://github.com/moby/moby/issues/38175#issuecomment-438048459

  • According to discussion it seems to a ARMv6 issue only (RPi0+1). But you use an RPi3, right? Would be good to know for which devices/architectures exactly we need to pin the version.
25

Correct.

26

Okay, failsafe pin version for all RPis.

Just recognized I did not yet link our GitHub issue here :wink:: https://github.com/Fourdee/DietPi/issues/2282

27

Ah have a better idea:

  • Instead of pinning to v18.06.1, it is better to pin priority of v18.09.0 to negative value?
  • This allows APT to update, as fast as a newer version than v18.09.0 is available:
cat << _EOF_ > /etc/apt/preferences.d/docker-ce
Package: docker-ce
Pin: version 18.09.0*
Pin-Priority: -1
_EOF_

iaresee
Could you test, if this works?

28

MichaIng If what you expect is true than your solution I agree would be better.

Btw I actually have a rpi1 but on Wednesday I am getting a new rpi3b+ :sunglasses:

29

If you guys can verify this works (v18.09.0 is not installed), then I will add it to a hotfix update tonight, that is required for another more critical issue anyway.

Sadly I have no RPi free for testing and Fourdee is away for some days.

30

I could some testing on the rpi1 when I get my rpi3 from Wednesday-Thursday because right now I have all my setup there and cannot play too much. Sorry.

31

The test can be done safely without risking any system change:

mv /etc/apt/preferences.d/docker-ce /mnt/dietpi_userdata/docker-ce.bak
cat << _EOF_ > /etc/apt/preferences.d/docker-ce
Package: docker-ce
Pin: version 18.09.0*
Pin-Priority: -1
_EOF_
apt upgrade
# Now check, whether docker-ce update to v18.09.0 is offered or not.
# If it is offered, simply press "n" (or anything NOT "y") to abort and revert via:
mv /mnt/dietpi_userdata/docker-ce.bak /etc/apt/preferences.d/docker-ce
# Else you can remove the backup
rm /mnt/dietpi_userdata/docker-ce.bak

But I fully understand, if you don’t want to play with your running system :slight_smile:.

32

It installed 18.09.0:

root@pi-hole:~# docker --version
Docker version 18.09.0, build 4d60db4

That version core dumps on me shortly after startup:

root@pi-hole:~# systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/docker.service.d
           └─dietpi-simple.conf
   Active: failed (Result: core-dump) since Sun 2018-12-09 22:40:35 GMT; 1min 3s ago
     Docs: https://docs.docker.com
  Process: 14925 ExecStart=/usr/bin/dockerd -H unix:// (code=dumped, signal=SEGV)
 Main PID: 14925 (code=dumped, signal=SEGV)
      CPU: 638ms

Dec 09 22:40:32 pi-hole systemd[1]: Started Docker Application Container Engine.
Dec 09 22:40:35 pi-hole systemd[1]: Stopping Docker Application Container Engine...
Dec 09 22:40:35 pi-hole systemd[1]: docker.service: Main process exited, code=dumped, status=11/SEGV
Dec 09 22:40:35 pi-hole systemd[1]: Stopped Docker Application Container Engine.
Dec 09 22:40:35 pi-hole systemd[1]: docker.service: Unit entered failed state.
Dec 09 22:40:35 pi-hole systemd[1]: docker.service: Failed with result 'core-dump'.

My /lib/systemd/system/docker.service remained unchange and patched correctly after the upgrade.

33

I assume the systemd unit does not change with the update.

But does it mean, apt update still offers docker-ce v18.09.0 even with

Package: docker-ce
Pin: version 18.09.0*
Pin-Priority: -1

inside APT preference file?

Ah saw your post update. Okay have to check further how to prevent update for a single package version.
Hmm generally that method should work:

root@VM-Buster:~# apt list --upgradable
Listing... Done
libgcrypt20/testing 1.8.4-4 amd64 [upgradable from: 1.8.4-3]
libgmp10/testing 2:6.1.2+dfsg-4 amd64 [upgradable from: 2:6.1.2+dfsg-3]
linux-image-amd64/testing 4.18+100 amd64 [upgradable from: 4.18+99]
root@VM-Buster:~# nano /etc/apt/preferences.d/test
root@VM-Buster:~# cat /etc/apt/preferences.d/test
Package: libgcrypt20
Pin: version 1.8.4*
Pin-Priority: -1
root@VM-Buster:~# apt list --upgradable
Listing... Done
libgmp10/testing 2:6.1.2+dfsg-4 amd64 [upgradable from: 2:6.1.2+dfsg-3]
linux-image-amd64/testing 4.18+100 amd64 [upgradable from: 4.18+99]
34

iaresee
Okay I found the issue: There is a leading 5: in current docker-ce version string :roll_eyes:.

The following should work then. Might I ask you to retest on RPi?

echo 'Package: docker-ce
Pin: version 5:18.09.0~3-0~raspbian-stretch
Pin-Priority: -1' > /etc/apt/preferences.d/docker-ce
G_AGI docker-ce=18.06.1~ce~3-0~raspbian
G_AGA
  • Updates pin version string
  • Downgrades docker again
  • Uninstalls obsolete packages

docker-ce-cli is not available for this lower version. Is this an issue or is it included in the main package that lower version? The same for containerd.io, will will be removed as obsolete after downgrade, at least here on Debian Stretch. But should be the same on Raspbian.

35

iaresee Garret
We just released v6.19.7 which hotfixes the docker issue. On this update, just pre-patches run, like it was for v6.19.6, so no reboot required, no full APT update and DietPi core code download/merge done.

The preferences.d file is named dietpi-docker_fix, so you might want to remove your preferences.d/docker-ce. The one provided with the update allows APT do update, as fast as a new version is available in the repo. I hope/expect a new version will fix the issue on RPi.

36

MichaIng ack. I’ll be able to test it out tonight, west coast USA time. Thanks for the tenacity working through this one!

37

MichaIng I seem be running a-okay with this update. No need to reboot after patching and docker-ce is happy with the apt preferences file that’s be deposited. I moved aside the existing file I had.

Thank you!

38

Yes, thanks a lot for this issue which was actually not really due to dietpi.

39

Garret iaresee
Thanks very much for your testing and feedback guys :sunglasses:.