k3s kubernetes fails to start on Odroid Bullseye

k3s kubernetes can’t start on Odroid HC2 due to missing kernel options as it looks like:

level=fatal msg=“PIDS cgroup support not found”

So looks like cgroup configuration is missing there.

k3s check-config command shows these as missing:

  • CONFIG_CGROUP_PIDS: missing
  • CONFIG_CGROUP_HUGETLB: missing


    And in the kernel config there’s # CONFIG_CGROUP_PIDS is not set

Looks like this needs a kernel recompile or is it possible to run it somehow?

hmm not sure if we can do something on this if this it’s a kernel issue or missing kernel featur as we don’t do any kernel development. What kind of system you are running?

  • DietPi version | cat /boot/dietpi/.version
  • Distro version | echo $G_DISTRO_NAME $G_RASPBIAN
  • Kernel version | uname -a
  • SBC model | echo $G_HW_MODEL_NAME or (EG: RPi3)

DietPi version | cat /boot/dietpi/.version
G_DIETPI_VERSION_CORE=8
G_DIETPI_VERSION_SUB=3
G_DIETPI_VERSION_RC=1
G_GITBRANCH=‘master’
G_GITOWNER=‘MichaIng’
Distro version | echo $G_DISTRO_NAME $G_RASPBIAN
bullseye
Kernel version | uname -a
Linux weakman 4.14.241+ #1 SMP PREEMPT Wed Jul 28 16:55:16 UTC 2021 armv7l GNU/Linux
SBC model | echo $G_HW_MODEL_NAME or (EG: RPi3)
Odroid XU3/XU4/MC1/HC1/HC2 (armv7l)

I see. It’s an older kernel version 4.14. Best to my knowledge, there is no other version/update available.

MichaIng
is there a way to create an Armbian image for the XU3/XU4/MC1/HC1/HC2 as well? Or does it not work as it is a 32bit architecture?

Actually I tried Armbian on this HC1 and k3s works ok. So it would be cool to have that clean DietPi touch :slight_smile:

If you are able to get Armbian working, how does it behave if you try to run PREP script afterwards https://dietpi.com/docs/hardware/#make-your-own-distribution

Does it still boot?

Yes, the prep script works on Armbian! Launched it and got proper looking dietpi installation on my HC1. And managed to run k3s with no problems. So that’s awesome! When I set up Armbian first, I also used their optimized DTB for my HC1 board.

What I had to do after the install was to go to dietpi-config > Advanced Options > Serial/UART, make all consoles to off and then make ttySAC2 console to on again. This solved my serial console not showing anything after kernel boot.

Also I’ve set iptables-nft to iptables-legacy, as k3s doesn’t work well with nftables

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

And there’s one fail message in boot log regarding haveged:

[FAILED] Failed to start Entropy Da…based on the HAVEGE algorithm. See 'systemctl status haveged.service' for details.

But that’s a non-issue.

For haveged you might could have a look to following GitHub issue https://github.com/MichaIng/DietPi/issues/4710#issuecomment-913840675

Great to hear that Armbian’s XU4 kernel works well. A new image our end is on the way. Could you check whether the hardware random generator works:

apt install rng-tools5
systemctl status rngd

haveged however should always work. What is the issue it reports?

journalctl -u haveged

rngd started successfully:

● rngd.service - Start entropy gathering daemon (rngd)
     Loaded: loaded (/lib/systemd/system/rngd.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-04-25 16:34:16 BST; 5s ago
       Docs: man:rngd(8)
   Main PID: 12160 (rngd)
      Tasks: 1 (limit: 4449)
     Memory: 188.0K
        CPU: 459ms
     CGroup: /system.slice/rngd.service
             └─12160 /usr/sbin/rngd -f

Apr 25 16:34:16 hc1 systemd[1]: Started Start entropy gathering daemon (rngd).

haveged just crashes:

Apr 25 09:11:06 hc1 systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:22 hc1 systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:22 hc1 systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Apr 25 16:36:22 hc1 systemd[1]: haveged.service: Failed with result 'signal'.
Apr 25 16:36:22 hc1 systemd[1]: haveged.service: Scheduled restart job, restart counter is at 1.
Apr 25 16:36:22 hc1 systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:22 hc1 systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:23 hc1 systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Apr 25 16:36:23 hc1 systemd[1]: haveged.service: Failed with result 'signal'.
Apr 25 16:36:23 hc1 systemd[1]: haveged.service: Scheduled restart job, restart counter is at 2.
Apr 25 16:36:23 hc1 systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:23 hc1 systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:23 hc1 systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Apr 25 16:36:23 hc1 systemd[1]: haveged.service: Failed with result 'signal'.
Apr 25 16:36:23 hc1 systemd[1]: haveged.service: Scheduled restart job, restart counter is at 3.
Apr 25 16:36:23 hc1 systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:23 hc1 systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Failed with result 'signal'.
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Scheduled restart job, restart counter is at 4.
Apr 25 16:36:24 hc1 systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:24 hc1 systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Main process exited, code=killed, status=31/SYS
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Failed with result 'signal'.
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Scheduled restart job, restart counter is at 5.
Apr 25 16:36:24 hc1 systemd[1]: Stopped Entropy Daemon based on the HAVEGE algorithm.
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Start request repeated too quickly.
Apr 25 16:36:24 hc1 systemd[1]: haveged.service: Failed with result 'signal'.
Apr 25 16:36:24 hc1 systemd[1]: Failed to start Entropy Daemon based on the HAVEGE algorithm.

Ahh, it is this bug: https://bugs.debian.org/985196
Nasty that it wasn’t solved until now. We applied a workaround, but missed to apply it for new images as well. I just did this and brought the workaround patch forward for upcoming release: https://github.com/MichaIng/DietPi/commit/e316744

Now I remember that rngd didn’t work on the old Odroid XU4 iamge: https://github.com/MichaIng/DietPi/issues/4318
However, based on your status it seems to work fine with the new image/kernel. This is actually much better, less overhead. So instead of fixing haveged, I’d purge it and observe whether there are entropy related issues (I do not expect as of clean rngd logs). Issues are usually handing boot, hanging service starts or commands, especially if any kind of cryptography is involved, which often makes use of random characters from /dev/random. So:

G_AGP haveged

I just uploaded a new image for testing with the Armbian kernel you use. If you could give it a try, that would be awesome: https://dietpi.com/downloads/images/testing/
But it has still haveged installed, without the workaround (done the image before the above commit), so when it boots, the first thing I suggest is switching to rngd:

G_AGI rng-tools5
G_AGP haveged

If it keeps running smooth like that, I’ll adjust our build script accordingly and re-create the image.

I’ve downloaded and flashed the testing image DietPi_OdroidXU4-ARMv7-Bullseye.7z and it fails to boot. Here’s the log I got on serial console, looks like boot files are missing. Another thing is that the image doesn’t have the traditional FAT partition where all the dietpi config files are stored. Plain Armbian doesn’t have such partition too by the way.

U-Boot 2017.05-armbian (Feb 27 2022 - 08:50:07 +0000) for ODROID-XU4

CPU:   Exynos5422 @ 800 MHz
Model: Odroid XU4 based on EXYNOS5422
Board: Odroid XU4 based on EXYNOS5422
Type:  xu4
DRAM:  2 GiB
MMC:   EXYNOS DWMMC: 0, EXYNOS DWMMC: 1
MMC Device 0 ( SD ): 28.9 GiB
Card did not respond to voltage select!
mmc_init: -95, time 12
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Press quickly 'Enter' twice to stop autoboot:  0
** File not found /boot.ini **
## Executing script at 43e00000
Wrong image format for "source" command
** File not found /boot/boot.ini **
## Executing script at 43e00000
Wrong image format for "source" command
** File not found /boot.scr **
## Executing script at 43e00000
Wrong image format for "source" command
** Invalid partition 2 **
## Executing script at 43e00000
Wrong image format for "source" command
** Invalid partition 2 **
## Executing script at 43e00000
Wrong image format for "source" command
** Invalid partition 2 **
## Executing script at 43e00000
Wrong image format for "source" command
mmc_init: -110, time 122
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
starting USB...
USB0:   USB EHCI 1.00
USB1:   Register 2000140 NbrPorts 2
Starting the controller
USB XHCI 1.00
USB2:   Register 2000140 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 1 for devices... 1 USB Device(s) found
scanning bus 2 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
       scanning usb for ethernet devices... 1 Ethernet Device(s) found

USB device 0: unknown device
Waiting for Ethernet connection... done.
BOOTP broadcast 1

Found the issue, image has been rebuilt :slight_smile:.

yes the Armnian image (we use same one) is build on a single ext4 partition nowadays. There is no FAT partition anymore.

Since the bootloader is looking for boot.ini in root directory as well, probably it would be even possible to generate the image with dedicated boot partition. But it has several downsides, too.

Flashed the latest testing image and as far as I see it’s working well! For tests I installed k3s and it works, haveged is not crashing, serial console is working, installed Docker and it works well too. So looks like a fine release.

One thing I noticed on serial console, it doesn’t print the classic DietPi first boot greeting screen with IP address, default logins and all the good info. So on this testing image when the image boots, it just displays plain boot prompt:

[   13.265697] systemd-journald[304]: Received client request to flush runtime journal.
[   13.384509] proc: Bad value for 'hidepid'
[   22.212152] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   22.218389] r8152 6-1:1.0 eth0: carrier on

Debian GNU/Linux 11 DietPi ttySAC2

DietPi login: [   36.958116] vdd_ldo12: disabling

For headless boards it’s really good to see that IP address to connect and finish the install.

I mean it’s missing this screen :slight_smile:

 
 ─────────────────────────────────────────────────────
 DietPi v7.9.3 : 03:00 - Fri 12/17/21
 ─────────────────────────────────────────────────────
 - LAN IP : 192.168.0.124 (eth0)

 Default Login:
 Username = root
 Password = dietpi (or custom dietpi.txt entry)

 Please hit <return> to login


Debian GNU/Linux 11 DietPi ttySAC2

DietPi login:

Also found another thing missing in drive manager - there’s no option to transfer the whole OS to the attached SSD. Only user data can be moved, but not the whole linux installation. Anyway I managed to copy the linux files to my SSD and then changed UUID on boot.ini and got DietPi running from SSD. Manual work, but all works well :slight_smile:

Also found another thing missing in drive manager - there’s no option to transfer the whole OS to the attached SSD

That’s a little bit as expected as such an option did not exist. I guess you had in mind the option to transfer the RootFS to another dive. Indeed this option is missing on your image because you have a single partition only now. In the past your image contains a FAT partition (bootFS) and the ext4 partition (RootFS).

Yes, the transfer RootFS, but I did that manually, so no problems there.

Should work to flash the image to USB, eMMC, SATA etc directly, so no rootfs transfer required. But I’m not 100% sure whether it supports booting from all those drive opens, an interesting info as well for our download page.

Btw, just to be sure, could you try to replace haveged with rng-tools5 as suggested above:

G_AGI rng-tools5
G_AGP haveged

Just as mind/long term observation whether it really works without issues (I do not expect any). I’ll recreate the image now with this pre-installed.