How do Kernel upgrades work in DietPi

Hey there,
unfortunately I got the same seg fault error within the kernel due to BTRFS raid volumes. I have the newest bullseye C4 image. Did I get you right that just using a newer btrfs-progs version wont help because the kernel itself also has the faulty version compiled?

In my situation it is very strange:

  1. First I setup the BTRFS volumes, setup nextcloud and everything. The system runs as expected.
  2. After couple of days e.g. I need to shutdown the odroid because of some maintenance stuff on the case. After the system boots again, every access to the raid volume freezes the whole system because of a segmentation fault.
  3. When I
umount /mnt/volume

i get the result code

Segmentation fault

The funny thing is, I can restore all data without any loss with

sudo btrfs -v restore /dev/sda1 /mnt/volume_restore/

. After the restore I can erase the partitions and create the raid again. But of course this can’t be the way to go for all time, right?

I wrote a runbook for the whole BTRFS maintenance because it is quite complex :frowning:

what is the actual kernel you are running

uname -a



Linux DietPi 4.9.241-arm64 #1 SMP PREEMPT Thu Feb 25 17:57:15 CET 2021 aarch64 GNU/Linux

MichaIng
can you have a look

Not sure, the old Odroid N2 image/kernel at least is known to have issues with USB in cases. You could check dmesg -l emerg,alert,crit,err for kernel errors but I fear we wouldn’t be able to do much about it.

If possible, I suggest to use our new mainline kernel based image. It is still in testing stage, but so far we got only positive feedback: https://github.com/MichaIng/DietPi/issues/5039

MichaIng
The connection is through SATA directly with the HC4. Perhaps this is the issue that the boot kernel of the HC4 image is the odroid N2 boot image or is this as intended?

After

xxxx@DietPi:~$ sudo umount /mnt/data
Segmentation fault

i get this kernel logs

[  241.608993] Unable to handle kernel NULL pointer dereference at virtual address 00000044
[  241.611623] pgd = ffffffc0b8efd000
[  241.615273] [0000000000000044] *pgd=0000000000000000, *pud=0000000000000000
[  241.622280] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[  242.204936] Process umount (pid: 3798, stack limit = 0xffffffc0b742c000)
[  242.211782] Stack: (0xffffffc0b742f570 to 0xffffffc0b7430000)
[  242.217677] f560:                                   ffffffc0b742f640 ffffff80091cc0cc
[  242.225651] f580: ffffffc0b97db080 0000000000000000 ffffffc07ce90360 0000000000000000
[  242.233624] f5a0: 000000000000003f 0000000000000400 0000000000000000 0000000000000848
[  242.241597] f5c0: 000000000000000d 000000000000330e ffffffc0b742f600 0000000000000000
[  242.249570] f5e0: ffffffc07ce90360 0000000000000189 0000000000000000 ffffffc0a41b7000
[  242.257544] f600: ffffffc0b742f630 ffffff8009215f1c ffffffc0b742f640 ffffff80090d6150
[  242.265517] f620: ffffff8009f89ba0 0000000000000189 0000000000000000 ffffff8001fdb2d8
[  242.273490] f640: ffffffc0b742f6a0 ffffff80091cc470 0000000000000000 ffffffc0b97db080
[  242.281464] f660: ffffffc07ce90360 0000000000000000 000000000000003f 0000000001d00000
[  242.289437] f680: ffffffc095929300 0000000000000848 ffffffc07ce901f0 ffffff800a8aada0
[  242.297410] f6a0: ffffffc0b742f6e0 ffffff8001fdf34c ffffffc0b742f758 ffffffc0b97db080
[  242.305384] f6c0: ffffffc07ce901f0 ffffff800a8aada0 ffffffc0a41b7000 ffffffc0b97db080
[  242.313357] f6e0: ffffffc0b742f7d0 ffffff8001fdf900 ffffffc071005800 ffffffc071005840
[  242.321330] f700: 0000000000000000 ffffffc088ad20e0 ffffffc0c93a8000 ffffffc095929300
[  242.329303] f720: 000000000005c000 ffffffc07ce901f0 ffffffc0c9baa290 0000000000000000
[  242.337277] f740: ffffffc0b742f760 0000000000000000 ffffffc0b742f770 ffffffc0b742f758
[  242.345250] f760: ffffffc0b742f758 fffffffffffffff5 00000001d0000000 ffffff8001fde200
[  242.353223] f780: 0000000000000000 0000000000000000 0000000000000000 ffffffc0a427a400
[  242.361197] f7a0: ffffffc0a41b7000 ffffffc07ce901f0 0000000000000000 0000004000000000
[  242.369170] f7c0: 0000000000000000 0000000000000001 ffffffc0b742f820 ffffff8001f6f690
[  242.377144] f7e0: ffffff8002042500 ffffffc071005800 ffffffc071005108 ffffffc071005840
[  242.385117] f800: ffffffc0c9baa240 ffffffc0c93a8000 ffffffc0c9baa278 ffffffc0c9baa250
[  242.393090] f820: ffffffc0b742f8b0 ffffff8001f7edd4 ffffffc071005800 ffffffc0c93a8b10
[  242.401063] f840: ffffffc071005108 ffffffc071005000 0000000000004000 0000000001d00000
[  242.409037] f860: 0000000000000001 0000000000000000 0000000000000014 ffffffc0a41b7000
[  242.417010] f880: ffffffc071005108 0000000000000000 ffffffc0b7496200 ffffff80090f9c98
[  242.424984] f8a0: ffffffc0b742f8a0 ffffffc0b742f8a0 ffffffc0b742f9a0 ffffff8001f7fa68
[  242.432957] f8c0: 0000000000002000 ffffffc0a41b7000 0000000000000000 0000000000000014
[  242.440930] f8e0: 0000000000000000 0000000000000000 0000000000004000 ffffffc0b742faa8
[  242.448903] f900: 0000000000000001 0000000000004000 ffffffc0a41b7000 ffffffc071005000
[  242.456877] f920: ffffffc0a4040000 ffffffc0c93a8b14 0000000000004000 00000001090ecc80
[  242.464850] f940: 0000000000004000 ffffffc0710058c4 0000000000000000 0000000100000000
[  242.472824] f960: ffffffc0710050d0 ffffffc0b742faa8 0000000100000001 ffffff8000000000
[  242.480797] f980: 0000000000000000 0000000000000000 0000000000000000 0000000000010000
[  242.488770] f9a0: ffffffc0b742fa20 ffffff8001f7fc84 0000000000004000 ffffffc0c93a8000
[  242.496744] f9c0: 0000000000000000 ffffffc07ce3f000 0000000000000000 0000000000000000
[  242.504717] f9e0: 0000000000000001 ffffffc0c93a8130 ffffffc0b742fb48 ffffffc0a41b7000
[  242.512690] fa00: 0000000000000000 ffffffc0c93a8000 0000000000000000 ffffffbebffb8f30
[  242.520664] fa20: ffffffc0b742fac0 ffffff8001f6582c ffffffc0a41b7000 ffffffc0c59f6000
[  242.528637] fa40: ffffffc07ce3f000 0000000000000000 ffffffc0a41b7200 ffffffc0b742fc38
[  242.536610] fa60: 0000000000000000 0000000000000000 ffffffc0b742fb48 0000000000000000
[  242.544584] fa80: ffffffc0b742fac0 0000000000000161 00000000a41b7000 0000000000000000
[  242.552557] faa0: ffffffc07ce3f000 0000000000000000 00000000000000a8 ffffffc0b742fc00
[  242.560530] fac0: ffffffc0b742fb60 ffffff8001f65d7c ffffffc07ce3f000 ffffffc0c59f6000
[  242.568504] fae0: ffffffc0a41b7000 ffffffc0b742fc38 0000000000000000 0000000000000000
[  242.576477] fb00: 0000000000000000 0000000000000027 ffffff8009c42000 ffffffc0b7496200
[  242.584450] fb20: ffffffc0b742fb40 0000000001fd0c3c ffffffc0c59f6000 0000000000000000
[  242.592423] fb40: 00000000b742fb60 0000000000000002 0000000000000084 ffffffc0c59f6000
[  242.600397] fb60: ffffffc0b742fbb0 ffffff8001f8f7d8 ffffffc07ce3f000 ffffffc0c93a8000
[  242.608370] fb80: ffffffc0a41b7000 ffffffc0b527a000 ffffff800a7f8928 ffffff800202b6d0
[  242.616344] fba0: 0000000000000123 ffffffc0c93a8000 ffffffc0b742fc40 ffffff8001f9217c
[  242.624317] fbc0: ffffffc07ce3f000 ffffffc0a41b7000 0000000000000000 ffffffc0b527a000
[  242.632290] fbe0: ffffff800a7f8928 ffffff800202b6d0 0000000000000123 0000000000000027
[  242.640264] fc00: ffffff8009c42000 ffffffc0b7496200 0000000000000123 0000000000000027
[  242.648237] fc20: ffffff8009c42000 ffffffc0b7496200 ffffffc07ce3f000 ffffffc0c59f6000
[  242.656210] fc40: ffffffc0b742fcb0 ffffff8001f8b82c ffffffc0a41b7000 ffffffc0c93a8f70
[  242.664184] fc60: ffffffc0a41b7000 ffffffc0c93a8020 ffffff800a7f8928 0000000000000015
[  242.672157] fc80: ffffff8009c42000 ffffffc0b7496200 ffffffc0b742fcb0 ffffff8001f8b81c
[  242.680130] fca0: ffffffc0a41b7000 ffffffc0c93a8020 ffffffc0b742fcd0 ffffff8001f8d5cc
[  242.688104] fcc0: ffffffc0c93a8000 ffffffc0c93a8020 ffffffc0b742fd40 ffffff8001f5f224
[  242.696077] fce0: ffffffc0a41b6800 ffffff8002021f10 ffffffc0c93a8000 ffffff8009f74688
[  242.704050] fd00: ffffff800a7f8928 ffffff8002021f10 ffffffc0c93a8000 ffffff8009f74688
[  242.712024] fd20: ffffff800a7f8928 0000000000000015 0000000000000123 0000000000000027
[  242.719997] fd40: ffffffc0b742fd60 ffffff8009239460 ffffffc0a41b6800 ffffffc0b742fd50
[  242.727971] fd60: ffffffc0b742fd80 ffffff80092397e4 ffffffc0a41b6800 0000000000000029
[  242.735944] fd80: ffffffc0b742fda0 ffffff8001f5ee8c ffffffc0a41b6800 ffffffc0c93a9000
[  242.743917] fda0: ffffffc0b742fdd0 ffffff8009239b7c ffffffc0a41b6800 ffffff8002038960
[  242.751890] fdc0: ffffffc0b7496200 ffffff8002038960 ffffffc0b742fdf0 ffffff800923aa70
[  242.759864] fde0: ffffffc0a41b6800 ffffffc0b74969c0 ffffffc0b742fe10 ffffff800925be64
[  242.767837] fe00: ffffffc0b970ae00 ffffffc0b74969c0 ffffffc0b742fe30 ffffff800925bf18
[  242.775810] fe20: ffffffc0b970ae40 ffffff80090c85fc ffffffc0b742fe50 ffffff80090c85ec
[  242.783784] fe40: 0000000000000000 ffffff8009083740 ffffffc0b742fe90 ffffff800908a6ac
[  242.791757] fe60: 0000000000000004 ffffffc0b7496200 ffffffc0b7496200 ffffffc0b742fec0
[  242.799730] fe80: 0000000060000000 0000000000000027 0000000000000000 ffffff800908361c
[  242.807704] fea0: ffffffffffffff00 00000040c515e000 ffffffffffffffff 0000007f841fa048
[  242.815677] fec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  242.823650] fee0: 00000055b831000a 00000055b831017a 7461642f746e6d2f 00617461642f746e
[  242.831623] ff00: 0000000000000027 00000055b8310160 0000000000000000 0000000000000000
[  242.839597] ff20: 0000000000000000 0000000000000001 0000000000000002 0000000000000020
[  242.847570] ff40: 0000007f843095c0 0000007f841fa030 0000000000000000 00000055b830da30
[  242.855543] ff60: 0000000000000000 00000055b830db40 0000007f84309000 00000055b830dc60
[  242.863517] ff80: 00000055b8310000 0000005587ca4430 0000000000000000 0000000000000000
[  242.871490] ffa0: 00000000ffffffff 0000007fde5c5760 0000007f842c99bc 0000007fde5c5760
[  242.879463] ffc0: 0000007f841fa048 0000000060000000 00000055b8310000 0000000000000027
[  242.887436] ffe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  243.146053] Code: a906cfa1 aa0303f7 aa0403f4 d503201f (b94046a2)
dietpi@DietPi:~$ sudo btrfs check /dev/sda1
Opening filesystem to check...
Checking filesystem on /dev/sda1
UUID: 3071c9b1-fbd3-49cd-be54-cb6ad296fab6
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 2054799360 bytes used, no error found
total csum bytes: 2000464
total tree bytes: 5275648
total fs tree bytes: 2342912
total extent tree bytes: 311296
btree space waste bytes: 1030953
file data blocks allocated: 2049523712
 referenced 2047770624
dietpi@DietPi:~$ sudo btrfs check /dev/sda2
Opening filesystem to check...
Checking filesystem on /dev/sda2
UUID: bdaf3535-0d3d-405d-9260-4bcc8aad1802
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 15825326080 bytes used, no error found
total csum bytes: 15259892
total tree bytes: 194478080
total fs tree bytes: 167526400
total extent tree bytes: 8486912
btree space waste bytes: 31531975
file data blocks allocated: 15630848000
 referenced 15630843904
 dietpi@DietPi:~$ sudo btrfs check /dev/sdb1
Opening filesystem to check...
Checking filesystem on /dev/sdb1
UUID: 3071c9b1-fbd3-49cd-be54-cb6ad296fab6
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 2054799360 bytes used, no error found
total csum bytes: 2000464
total tree bytes: 5275648
total fs tree bytes: 2342912
total extent tree bytes: 311296
btree space waste bytes: 1030953
file data blocks allocated: 2049523712
 referenced 2047770624
dietpi@DietPi:~$ sudo btrfs check /dev/sdb2
Opening filesystem to check...
Checking filesystem on /dev/sdb2
UUID: bdaf3535-0d3d-405d-9260-4bcc8aad1802
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 15825326080 bytes used, no error found
total csum bytes: 15259892
total tree bytes: 194478080
total fs tree bytes: 167526400
total extent tree bytes: 8486912
btree space waste bytes: 31531975
file data blocks allocated: 15630848000
 referenced 15630843904

Btrfs check shows no errors for all paritions.

Would it be possible to use the kernel from backports?

sudo apt-get install -t bullseye-backports linux-image-arm64
sudo apt-get install -t bullseye-backports firmware-linux

MichaIng after some further analysis, I think it is still the same issue as https://github.com/MichaIng/DietPi/issues/4062. I can help in testing newer versions but for now, we should highlight that BTRFS is not working for C4 image. I also switched to LVM now until BTRFS is working.

What? The bootloader, kernel and device tree is for Odroid C4 and HC4, not for N2. I just mentioned the known issues with Odroid N2 since it could be similar with C4/HC4. But since you use SATA this shouldn’t apply.

Interesting, does this happen in general with the umount command or with this particular filesystem only? Probably the kernel indeed has some limitations regarding Btrfs, would be nasty.

No. The kernel is not the one from Debian but one from Hardkernel. The one from Debian won’t work with the bootloader and configs. We are however planning to migrate to a mainline kernel build from Armbian soon, like we just did for Odroid N2. But this requires reflashing the system since Armbian’s U-Boot build expects everything to be on a single ext4 partition without a dedicated boot FAT partition.