Almost all services dead after some time

Troubleshooting
1

Maybe this is the problem… but whats the cause?

Dez 04 18:53:25 server kernel: EXT4-fs error (device mmcblk1p2): __ext4_find_entry:1622: inode #136832: comm squeezeboxserve: checksumming directory block 0


In this state I got errors like this when execution everyday tasks:

root@server:/# dmesg -T
-bash: /usr/bin/dmesg: Kann die Binärdatei nicht ausführen: Fehler im Format der Programmdatei


translated by me:

cannot execute binary: error in programfile format



after a reboot everything is running again for some time...
2

Anything wrong with our trouble shooting template or why did you ignored it?

Can you share some more details on your system pls

Required

  • DietPi version | cat /boot/dietpi/.version
  • Distro version | echo $G_DISTRO_NAME $G_RASPBIAN
  • Kernel version | uname -a
  • Architecture | dpkg --print-architecture
  • SBC model | echo $G_HW_MODEL_NAME or (EG: RPi3)
  • SDcard used | (EG: SanDisk ultra)

Especially about the used storage device, SD card, USB attached HDD/SSD, eMMC aso.

As well following would be helpful

df -h
lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid
3

deleted

4

for diag i changed the following in /etc/systemd/journald.conf

[...]
[Journal]
**Storage=persistent**
#Compress=yes

But after hard reset (as ssh doesn’t work either after some point in time) there’s always only on boot to select:

root@server:~# journalctl --list-boots
Journal file /var/log/journal/bffaf240bb144a43a216fe42d351c598/system@00000000000000000000000000000000-0000000000000000-0000000000000000.journal is truncated, ignoring file.
IDX BOOT ID                          FIRST ENTRY                 LAST ENTRY
  0 fa94be513ce5487d86e5718942b7d9be Thu 2024-12-05 10:17:02 CET Thu 2024-12-05 10:21:45 CET

Is there anything more I have to modify for logging more than one boot process?

5

Hmm how is the external drive powered? We recommend using powered USB hub or enclosure.

6

there’s no external drive (except usb-stick for backup)… it’s a radxa 5b with m2 ssd

7

if system is hanging, would be good to check for kernel error messages. At least if you still have access

dmesg -l 0,1,2,3

As well you can try to enable persistente logs

persistent system logs:

dietpi-software uninstall 103 # uninstalls DIetPi-RAMlog
mkdir /var/log/journal # triggers systemd-journald logs to disk
reboot # required to finalise the RAMlog uninstall

Then you can check system logs via:

journalctl

which will then show as well logs from previous boot sessions. To limit the size, you can additionally e.g. apply the following:

mkdir -p /etc/systemd/journald.conf.d
cat << '_EOF_' > /etc/systemd/journald.conf.d/99-custom.conf
[Journal]
SystemMaxFiles=2
MaxFileSec=7day
_EOF_

This will limit logs to 14 days split across two journal files, so that with rotation you will always have between 7 and 14 days of logs available.

8

OK. I think I got the root cause for this problem.

After transfer of the image from emmc to micro sd erverything is running fine again.

So the emmc seems to be damaged…

Thank you for your help!

1 Like
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.