Random lock-ups Nanopi Neo 3

General Discussion
1

I use a nanopi neo 3 for basic home services for about 2 years. It runs Dietpi and have a handful containers running on it using docker/portainer. I keep Dietpi updated.

I always experienced random lock-ups. Once every 2 months on average I think. At first I thought this was related to power surges ( info from a home electricity power usage logging container) so I installed a tiny UPS device ( lipo battery charger/ups) to rule out the power surges.

However it made not much difference.
I still have the random lock-ups.

Luckily the system recovers well until now after a lock-up and a power cycle. The partial loss of the home electricity logging is not crucial for me.
I would like to add some control function for home battery storage to control charging and discharging the battery. But with the random lock-ups I find this setup not reliable enough to go on with it.

I also have tried to preserve a few previous bootlogs but I could not figure out how to get that happening. Although I do not expect that older boot logs will show much info to help. But you never know…

My system runs on USB but boots from SD as described here:

I also assume I have ruled out that a bad power supply may be the cause. The NPN3 will even run when powered from a USB hub but it is now powered from a tiny lipo UPS/charger. Running heavy tasks: e.g. zip7 compression using 4 cores for 30 minutes works well. Overheating is no problem either because a regulated cooling fan, I use a cron-job for it, is installed. It does it’s job as expected.

Applications running in docker:

  • dsmrreader (home power usage logging)
  • database (postgres) for previous
  • whoogle ( rarely used)
  • Pihole
  • wireshark server for remote access ( I never had a lock-up related to remote access)
  • filebrowser
    I have no services running on Dietpi natively, only a cron-job to regulate the fan.

The random lock-ups don’t seem to be related to acces or program tasks, but I cannot rule this out 100%

I know this “problem” is not very well prescribed with this, but I have no additional information that seems to be related to these random lock-ups.

Any suggestion is welcome to troubleshoot this.

2

Without an error message or log, it is almost impossible to determine the cause. Unfortunately, on a standard system the log is deleted after a reboot. You can therefore try to activate permanent logging. There may be some information after the next system freeze.

persistent system logs:

dietpi-software uninstall 103 # uninstalls DIetPi-RAMlog
mkdir /var/log/journal # triggers systemd-journald logs to disk
reboot # required to finalise the RAMlog uninstall

Then you can check system logs via:

journalctl

which will then show as well logs from previous boot sessions. To limit the size, you can additionally e.g. apply the following:

mkdir -p /etc/systemd/journald.conf.d
cat << '_EOF_' > /etc/systemd/journald.conf.d/99-custom.conf
[Journal]
SystemMaxFiles=2
MaxFileSec=7day
_EOF_

This will limit logs to 14 days split across two journal files, so that with rotation you will always have between 7 and 14 days of logs available.

3

Thank for that!

I had tried before to preserve previous boot logs using the generic Linux way, but that did not work.

With your info I will try again.
I do not expect storage issues. I have a 240Gb USB disk with a 20Gb rootfs partion , currently only 17% of it used. All data including docker files are stored on the remaining 220GB partition. (also only 17% used…)

It may take a while before the next lock-up will happen, however it may also manifest itself tomorrow :slightly_smiling_face:

4

Take your time and report back if it happens again