Daily Dietpi-sync fails half the time

Troubleshooting
1

Hi,

I am running a daily backup followed by a sync, all set up via dietpi menu.

The backup always runs through, but the sync to my second HDD fails regularly, up to every 2nd day.

I recently activated a hardware watchdog and this restarts my pi3b+ whilst the sync should be running indicating something made the sync fail or putting too much strain on my PI3b+ whilst syncing.

If the sync succeeds, it takes around 2 hours.
Although only a handful of files actually change from day to day. Shouldn’t the sync be much fast then?

How to troubleshoot?

Thanks
T

2

how is the HDD powered? Does it have an own PSU or just powered via USB port of RPI?

You might need to disable your watchdog to have a chance to see what the actual issue is.

3

HDD has its own PSU.

When the sync fails, I can no longer ssh into my Pi to check what is going on

The pi seems to be dead and doesn’t recover. That’s the reason for the watch dog in the first place.

4

you could try to activate persistent logging. This might allow to check what was the last action before device become unavailable.

persistent system logs:

dietpi-software uninstall 103 # uninstalls DIetPi-RAMlog
mkdir /var/log/journal # triggers systemd-journald logs to disk
reboot # required to finalise the RAMlog uninstall

Then you can check system logs via:

journalctl

which will then show as well logs from previous boot sessions. To limit the size, you can additionally e.g. apply the following:

mkdir -p /etc/systemd/journald.conf.d
cat << '_EOF_' > /etc/systemd/journald.conf.d/99-custom.conf
[Journal]
SystemMaxFiles=2
MaxFileSec=7day
_EOF_

This will limit logs to 14 days split across two journal files, so that with rotation you will always have between 7 and 14 days of logs available.

1 Like
5

I will give that a try. Thank you.

Before I do that, how do I revert back i.e. disable persistent logging again?

6

You can change back to another logging method with dietpi-software, there is a separate menu entry for it.

1 Like
7

simple revert the steps done in opposite order :slight_smile:

1 Like
8

Just to give an update:

My pi is up 6 days now and there was not a single crash after I enabled logging as per above (and uninstalling ram log)
Everything else (backup/sync) is as before.

I will continue observing to see if that is a coincidence or not.

9

Got another restart but journalctl only shows logs starting from the restart.

Not really helpful and not sure why the older entries are gone.

10

@MichaIng any ideas why journalctl are not surviving the reboot? On my test system, logs did not keep most of the time. On some reboots, they survive. But not that often. :thinking:

11

No idea. Works reliable on dietpi.com and my home server. With the applied settings there should be always two files in /var/log/journal/*/, one active system.journal and one with a timestamp of up to 7 days age. /run/log/journal/ on the other hand should be empty.