Dietpi-backup stops with rsync warning: some files vanished

Troubleshooting
1

Creating a bug report/issue

Required Information

  • DietPi version |
    G_DIETPI_VERSION_CORE=8
    G_DIETPI_VERSION_SUB=16
    G_DIETPI_VERSION_RC=2
    G_GITBRANCH=‘master’
    G_GITOWNER=‘MichaIng’
    G_LIVE_PATCH_STATUS[0]=‘not applicable’

  • Distro version | bullseye 0

  • Kernel version | Linux THORIN 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

  • Architecture | arm64

  • SBC model | RPi 4 Model B (aarch64)

  • Power supply used | 5V 2.4A

  • SD card used | (EG: SanDisk ultra)

I run dietpi-backup via script calling G_INTERACTIVE=0 /boot/dietpi/dietpi-backup 1 || exit 1
Sometimes (often) it happens that it exits with a rsync error 24, looking like this (from dietpi-backup.log):

2023/05/24 22:34:23 [429429] .d..t...... var/spool/cron/crontabs/
2023/05/24 22:34:23 [429429] >f..t...... var/spool/cron/crontabs/root
2023/05/24 22:34:23 [429429] sent 605,254,322 bytes  received 45,546 bytes  2,945,498.14 bytes/sec
2023/05/24 22:34:23 [429429] total size is 8,458,282,478  speedup is 13.97
2023/05/24 22:34:23 [429429] sent 605254327 bytes  received 45551 bytes  total size 8458282478
2023/05/24 22:34:23 [429429] rsync warning: some files vanished before they could be transferred (code 24) at main.c(1333) [sender=3.2.3]

No idea what that would be.
However my destination is a nfs mounted dir on my nas.
Include/excludes are:
“+ /mnt/dietpi_userdata/”
“- /mnt/*”
“- /media/”
“- /var/tmp/”

2

You can probably ignore it as it is just a warning. I think you have an application running that creates temp files. These may be quickly deleted before they can be saved.

3

Unfortunately no, because it will exit at this point.
And also as stated above I have /var/tmp/ excluded.
And then my script is dead too

4

there might be apps using different location for their temporary files.

5

So exiting is just fine?
Isn’t rsync supposed to ignore such things?

6

not sure if there is an option within rsync to ignore such files. Probably @MichaIng knows.

7

What I meant was “ignore such errors” .
The point here is that this error breaks dietpi-sync.
It just aborts, quits, exits without message. And so does my script which invoked dietpi-sync.
My snippet from dietpi-backup.log above is the last entry in the log.

8

How do you know that it all exits? Since the sync has finished, you could ignore it and even if rsync returned an error code so that dietpi-backup would exit (loudly with an error message), that would not exit your wrapping script unless you actively set -e or otherwise handle dietpi-backups exit code.

You could check the log to see which files exactly did vanish. This is a bit uncommon since /tmp should be used for such short term temporary files, which is excluded OOTB. /var/tmp is more aimed for files which should survive at least reboots, so those should usually better not be excluded. DietPi uses it for some logs (first run setup, update, backup, RAM log, …) that you may want to review also at a later date.

9

I know it because I see it in the logfile of my script. It calls dietpi-backup and from there it never comes back.
It shows “22:30:56 performing dietpi-backup…”
and then ends at this point.

the lines are

echo `date +%T` "performing dietpi-backup..."  >> $MYLOGFILE
G_INTERACTIVE=0 /boot/dietpi/dietpi-backup 1 || exit 1
echo `date +%T` "dietpi-backup completed"  >> $MYLOGFILE

and “dietpi-backup completed” never shows up if this error (warning) occurs.
And the rsync warning is the last line in dietpi-backup.log

So that’s at least what I assumed.
Since it doesn’t happen every time I run my script, but only every other day.

10

Ah okay. We could ignore exit code 24: https://manpages.debian.org/bullseye-backports/rsync/rsync.1.en.html#EXIT_VALUES

  • 24 - Partial transfer due to vanished source files

The whole thing is discussed here since a long time: 3653 – Reduce the need for the "vanished files" warning

Still, would be interesting to know which files exactly vanished. Probably there is a common temporary location which we could exclude by default to prevent this error in the first place.

11

What would you suggest?
I don’t know which files vanished.
I added /tmp to exclude though.

12

Does the log file not show them? I know it is difficult to find the relevant lines, since the file is usually large, but probably you are luckly.

/tmp is excluded by default, along with all other known tmpfs filesystems, like /sys, /run, /proc etc.

13

Hi,
so I had to wait several days, bit today it happened again.
So the log isn’t that long, about 300 lines.
It looks like a normal rsync log to me. No mentioning of vanished files except the last line which I posted here.
And here they are again

1771343] >f.st...... var/log/samba/log.nmbd
2023/06/10 22:33:58 [1771343] >f.st...... var/log/samba/log.orthanc
2023/06/10 22:33:58 [1771343] >f.st...... var/log/samba/log.smbd
2023/06/10 22:33:59 [1771343] >f.st...... var/log/unattended-upgrades/unattended-upgrades.log
2023/06/10 22:33:59 [1771343] sent 620,968,266 bytes  received 46,497 bytes  3,421,568.94 bytes/sec
2023/06/10 22:33:59 [1771343] total size is 9,074,319,964  speedup is 14.61
2023/06/10 22:33:59 [1771343] sent 620968271 bytes  received 46502 bytes  total size 9074319964
2023/06/10 22:33:59 [1771343] rsync warning: some files vanished before they could be transferred (code 24) at main.c(1333) [sender=3.2.3]

It is strange because it looks like it ended normally with giving a summary sent received etc.

I’ve also had look into .dietpi-backup_stats and there is also confirmation that the backup didn’t end normally:

2023-06-09_22:31:03  Backup starting
2023-06-09_22:43:42  Backup completed
2023-06-10_22:30:58  Backup starting

So here we are.

14

So are you implementing ignoring of exit code 24?