DietPi-Backup filter options only work with a directory depth of 2?

Troubleshooting
1

Creating an issue

rsync (which is called from dietpi-backup) seems not to backup a directory depth > 2.
Q: Is this a known behaviour or can this be overcome by a different “include” line in the file /boot/dietpi/.dietpi_backup_inc_exc?

Required Information

  • DietPi version | v8.11
  • Debian Bullseye
  • Seen on several different hardwares (ODROID, PC, VM,…)

Additional Information (if applicable)

  • Software title | Dietpi-Backup (-> rsync -V shows version 3.2.3)
  • Tested on minimal installation

Steps to reproduce

dietpi-backup was executed with a filter contents like

+ /mnt/backuptest1/
+ /mnt/backuptest2/dir2/
+ /mnt/backuptest3/dir3/subdir3/

In every directory is a file stored to see the backup result in the data directory of the backup (rsync) target directory.

Expected behaviour

All three directory structures should be backed up with the contained files.

Actual behaviour

Only the directory structure /mnt/backuptest1 is backed up (tested only with absolute path beginning with a “/”).

I.e., directories based on more than a depth of 2 are not backed up. Also they are not present as empty directories in the backup.

Extra details

I did not find any hints in the rsync man pages resp. documentation.
Also tested with Debian Bookworm, which contains rsync version 3.2.6: No difference.

2

nope, we backup the entire system using rsync, which include directories with a depth > 2 as well. Other than that, the backup would be inconsistent and not usable for restore.

How did you setup the filter? For me following filter setting is working fine

+ /mnt/dietpi_userdata/
+ /mnt/backuptest*
- /mnt/*
- /media/

Backup log

 Backup log from 2022-12-16_10:13:41                                                       
                                                                                           
 2022/12/16 10:13:41 [1438] building file list                                             
 2022/12/16 10:13:42 [1438] .d..t...... ./                                                 
 2022/12/16 10:13:42 [1438] .d..t...... boot/dietpi/                                       
 2022/12/16 10:13:42 [1438] >f..t...... boot/dietpi/.dietpi-backup_inc_exc                 
 2022/12/16 10:13:42 [1438] cd+++++++++ mnt/backuptest1/                                   
 2022/12/16 10:13:42 [1438] >f+++++++++ mnt/backuptest1/demo.file1                         
 2022/12/16 10:13:42 [1438] cd+++++++++ mnt/backuptest2/                                   
 2022/12/16 10:13:42 [1438] cd+++++++++ mnt/backuptest2/dir2/                              
 2022/12/16 10:13:42 [1438] >f+++++++++ mnt/backuptest2/dir2/demo.file2                    
 2022/12/16 10:13:42 [1438] cd+++++++++ mnt/backuptest3/                                   
 2022/12/16 10:13:42 [1438] cd+++++++++ mnt/backuptest3/dir3/                              
 2022/12/16 10:13:42 [1438] cd+++++++++ mnt/backuptest3/dir3/subdir3/                      
 2022/12/16 10:13:42 [1438] >f+++++++++ mnt/backuptest3/dir3/subdir3/demo.file3            
 2022/12/16 10:13:43 [1438] sent 419,401 bytes  received 1,596 bytes  280,664.67 bytes/sec 
 2022/12/16 10:13:43 [1438] total size is 587,454,187  speedup is 1,395.39

Content of backup folder

root@DietPiR5S:~# ls -la /mnt/dietpi-backup-test/data/mnt/backuptest1/ /mnt/dietpi-backup-test/data/mnt/backuptest2/dir2/ /mnt/dietpi-backup-test/data/mnt/backuptest3/dir3/subdir3//mnt/dietpi-backup-test/data/mnt/backuptest1/:
total 8
drwxr-xr-x 2 root root 4096 Dec 16 10:04 .
drwxr-xr-x 6 root root 4096 Dec 16 10:06 ..
-rw-r--r-- 1 root root    0 Dec 16 10:04 demo.file1

/mnt/dietpi-backup-test/data/mnt/backuptest2/dir2/:
total 8
drwxr-xr-x 2 root root 4096 Dec 16 10:04 .
drwxr-xr-x 3 root root 4096 Dec 16 10:04 ..
-rw-r--r-- 1 root root    0 Dec 16 10:04 demo.file2

/mnt/dietpi-backup-test/data/mnt/backuptest3/dir3/subdir3/:
total 8
drwxr-xr-x 2 root root 4096 Dec 16 10:04 .
drwxr-xr-x 3 root root 4096 Dec 16 10:04 ..
-rw-r--r-- 1 root root    0 Dec 16 10:04 demo.file3
root@DietPiR5S:~#

I guess order of includes / excludes matters.

1 Like
3

The given directory structure was a structure for testing the max depth in the .dietpi_backup_inc_exc file.
The real structure is:
/mnt/nvme_disk/RaidBackupDir (shall not be backed up) and
/mnt/nvme_disk/samba-server (shall be backed up)
With /mnt/nvme_disk is a mounted disk.

The filter contents I wanted is
+ /mnt/nvme_disk/samba-server/
which did not work due to the 3rd entry samba-server of the path value.
(Also a test with samba_server did not work.)

So the only thing could be to mount the nvme_disk not to mnt and mount it into / instead to get rid of one directory level.

4

You could try to work around by including whole /mnt and excluding things you don’t want to backup

- /mnt/nvme_disk/RaidBackupDir*
- /mnt/dietpi-backup*
- /media/
5

IMO the filter works line by line, see config file:

# Since the list is processed from top to bottom and the first match defines the result,
#   includes need to be defined before their wildcard exclude rule
#   and in case excludes before their wildcard include rule.

So a

+ /mnt
- /mnt/dietpi-backup*

should not work because the +/mnt overrides the following line.

6

No need to include the directory. Should be in by default.

7

Aaah, I understand. You are right (like ever :smiley:).

8

Not always :sweat_smile:

Maybe @MichaIng has some thoughts on this as well

9

Just for confirmation, my test also failed.
Filter:

+ /mnt/dietpi_userdata/test1/sub2
- /mnt/*

but in the backup the directory /mnt is completly empty.

10

Every (otherwise excluded) parent directory of an included directory needs to be included as well. In this case rsync skips /mnt/backuptest2 and /mnt/backuptest3 all together, so the 2nd and 3rd rule have no effect. Instead try:

+ /mnt/backuptest1/
+ /mnt/backuptest2/
+ /mnt/backuptest2/dir2/
- /mnt/backuptest2/*
+ /mnt/backuptest3/
+ /mnt/backuptest3/dir3/
+ /mnt/backuptest3/dir3/subdir3/
- /mnt/backuptest3/*

So quite nasty to include individual sub directories within an excluded parent dir.

3 Likes