Dietpi-backup: How can I give a specific name to a backup fileset?

gjwAudio · 11 June 2024 00:39

Hello All

My dietpi machine is a single-use “appliance”, used as music playback source in a HiFi system. With the past few version upgrades, the quality of sound has improved (to my ears). As such, I make a backup of the current state of the machine before applying an update, so I can roll back to the previous version if the “new sound” does not suit my taste.

My dietpi-backup limit is set to 3 copies. It would be very helpful if I could assign a meaningful name (ie: v9.4.2) to the backup fileset, so that I am confident of which version I am restoring. This will facilitate toggling between versions to evaluate the impact an update has on SQ (sound quality).

Q: is it as easy as manually specifying an additional element in the path before running the backup, like this /mnt/dietpi-backup/v9.4.2 ?

Thank you for helping an old music enthusiast with some modern computer guidance

Joulinar · 11 June 2024 06:10

simply open dietpi-backup and set a target location manually.

gjwAudio · 15 June 2024 13:57

Thank you for that confirmation.

Now that I have my two backups of interest created

/mnt/dietpi-backup/v9.3.0
/mnt/dietpi-backup/v9.4.2

How can I get them both to display in the backup list when I wish to restore one or the other version ?

Also, does the Amount (# of backups, set to 3 in my case) affect these files when they are named in this way ?

Curiously, the du command shows different filesets for the various backup versions - is this expected ?

root@DietPi:~# du -ah / --exclude /mnt/FREENAS | sort -n -r | head -n 20
du: cannot access '/proc/1045/task/1045/fd/4': No such file or directory
du: cannot access '/proc/1045/task/1045/fdinfo/4': No such file or directory
du: cannot access '/proc/1045/fd/3': No such file or directory
du: cannot access '/proc/1045/fdinfo/3': No such file or directory
du: cannot read directory '/mnt/2TB_Music': No such device
du: cannot read directory '/mnt/WTF_Music': No such device
1020K   /usr/lib/modules/5.19.17-100HZv2/kernel/fs/f2fs
1020K   /usr/lib/modules/5.19.17-100HZv2/kernel/drivers/nvme
1020K   /usr/lib/firmware/iwlwifi-cc-a0-46.ucode
1020K   /root/linux-5.19.17/fs/ksmbd
1020K   /root/linux-5.19.17/arch/s390/include/asm
1020K   /mnt/dietpi-backup/v9.4.2/data/usr/lib/modules/5.19.17-100HZv2/kernel/fs/f2fs
1020K   /mnt/dietpi-backup/v9.4.2/data/usr/lib/modules/5.19.17-100HZv2/kernel/drivers/nvme
1020K   /mnt/dietpi-backup/v9.4.2/data/usr/lib/firmware/iwlwifi-cc-a0-46.ucode
1020K   /mnt/dietpi-backup/v9.4.2/data/root/linux-5.19.17/fs/ksmbd
1020K   /mnt/dietpi-backup/v9.4.2/data/root/linux-5.19.17/arch/s390/include/asm
1020K   /mnt/dietpi-backup/v9.3.0/data/root/linux-5.19.17/arch/s390/include/asm
1020K   /mnt/dietpi-backup/data_3/usr/lib/modules/5.19.17-100HZv2/kernel/fs/f2fs
1020K   /mnt/dietpi-backup/data_3/usr/lib/modules/5.19.17-100HZv2/kernel/drivers/nvme
1020K   /mnt/dietpi-backup/data_3/usr/lib/firmware/iwlwifi-cc-a0-46.ucode
1020K   /mnt/dietpi-backup/data_3/root/linux-5.19.17/fs/ksmbd
1020K   /mnt/dietpi-backup/data_3/root/linux-5.19.17/arch/s390/include/asm
1020K   /mnt/dietpi-backup/data_2/usr/lib/modules/5.19.17-100HZv2/kernel/fs/f2fs
1020K   /mnt/dietpi-backup/data_2/usr/lib/modules/5.19.17-100HZv2/kernel/drivers/nvme
1020K   /mnt/dietpi-backup/data_2/usr/lib/firmware/iwlwifi-cc-a0-46.ucode
1020K   /mnt/dietpi-backup/data_2/root/linux-5.19.17/fs/ksmbd
root@DietPi:~#

Presumably I can remove the data_2 and data_3 folders (to conserve drive space), now that v9.3.0 and v9.4.2 exist.

Thanks for your help.

gjwAudio · 15 June 2024 23:09

Uh-oh… diertpi-backup RESTORE job failed and dietpi-launcher will not run.

the end of failed restore output:

sent 40,938,918 bytes  received 2,069 bytes  81,881,974.00 bytes/sec
total size is 2,152,728,633  speedup is 52.58
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1333) [sender=3.2.3]
/boot/dietpi/func/dietpi-globals: line 265: tput: command not found
/boot/dietpi/func/dietpi-globals: line 266: ( 94 + 5 ) /  : syntax error: operand expected (error token is "/  ")
/boot/dietpi/func/dietpi-globals: line 148: /usr/bin/rm: No such file or directory
[FAILED] DietPi-Backup | Failed to remove scripts working directory: /tmp/DietPi-Backup
root@DietPi:~#

and subsequent attempt at the launcher:

root@DietPi:~# dietpi-launcher
-bash: /boot/dietpi/dietpi-launcher: /bin/bash: bad interpreter: No such file or directory
root@DietPi:~#

I still have access through my PuTTY session, but fear a reboot will stop that from working.

How can I recover from this disaster ? My three backups generated from the backup utility are on the same SSD as the OS.

Please help me fix this dead computer setup

Joulinar · 16 June 2024 10:02

can you share kernel error messages

dmesg -l 0,1,2,3

gjwAudio · 16 June 2024 15:53

Thanks for your reply. Unfortunately, the system is compromised beyond running any commands.

I think that somehow the v9.3.0 backup was incomplete (2.5GB versus 5.5GB for v9.4.2 & v9.5.1), even though it was generated by the backup utility.

Today I have “imagined” this strategy to restore to a working setup (containing all the tweaks & optimizations that were in force before the failed restore). Please advise if it has any flaws/pitfalls.

reinstall video card to be able to work locally on affected machine (usually it is run “headless” via SSH)
create Linux Live USB with persistent storage option
create fresh Dietpi installation USB
boot Linux Live and copy v9.4.2 & v9.5.1 filesets from OS drive to persistent USB storage
boot Dietpi USB, install fresh version
boot fresh dietpi OS and run dietpi-backup to restore from v9.4.2 on USB

Presumably - since the backup is a full copy of the installation at the time it was created - the tweaks & optimizations will return with the next reboot.

Thanks for help and guidance.

gjwAudio · 17 June 2024 00:57

Arrrrrgh… total crash & burn I have not the linux-skill to get my previous backups to cooperate with a fresh install of Dietpi.

“Invested” the whole day in the attempt, now only reasonable alternative is to start again from scratch.

Failure = greater learning, so I hope to learn something new and useful from all this pain

Thanks anyway for playing…

Joulinar · 17 June 2024 14:51

what SBC you are running on? Theoretically you could boot of a DietPi SD card or USB pen drive and copy all your backup data from old SSD to a separate drive. Important to copy whole backup folder including all file system permissions and symbolic links to an ext4 formatted device. This should allow to restore the backup during initial setup or right after system has been finished setup

gjwAudio · 18 June 2024 00:18

Hi @Joulinar, I appreciate you trying to help.

Well, it’s a bit extreme - kind of Frankenstein affair. Asus Crosshair VII Hero gaming mobo with AMD Ryzen 7 5700X cpu… no SD cards used here

…all in the service of beautiful music playback !

As noted above, I tried copying the backup(s) from original 32G OS drive to USB (ext4 formatted by Linux Mint Live) but wasn’t confident in the result. Switched strategy.

I installed a fresh DietPi instance to another 128GB SSD, then rebooted, mounted the 32GB drive and attempted to run the restore, pointing to the 32GB backup folder. It failed with this message

Restore failed:

/mnt/oldBlitz/dietpi-backup/.dietpi-backup_stats does not exist

Have you created a backup?

…however, I can see that file does exist by looking at drive contents via WinSCP.

And… just to have something of a placeholder, I did run a backup of the base install (stored in the default location on the 128GB drive). Seems to have no impact on the process.

How can I hack this into running successfully ?

Jappe · 18 June 2024 05:10

You need to select the correct path, you tried
/mnt/oldBlitz/dietpi-backup/ but the screenshot shows /mnt/oldBlitz/dietpi-backup/v9.4.2.

gjwAudio · 21 June 2024 02:27

@Jappe - thanks for chiming in. Yes… and no. You are correct about the mismatch between screenshot path and the path seen in the failure message.

However, I tried specifying all three path combinations…

/mnt/oldBlitz/dietpi-backup
/mnt/oldBlitz/dietpi-backup/v9.4.2
/mnt/oldBlitz/dietpi-backup/v9.5.1

and not one of them would run a restore operation. Each gave the error noted above.

I expected to be presented a list of versions to choose from, when setting the backup path to /mnt/oldBlitz/dietpi-backup , since you can clearly see more than one folder in that path. No luck.

Perhaps the backup procedure is not as flexible as I’m assuming, and has some fixed pathnames that cannot be altered through the DietPi-Backup interface.

Any suggestions on how to “twist its arm” into running with my v9.4.2 backup ?
Thanks

Jappe · 21 June 2024 05:06

So did you make the first backup on version 9.4.2 and made the next (which builds on the first one) in another path (9.5.1) or did you make another full backup?
Maybe you can use the imager to make full images and not incremental backups, if this is an option for you.

Joulinar · 21 June 2024 06:33

dietpi-backup > Location > Search

It’s quite flexible and a backup can be done to any path on disk or even on NFS.

gjwAudio · 22 June 2024 16:24

@Jappe: to my knowledge, every backup I made was a full image, not incremental. However, I think I unwittingly corrupted the 9.3.0 image by running dietpi-cleanup before making the backup. I base this on the much smaller 9.3.0 folder size (2.5GB) versus the size of 9.4.2 & 9.5.1 (5.5GB)

Of course, 9.3.0 is the one I really wish I could roll back to, because of the particular “sound” it gave to the music playback. A vanilla install of 9.3.0 would lack the many tweaks & optimizations done a year ago when I began with DietPi, so obviously hope of restoring continues…

gjwAudio · 22 June 2024 17:06

@Joulinar: thank you for correcting me on how to select from the group of backups I’m working with.

I chose the 9.4.2 version and received this warning:

OK, I launched the restore and it ran without incident, recommending a reboot (of course).

Unfortunately, the system is now hung with this displayed on the directly-connected monitor:

Can this problem be overcome somehow ?

OR… if I am forced into rebuilding from scratch, can I still download the 9.3.0 version of dietpi install and recreate that environment (and capture a valid working backup) before upgrading to 9.4.2 and beyond ?

gjwAudio · 24 June 2024 05:09

Hi Folks - between Saturday & Sunday the “rescue” has gone from bad, to worse, to major disaster. Abandoning the attempts to restore from 9.4.2 backup, I ran a fresh DietPi install over the 128GB drive and started from scratch.

The “recipe” for this wondrous music playback calls for compiling a very specific kernel from kernel.org (https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.19.17.tar.gz). The original “author” of this concoction specified many obscure settings to make in the config before compiling - no fun to redo when the compile job fails, and needs to resume from the top.

When I first did this a year ago, the compile went smoothly and I got on with the rest of the tinkering adjustments just fine.

Not so today. For some reason, both apt-utils and bison were missing, thus preventing make menuconfig from running. Installed them both and continued. In case it holds a clue this is from a good run:

root@DietPi:~/linux-5.19.17# make menuconfig
  HOSTCC  scripts/basic/fixdep
  UPD     scripts/kconfig/mconf-cfg
  HOSTCC  scripts/kconfig/mconf.o
  HOSTCC  scripts/kconfig/lxdialog/checklist.o
  HOSTCC  scripts/kconfig/lxdialog/inputbox.o
  HOSTCC  scripts/kconfig/lxdialog/menubox.o
  HOSTCC  scripts/kconfig/lxdialog/textbox.o
  HOSTCC  scripts/kconfig/lxdialog/util.o
  HOSTCC  scripts/kconfig/lxdialog/yesno.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/mconf
#
# using defaults found in /boot/config-6.1.0-21-amd64
#
.config:9250:warning: symbol value 'm' invalid for ANDROID_BINDER_IPC

*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

Then came the compile failures - two times now. First fatal error: #include <openssl/opensslv.h> was not found. Installed that with apt-get install libssl-dev and began again.

Next, two fatal errors in one go, as seen here:

root@DietPi:~/linux-5.19.17# make
  SYNC    include/config/auto.conf.cmd
  HOSTCC  scripts/kconfig/conf.o
  HOSTLD  scripts/kconfig/conf
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_x32.h
  HYPERCALLS arch/x86/include/generated/asm/xen-hypercalls.h
  HOSTCC  arch/x86/tools/relocs_32.o
  HOSTCC  arch/x86/tools/relocs_64.o
  HOSTCC  arch/x86/tools/relocs_common.o
  HOSTLD  arch/x86/tools/relocs
  HOSTCC  scripts/genksyms/genksyms.o
  YACC    scripts/genksyms/parse.tab.[ch]
  HOSTCC  scripts/genksyms/parse.tab.o
  LEX     scripts/genksyms/lex.lex.c
  HOSTCC  scripts/genksyms/lex.lex.o
  HOSTLD  scripts/genksyms/genksyms
  HOSTCC  scripts/selinux/genheaders/genheaders
  HOSTCC  scripts/selinux/mdp/mdp
  HOSTCC  scripts/bin2c
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/sorttable
  HOSTCC  scripts/asn1_compiler
  HOSTCC  scripts/sign-file
  WRAP    arch/x86/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/x86/include/generated/uapi/asm/errno.h
  WRAP    arch/x86/include/generated/uapi/asm/fcntl.h
  WRAP    arch/x86/include/generated/uapi/asm/ioctl.h
  WRAP    arch/x86/include/generated/uapi/asm/ioctls.h
  WRAP    arch/x86/include/generated/uapi/asm/ipcbuf.h
  WRAP    arch/x86/include/generated/uapi/asm/param.h
  WRAP    arch/x86/include/generated/uapi/asm/poll.h
  WRAP    arch/x86/include/generated/uapi/asm/resource.h
  WRAP    arch/x86/include/generated/uapi/asm/socket.h
  WRAP    arch/x86/include/generated/uapi/asm/sockios.h
  WRAP    arch/x86/include/generated/uapi/asm/termbits.h
  WRAP    arch/x86/include/generated/uapi/asm/termios.h
  WRAP    arch/x86/include/generated/uapi/asm/types.h
  WRAP    arch/x86/include/generated/asm/early_ioremap.h
  WRAP    arch/x86/include/generated/asm/export.h
  WRAP    arch/x86/include/generated/asm/mcs_spinlock.h
  WRAP    arch/x86/include/generated/asm/irq_regs.h
  WRAP    arch/x86/include/generated/asm/kmap_size.h
  WRAP    arch/x86/include/generated/asm/local64.h
  WRAP    arch/x86/include/generated/asm/mmiowb.h
  WRAP    arch/x86/include/generated/asm/module.lds.h
  WRAP    arch/x86/include/generated/asm/platform-feature.h
  WRAP    arch/x86/include/generated/asm/rwonce.h
  WRAP    arch/x86/include/generated/asm/unaligned.h
  UPD     include/config/kernel.release
  UPD     include/generated/uapi/linux/version.h
  UPD     include/generated/utsrelease.h
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC      scripts/mod/devicetable-offsets.s
  UPD     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  CC      kernel/bounds.s
  UPD     include/generated/bounds.h
  UPD     include/generated/timeconst.h
  CC      arch/x86/kernel/asm-offsets.s
  UPD     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CALL    scripts/atomic/check-atomics.sh
  DESCEND objtool
<stdin>:1:10: fatal error: libelf.h: No such file or directory
compilation terminated.
  HOSTCC  /root/linux-5.19.17/tools/objtool/fixdep.o
  HOSTLD  /root/linux-5.19.17/tools/objtool/fixdep-in.o
  LINK    /root/linux-5.19.17/tools/objtool/fixdep
  CC      /root/linux-5.19.17/tools/objtool/exec-cmd.o
  CC      /root/linux-5.19.17/tools/objtool/help.o
  CC      /root/linux-5.19.17/tools/objtool/pager.o
  CC      /root/linux-5.19.17/tools/objtool/parse-options.o
  CC      /root/linux-5.19.17/tools/objtool/run-command.o
  CC      /root/linux-5.19.17/tools/objtool/sigchain.o
  CC      /root/linux-5.19.17/tools/objtool/subcmd-config.o
  LD      /root/linux-5.19.17/tools/objtool/libsubcmd-in.o
  AR      /root/linux-5.19.17/tools/objtool/libsubcmd.a
  CC      /root/linux-5.19.17/tools/objtool/arch/x86/special.o
In file included from /root/linux-5.19.17/tools/objtool/include/objtool/objtool.h:13,
                 from /root/linux-5.19.17/tools/objtool/include/objtool/arch.h:11,
                 from /root/linux-5.19.17/tools/objtool/include/objtool/check.h:11,
                 from /root/linux-5.19.17/tools/objtool/include/objtool/special.h:10,
                 from arch/x86/special.c:4:
/root/linux-5.19.17/tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
   10 | #include <gelf.h>
      |          ^~~~~~~~
compilation terminated.
make[4]: *** [/root/linux-5.19.17/tools/build/Makefile.build:97: /root/linux-5.19.17/tools/objtool/arch/x86/special.o] Error 1
make[3]: *** [/root/linux-5.19.17/tools/build/Makefile.build:139: arch/x86] Error 2
make[2]: *** [Makefile:54: /root/linux-5.19.17/tools/objtool/objtool-in.o] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1347: tools/objtool] Error 2
root@DietPi:~/linux-5.19.17#

Gee, could this be happening because this kernel (linux-5.19.17) is so “old” ? If it won’t compile now, I’m back to restoring my backup. Please help the noobie help himself .
Thank you.

Joulinar · 24 June 2024 06:04

you would need to install missing libraries for gelf.h

apt install libelf-dev

gjwAudio · 24 June 2024 13:47

you would need to install missing libraries…

Yes indeed. Is there a way to scan for missing elements before running/crashing the compile job, instead of finding them onesey-twosey and suffering 45 minutes of cleanup & reset before attempting the compile again ?

Of course, I’m curious why these files are absent today, when they were all available for the original compile a year ago. Hmmm… ?

From the RESTORE my backup approach, can I remedy the

/sbin/init: No such file or directory

failure (shown above), and get my system running that way ?

Thank you Joulinar, for answers and insights.

UPDATE: compile is now running - I hope Music is just around the corner.

Joulinar · 24 June 2024 15:38

I’m not aware on how to do so. Usually application developer list their dependency on their own web site.

gjwAudio · 24 June 2024 18:11

Well… no Music yet. After a very long time running, the compile ended this way:

  AS      arch/x86/lib/hweight.o
  CC      arch/x86/lib/iomem.o
  AS      arch/x86/lib/iomap_copy_64.o
arch/x86/lib/iomap_copy_64.S: Assembler messages:
arch/x86/lib/iomap_copy_64.S:13: Warning: found `movsd'; assuming `movsl' was meant
  AR      arch/x86/lib/built-in.a
  GEN     .version
  CHK     include/generated/compile.h
  LD      vmlinux.o
  MODPOST vmlinux.symvers
  MODINFO modules.builtin.modinfo
  GEN     modules.builtin
  CC      .vmlinux.export.o
BTF: .tmp_vmlinux.btf: pahole (pahole) is not available
Failed to generate BTF for vmlinux
Try to disable CONFIG_DEBUG_INFO_BTF
make: *** [Makefile:1168: vmlinux] Error 1
root@DietPi:~/linux-5.19.17# make modules_install
sed: can't read modules.order: No such file or directory
make: *** [Makefile:1477: __modinst_pre] Error 2
root@DietPi:~/linux-5.19.17#

No clue how to remedy this one… (or is it two).

UPDATE: two steps forward, one step back…

apt-get -y install pahole

got my compile to complete without incident. Then these to finish up and make the “new” kernel active…

make modules_install
make install
update-grub
reboot

and when the system comes up, uname -r should display the “new” kernal ID, but it does not - instead showing the 6.1.0-21-amd64 of the original DietPi installation.

What am I missing to get this other kernel to be active ?

Thanks.