Automate dietpi-backup

johnvick · 19 May 2018 01:14

Nice one MichaIng I’ll adopt your method.

Thanks, John

johnvick · 19 May 2018 05:06

I get this error - any ideas? MichaIng

I’m not very good at scripts or coding!

root@Odroid:/home/john/scripts# ./backup.sh
./backup.sh: line 5: syntax error near `;'
./backup.sh: line 5: `  for ((i=$MAX_BACKUPS-1;i>1;i--); do'

MichaIng · 19 May 2018 08:53

johnvick
Forgot the second closing parenthesis: ((…i–));

Also mv inside same folder should really just rename files, but to assure you could add time output around the loop to check if it really just takes a second or so:
echo $(date +’%H:%M:%S’)

Would be nice to keep timestamps inside the file names actually for better archiving/review, but then there is a safe way needed to assure that really the oldest file is removed. Will try around a bid .

I opened issue on github, would like to have this implemented into DietPi-Backup: https://github.com/Fourdee/DietPi/issues/1785

johnvick · 19 May 2018 18:26

All working well thanks again MichaIng

unixmit · 10 July 2018 15:42

I am trying and failing with this. I want a cronjob to run a script that runs the backup automatically.

In my script I have:

#!/bin/bash

/DietPi/dietpi/dietpi-backup 1

In crontab I have:

* 3  *   *   *     sh /home/unix/dietpi-backup.sh

It never runs the backup utility. The script runs as I have set it to create a log file when it runs. If I run the script manually it works too.

Can anyone see what I’m missing?
Thank you

MichaIng · 10 July 2018 15:56

unixmit
Jep, somehow cron does not allow tput anymore. Fix: https://github.com/Fourdee/DietPi/commit/82ac7b32d32dca9db4fdb824c7ead80174844090

brightwolf · 10 February 2019 13:58

Thanks for the script. However, when I run it as published here, I end up with a ‘dietpi-backup-’ directory, i.e. a dietpi backup without a number. If you’re a Linux scripting beginner like me, this could cause some headaches. I solved it by isolating the new backup number calculation on a seperate line. Here’s my version of the script:

MAX_BACKUPS=10
cd /mnt/usbstick
if [[ -d dietpi-backup ]]; then
  rm dietpi-backup-$MAX_BACKUPS
  for ((i=$MAX_BACKUPS-1;i>1;i--)); do
    j=$((i+1))
    mv dietpi-backup-$i dietpi-backup-$j
  done
  mv dietpi-backup dietpi-backup-2
fi
/DietPi/dietpi/dietpi-backup 1

MichaIng · 10 February 2019 15:48

rm dietpi-backup-$MAX_BACKUPS does not work for directories. This requires the -R command. As well to reduce disk writes, I would use the oldest backup as a start point for the new one, so incremental sync can avoid unnecessary writes.
As well check for existing dirs, before moving them, to avoid error messages.

MAX_BACKUPS=10
cd /mnt/usbstick
[[ -d dietpi-backup-$MAX_BACKUPS ]] && mv dietpi-backup-$MAX_BACKUPS dietpi-backup-tmp
if [[ -d dietpi-backup ]]; then
  for ((i=$MAX_BACKUPS-1;i>1;i--)); do
    [[ -d dietpi-backup-$i ]] && mv dietpi-backup-$i dietpi-backup-$((i+1))
  done
  mv dietpi-backup dietpi-backup-2
fi
[[ -d dietpi-backup-tmp ]] && mv dietpi-backup-tmp dietpi-backup
/boot/dietpi/dietpi-backup 1

brightwolf

Ah I see, but the result will be exactly the same?

    j=$((i+1))
    mv dietpi-backup-$i dietpi-backup-$j

and

    mv dietpi-backup-$i dietpi-backup-$((i+1))

leads to the same backup dir structure.

Currently /path/to/dietpi-backup (without appending number) is required to allow the dietpi-backup script restoring this backup. You could tweak the naming a bid, e.g. starting the rotated backup dirs with dietpi-backup-1 or dietpi-backup.1, but for easy rotation a number is required and as said the initial dir without number as well.

I see if I can add this natively with v6.22 to have a more consistent/user friendly solution and find a way to use time/date format instead of numbers. So e.g. every backup will have a time/data appendix and when recovering you can choose from which date to recover. It just causes me headache when thinking about how to reliably find the oldest date .

noobian · 24 March 2019 08:33

Hi

I’m running the “Native PC for UEFI”-DietPi on a Z83-F & love it.

Since I am new to Linux, I don’t know where to save or call the .sh/script mentioned in this thread

I would need to generate versioned (named by timestamp?) backups in /mnt/mybackuplocation
… so I could return to different states of the DietPi. Instead of re-installing the whole system over and over again.
But whenever I run “dietpi-backup 1” my existing backup just gets overwritten.

An automated system-backup would be great (like a timeshift for DietPi), but I even would call it manually, mainly the versioned backups would be helpfull. Any tip on how to achieve this?

MichaIng · 25 March 2019 03:35

noobian
The below will automatically create a backup every day:

cat << '_EOF_' > /etc/cron.daily/backup
#!/bin/bash
MAX_BACKUPS=10
cd /mnt/mybackuplocation
[[ -d dietpi-backup-$MAX_BACKUPS ]] && mv dietpi-backup-$MAX_BACKUPS dietpi-backup-tmp
if [[ -d dietpi-backup ]]; then
  for ((i=$MAX_BACKUPS-1;i>1;i--)); do
    [[ -d dietpi-backup-$i ]] && mv dietpi-backup-$i dietpi-backup-$((i+1))
  done
  mv dietpi-backup dietpi-backup-2
fi
[[ -d dietpi-backup-tmp ]] && mv dietpi-backup-tmp dietpi-backup
/boot/dietpi/dietpi-backup 1
_EOF_
chmod +x /etc/cron.daily/backup

You can adjust the backup location via dietpi-backup directly. Then assure that the parent directory matches the one above: /mnt/mybackuplocation
You can adjust the time (hour + minute) of the daily cron job via dietpi-cron.
And of course adjust MAX_BACKUPS to your needs.
Place the script into /etc/cron.weekly instead if this is regular enough. It would be possible as well to define a more individual schedule via crontab.

noobian · 25 March 2019 11:29

MichaIng

wow … thanks so much!
never heard of “cat << EO” before … nice

is it possible to run a daily & a weekly backupscript next to each other, without them overwriting each other? so I could keep 2 weekly backups, and 7 daily ones?

MichaIng · 28 March 2019 13:05

The one weekly backup would match one of the daily backups but of course it is possible. You could add a weekly cron job that syncs the 7th daily backup to another sub dir and before that the existing weekly backup to another one. So you would have 7 daily backups and two weekly that are up to two weeks older than the last daily. Since cron jobs are executed concurrently we have to think about how to assure the same dir is not synced concurrently. Perhaps add it to the same script and check for a certain day of week.

hemertje · 3 January 2021 13:35

Hi MichaIng

Above is not included yet inside the menu options?

So the above coding is the way to go?

MichaIng · 3 January 2021 15:25

Yes to both, the above should still work fine, now (after a required edit) via copy&paste the whole code block as well, only the “/mnt/mybackuplocation” needs to be adjusted of course, so that you are in the root of the drive where the dietpi-backup directory is stored.

hemertje · 3 January 2021 15:59

I assume that this

chmod +x /etc/cron.daily/backup

needs to be outside the script / code tags and has to be done after editing and saving the backup script?

hemertje · 3 January 2021 16:06

Thx!

Some things are important to me:

An exact copy of my Data / Files to a 2nd NAS
Encryption isn’t necessary so I can browse it directly on the 2nd NAS
(For me that is a Synology system at a physical different location in case something happens to the 1st NAS (in any form: Data corruption, Virus, Burglary, Fire, Water damage etc.)
An exact copy of my (Nextcloud) Database to a 2nd NAS, Will this script also take care of the Nextcloud database ans not only the files?

-An exact copy of the Operating System (OS) to an USB-drive and Harddisk.
(In case the USB-drive dies before the Harddisk does and you don’t have a working Backup anymore)

I’ll have to create a script for the Data / Files / Database to a 2nd NAS ?
I’ll have to create two scripts for the backup of the DietPi OS to an USB-drive and Harddisk?

MichaIng · 3 January 2021 16:20

/mnt/dietpi_userdata is only included when it’s on the same drive. If you moved it to an external drive, it’s not included and you need to create a separate script for that. You could use dietpi-sync and a similar approach for multiple backup rotation.

But for the database, a raw file backup is not what I would do, as it can only be reliably restored to the exact same MariaDB version then, theoretically a newer one as well, but I would not count on that. Instead do a regular database SQL dump, so that you can import it on any SQL database server and even browse/edit the SQL commands, if required.

I use this as daily cron job:

mysqldump --log-error=/var/log/nextcloud-db-backup.log nextcloud > /mnt/sda/ncdata/database-backups/$(date +%Y-%m-%d_%T).sql

Reminds me that I need to add an automated removal of old backups, currently it’s growing until I manually clean them up .

This is then mirrored again to a second backup drive, together with the Nextcloud data, to protect against data loss through drive crash. I know, a RAID tower would be better .

Joulinar · 3 January 2021 17:59

Hi,

no, this is not part of the dietpi-backup functionality. But you can create it yourself.

hemertje · 5 January 2021 21:16

MichaIng

any hints for this removal?

how did you setup the mirrored backup to a second backup drive?

MichaIng · 10 January 2021 15:52

The following should work to keep 10 backups:

until (( $(find /mnt/sda/ncdata/database-backups/ -name '*.sql' | wc -l) < 11 ))
do
rm -v "$(find /mnt/sda/ncdata/database-backups/ -name '*.sql' | sort -n | head -1)"
done

I use simple rsync to backup all Nextcloud data afterwards. Could be dietpi-sync as well, but I wrote that script before using/joining DietPi .

rsync -aH --delete /mnt/sda/ncdata /mnt/sdb/backup/