DietPi Sync Request

I know there is an option to perform a Dry Run, but when performing an actual sync would it be possible to first perform a quick comparison between sync source and sync destination and then have a simple prompt stating the following:

"Number of files to be deleted: X
Number of files to be copied: X

Do you wish to continue? y/n"

It would be nice to have a simple comparison with a y/n prompt before the sync/backup instead it having to do a full fledged Dry Run.

DietPi-Sync already performs a dry run before the actual sync to check sufficient disk space. But the actual sync is done automatically if disk space test succeeds.

However we could indeed merge both:

  • Only offer sync, but add info that a dry run with dedicated question to go on is done.
  • Do dry run, show log to user and allow to continue or cancel. But this would be a full log. Perhaps we could add a summary as you suggested to the top of the shown log. But exactly those numbers are already printed at the end of the log by rsync.
  • However running the script via input dietpi-sync 1 of course would still automate the sync and skip the user prompt, instead exit automatically with error, if free space check did not succeed.

I like the idea of merging them both. If space is deemed sufficient, the user prompt could be something like:

"Number of files to be deleted: X
Number of files to be copied: X

Do you wish to continue? "

If the user chooses to View Log, only then would DietPi-Sync run a full log (I’m assuming this would take a little more time than a simple files to be copied and files to be deleted comparison).

The log is generated anyway. We use it to determine required disk space. Showing it does not take any recognisable time, but yeah perhaps distracts user, so only sum up the most important info by default.

Sounds good. It would help people like me who are sometimes too lazy to do a dry run from accidentally deleting the destination drive. (Last week my source drive wasn’t mounted properly like I thought. DietPi-Sync thought it was empty and deleted all 7 terabytes of the destination drive. After I figured out what happened I mounted the source drive and did a sync, which took nearly two days to complete).

:open_mouth:

Ui jep. Perhaps we should also add a check, if the source dir is empty, which is in very most cases not what users want to sync :open_mouth:.

Excellent idea.

Should I add this to FeatHub?

Not required. I will add this with v6.22:

  • Empty source check
  • Prompt info after dry run before real sync with rough summary and options to show full sync log, cancel or continue with real sync.
  • Remove dedicated dry run option instead.
  • When calling automated sync with cmd argument, do real sync directly, in case dry run succeeded and drive space + empty source checks passed.

https://github.com/MichaIng/DietPi/issues/2602

midnightwatcher
I finished the request, including some other minor fixes and enhancements. You can test the new script if you want:

wget https://raw.githubusercontent.com/MichaIng/DietPi/dev/dietpi/dietpi-sync -O /DietPi/dietpi/dietpi-sync
chmod +x /DietPi/dietpi/dietpi-sync

The changes do not depend on other changes made with v6.22 or even this PR, so you can test this individually. If you face any error or quirks, please report back.

You can always revert to current stable version:

wget https://raw.githubusercontent.com/MichaIng/DietPi/master/dietpi/dietpi-sync -O /DietPi/dietpi/dietpi-sync
chmod +x /DietPi/dietpi/dietpi-sync

Thank you! This is a very good improvement over the previous version imo. If I were to make any suggestions for future tweaks, I’d recommend the following:

  • Simplify the summary to only present the user with the number of regular files to be transferred (with file size in MB or GB for simplicity) and deleted (with file size in MB or GB). Any additional details the user may require can be viewed in the log.

  • If “Continue with real sync” is selected it would be nice to see the filename(s) being transferred (I recall this info was presented a few versions prior). Maybe have it as an On/Off setting in the DietPi-Sync Options?

  • Include a Restore option from Target Location to Source Location in the event of a source drive failure or accidental deletion of data.

Keep up the great work!

midnightwatcher
Thanks for your suggestions.

About the summary:
I guess some users would like to see more details, others less. Currently we simply paste the related lines from the command output itself without any special formatting. This needs to be in bytes, as this is required for the free space check we do based on it. So to keep it simply and as this is a question of personal taste, I will leave it as is.

About output verbosity:
Yeah in the past we showed the full output and skip the log, that now contains this. This was due to other user requests to not fill the screen with the detailed file-by-file log. I already thought the same that time that a verbose on/off switch would be nice, to either show the full command output directly and skip the log prompt instead, or have it as now. Will see if I can implement this as well, but most likely with v6.23 then since we want to release v6.22 soon.

About restore option
Currently restoring can be done simply by switching source and target dir. A larger plan already in the pipe is, to merge dietpi-backup and dietpi-sync into one script, since they basically do the same. This will include the merge of features both offer, including the restore option from dietpi-backup. I just have not yet a clue whether to name the new script dietpi-sync or dietpi-backup or another new name. It would be a general sync script that allows multiple sync slots, including a pre-defined one for system backups. So “sync” actually fit’s better. On the other hand dietpi-backup is the more prominent/better known one currently and properly what most users seek for. At least an alias needs to stay :wink:.

Could the verbosity option be added to the 6.24 release?

Let me see what I can do with v6.25. But there are some other requests in the first line.