Dietpi Restore failed, even tried on a new flash of Dietpi

cyy · July 25, 2022, 5:47am

Wanted to restore my dietpi backup, after messing up with some software installation.
Now won’t be able to boot.
Connected to HDMI, but screen output is like a very zoomed-in screen, can only see a few characters.
Then flash dietpi into another SSD and try to restore from there. Still getting this error message.
Thinking that if I can’t save the restore, is there anyway to extract my db out? Am using mariadb.

DietPi version 8.4.2
Distro version Buster 1
Kernel version Linux DietPi 5.15.32-v7l+ #1538 SMP Thu Mar 31 19:39:41 BST 2022 armv7l GNU/Linux
SBC model Rpi4 4GB
Power supply used 5V 3A
SD card used | Samsung 860 Evo 256GB with External Case
Bug report ID 9993944a-5530-4311-bb0a-b250aa5193b6
HW_UUID - ea8cf344-5651-4d5a-bee7-a8d2fb05fd9a

Error Message after scrolling up in terminal

building file list ... done
 19,597,241,294  85%   23.43MB/s    0:13:17 (xfr#233305, to-chk=115773/389974)
deleting usr/share/doc/perl-base/copyright
deleting usr/share/doc/perl-base/changelog.gz
deleting usr/share/doc/perl-base/changelog.Debian.gz
 20,429,220,574  89%   18.51MB/s    0:02:09  xfr#316375, to-chk=11122/389974)
rsync: [sender] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(823) [sender=3.2.3]
rsync error: received SIGUSR1 (code 19) at main.c(1595) [generator=3.2.3]
lsblk: /lib/arm-linux-gnueabihf/libsmartcols.so.1: version `SMARTCOLS_2.34' not found (required by lsblk)
lsblk: /lib/arm-linux-gnueabihf/libsmartcols.so.1: version `SMARTCOLS_2.34' not found (required by lsblk)
[  OK  ] DietPi-Backup | sed -Ei s/(^|[[:blank:]])root=[^[:blank:]]*/\1root=PARTUUID=d5900763-02/ /boot/cmdline.txt
[FAILED] DietPi-Backup: Restore | Exited with error
umount: /lib/arm-linux-gnueabihf/libmount.so.1: version `MOUNT_2_35' not found (required by umount)
rmdir: failed to remove '/mnt/dietpi-backup0': Device or resource busy
mkdir: /lib/arm-linux-gnueabihf/libselinux.so.1: no version information available (required by mkdir)
[ SUB1 ] DietPi-Set_software > passwords
[  OK  ] passwords  | Completed
mkdir: /lib/arm-linux-gnueabihf/libselinux.so.1: no version information available (required by mkdir)
[ SUB1 ] DietPi-Set_hardware > serialconsole (disable)
[ INFO ] DietPi-Set_hardware | Disabling serial-getty on: /dev/serial0
[FAILED] DietPi-Set_hardware | systemctl unmask serial-getty@serial0
mkdir: /lib/arm-linux-gnueabihf/libselinux.so.1: no version information available (required by mkdir)
[ INFO ] DietPi-BugReport | Generating informative command outputs, please wait...
mount: /lib/arm-linux-gnueabihf/libmount.so.1: version `MOUNT_2.34' not found (required by mount)
mount: /lib/arm-linux-gnueabihf/libmount.so.1: version `MOUNT_2_35' not found (required by mount)
ls: /lib/arm-linux-gnueabihf/libselinux.so.1: no version information available (required by ls)
ls: /lib/arm-linux-gnueabihf/libselinux.so.1: no version information available (required by ls)

Joulinar · July 25, 2022, 8:21am

That looks like issues with the attached SSD. How do you power that device? With an own PSU? Or just via the USB port?

Can you check for kernel error messages

dmesg -l err,crit,alert,emerg

cyy · July 25, 2022, 8:35am

Oh SSD to Pi with just a USB. I have tried restoring to two different SSD, it is still having issue.
Been running on the SSD for at least 1 year without any issue.

I have no access to SSH anymore. Connection refused.
If i connect it via HDMI I can only see a very big blinking underscore, as if the resolution is super low, enough to only fit one character at one time.

I’m able to mount the disk, is there anyway I can get the log from there?

cyy · July 25, 2022, 9:31am

2nd run with HDD Case with Y Spitter to PSU

building file list ... done
 19,597,245,390  85%   24.38MB/s    0:12:46 (xfr#233305, to-chk=115773/389974)s
deleting usr/share/doc/perl-base/copyright
deleting usr/share/doc/perl-base/changelog.gz
deleting usr/share/doc/perl-base/changelog.Debian.gz
 20,437,023,454  89%   21.72MB/s    0:01:49  xfr#316375, to-chk=11122/389974)
rsync: [sender] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(823) [sender=3.2.3]
rsync error: received SIGUSR1 (code 19) at main.c(1595) [generator=3.2.3]
lsblk: /lib/arm-linux-gnueabihf/libsmartcols.so.1: version `SMARTCOLS_2.34' not found (required by lsblk)

am prepare to do a reinstallation of everything. Is it possible to extract mariadb .db files?

Joulinar · July 25, 2022, 10:29am

are you able to check for kernel error messages? And from what source the backup is trying to restore?

cyy · July 25, 2022, 10:40am

Negative, I am not able to boot the system to enter the comment. Not even with HDMI. Screen will only show some random characters and underscore blinking that takes up the whole screen for 1 characters. It’s like it’s in some boot loop.
Can only read the file content via USB.
I do have my previous 10 copies of dietpi backup.

For source do you mean:
another external powered 3.5" hdd. in there there is backup which contains “data” to “data10”.

Joulinar · July 25, 2022, 11:04am

it still looks like issues with the external drives while performing the restore causing the process to fail

rsync: [sender] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(823) [sender=3.2.3]
rsync error: received SIGUSR1 (code 19) at main.c(1595) [generator=3.2.3]

Are both disk powered by a PSU?

cyy · July 25, 2022, 11:20am

OS Disk - Y Cable USB, 1 to pi, 1 to 2A plug
Backup Disk - USB directly to Pi, DC Power

New finding…
I managed to restore an older backup.
So possible that latest backup might be corrupted.
Is there anyway I can find out what files is having issue? so maybe I can delete off certain files in diet-pi backup and restore it with the healthy one in other backup?

Another finding is.

Restore to latest backup, it will fail. Then I Ctrl + C
At this point if I do any bash command like “rm”, “sysctl”, “mount” it will give me something like “mount: armxxx.lib.so Version Not Found”. (Will repeat the process to get the exact error message).

Usually when it failed, I realised it is a around 89% for the latest back up.

Afterwards, I try restoring to another backup.
I will get error saying /var/tmp/logs not enough space. This time it failed much faster at 30%+.
Repeat the process again, but this time I change fstab tmp mount size from 50M to 1G.
Will get the same error /var/tmp/logs not enough space, during the backup. Check df, it will show that /var/tmp/logs revert back to 2000+

Joulinar · July 25, 2022, 11:36am

you could try to perform some file system checks on your backup storage. Just unmound the disk and try

fsck -a /dev/xxx

replace xxx with the device name of your backup disk. To find out

lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid

/var/log is specified as tmpfs in /etc/fstab. There you should be able to adjust an remount.

cyy · July 25, 2022, 1:05pm

Will try the check disk again, right still still trying another restore.
So i made 2 backup yesterday
so unfortunately i think both backups are corrupted.

What I have tried. Restore from backup A, once restored, copy lib from backup B “lib” directly. With this I am able to restore some service in “dietpi-services”
i copied with “cp -r -p”
from what I understand r is for recursive, p is for permission.

Am thinking is it possible to compared A and B. and give me another “backup C”

Whatever not in A will get copied to C
Whatever not in B will get copied to C
With the clause, whichever is later
Once that is done can I repeat it with a full working backup to copy whatever is still missing in C?

I think system wise not much changes, it’s the db data that it is important.

cyy · July 25, 2022, 1:16pm

Update, i tried another restore and i got 90% back the services back online.
I think there will be error here and there because I don’t think I am doing it the right way. I think as of now this is sufficient.

Might want to take this chance to migrate to the 64bit as well. So can do a fresh installation.

Thank you so much!

Is there anyway to back up to 2 sources for dietpi backup?

cyy · July 25, 2022, 1:21pm

Have ran the fsck disk, but it ran a bit too fast. completed within 1 second… is this an issue?

root@DietPi:~# fsck -a /dev/sda1
fsck from util-linux 2.33.1
/dev/sda1: clean, 2776271/122101760 files, 387138189/488378368 blocks

MichaIng · July 25, 2022, 3:50pm

The main issue is that you are trying to restore a Buster backup onto a Bullseye system. If this breaks in the middle, there may be libraries and binaries with incompatible versions, leading to such missing library error messages.

To find the broken file(s) did you try an fsck on the backup filesystem?

It is possible to simply copy the /mnt/dietpi_userdata/mysql from the backup onto the new system before installing any software/MariaDB. It will be picked up and migrated automatically. Going this path has the benefit that you are on a clean recent Debian version already, saving you the upgrade when we drop support for Buster sometime next year.

Joulinar · July 25, 2022, 7:58pm

uhhh I totally overlooked that it was a Debian Buster system actually. And yeah, we don’t offer Buster images anymore officially. Of course our new images are Bullseye and the restore will not work. How could I have overlooked this.

Maybe you have the old Buster image flying around to try the backup there.

cyy · July 26, 2022, 6:31am

@MichaIng @Joulinar Oh my, totally missed out that part. Have already dropped the backup, move over to 64bit Bulleye image completely. Managed to back backup and restore the Mariadb with Mariadb Backup. Didn’t know I could just copy directly.

However, now I am having issues with Wireguard.
Previous release I was just using diet pi-software and install Wireguard, follow the guy to generate key and config. It was working.

Tried installing just Wireguard only, copy pasted the config files in /etc/wireguard. I have two profiles for each device.
a. AllowedIPs = 0.0.0.0/0, ::/0 - Forward All connection
b. AllowedIPs = 10.0.0.0/21 - Forward Local only

When I connect using a.
I am able to get a handshake, but there is no connection at all. My phone just can’t browse any site. Tried direct IP 1.1.1.1 as well.

When connected using b.
Total no handshake.

I just systemctl status wg-quick@wg0, it is showing me “Active(Status)” can’t see any error message.

So I thought it could be my old configurations. Uninstalled Wireguard. Install PiVPN (Just saw that this can sort of automate the process). Also think that it might be good to start fresh.
Setup one device with PiVPN, still having the issue, there is handshake but no network.

● wg-quick@wg0.service - WireGuard via wg-quick(8) for wg0
     Loaded: loaded (/lib/systemd/system/wg-quick@.service; enabled; vendor pres
et: enabled)
     Active: active (exited) since Tue 2022-07-26 06:51:10 +08; 7h ag
o
       Docs: man:wg-quick(8)
             man:wg(8)
             https://www.wireguard.com/
             https://www.wireguard.com/quickstart/
             https://git.zx2c4.com/wireguard-tools/about/src/man/wg-quick.8
             https://git.zx2c4.com/wireguard-tools/about/src/man/wg.8
    Process: 16092 ExecReload=/bin/bash -c exec /usr/bin/wg syncconf wg0 <(exec 
/usr/bin/wg-quick strip wg0) (code=exited, status=0/SUCCESS)
   Main PID: 14290 (code=exited, status=0/SUCCESS)
        CPU: 42ms

Jul 26 06:51:10 DietPi systemd[1]: Starting WireGuard via wg-quick(8) for wg0...
Jul 26 06:51:10 DietPi wg-quick[14290]: [#] ip link add wg0 type wireguard
Jul 26 06:51:10 DietPi wg-quick[14290]: [#] wg setconf wg0 /dev/fd/63
Jul 26 06:51:10 DietPi wg-quick[14290]: [#] ip -4 address add 10.8.196.1/24 dev 
wg0
Jul 26 06:51:10 DietPi wg-quick[14290]: [#] ip -6 address add fd11:5ee:bad:c0de:
:1/64 dev wg0

...skipping 1 line
Jul 26 06:51:10 DietPi systemd[1]: Finished WireGuard via wg-quick(8) for wg0.
Jul 26 06:52:21 DietPi systemd[1]: Reloading WireGuard via wg-quick(8) for wg0.
Jul 26 06:52:21 DietPi systemd[1]: Reloaded WireGuard via wg-quick(8) for wg0.

In /etc/wireguard/wg0.conf

root@DietPi:/etc/wireguard# cat wg0.conf 
[Interface]
PrivateKey = OHxx
Address = 10.8.196.1/24,fd11:5ee:bad:c0de::1/64
MTU = 1420
ListenPort = 51820
### begin Phone-All ###
[Peer]
PublicKey = DQxx
PresharedKey = spxx
AllowedIPs = 10.8.196.2/32,fd11:5ee:bad:c0de::2/128
### end Phone-All ###

Phone config

[Interface]
PrivateKey = iDxx
Address = 10.8.196.2/24,fd11:5ee:bad:c0de::2/64
DNS = 9.9.9.9, 149.112.112.112

[Peer]
PublicKey = +/ixx
PresharedKey = spxx
Endpoint = 10.0.0.11:51820
AllowedIPs = 0.0.0.0/0, ::0/0

Have also tried Endpoint instead of putting the domain, I tried using my Local IP to connect to Wireguard locally. There is handshake but still not able to surf.

However one odd thing I realised is on my phone

Latest Handshake: 01:00 minute ago 
Data Sent: 1.81 GiB

Like a bit too much data is being transferred.
Difference I found in the config file is

It doesn’t have the Postup and PostDown Script.
There seems to be IPv6 IP now.

cyy · July 26, 2022, 9:20am

Try disabling IPv6, since previously not using it and not very sure how to deal with it. One thing i realised sometimes it will get a handshake for 1 second and lose it.

:::: PiVPN debug ::::

=============================================

:::: Latest commit ::::

Branch: master

Commit: f8cb945af15a1ca0cf063475c6e1557c6e8da06c

Author: 4s3ti

Date: Fri Jun 10 16:10:57 2022 +0200

Summary: Merge branch 'test'

=============================================

:::: Installation settings ::::

PLAT=Debian

OSCN=bullseye

USING_UFW=0

pivpnforceipv6route=0

IPv4dev=eth0

dhcpReserv=1

IPv4addr=10.0.0.11/24

IPv4gw=10.0.0.1

install_user=dietpi

install_home=/home/dietpi

VPN=wireguard

pivpnPORT=51820

pivpnDNS1=1.1.1.1

pivpnDNS2=1.0.0.1

pivpnHOST=REDACTED

INPUT_CHAIN_EDITED=0

FORWARD_CHAIN_EDITED=1

INPUT_CHAIN_EDITEDv6=

FORWARD_CHAIN_EDITEDv6=

pivpnPROTO=udp

pivpnMTU=1420

pivpnDEV=wg0

pivpnNET=10.12.49.0

subnetClass=24

pivpnenableipv6=0

ALLOWED_IPS="0.0.0.0/0"

INSTALLED_PACKAGES=(dhcpcd5 wireguard-tools)

=============================================

:::: Server configuration shown below ::::

[Interface]

PrivateKey = server_priv

Address = 10.12.49.1/24

MTU = 1420

ListenPort = 51820

### begin Phone-All ###

[Peer]

PublicKey = Phone-All_pub

PresharedKey = Phone-All_psk

AllowedIPs = 10.12.49.2/32

### end Phone-All ###

=============================================

:::: Client configuration shown below ::::

[Interface]

PrivateKey = Phone-All_priv

Address = 10.12.49.2/24

DNS = 1.1.1.1, 1.0.0.1

[Peer]

PublicKey = server_pub

PresharedKey = Phone-All_psk

Endpoint = REDACTED:51820

AllowedIPs = 0.0.0.0/0

=============================================

:::: Recursive list of files in ::::

:::: /etc/wireguard shown below ::::

/etc/wireguard:

configs

keys

wg0.conf

/etc/wireguard/configs:

Phone-All.conf

clients.txt

/etc/wireguard/keys:

Phone-All_priv

Phone-All_psk

Phone-All_pub

server_priv

server_pub

cyy · July 26, 2022, 9:58am

Realised that I am also not able to reach my other services.
I did a restart, fireguard started to work.
Have no issue now thank you so much.

However, when I check

systemctl status wg-quick@wg0.service

Status it is showing active(exited)
should I be concern about the “exited”?

 wg-quick@wg0.service - WireGuard via wg-quick(8) for wg0
     Loaded: loaded (/lib/systemd/system/wg-quick@.service; enabled; vendor preset: enabled)
     Active: active (exited) since Tue 2022-07-26 17:43:34 +08; 13min ago
       Docs: man:wg-quick(8)
             man:wg(8)
             https://www.wireguard.com/
             https://www.wireguard.com/quickstart/
             https://git.zx2c4.com/wireguard-tools/about/src/man/wg-quick.8
             https://git.zx2c4.com/wireguard-tools/about/src/man/wg.8
    Process: 646 ExecStart=/usr/bin/wg-quick up wg0 (code=exited, status=0/SUCCESS)
   Main PID: 646 (code=exited, status=0/SUCCESS)
        CPU: 139ms

Jul 26 17:43:34 DietPi systemd[1]: Starting WireGuard via wg-quick(8) for wg0...
Jul 26 17:43:34 DietPi wg-quick[646]: [#] ip link add wg0 type wireguard
Jul 26 17:43:34 DietPi wg-quick[646]: [#] wg setconf wg0 /dev/fd/63
Jul 26 17:43:34 DietPi wg-quick[646]: [#] ip -4 address add 10.12.49.1/24 dev wg0
Jul 26 17:43:34 DietPi wg-quick[646]: [#] ip link set mtu 1420 up dev wg0
Jul 26 17:43:34 DietPi systemd[1]: Finished WireGuard via wg-quick(8) for wg0.

Joulinar · July 26, 2022, 10:14am

The service doesn’t give that much information. Better to use the wg command to show the available peers and what time they got connected last time.

cyy · July 26, 2022, 10:23am

root@DietPi:~# wg
interface: wg0
  public key: KOxx
  private key: (hidden)
  listening port: 51820

peer: RNxx
  preshared key: (hidden)
  endpoint: 122.xx:28511
  allowed ips: 10.12.49.4/32
  latest handshake: 27 minutes, 47 seconds ago
  transfer: 52.60 MiB received, 352.47 MiB sent

peer: mkxx
  preshared key: (hidden)
  endpoint: 122.xx:51458
  allowed ips: 10.12.49.5/32
  latest handshake: 31 minutes, 19 seconds ago
  transfer: 83.67 KiB received, 313.16 KiB sent

peer: Joxx
  preshared key: (hidden)
  endpoint: 122.xx:37233
  allowed ips: 10.12.49.2/32
  latest handshake: 33 minutes, 36 seconds ago
  transfer: 763.14 KiB received, 14.15 MiB sent

peer: FSxx
  preshared key: (hidden)
  endpoint: 122.xx:54732
  allowed ips: 10.12.49.3/32
  latest handshake: 37 minutes, 12 seconds ago
  transfer: 243.52 KiB received, 10.22 MiB sent

It’s working for all my 4 clients now. Thank you very much!

Joulinar · July 26, 2022, 12:57pm

Uhh I just realised that you shared PublicKey and PrivateKey of your configuration. Something you should never do. It’s like writing your pin for your credit card on it and leave the card on public transport.

I highly recommend to change your entire configuration keys (security reason) to avoid someone will try to get access.