Help - DietPi Completely FUBAR'd?

Troubleshooting
1

I noticed this evening that one of my external drives was suddenly no longer mounted properly for some reason, almost as though something changed fstab (this happened last week right around the exact same time). Tried updating fstab to mount the attached drive properly using its UUID to /mnt/KODI_Media, but now a reboot is giving me this type of login:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
-bash: /DietPi/dietpi/func/dietpi-globals: No such file or directory
-bash: /DietPi/dietpi/login: No such file or directory
grep: /DietPi/dietpi.txt: No such file or directory
root@DietPi:~#

Other than the “root@dietpi” it almost looks as though DietPi isn’t even installed. I can’t run DietPi config or launcher or anything. Any ideas? Here is my fstab:

#-----------------------------------------------------------
#NETWORK
#-----------------------------------------------------------
#Please use DietPi-Drive_Manager to setup network mounts



#-----------------------------------------------------------
#TMPFS
#-----------------------------------------------------------
tmpfs /tmp tmpfs defaults,size=1023M,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /DietPi tmpfs defaults,size=10m,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /var/log tmpfs defaults,size=50m,noatime,nodev,nosuid,mode=1777 0 0
#-----------------------------------------------------------
#MISC (bind)
#-----------------------------------------------------------

#-----------------------------------------------------------
#SWAPFILE
#-----------------------------------------------------------
/var/swap    none    swap    sw    0   0

#-----------------------------------------------------------
#PHYSICAL DRIVES
#-----------------------------------------------------------
UUID=aa010bdf-3471-4ca4-ab17-d1488d382949 /mnt/KODI_Media auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
UUID=bc2a05cc-caa2-43cb-ae5c-9f2e840fc9cb /mnt/KODI_Backup auto defaults,noatime,rw,nofail,x-systemd.automount 0 0  
#UUID=CA7A-8CED /mnt/CA7A-8CED auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
#UUID=0dca34ad-cbea-40cc-9378-19326ee894f0 /mnt/0dca34ad-cbea-40cc-9378-19326ee894f0 auto defaults,noatime,rw,nofail

Using Native PC x86_64.

2

midnightwatcher
Thanks for your report.

Does your fstab really look like this? Because some linebreaks are missing. It should be:

#-----------------------------------------------------------
#NETWORK
#-----------------------------------------------------------
#Please use DietPi-Drive_Manager to setup network mounts



#-----------------------------------------------------------
#TMPFS
#-----------------------------------------------------------            
tmpfs /tmp tmpfs defaults,size=1023M,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /DietPi tmpfs defaults,size=10m,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /var/log tmpfs defaults,size=50m,noatime,nodev,nosuid,mode=1777 0 0
#-----------------------------------------------------------
#MISC (bind)
#-----------------------------------------------------------            
#-----------------------------------------------------------            
#SWAPFILE
#-----------------------------------------------------------
/var/swap    none    swap    sw    0   0

#-----------------------------------------------------------
#PHYSICAL DRIVES
#-----------------------------------------------------------
UUID=aa010bdf-3471-4ca4-ab17-d1488d382949 /mnt/KODI_Media auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
UUID=bc2a05cc-caa2-43cb-ae5c-9f2e840fc9cb /mnt/KODI_Backup auto defaults,noatime,rw,nofail,x-systemd.automount 0 0  
#UUID=CA7A-8CED /mnt/CA7A-8CED auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
#UUID=0dca34ad-cbea-40cc-9378-19326ee894f0 /mnt/0dca34ad-cbea-40cc-9378-19326ee894f0 auto defaults,noatime,rw,nofail

At least something must have gone wrong with it, since the /DietPi has not been mounted properly or dietpi-ramdisk.service has failed.

So please check

df # If tmpfs /DietPi mount is available
systemctl status dietpi-ramdisk # RAMdisk service status
cat /var/tmp/dietpi/logs/dietpi-ramdisk.log # RAMdisk logs
3

I updated my original post above to fix the bad formatting when I copied/pasted fstab.

Here’s what I get when I run the commands you suggested:

root@DietPi:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              977896       0    977896   0% /dev
tmpfs             198016    8556    189460   5% /run
/dev/mmcblk0p2  29254192 6147188  23090620  22% /
tmpfs             990076       0    990076   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             990076       0    990076   0% /sys/fs/cgroup
tmpfs              51200       8     51192   1% /var/log
tmpfs            1047552       0   1047552   0% /tmp
tmpfs              10240       0     10240   0% /DietPi
tmpfs             198012       0    198012   0% /run/user/0
root@DietPi:~# systemctl status dietpi-ramdisk
● dietpi-ramdisk.service - DietPi-RAMdisk
   Loaded: loaded (/etc/systemd/system/dietpi-ramdisk.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2019-02-25 06
:42:38 GMT; 8h ago
  Process: 357 ExecStart=/bin/bash -c /boot/dietpi/func/dietpi-ramdisk 0 &>> /var/tmp/dietpi/logs/dietpi-ramdisk.log (code=exited, statu
s=1/FAILURE)
  Process: 344 ExecStartPre=/bin/mkdir -p /var/tmp/dietpi/logs (code=exited, status=0/SUCCESS)

Feb 25 06:42:38 DietPi systemd[1]: Starting DietPi-RAMdisk...
Feb 25 06:42:38 DietPi bash[357]: /bin/bash: /var/tmp/dietpi/logs/dietpi-ramdisk.log: Read-only file system
Feb 25 06:42:38 DietPi systemd[1]: dietpi-ramdisk.service: Contr
ol process exited, code=exited status=1
Feb 25 06:42:38 DietPi systemd[1]: Failed to start DietPi-RAMdis
k.
Feb 25 06:42:38 DietPi systemd[1]: dietpi-ramdisk.service: Unit
entered failed state.
Feb 25 06:42:38 DietPi systemd[1]: dietpi-ramdisk.service: Faile
d with result 'exit-code'.
root@DietPi:~# cat /var/tmp/dietpi/logs/dietpi-ramdisk.log
Mon 25 Feb 05:39:53 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 05:39:54 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 05:42:01 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:01 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 05:42:01 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:01:01 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 06:01:01 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:01:01 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 06:01:02 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:22:23 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 06:22:23 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:22:23 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 06:22:24 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:33:45 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 06:33:45 GMT 2019 | DietPi-RAMdisk: Completed
root@DietPi:~#
4

Not sure why the problems all of a sudden. The only real change I did this past month (aside from performing an update) that I recall was installing Webmin to set up an NFS server. All seemed well until a week ago when one of my USB drives was no longer mounted, as though something changed the setting in fstab. I updated fstab to mount it again and all seemed well until exactly one week later. When I tried to mount it again, the above is what happened. I’d like to find out the root cause so that I can fix it and make sure it doesn’t happen again.

5

Ok, so I did a completely new reinstall (6.21.1 version), set up the USB drives in fstab correctly, set up the FTP and NFS server. Webmin not installed. All was well. After about an hour of watching a movie, it stopped. I checked the FTP server, and it’s now empty. I go back to look at fstab and, sure enough, something removed the UUID of my drive. Wasn’t mounted.

Any ideas what’s doing this?? How can I find out??

Fortunately I made a backup of fstab and it works again. This time I made it Read Only.

There has to be a serious bug somewhere or a setting that needs to be adjusted, but I’ve no idea where to look.

6

midnightwatcher
Very strange. You could monitor that file for processes that access it

apt install auditd
auditctl -w /etc/fstab -p wa

Source: https://unix.stackexchange.com/a/99091

7

Thanks, I’ve installed auditd. Will the log still record results even though I’ve made fstab read only, or is it only recorded if actual changes are made?

8

I think it only records changes, but not 100% sure. On DietPi in fstab you will find the noatime flag on all drives/mounts. This means that no read access time is recorded (for performance and I/O reasons) but only write access/modify times. There are other ways to check read access, especially when it’s also about WHO accesses, but as said not sure how it does this.

Check out the auditctl man in case: https://linux.die.net/man/8/auditctl

To really debug things, I guess making it R/W-able again (backup first) is best.