Help - DietPi Completely FUBAR'd?

Having issues with your DietPi installation, or, found a bug? Post it here.
Post Reply
midnightwatcher
Posts: 54
Joined: Mon Dec 26, 2016 7:54 am

Help - DietPi Completely FUBAR'd?

Post by midnightwatcher »

I noticed this evening that one of my external drives was suddenly no longer mounted properly for some reason, almost as though something changed fstab (this happened last week right around the exact same time). Tried updating fstab to mount the attached drive properly using its UUID to /mnt/KODI_Media, but now a reboot is giving me this type of login:

Code: Select all

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
-bash: /DietPi/dietpi/func/dietpi-globals: No such file or directory
-bash: /DietPi/dietpi/login: No such file or directory
grep: /DietPi/dietpi.txt: No such file or directory
root@DietPi:~#
Other than the "root@dietpi" it almost looks as though DietPi isn't even installed. I can't run DietPi config or launcher or anything. Any ideas? Here is my fstab:

Code: Select all

#-----------------------------------------------------------
#NETWORK
#-----------------------------------------------------------
#Please use DietPi-Drive_Manager to setup network mounts



#-----------------------------------------------------------
#TMPFS
#-----------------------------------------------------------
tmpfs /tmp tmpfs defaults,size=1023M,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /DietPi tmpfs defaults,size=10m,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /var/log tmpfs defaults,size=50m,noatime,nodev,nosuid,mode=1777 0 0
#-----------------------------------------------------------
#MISC (bind)
#-----------------------------------------------------------

#-----------------------------------------------------------
#SWAPFILE
#-----------------------------------------------------------
/var/swap    none    swap    sw    0   0

#-----------------------------------------------------------
#PHYSICAL DRIVES
#-----------------------------------------------------------
UUID=aa010bdf-3471-4ca4-ab17-d1488d382949 /mnt/KODI_Media auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
UUID=bc2a05cc-caa2-43cb-ae5c-9f2e840fc9cb /mnt/KODI_Backup auto defaults,noatime,rw,nofail,x-systemd.automount 0 0  
#UUID=CA7A-8CED /mnt/CA7A-8CED auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
#UUID=0dca34ad-cbea-40cc-9378-19326ee894f0 /mnt/0dca34ad-cbea-40cc-9378-19326ee894f0 auto defaults,noatime,rw,nofail
Using Native PC x86_64.
Last edited by midnightwatcher on Mon Feb 25, 2019 7:58 pm, edited 2 times in total.
User avatar
MichaIng
Site Admin
Posts: 2276
Joined: Sat Nov 18, 2017 6:21 pm

Re: Help - DietPi Completely FUBAR'd?

Post by MichaIng »

@midnightwatcher
Thanks for your report.

Does your fstab really look like this? Because some linebreaks are missing. It should be:

Code: Select all

#-----------------------------------------------------------
#NETWORK
#-----------------------------------------------------------
#Please use DietPi-Drive_Manager to setup network mounts



#-----------------------------------------------------------
#TMPFS
#-----------------------------------------------------------            
tmpfs /tmp tmpfs defaults,size=1023M,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /DietPi tmpfs defaults,size=10m,noatime,nodev,nosuid,mode=1777 0 0
tmpfs /var/log tmpfs defaults,size=50m,noatime,nodev,nosuid,mode=1777 0 0
#-----------------------------------------------------------
#MISC (bind)
#-----------------------------------------------------------            
#-----------------------------------------------------------            
#SWAPFILE
#-----------------------------------------------------------
/var/swap    none    swap    sw    0   0

#-----------------------------------------------------------
#PHYSICAL DRIVES
#-----------------------------------------------------------
UUID=aa010bdf-3471-4ca4-ab17-d1488d382949 /mnt/KODI_Media auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
UUID=bc2a05cc-caa2-43cb-ae5c-9f2e840fc9cb /mnt/KODI_Backup auto defaults,noatime,rw,nofail,x-systemd.automount 0 0  
#UUID=CA7A-8CED /mnt/CA7A-8CED auto defaults,noatime,rw,nofail,x-systemd.automount 0 0
#UUID=0dca34ad-cbea-40cc-9378-19326ee894f0 /mnt/0dca34ad-cbea-40cc-9378-19326ee894f0 auto defaults,noatime,rw,nofail
At least something must have gone wrong with it, since the /DietPi has not been mounted properly or dietpi-ramdisk.service has failed.

So please check

Code: Select all

df # If tmpfs /DietPi mount is available
systemctl status dietpi-ramdisk # RAMdisk service status
cat /var/tmp/dietpi/logs/dietpi-ramdisk.log # RAMdisk logs
midnightwatcher
Posts: 54
Joined: Mon Dec 26, 2016 7:54 am

Re: Help - DietPi Completely FUBAR'd?

Post by midnightwatcher »

I updated my original post above to fix the bad formatting when I copied/pasted fstab.

Here's what I get when I run the commands you suggested:

Code: Select all

root@DietPi:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              977896       0    977896   0% /dev
tmpfs             198016    8556    189460   5% /run
/dev/mmcblk0p2  29254192 6147188  23090620  22% /
tmpfs             990076       0    990076   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             990076       0    990076   0% /sys/fs/cgroup
tmpfs              51200       8     51192   1% /var/log
tmpfs            1047552       0   1047552   0% /tmp
tmpfs              10240       0     10240   0% /DietPi
tmpfs             198012       0    198012   0% /run/user/0
root@DietPi:~# systemctl status dietpi-ramdisk
● dietpi-ramdisk.service - DietPi-RAMdisk
   Loaded: loaded (/etc/systemd/system/dietpi-ramdisk.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2019-02-25 06
:42:38 GMT; 8h ago
  Process: 357 ExecStart=/bin/bash -c /boot/dietpi/func/dietpi-ramdisk 0 &>> /var/tmp/dietpi/logs/dietpi-ramdisk.log (code=exited, statu
s=1/FAILURE)
  Process: 344 ExecStartPre=/bin/mkdir -p /var/tmp/dietpi/logs (code=exited, status=0/SUCCESS)

Feb 25 06:42:38 DietPi systemd[1]: Starting DietPi-RAMdisk...
Feb 25 06:42:38 DietPi bash[357]: /bin/bash: /var/tmp/dietpi/logs/dietpi-ramdisk.log: Read-only file system
Feb 25 06:42:38 DietPi systemd[1]: dietpi-ramdisk.service: Contr
ol process exited, code=exited status=1
Feb 25 06:42:38 DietPi systemd[1]: Failed to start DietPi-RAMdis
k.
Feb 25 06:42:38 DietPi systemd[1]: dietpi-ramdisk.service: Unit
entered failed state.
Feb 25 06:42:38 DietPi systemd[1]: dietpi-ramdisk.service: Faile
d with result 'exit-code'.
root@DietPi:~# cat /var/tmp/dietpi/logs/dietpi-ramdisk.log
Mon 25 Feb 05:39:53 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 05:39:54 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:00 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 05:42:01 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 05:42:01 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 05:42:01 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:01:01 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 06:01:01 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:01:01 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 06:01:02 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:22:23 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 06:22:23 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:22:23 GMT 2019 | DietPi-RAMdisk: Starting...
rm: cannot remove '/DietPi': Device or resource busy
Mon 25 Feb 06:22:24 GMT 2019 | DietPi-RAMdisk: Completed
Mon 25 Feb 06:33:45 GMT 2019 | DietPi-RAMdisk: Stopping...
Mon 25 Feb 06:33:45 GMT 2019 | DietPi-RAMdisk: Completed
root@DietPi:~#
midnightwatcher
Posts: 54
Joined: Mon Dec 26, 2016 7:54 am

Re: Help - DietPi Completely FUBAR'd?

Post by midnightwatcher »

Not sure why the problems all of a sudden. The only real change I did this past month (aside from performing an update) that I recall was installing Webmin to set up an NFS server. All seemed well until a week ago when one of my USB drives was no longer mounted, as though something changed the setting in fstab. I updated fstab to mount it again and all seemed well until exactly one week later. When I tried to mount it again, the above is what happened. I'd like to find out the root cause so that I can fix it and make sure it doesn't happen again.
Last edited by midnightwatcher on Tue Feb 26, 2019 6:29 am, edited 1 time in total.
midnightwatcher
Posts: 54
Joined: Mon Dec 26, 2016 7:54 am

Re: Help - DietPi Completely FUBAR'd?

Post by midnightwatcher »

Ok, so I did a completely new reinstall (6.21.1 version), set up the USB drives in fstab correctly, set up the FTP and NFS server. Webmin not installed. All was well. After about an hour of watching a movie, it stopped. I checked the FTP server, and it's now empty. I go back to look at fstab and, sure enough, something removed the UUID of my drive. Wasn't mounted.

Any ideas what's doing this?? How can I find out??

Fortunately I made a backup of fstab and it works again. This time I made it Read Only.

There has to be a serious bug somewhere or a setting that needs to be adjusted, but I've no idea where to look.
User avatar
MichaIng
Site Admin
Posts: 2276
Joined: Sat Nov 18, 2017 6:21 pm

Re: Help - DietPi Completely FUBAR'd?

Post by MichaIng »

@midnightwatcher
Very strange. You could monitor that file for processes that access it

Code: Select all

apt install auditd
auditctl -w /etc/fstab -p wa
Source: https://unix.stackexchange.com/a/99091
midnightwatcher
Posts: 54
Joined: Mon Dec 26, 2016 7:54 am

Re: Help - DietPi Completely FUBAR'd?

Post by midnightwatcher »

Thanks, I've installed auditd. Will the log still record results even though I've made fstab read only, or is it only recorded if actual changes are made?
User avatar
MichaIng
Site Admin
Posts: 2276
Joined: Sat Nov 18, 2017 6:21 pm

Re: Help - DietPi Completely FUBAR'd?

Post by MichaIng »

I think it only records changes, but not 100% sure. On DietPi in fstab you will find the noatime flag on all drives/mounts. This means that no read access time is recorded (for performance and I/O reasons) but only write access/modify times. There are other ways to check read access, especially when it's also about WHO accesses, but as said not sure how it does this.

Check out the auditctl man in case: https://linux.die.net/man/8/auditctl

To really debug things, I guess making it R/W-able again (backup first) is best.
Post Reply