Apt upgrade gone very wrong, second time in as many upgrades

I have searched the existing open and closed issues: yes

Required Information

  • DietPi version
    G_DIETPI_VERSION_CORE=9
    G_DIETPI_VERSION_SUB=6
    G_DIETPI_VERSION_RC=1
    G_GITBRANCH=‘master’
    G_GITOWNER=‘MichaIng’

  • Distro version | bookworm

  • Kernel version | Linux DietPi 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux

  • Architecture | amd64

  • SBC model |Native PC (x86_64), Intel NUC

Additional Information (if applicable)

  • Software title | Dietpi apt upgrade
  • Was the software title installed freshly or updated/migrated: updated to DietPi v9.6.1
  • Can this issue be replicated on a fresh installation of DietPi? N/A
    ← If you sent a “dietpi-bugreport”, please paste the ID here → N/A
  • Bug report ID | N/A

Steps to reproduce

  1. … I ran sudo apt upgrade, in response to notice in Message of the Day.
  2. … Once update completed (no errors I could see), I issued a reboot.
  3. … Upon successful restart, no matter what command I issue, for example ‘sudo detpi-launcher’, I get a dialog that says "RootFS is currently set to “Read Only…etc.” . Screen shot of the dialog attached.

This exact thing happened the last time I did an ‘upgrade’, and in an effort to resolve it myself by hitting up google, I managed to make the situation much worse by, among other things, completely wiping out a 4TB drive with all my movies/shows//music on it. I didn’t want to make that mistake again, so I came here right away. (I had to re-install dietpi to recover the first time this happened)

Expected behaviour

I have also attached the output from ‘sudo journalctl’, issued immediately after NUC startup.

journalctl from dietpi.txt (98.5 KB)

can you share following

df -h
lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid
dmesg -l 0,1,2,3

Note that I had to issue some of these commands with sudo, otherwise I get Operation not permitted’. I assume you know this, but mentioned it just in case.

Note also that I have three external USB 3 hard drives mounted but physically disconnected from the NUC…didn’t want to risk losing data on them.

dietpi@DietPi:~$ sudo df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.8G     0  3.8G   0% /dev
tmpfs           779M   11M  769M   2% /run
/dev/mmcblk0p2   29G  2.0G   26G   8% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.9G     0  3.9G   0% /tmp
tmpfs            50M  8.0K   50M   1% /var/log
/dev/mmcblk0p1   63M   12M   52M  19% /boot/efi
/dev/sda1       220G   56K  219G   1% /mnt/INTERNAL-SSD/26c76e42-f1c0-41f7-baf8-36c20cf1ac4c
dietpi@DietPi:~$ sudo lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid
NAME FSTYPE LABEL   SIZE RO TYPE MOUNTPOINT PARTUUID                             UUID
sda               223.6G  0 disk                                                 
└─sda1
     ext4         223.6G  0 part /mnt/INTER d2a6b0ac-e10d-4194-94fc-364deed9acc0 26c76e42-f1c0-41f7-baf8-36c20cf1ac4c
mmcblk0
                   29.1G  0 disk                                                 
├─mmcblk0p1
│    vfat            64M  0 part /boot/efi  ae3df4d6-d79f-423a-8a44-3e14f2bd76e9 9BE9-569B
└─mmcblk0p2
     ext4          29.1G  0 part /          80183ef5-2ac4-41fc-a915-96ea54d6c075 a2441233-19dc-4245-94a8-7919b5fdc0e8
mmcblk0boot0
                      4M  1 disk                                                 
mmcblk0boot1
                      4M  1 disk
dietpi@DietPi:~$ sudo dmesg -l 0,1,2,3
[    2.561751] mmc0: mmc_select_hs400 failed, error -84
[    2.562194] mmc0: mmc_hs400_to_hs200 failed, error -110
[    4.123675] mmc0: switch to high-speed from hs200 failed, err:-110
[    5.424509] platform regulatory.0: firmware: failed to load regulatory.db (-2)
[    5.424528] firmware_class: See https://wiki.debian.org/Firmware for information about missing firmware
[    5.424564] platform regulatory.0: firmware: failed to load regulatory.db (-2)
[    6.252360] EDAC pnd2: Failed to register device with error -22.

dietpi@DietPi:~$

can you share following as well

cat /etc/fstab
dietpi@DietPi:~$ cat /etc/fstab
# You can use "dietpi-drive_manager" to setup mounts.
# NB: It overwrites and re-creates physical drive mount entries on use.
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------


#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=3893M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid

#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf, glusterfs, mergerfs, bind, Btrfs subvolume
#----------------------------------------------------------------


#----------------------------------------------------------------
# SWAP SPACE
#----------------------------------------------------------------


#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=a2441233-19dc-4245-94a8-7919b5fdc0e8 / ext4 noatime,lazytime,rw 0 1
UUID=9BE9-569B /boot/efi vfat noatime,lazytime,rw 0 2
UUID=b3d7ef85-0c99-42ff-99af-bbbd62886a4e /mnt/DATA/b3d7ef85-0c99-42ff-99af-bbbd62886a4e ext4 noatime,lazytime,rw,nofail,noauto,x-systemd.automount
UUID=26c76e42-f1c0-41f7-baf8-36c20cf1ac4c /mnt/INTERNAL-SSD/26c76e42-f1c0-41f7-baf8-36c20cf1ac4c ext4 noatime,lazytime,rw,nofail,noauto,x-systemd.automount
UUID=533961e7-d051-449d-a4df-b43e42fb69ed /mnt/BACKUPS/533961e7-d051-449d-a4df-b43e42fb69ed ext4 noatime,lazytime,rw,nofail,noauto,x-systemd.automount
UUID=2d8b6179-9443-44b6-b1b8-2602db13d15a /mnt/MEDIA/2d8b6179-9443-44b6-b1b8-2602db13d15a xfs noatime,lazytime,rw,nofail,noauto,x-systemd.automount
UUID=b3d7ef85-0c99-42ff-99af-bbbd62886a4e /mnt/DATA/b3d7ef85-0c99-42ff-99af-bbbd62886a4e ext4 
UUID=26c76e42-f1c0-41f7-baf8-36c20cf1ac4c /mnt/INTERNAL-SSD/26c76e42-f1c0-41f7-baf8-36c20cf1ac4c 
UUID=533961e7-d051-449d-a4df-b43e42fb69ed /mnt/BACKUPS/533961e7-d051-449d-a4df-b43e42fb69ed 
UUID=2d8b6179-9443-44b6-b1b8-2602db13d15a /mnt/MEDIA/2d8b6179-9443-44b6-b1b8-2602db13d15a

Hmm you habe quite some drives configured to be mounted but just one seems to be present.

At least with one of the missing HDDs Samba tries to mount it

Jul 17 19:28:55 DietPi smbd[1359]: pam_unix(samba:session): session opened for user dietpi(uid=1000) by (uid=0)
Jul 17 19:28:55 DietPi systemd[1]: mnt-DATA-b3d7ef85\x2d0c99\x2d42ff\x2d99af\x2dbbbd62886a4e.automount: Got automount request for /mnt/DATA/b3d7ef85-0c99-42ff-99af-bbbd62886a4e, triggered by 1359 (smbd)
Jul 17 19:28:55 DietPi systemd[1]: Expecting device dev-disk-by\x2duuid-b3d7ef85\x2d0c99\x2d42ff\x2d99af\x2dbbbd62886a4e.device - /dev/disk/by-uuid/b3d7ef85-0c99-42ff-99af-bbbd62886a4e...
Jul 17 19:30:25 DietPi systemd[1]: mnt-DATA-b3d7ef85\x2d0c99\x2d42ff\x2d99af\x2dbbbd62886a4e.mount: Job mnt-DATA-b3d7ef85\x2d0c99\x2d42ff\x2d99af\x2dbbbd62886a4e.mount/start failed with result 'dependency'.
Jul 17 19:30:25 DietPi systemd[1]: dev-disk-by\x2duuid-b3d7ef85\x2d0c99\x2d42ff\x2d99af\x2dbbbd62886a4e.device: Job dev-disk-by\x2duuid-b3d7ef85\x2d0c99\x2d42ff\x2d99af\x2dbbbd62886a4e.device/start failed with result 'timeout'.

Anyway let’s check for ro file system

grep "[[:space:]]ro[[:space:],]" /proc/mounts 

Your system is booting from eMMC module or SD card?

I noted above that I had several external USB drives mounted but disconnected to avoid possible data loss. But note that the issue I described in my first post occurs whether ort not the drives are physically attached.

My system is booting from eMMC. Note that the last time this problem occurred on this NUC (which was two Dietpi versions ago), I had the NUC booting from an SSD mounted inside the NUC, so on the surface, it appears that this ‘Upgrade’ problem is unrelated to what media the OS is installed on.

Output from the command you requested:

dietpi@DietPi:~$ grep "[[:space:]]ro[[:space:],]" /proc/mounts
ramfs /run/credentials/systemd-sysctl.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
dietpi@DietPi:~$

Thank you, BTW…

it seems there is no file system mounted ro actually :thinking:

can you write to rootFS? Pls use root user for testing.

touch /demo.file
dietpi@DietPi:/$ sudo touch /demo.file
dietpi@DietPi:/$ ls
bin  boot  demo.file  dev  etc	home  lib  lib64  lost+found  media  mnt  opt  proc  root  run	sbin  srv  sys	tmp  usr  var
dietpi@DietPi:/$

as you can see, demo file is there. Means your rootFS is not set to ro.

OK, so where does that leave me? If I ask Google what to do, it gives me a command to try to mount the rootFS, and I recall issuing that command the last time this happened, and it didn’t help.

Do you figure I’m out of luck again? Re-install Dietpi?

Why you like to reinstall your system. It seems fine for me, nothing is mounted ro actually. You are able to create files on root partition.

Does the message still appears when you run dietpi-drive_manager?

I plugged all three of my USB hard drives in, and ran:

sudo dietpi-drive_manager

It starts without error and lists my drives, but none of them are mounted to the folders they used to be. Screenshot attached.

It actually appears that it’s all working again. Here’s the sequence of events that led to a “fix”:

  1. Issue “sudo apt upgrade”. Appears to succeed without error. Reboot NUC (for good measure)
  2. After restart, issue “sudo dietpi- launcher”. Error dialog, as per my first post, above.
  3. Shut down NUC, then unplug all three of the USB drives.
  4. Restart the NUC. Executing “sudo dietpi-” still causes the error shown in my first post.
  5. Shut down the NUC.
  6. Plug all three USB drives back in again, same port each was in before.
  7. Restart NUC, and all seems OK now. I can issue commands fine.

I will report back after I’ve sorted out the drive mounts again. Crossing my fingers that the simple act of mounting one or more of those drives doesn’t wipe them clean…

Replying to self to note that the story gets a bit weirder: I rebooted the NUC once again, and now all the original USB drive mounts are restored, so I didn’t have to re-establish them using dietpi-drive_manager.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.