Cannot mount USB drive

Hello everyone,
RPi4 user here. I keep some of my userdata on a USB thumb drive, partly to minimize the writes/ rw on the SD card containing the root FS (though I think I have an SD card designed for dashcams and hence have higher r/rw scores), and partly to avoid running into lack of space which can happen if you are using Sonar, Radar, etc.).
Now a couple of weeks ago, I noticed that my USB drive was not getting mounted and was flagged as read-only. It has happened to me once before and I replaced the USB thinking that the previous one had gone bad. However now on this second one, I have the same problem, and as a result of which certain apps cannot run since their userdata is on the USB drive. The system itself is running fine. The USB is ext4 and the check shows issues. Any idea how to fix these? I have access to an Ubuntu machine if need be. Many thanks in advance.
Faisal


│ e2fsck 1.44.5 (15-Dec-2018)
│ Warning: skipping journal recovery because doing a read-only filesystem check.
│ Pass 1: Checking inodes, blocks, and sizes
│ Inode 2360134, end of extent exceeds allowed value
│         (logical block 256, physical block 3354368, len 26)
│ Clear? no
│
│ Inode 2360134, i_blocks is 2096, should be 2056.  Fix? no
│
│ Inode 2757330, end of extent exceeds allowed value
│         (logical block 512, physical block 3370496, len 532)
│ Clear? no
│
│ Inode 2757330, i_blocks is 5928, should be 4104.  Fix? no
│
│ Pass 2: Checking directory structure
│ Pass 3: Checking directory connectivity
│ Pass 4: Checking reference counts
│ Pass 5: Checking group summary information
│ Block bitmap differences:  -(3157704--3158378) +(3342448--3342463)
│ +(3351744--3351807) +(3354405--3354623) +(3355136--3355647) +(3356672--3362540)
│ +(3362816--3363839) +(3369184--3369215) +(3370112--3370239) -(3370496--3371027)
│ +11047293 +(11296068--11296070) +(11296422--11296437) -(11296438--11296445)
│ +(11303860--11303865) -11303868 +(12719986--12719992) -(12719993--12720006)
│ Fix? no
│
│ Free blocks count wrong for group #96 (19013, counted=19010).
│ Fix? no
│
│ Free blocks count wrong for group #102 (16040, counted=16034).
│ Fix? no
│
│ Free blocks count wrong (12952171, counted=12976295).
│ Fix? no
│
│ Free inodes count wrong (3701304, counted=3701878).
│ Fix? no

I guess these errors would need to be corrected. But let’s ask developer first. Usually he has some ideas on these issues.

MichaIng
could you have a look pls

Yes, fix those better earlier than later. But good to try a backup of the content first. Run fsck interactively or with -p flag first to have errors resolved automatically which can be resolved safely, then again without the flag to check whether there are still some left. Afterwards check the lost+found directory for files/inodes which lost their links to the filesystem.

Thanks but that is the problem. The system has marked it as read only so whenever I run fsck, I get:

root@DietPi:~# fsck /dev/sda
fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
fsck.ext2: Read-only file system while trying to open /dev/sda
Disk write-protected; use the -n option to do a read-only
check of the device.


Any idea how to remove the read-0nly flag? Thanks.

Firstly, it is better to specify the partition (/dev/sda1) you want to check and fix the file system, not on the whole drive (/dev/sda), as there might be other file systems there.
Second, make sure it is not mounted.
Third, if everything else fails, you can always run the fsck on the Ubuntu.

Thanks. I think there is only one partition.

root@DietPi:~# lsblk /dev/sda
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 57.7G 1 disk
└─sda1 8:1 1 57.7G 1 part

Or:
root@DietPi:~# fdisk -l /dev/sda
Disk /dev/sda: 57.7 GiB, 61983424512 bytes, 121061376 sectors
Disk model: USB 3.0 FD
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 10E5615A-CF44-4CCF-AB12-6C7ECD7C5305

Device Start End Sectors Size Type
/dev/sda1 2048 121059327 121057280 57.7G Linux filesystem

Or:
root@DietPi:~# blkid -o list
device fs_type label mount point UUID

/dev/mmcblk0p1 vfat boot /boot F661-303B
/dev/mmcblk0p2 ext4 rootfs / 8d008fde-f12a-47f7-8519-197ea707d3d4
/dev/sda1 ext4 (not mounted) 576d5fbf-697e-4b5b-81ab-ac4335404da5
/dev/mmcblk0 (in use)
root@DietPi:~#

I think I need to safely remove the read-only flag and then run a disk check. Any idea how to do that? Thanks.

make sure the disk is not mounted or specified in /etc/fstab. Reboot your system and check for kernel error messages

dmesg -l err,crit,alert,emerg

Thanks, here it is.
root@DietPi:~# dmesg -l err,crit,alert,emerg
[ 3.043327] vc4_hdmi fef00700.hdmi: Could not register sound card: -517
[ 12.524885] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.525857] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.526697] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.526757] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.526933] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.527106] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.527302] FS-Cache: Duplicate cookie detected
[ 12.527315] FS-Cache: O-cookie c=ca323e76 [p=2989db56 fl=222 nc=0 na=1]
[ 12.527322] FS-Cache: O-cookie d=aaa71abd n=d3724bd7
[ 12.527328] FS-Cache: O-key=[8] ‘020001bdc0a80101’
[ 12.527389] FS-Cache: N-cookie c=ba240988 [p=2989db56 fl=2 nc=0 na=1]
[ 12.527396] FS-Cache: N-cookie d=aaa71abd n=56c03d4b
[ 12.527402] FS-Cache: N-key=[8] ‘020001bdc0a80101’
[ 12.527681] FS-Cache: Duplicate cookie detected
[ 12.527692] FS-Cache: O-cookie c=822c15a9 [p=2989db56 fl=222 nc=0 na=1]
[ 12.527699] FS-Cache: O-cookie d=aaa71abd n=d60eb048
[ 12.527705] FS-Cache: O-key=[8] ‘020001bdc0a801de’
[ 12.527759] FS-Cache: N-cookie c=ba240988 [p=2989db56 fl=2 nc=0 na=1]
[ 12.527765] FS-Cache: N-cookie d=aaa71abd n=1ad3a06e
[ 12.527771] FS-Cache: N-key=[8] ‘020001bdc0a801de’
[ 12.527893] FS-Cache: Duplicate cookie detected
[ 12.527903] FS-Cache: O-cookie c=822c15a9 [p=2989db56 fl=222 nc=0 na=1]
[ 12.527912] FS-Cache: O-cookie d=aaa71abd n=d60eb048
[ 12.527920] FS-Cache: O-key=[8] ‘020001bdc0a801de’
[ 12.527977] FS-Cache: N-cookie c=ba240988 [p=2989db56 fl=2 nc=0 na=1]
[ 12.527984] FS-Cache: N-cookie d=aaa71abd n=7e44e0e2
[ 12.527989] FS-Cache: N-key=[8] ‘020001bdc0a801de’
[ 12.528184] FS-Cache: Duplicate cookie detected
[ 12.528196] FS-Cache: O-cookie c=ca323e76 [p=2989db56 fl=222 nc=0 na=1]
[ 12.528206] FS-Cache: O-cookie d=aaa71abd n=d3724bd7
[ 12.528212] FS-Cache: O-key=[8] '020001bdc0a80101
[ 12.528259] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers
[ 12.528266] FS-Cache: N-cookie c=3ede2965 [p=2989db56 fl=2 nc=0 na=1]
[ 12.528492] FS-Cache: N-cookie d=aaa71abd n=4e279247
[ 12.528499] FS-Cache: N-key=[8] ‘020001bdc0a80101’
[ 12.529682] FS-Cache: Duplicate cookie detected
[ 12.529694] FS-Cache: O-cookie c=822c15a9 [p=2989db56 fl=222 nc=0 na=1]
[ 12.529701] FS-Cache: O-cookie d=aaa71abd n=d60eb048
[ 12.529708] FS-Cache: O-key=[8] ‘020001bdc0a801de’
[ 12.529764] FS-Cache: N-cookie c=8d81e269 [p=2989db56 fl=2 nc=0 na=1]
[ 12.529771] FS-Cache: N-cookie d=aaa71abd n=0c111dad
[ 12.529777] FS-Cache: N-key=[8] ‘020001bdc0a801de’
[ 12.534523] FS-Cache: Duplicate cookie detected
[ 12.534535] FS-Cache: O-cookie c=ca323e76 [p=2989db56 fl=222 nc=0 na=1]
[ 12.534543] FS-Cache: O-cookie d=aaa71abd n=d3724bd7
[ 12.534549] FS-Cache: O-key=[8] ‘020001bdc0a80101’
[ 12.534604] FS-Cache: N-cookie c=7542404e [p=2989db56 fl=2 nc=0 na=1]
[ 12.534611] FS-Cache: N-cookie d=aaa71abd n=ccd717ea
[ 12.534617] FS-Cache: N-key=[8] ‘020001bdc0a80101’
root@DietPi:~#

You must use the partition:

fsck /dev/sda1

There is no filesystem written to /dev/sda but a partition table.

I did try that several times :slight_smile: Here it is:

root@DietPi:~# fsck /dev/sda1
fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
fsck.ext4: Read-only file system while trying to open /dev/sda1
Disk write-protected; use the -n option to do a read-only
check of the device.

Also, when I did cat /var/log/messages, it showed, among other things:

Oct 12 21:46:16 DietPi kernel: [ 2.729210] scsi 0:0:0:0: Direct-Access PNY USB 3.0 FD PMAP PQ: 0 ANSI: 6
Oct 12 21:46:16 DietPi kernel: [ 2.730039] sd 0:0:0:0: [sda] 121061376 512-byte logical blocks: (62.0 GB/57.7 GiB)
Oct 12 21:46:16 DietPi kernel: [ 2.730514] sd 0:0:0:0: [sda] Write Protect is on
Oct 12 21:46:16 DietPi kernel: [ 2.730918] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn’t support DPO or FUA

Okay, at least now fsck.ext4 is correctly invoked, previously it showed fsck.ext2. The read-only is on a different level, because fsck doesn’t require a filesystem to be mounted at all. [sda] Write Protect is on sounds like the drive itself is somehow write protected, but not sure if this is still due to filesystem or partition table corruption :thinking:.

Can you try:

hdparm -r0 /dev/sda
fsck /dev/sda1

Thanks, I read somewhere that if the drive is failing or encounters errors, the system marks it as read-only. Anyway, here is the rather strange result of your suggestion :frowning:

root@DietPi:~# root@DietPi:~# hdparm -r0 /dev/sda

/dev/sda:
setting readonly to 0 (off)
readonly = 0 (off)
root@DietPi:~# fsck /dev/sda1
fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
fsck.ext4: Read-only file system while trying to open /dev/sda1
Disk write-protected; use the -n option to do a read-only
check of the device.
root@DietPi:~#

Also,

root@DietPi:~# cat /var/log/messages
Oct 12 23:35:54 DietPi kernel: [ 6568.733260] EXT4-fs (sda1): INFO: recovery required on readonly filesystem
root@DietPi:~#

Try to do the filesystem check on the Ubuntu. Remember not to mount the stick upon connecting it.

What I know is that a mounted filesystem is re-mounted as read-only, when critical filesystem errors are found, but I didn’t see this kind of write-protection yet that blocks all kind of writes outside of the mount. hdparm -r allows to set/unset a device-level read-only flag, but obviously that either was not the case of reverted directly.

Yes makes sense to try it on a different system. Next would be to try rewriting the partition table, if not prevented for the same reason.

Thanks everyone. Well the result is the same. Ubuntu also recognizes the USB as in need of repair but it can neither mount it nor can it remove the read-only flag :frowning: Seems like quite a few people had this issue but with little success. Most cases just ended up formatting the USB.
Is there any way I can salvage the data from it? Or is there a possibility of a backup of userdata might be in Dietpi?
Thank you and looking for suggestions.

You could try to make a dump of the usb stick with dd.
Then try to mount the image as a loop device and copy what you can salvage.

Good idea, probably a loop device does not suffer from such read-only flag. Whether it can then be mounted is another question, but probably it can be repaired (fsck) then.

Thanks everyone, so I have managed to make a couple of images of the said USB. The USB itself is a write off I guess. I mounted the image in Ubuntu but it wont let me do anything probably due to ownership. I then mounted the same image to dietpi and copied the files from dietpi-userdata folder to another USB device, albeit with some copying errors. In the end however, the apps with their user data on the USB (Sonarr, Radarr, NzbGet, Jellyfin, etc) still cannot be accessed, though unlike before, they are running and not inactive.
Is there a way to fix those? Or the files within the image? The image still mounts as a read-only device (just like the USB it came from) if I remember correctly.
Many thanks for your help and time.

Check the logs of the services which are not accessible and whether they are listening to network ports as expected:

journalctl -u sonarr -u radarr -u jellyfin -u nzbget
ss -tulpn