USB3 Boot Hard Drive Corruption Issues

Troubleshooting
1

Hey guys!

Long time desktop Linux user, fairly new to Raspberry Pi and DietPi.
Really thankful for all the effort you guys have put into this distro!

I’ve got a RPI 4 8GB I’m using as a home server and I’ve been keen to get it booting off USB instead of SD card.
So I bought a 1TB Crucial MX300 SSD and a cheap unpowered Wavlink USB3 enclosure from Amazon. It’s booting fine, but I seem to be running into occasional issues with corruption and I’m not entirely sure what the cause is.

Today I was mid-way through building offline Mimic TTS for Mycroft (which I know is a bit of an intensive process) and it suddenly failed, spitting out errors. When I tried to diagnose over SSH I was greeted with “input/output error” for every bash command. When I went to look at the Pi’s display, it was riddled with EXT4 read errors - I can’t quite remember now the exact error, but if it happens again I’ll make a note.

I’ve since run fsck on the drive and it seems to be up and running with no dramas. I have a PlayStation 3 Eye that I’m using as a mic for Mycroft and a camera for MotionEye and I noticed that I’d get those same EXT4 errors on boot if the PS3 Eye was plugged in. I unplugged it and had no issues with booting or corruption until today, when I decided to get Mycroft up and running again and plugged it in.

My best guess is that maybe I’m maxing out the Pi’s power output? I am running it off the official power supply though.

I haven’t seen to many other forum posts or videos of others complaining about compatibility or corruption issues with certain drives, so I’ve just assumed that my drive enclosure is working fine.

Any ideas or similar experiences would be much appreciated!

2

Hi,

First thing, don’t run the external disk unpowered. We always recommend to use an own power supply for USB disk. Otherwise it could lead to under voltage issue and data corruption in the external disk

You could have a look to kernel error messages as follow

dmesg -l err,crit,alert,emerg
3

Thanks for the reply, Joulinar

The only error dmesg brought up was:

vc4_hdmi fef00700.hdmi: Could not register sound card: -517

I have been running for a good 12 hours since the crash though. If it happens again I’ll run dmesg sooner.
Thanks for the tip RE external power. I’ve got a desktop drive enclosure I can try, just not a snug-fit for the 2.5" drive. If it seems stable I’ll get a proper enclosure.

Will keep you posted!

4

From our experience, such issues are gone in most cases while the external disk was powered in addition. Let’s see how it’s going for you.

5

Hi guys,

Sorry for the delay.
I tried my externally powered enclosure (Seagate Expansion 16TB with the drive swapped)
Unfortunately I get a kernel panic error:

VFS: Cannot open root device "sda1" or unknown-block(0,0)
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

Is this a driver issue perchance?
I can see the partitions on both my PC and Linux machine.

I know my current unpowered enclosure works, besides the presumed low-power corruption issues, so my current thinking is to get a USB 3 micro ‘Y’ cable and plug the powered cable into a decent USB phone charger. Any perceived problems with this? Or recommended powered drive enclosures?

6

hmm there could be some corruption on the disk. Therefore the system is not able to boot. The ‘Y’ cable should be fine. We had another user who could solve his voltage issues with such a cable.

MichaIng could you have a look pls

7

Ahh, sorry I should’ve been more clear. After receiving that mount error I popped the drive into the unpowered enclosure and it worked just fine. I didn’t run fsck or anything and it seemed to boot fine.

Thanks for your help so far, Joulinar! I’ll buy that cable and see how it goes.

8

Looks like I spoke too soon. I can’t seem to find any decently priced that won’t arrive before next month.
Ended up getting this:
https://www.ebay.com.au/itm/403040186383

It should arrive in a week and a bit :slight_smile:
Will post back with results.

9

ok crossing fingers it will help on your issue.

10

Generally, the USB ports of the Raspberry Pi (and most SBCs) are not designed to reliably power 2.5" drives or larger. The absolute maximum power they can serve, shared across all USB ports, is 1.2A/6W: https://www.raspberrypi.org/documentation/computers/raspberry-pi.html#power-supply
A single 2.5" SSD can peak this when powering on, or during some usage peaks or so, and then sudden unmounts or I/O errors may happen.

When powering the drive hence externally, please don’t use a phone charger to power the drive, but a PSU like you use to power the SBC. Charging does not require a stable voltage, so cheap phone chargers, even when theoretically serving sufficient current, may not reliably power a device, as they are simply not designed to cover this need. So use a PSU which is explicitly designed to power devices, holding voltage in a sufficiently tight range even on usage fluctuations.

11

Hey! Thanks for that, MichaIng. That’s great to know.
I think I was definitely hitting that power limit when I got all those I/O errors.
I’ll make sure I’m getting stable power to the drive then, no phone charger.

Finally my powered drive adapter came!
I powered off the Pi, swapped the adapters, but alas… It refused to boot. I believe I got the same error as earlier or maybe a variation:

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

I know the exact error is important to know, so I took a photo on my phone, but as luck would have it, my photo came out corrupt…

I got the same error on all USB ports 2.0 and 3.0, and when I switched back to the other enclosure just to be sure (it’s been running fine for weeks now, but I haven’t been pushing it hard), same error. So I popped the drive out and repaired the filesystems with my other Linux machine (I can’t seem to get the hold shift trick working on my Pi), but now when I attempt to boot I am greeted with a blank screen.

I wish I had a more straightforward problem, sorry guys. I was under the impression that booting from SSD would be simple? Am I doing something dreadfully wrong? Is there an official Pi guide with a list of compatible drives and enclosures? It seems like I’m walking a tightrope over corruption city here, haha.

The only reason I switched to SSD was for longevity. I’ve heard the SD cards wear out quickly if you run busy webservers or the like. I want to run Caddy, Nextcloud, Syncthing, Home Assistant and Mycroft, maybe a few other things to fiddle with, so I thought it would be worth the switch with all that data moving around.

In the interest of looking at alternatives, I’m aware I can move most of the DietPi offical apps’ userdata to an external drive which might be a good compromise? But some of the unofficial apps I run will still be writing with some regularity to the root partition. Are there any downsides to moving the whole root partition to SSD and just leaving boot on SD?

Any ideas welcome, guys.
Sorry for the messy confusing post :thinking:

12

Have you also check the FAT partition for errors? This can be done on Windows as well, including repairing it. When bootup doesn’t show any output, likely parts of the kernel or bootloader are damaged. As on RPi, all are actual files on the FAT partition, this can be quite easily repaired from Windows (or any other OS, of course). Can you paste the content (with file sizes) of the FAT partition, so we can check if something is missing or has the wrong size and then re-add or re-place it with a fresh file from GitHub: https://github.com/raspberrypi/firmware/tree/stable/boot

13

I remember there was some challenges for RPi to work together with specific adapter using JMicron Technology controller. As well there was a need to add an UAS blacklist quirk sometimes. Or to delay the start from SSD to give time for the disk to come up.

Another option is to boot from a USB stick and have RootFS transferred to the SSD?

14

Ah yes good point about the UAS issue. Let’s first see if the boot files are all there and healthy. As there was some filesystem corruption, not unlikely that the FAT partition was affected as well.

15

Hey guys, thanks for the responses. I have been repairing the boot partition as well as root. It took a few hours, but that blank screen eventually threw up some more errors:

EXT4-fs-error (device sda2): __ext4_find_entry:1536: inode #2: comm (sh): reading directory Iblock 0
EXT4-fs-error (device sda2): __ext4_find_entry:1536: inode #2: comm (sh): reading directory Iblock 0
EXT4-fs-error (device sda2): __ext4_find_entry:1536: inode #2: comm (sh): reading directory Iblock 0
EXT4-fs-error (device sda2): __ext4_find_entry:1536: inode #2: comm (modprobe): reading directory Iblock 0
EXT4-fs-error (device sda2): __ext4_find_entry:1536: inode #8108: comm (sh): reading directory Iblock 0
EXT4-fs-error (device sda2): __ext4_find_entry:1536: inode #8108: comm (sh): reading directory Iblock 0

And that just loops in a pattern.
So I left it off for the night and tried the fsck again this morning, seems we’re in business!
No other USB devices plugged in, using one of the USB2 ports.

The first unpowered enclosure I had is indeed JMicron chipset. I recall only setting

USB_MSD_PWR_OFF_TIME=0

in the EEPROM bootloader as recommended.

After success again with the JMicron enclosure, I tried the new powered one and finally have the kernel panic in its full form for all to see:

Kernel panic - not syncing with VFX: unable to mount root-fs on unknown block(8,2)
CPU: 1 PID: 1 Comm: swapper/O Not tainted 5.10.52-v8+ #1441
Hardware name: Raspberry  Pi 5 Model B Rev 1.4 (DT)
Call trace:
dump_backtrace+0x0/0x1b8
show_stack+0x20/0x30
dump_stack+0xf0/0x158
panic+0x18c/0x38c
mount_block_root+0x240/0x25c
mount_root+0x114/0x144
prepare_namespace+0x134/0x174
kernel_init_freeable+0x2a0/0x2e4
kernel_init+0x1c/0x120
ret_from_fork+0x10/0x30
SMP: stopping secondary CPUs
Kernel Offset: disabled
CPU features: 0x0240022, 6002000
Memory limit: none
--- [  end Kernel panic - not syncing: VFS: Unable to mount root-fs on unknown block(8,2)  ]---

Popped it back into the JMicron enclosure, started it again and I’m back at the login shell.
Looks like it doesn’t like the chipset of my new powered enclosure then hey?

Thanks Joulinar for that USB stick suggestion! I didn’t think of that. If this doesn’t pan out, I’ll probably go that route.

16

Hi guys, sorry to revive this thread.

I’ve put my Pi project on the back-burner for a whilst I’ve been figuring some other stuff out.
I don’t have a full solution to my problems yet, but I’ve had some developments, so I figured I’d post here in case it’s useful for others in the future.

After some research on my SSD, which took me a while as it’s older and discontinued, I found out it’s rated for 1.7A at peak! Supposedly boot-up can draw a lot of current with SSDs, which explains the frequent corruption errors I was having at boot time, especially when peripherals were attached.

The unpowered enclosures are no good for me, subject to the 1.2A limit MichaIng mentioned; the power supplies for the 3.5" enclosures I had were only good for 1.5A; the powered USB3 SATA adapter I bought off eBay is good for 2A, but shipped DOA.

So working with the best of what I already have, I went ahead and bought a 5V 6A power supply, a couple of SATA (power only) to Molex power cables, and a couple of SATA (data) extension cables. I’m going to run the drive off my unpowered enclosure’s SATA to USB3 chip, but power the drive itself from my new power supply. I’ve got a couple of these SSDs I’m thinking of using, so I went with 6 Amps to future proof.

I suppose I could have tried the USB Y-cable after all this, but I wasn’t sure if these unpowered enclosures current-limit their power ports and I didn’t want to take any chances.

The SATA extension cables are coming in the mail. I’ll post back with results.

17

So far so good!
Not the greatest pictures.
Hopefully get some time to rig it up properly.

18

Not the most beautiful solution but a very functional one…that’s all anyone can ask for!

Usually it’s almost always best to have a stable secondary power supply for media storage (I’ve never heard of a SSD needing that much power on startup…daggum)

You can also get those dual head USB power setups, that has data on one, and a 2nd power adapter that you can plug into a dedicated power source…you can even modify a regular cable by splicing in a 2nd USB cable and using the + and - wires as well, leave the data lines alone so it doesn’t interfere with data…either way having something that can better stable power