Raspi 4/4GB with OwnCloud crashing after some hours

Troubleshooting
1

Hi all,

I have a Raspi 4, 4 GB which is stored in an Argon M.2 case.

It’s booting from a 256 GByte M.2 disk and I have connected an external Samsung 1 TB SSD (ext4 formatted)

I’m using the Raspi mainly for Owncloud using Apache2 and Mariadb as database.
Other software (all from the dietpi optimized list):

  • mc
  • Let’s Encrypt/Certbot
  • Samba Server
  • PHPMyAdmin

There is no extra software installed.

After booting for some hours, sometimes for one day, everything is working perfectly.
But at some point the system starts failing.
Apache2 reports 403 error on all pages/URLs and obviously can’t access .htaccess anymore
Also SSH-connections via Putty are rejected, so it’s not only Apache2.

Ping still works.
The temperature of the system is ok and around 40 to 45 degrees Celsius.
Also the SSDs/M2 don’t show any failures when checked with drive manager.

Anybody here has similar experiences and how could I find out what’s causing the trouble?

Or which mointoring system could I install to trace what’s happening to the system?

Greetz
K.

PS: I’m more a Windows guy and used to Server administration. So tips and explanations please in a way, I can understand :wink:
Thank you.

2

Welcome to our forum, @sk-seo.
Do you use an extra power supply for the external SSD? Usually the USB ports of the Pi can’t handle external drives well. Sometimes you get lucky and you can use a SSD without extra power supply, but on heavy system loads and parallel writing/reading to the drive you can even get undervolt warnings with a SSD.
You can check on your running system for this undervolt warnings with
dmesg --level=warn,err,crit,alert,emerg.

Since your system freezes and by default logs aren’t saved to hard disk I suggest you switch to full logging with dietpi-software and after the next freeze we can have a look into the logs.

3

You could try following to get persistent system logs:

dietpi-software uninstall 103 # uninstalls DIetPi-RAMlog
mkdir /var/log/journal # triggers systemd-journald logs to disk
reboot # required to finalise the RAMlog uninstall

Then you can check system logs via:

journalctl

which will then show as well logs from previous boot sessions. To limit the size, you can additionally e.g. apply the following:

mkdir -p /etc/systemd/journald.conf.d
cat << '_EOF_' > /etc/systemd/journald.conf.d/99-custom.conf
[Journal]
SystemMaxFiles=2
MaxFileSec=7day
_EOF_

This will limit logs to 14 days split across two journal files, so that with rotation you will always have between 7 and 14 days of logs available.

4

Thanks a lot!
dmesg --level=warn,err,crit,alert,emerg

Didn’t show any undervolt warnings.

But as I applied the logging changes, the system didn’t reboot anymore and I couldn’t connect via SSH.
So I attached a HDMI monitor and saw a kernel panic error, and before something about

ext4 filesystem with invalid superblock checksum

Obviously the ext4 partition on the M.2 boot device failed.
So I created a new boot SDCard with the latest Dietpi 64 bit system and booted from it.

dietpi-drive_manager shows the M.2 device as /dev/something with ext4 file system

But even trying to repair it, fails and the whole system gets stuck showing a blue screen.

So, the M.2 device seems to be broke.
Is there a chance it will work again after formatting it or is it a piece of junk now?

Cheers
K.

5

quite hard to answer. We could try to repair the disc (without guarantee). At least you would need another Linux box where you could connect the disc on.

@MichaIng maybe you can have a look if there is a chance to fix the invalid superblock

6

When I connect the M.2 device - now built into a USB-3 adapter - to my Windows notebook and use DiskInternals LinuxReader (the free version), I can mount the Ext4 partition without any problem and also browse through the files.

For the Raspi 4 I meanwhile created a new microSD with dietpi on it.
When connecting the M.2 drive/USB-3 adapter there, the disk/Ext4 partition is recognized.

But hen I try to check/repair it with dietpi-drive_manager, the system hangs.
Isn’t there a better disk check & repair program?
I don’t think the M.2 is physically damaged.

Cheers
K.

7

When you attach the M.2, can you try to check it manually:

e2fsck /dev/sdb2

In case adjust the drive node if it’s not /dev/sdb.

If really the superblock is damaged, it can be restored from a backup, but I need to lookup the steps again as well.

8

Thank you for the advice, Micha!

The device (or better the partition) shows as
/dev/sda2

After applying the e2fsck command for the first time, it said that there were some errors and seemed to fix it.
After the second time it show the partition as good so I even could mount it and browse through the files.
But another
e2fsck
today showed a huge amount of errors. After fixing them e2fsck now says, the drive is clean.
I put the M.2 back in the case it was really booting again I could log in.
OwnCloud seems to work again.
But somehow I’m not really confident about the M.2 being stable.
So yesterday I got me another new M.2 drive and I’d like to transfer the whole device data to the new M.2 card.
What program (also Windows) would you recommend for that job?

Klaus

9

You could have a look to one of our blogs on how to move an entire system from one disk to another

Just as idea and starting point

10

If the new drive is larger than the old one, also https://rufus.ie/ or USBImager can be used on Windows.

11

It looks like the problem is solved.

It turned out to be the M.2 SATA Controller on the daughter board belonging to the Argon case.
It had issues from the day I installed it, but I thought it was the software or some kind of misconfiguration.
But back then I had to restart every 4 or 5 days what got worse and worse.

At the moment the system is running flawless for more than two days and I expect it to continue like this.

Thank you all again for you immediate help which I really appreciate.

Klaus

1 Like