mariadb fails to start

Don’t know if this is the same problem, but all of a sudden MaraDB isn’t working anymore.

 mariadb.service - MariaDB 10.3.27 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/

Mar 19 23:09:41 DietPi systemd[1]: mariadb.service: Service RestartSec=5s expired, scheduling restart.
Mar 19 23:09:41 DietPi systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 44.
Mar 19 23:09:41 DietPi systemd[1]: Stopped MariaDB 10.3.27 database server.
Mar 19 23:09:41 DietPi systemd[1]: Starting MariaDB 10.3.27 database server...
Mar 19 23:09:42 DietPi mysqld[14479]: 2021-03-19 23:09:42 0 [Note] /usr/sbin/mysqld (mysqld 10.3.27-MariaDB-0+deb10u1) s
tarting as process 14479 ...
Mar 19 23:09:42 DietPi mysqld[14479]: 2021-03-19 23:09:42 0 [Warning] Could not increase number of max_open_files to mor
e than 16384 (request: 32186)
Mar 19 23:09:42 DietPi systemd[1]: mariadb.service: Main process exited, code=killed, status=6/A
BRT
Mar 19 23:09:42 DietPi systemd[1]: mariadb.service: Failed with result 'signal'.
Mar 19 23:09:42 DietPi systemd[1]: Failed to start MariaDB 10.3.27 database server.
Mar 19 23:09:43 DietPi systemd[1]: Stopped MariaDB 10.3.27 database server.
root@DietPi:~# systemctl start mariadb
Job for mariadb.service failed because a fatal signal was delivered to the control process.
See "systemctl status mariadb.service" and "journalctl -xe" for details.
root@DietPi:~# systemctl status mariadb.service
● mariadb.service - MariaDB 10.3.27 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; disabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: signal) since Fri 2021-03-19 23:10:15 CET; 4s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 14743 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS
)
  Process: 14744 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCES
S)
  Process: 14746 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`cd /usr/bin/..; /usr/bin/ga
lera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/
SUCCESS)
  Process: 14802 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=killed
, signal=ABRT)
 Main PID: 14802 (code=killed, signal=ABRT)

Please show the error log:

cat /var/log/mysql/error.log

If it’s empty, try again to start the service to create a fresh error log.

I split the topic into it’s own one as usually there are different challenges. Let’s check the entire stack

systemctl restart mariadb.service
journalctl -u mariadb
cat /var/log/mysql/error.log
readlink /var/lib/mysql
readlink -f /var/lib/mysql

Atm this is obsulete now, since the Pi did die on me :frowning:

means the physical device is dead completely? Sorry for that.

so, the problem with Mariadb is back :frowning:

These are the output of the given commands.

cat /var/log/mysql/error.log
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             3570                 3570                 processes
Max open files            16384                16384                files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       3570                 3570                 signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
Core pattern: core



readlink /var/lib/mysql
/mnt/dietpi_userdata/mysql



readlink -f /var/lib/mysql
/mnt/dietpi_userdata/mysql

are you sure you copied the error log? As this is not a content I expect :thinking:

Just checked with

nano error.log

2021-05-17 12:17:04 0 [Note] InnoDB: Using Linux native AIO
2021-05-17 12:17:04 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-05-17 12:17:04 0 [Note] InnoDB: Uses event mutexes
2021-05-17 12:17:04 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-05-17 12:17:04 0 [Note] InnoDB: Number of pools: 1
2021-05-17 12:17:04 0 [Note] InnoDB: Using generic crc32 instructions
2021-05-17 12:17:04 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-05-17 12:17:04 0 [Note] InnoDB: Completed initialization of buffer pool
2021-05-17 12:17:04 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be ch$
2021-05-17 12:17:04 0 [ERROR] InnoDB: Space id and page no stored in the page, read in are [page id: space=4294967295, $
2021-05-17 12:17:04 0 [ERROR] InnoDB: Page [page id: space=4294967295, page number=4294967295] log sequence number 1844$
2021-05-17 12:17:04 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not$
2021-05-17 12:17:04 0 [ERROR] InnoDB: Database page corruption on disk or a failed file read of tablespace innodb_syste$
2021-05-17 12:17:04 0 [Note] InnoDB: Page dump in ascii and hex (16384 bytes):
 len 16384; hex fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff$InnoDB: End of page dump
2021-05-17 12:17:05 0 [Note] InnoDB: Uncompressed page, stored checksum in field1 4294967295, calculated checksums for $
2021-05-17 12:17:05 0 [Note] InnoDB: It is also possible that your operating system has corrupted its own file cache an$
2021-05-17 12:17:05 0 [ERROR] [FATAL] InnoDB: Aborting because of a corrupt database page.
210517 12:17:05 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs
[/cat]

Prev. I just copied what I was seeing on the screen.

Did you use a SD card? probably the same as before where your system already died on?

Because it seem you have data corruption.

[ERROR] InnoDB: Database page corruption on disk or a failed file read of tablespace

There could be physical issues on the SD card, if it is still the same one leading to data corruption again.

It was the same SD-card as before. Will try another one this weekend. Now I must find a way to get a backup of the WordPress database.

(sory for replying so late, but had some personal problems)

(sory for replying so late, but had some personal problems)

nothing to be sorry about. Hope you are healthy and ok

Now I must find a way to get a backup of the WordPress database.

Keep your SD card. Once you have setup the new system we could perform some file system check to see if there is a corruption and probably we are able to fix (without guarantee)

Thanks. Will do so later this week.

Took a little more longer, but finally having some time for this again.

I installed a new version of DietPi and WordPress on a fresh SD-card, and save the old one to investigate the old disk. If nothng works, no big problem.

If possible you can connect your old SD card via USB connector to your device and run a file system check

fsck -a /dev/sdxx

You would need to replace sdxx with the correct device name. probably sda2

fsck -a /dev/sda2
fsck from util-linux 2.33.1
fsck.ext2: No such file or directory while trying to open /dev/sda2
Possibly non-existent device?

Same with sda1 to sda4

when I tried to look at the card earlier, I got a ‘RAW’ error message.

let’s check where the SD card is

lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid

and kernel error messages

dmesg -l err,crit,alert,emerg

Then I get the following:

root@DietPi:~# lsblk -o name,fstype,label,size,ro,type,mountpoint,partuuid,uuid
NAME     FSTYPE LABEL  SIZE RO TYPE MOUNTPOINT PARTUUID                             UUID
sda                   29.5G  0 disk
mmcblk0               29.8G  0 disk
├─mmcblk0p1
│        vfat   boot   256M  0 part /boot      8f4dbd00-01                          4ADC-240F
└─mmcblk0p2
         ext4   rootfs
                      29.6G  0 part /          8f4dbd00-02                          89cef7ba-d45a-47e7-8d42-aff18afcec80



root@DietPi:~# dmesg -l err,crit,alert,emerg
[    5.100383] vc_sm_cma_vchi_init: failed to open VCHI service (-1)
[    5.100401] [vc_sm_connected_init]: failed to initialize shared memory service
[    5.127675] bcm2835_mmal_vchiq: Failed to open VCHI service connection (status=-1)
[    5.348096] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    5.642702] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[    5.642788] brcmfmac: brcmf_c_process_clm_blob: no clm_blob available (err=-2), device may have limited channels available
[    5.644028] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Oct 22 2019 01:59:28 version 7.45.98.94 (r723000 CY) FWID 01-3b33decd
[49113.118316] sd 0:0:0:0: [sda] No Caching mode page found
[49113.118338] sd 0:0:0:0: [sda] Assuming drive cache: write through
[49113.147751] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[49113.147767] Buffer I/O error on dev sda, logical block 0, async page read
[49113.149215] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[49113.149229] Buffer I/O error on dev sda, logical block 0, async page read
[49113.293662] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[49113.293690] Buffer I/O error on dev sda, logical block 0, async page read
[49113.310761] blk_update_request: critical medium error, dev sda, sector 61931392 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[49113.312113] blk_update_request: critical medium error, dev sda, sector 61931392 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[49113.312127] Buffer I/O error on dev sda, logical block 7741424, async page read
[50169.487676] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50169.487690] Buffer I/O error on dev sda, logical block 0, async page read
[50169.489028] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50169.489045] Buffer I/O error on dev sda, logical block 0, async page read
[50169.494424] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50169.494444] Buffer I/O error on dev sda, logical block 0, async page read
[50973.712374] sd 0:0:0:0: [sda] No Caching mode page found
[50973.712385] sd 0:0:0:0: [sda] Assuming drive cache: write through
[50973.732974] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50973.733001] Buffer I/O error on dev sda, logical block 0, async page read
[50973.735051] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50973.735074] Buffer I/O error on dev sda, logical block 0, async page read
[50973.736978] blk_update_request: critical medium error, dev sda, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50973.737003] Buffer I/O error on dev sda, logical block 0, async page read
[50973.752188] blk_update_request: critical medium error, dev sda, sector 61931392 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[50973.753692] blk_update_request: critical medium error, dev sda, sector 61931392 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[50973.753706] Buffer I/O error on dev sda, logical block 7741424, async page read
root@DietPi:~#

looks like quite some damage on the old SD card

critical medium error, dev sda

Was afraid of that. Well, no big problem. just starting over again with everything, and maybe I can still import from the old,old source

as well a good time to remember to create backups on regular basis on another disk/stick :wink: