No space left on sd-card Topic is solved

Have some feedback, questions, suggestions, or just fancy a chat? Pop it in here.
Post Reply
przemko
Posts: 170
Joined: Sun Mar 15, 2020 5:40 pm

No space left on sd-card

Post by przemko »

Hello, today I have strange situation. I cannot login to my Emby and cannot do anything in my Nextcloud. I see also redis-server stuck on restart services. I try issue from github about rootfs https://github.com/MichaIng/DietPi/issues/3511 but don't help.

Code: Select all

root@DietPi:/# systemctl status redis-server.service 
● redis-server.service - Advanced key-value store
   Loaded: loaded (/lib/systemd/system/redis-server.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2020-12-10 19:02:26 CET; 29min ago
     Docs: http://redis.io/documentation,
           man:redis-server(1)
  Process: 567 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exited, status=0/SUCCESS)
 Main PID: 568 (redis-server)
    Tasks: 4 (limit: 4442)
   CGroup: /system.slice/redis-server.service
           └─568 /usr/bin/redis-server 127.0.0.1:6379

Dec 10 19:31:15 DietPi redis[3474]: Write error saving DB on disk: No space left on device
Dec 10 19:31:15 DietPi redis[568]: Background saving error
How it's possible I don't have space?

Code: Select all

root@DietPi:/# df -a
Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/root        30393948   29134376          0 100% /
devtmpfs           947680          0     947680   0% /dev
sysfs                   0          0          0    - /sys
proc                    0          0          0    - /proc
securityfs              0          0          0    - /sys/kernel/security
tmpfs              981472          0     981472   0% /dev/shm
devpts                  0          0          0    - /dev/pts
tmpfs              981472       8544     972928   1% /run
tmpfs                5120          0       5120   0% /run/lock
tmpfs              981472          0     981472   0% /sys/fs/cgroup
cgroup2                 0          0          0    - /sys/fs/cgroup/unified
cgroup                  0          0          0    - /sys/fs/cgroup/systemd
none                    0          0          0    - /sys/fs/bpf
cgroup                  0          0          0    - /sys/fs/cgroup/freezer
cgroup                  0          0          0    - /sys/fs/cgroup/cpu,cpuacct
cgroup                  0          0          0    - /sys/fs/cgroup/net_cls,net_prio
cgroup                  0          0          0    - /sys/fs/cgroup/cpuset
cgroup                  0          0          0    - /sys/fs/cgroup/blkio
cgroup                  0          0          0    - /sys/fs/cgroup/perf_event
cgroup                  0          0          0    - /sys/fs/cgroup/devices
cgroup                  0          0          0    - /sys/fs/cgroup/pids
systemd-1               0          0          0    - /proc/sys/fs/binfmt_misc
systemd-1               -          -          -    - /mnt/Toshiba
sunrpc                  0          0          0    - /run/rpc_pipefs
mqueue                  0          0          0    - /dev/mqueue
debugfs                 0          0          0    - /sys/kernel/debug
configfs                0          0          0    - /sys/kernel/config
tmpfs             1048576          4    1048572   1% /tmp
tmpfs               51200          4      51196   1% /var/log
/dev/mmcblk0p1     258095      56606     201490  22% /boot
/dev/sda       2883221128 1705043156 1031648260  63% /mnt/Toshiba
nfsd                    0          0          0    - /proc/fs/nfsd
Regards,
Przemek
Top
User avatar
Joulinar
Posts: 5115
Joined: Sat Nov 16, 2019 12:49 am

Re: No space left on sd-card

Post by Joulinar »

You need to find out which files are blocking your RootFS as it's completely full.

Code: Select all

du -a / | sort -n -r | head -n 20
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
Top
przemko
Posts: 170
Joined: Sun Mar 15, 2020 5:40 pm

Re: No space left on sd-card

Post by przemko »

Hi, I have run dietpi-cleanrs and also stuck on stopping dietpi-services redis-server. In the same time I run Your command:

Code: Select all

root@DietPi:/# du -a / | sort -n -r | head -n 20
du: cannot access '/proc/10046/task/10046/fd/3': No such file or directory
du: cannot access '/proc/10046/task/10046/fdinfo/3': No such file or directory
du: cannot access '/proc/10046/fd/4': No such file or directory
du: cannot access '/proc/10046/fdinfo/4': No such file or directory
1732575043	/
1705629276	/mnt
1703333204	/mnt/Toshiba
1700857544	/mnt/Toshiba/dietpi_userdata
916151384	/mnt/Toshiba/dietpi_userdata/Video
602783060	/mnt/Toshiba/dietpi_userdata/Zapas
475674520	/mnt/Toshiba/dietpi_userdata/Video/Filmy
440476860	/mnt/Toshiba/dietpi_userdata/Video/Seriale
346885684	/mnt/Toshiba/dietpi_userdata/Zapas/Przemek-dell
147057092	/mnt/Toshiba/dietpi_userdata/Zapas/Elwirka-laptop
92962012	/mnt/Toshiba/dietpi_userdata/Pictures
92962008	/mnt/Toshiba/dietpi_userdata/Pictures/Zdjęcia
85089372	/mnt/Toshiba/dietpi_userdata/Zapas/Lusia-Laptop
47298688	/mnt/Toshiba/dietpi_userdata/nextcloud_data
39541012	/mnt/Toshiba/dietpi_userdata/nextcloud_data/admin
39335764	/mnt/Toshiba/dietpi_userdata/Music
36234420	/mnt/Toshiba/dietpi_userdata/nextcloud_data/admin/files
29280388	/mnt/Toshiba/dietpi_userdata/Pictures/Zdjęcia/Obrazy
24274344	/var
22384544	/mnt/Toshiba/dietpi_userdata/Zapas/Ciocia
Regards.
Top
User avatar
Joulinar
Posts: 5115
Joined: Sat Nov 16, 2019 12:49 am

Re: No space left on sd-card

Post by Joulinar »

can you detach your HDD for a moment?
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
Top
przemko
Posts: 170
Joined: Sun Mar 15, 2020 5:40 pm

Re: No space left on sd-card

Post by przemko »

I detach HDD and reboot. Now my Dietpi won't start. :/
Image

Regards.
Top
User avatar
Joulinar
Posts: 5115
Joined: Sat Nov 16, 2019 12:49 am

Re: No space left on sd-card

Post by Joulinar »

the idea was more to stop all services, just remove HDD and check again for large files
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
Top
przemko
Posts: 170
Joined: Sun Mar 15, 2020 5:40 pm

Re: No space left on sd-card

Post by przemko »

Joulinar wrote: ↑Thu Dec 10, 2020 9:14 pm the idea was more to stop all services, just remove HDD and check again for large files
OK, now I start Rpi with HDD and then:

Code: Select all

root@DietPi:/# du -a / | sort -n -r | head -n 20
du: cannot access '/proc/712/task/1860': No such file or directory
du: cannot access '/proc/1818/task/1818/fd/3': No such file or directory
du: cannot access '/proc/1818/task/1818/fdinfo/3': No such file or directory
du: cannot access '/proc/1818/fd/4': No such file or directory
du: cannot access '/proc/1818/fdinfo/4': No such file or directory
1732590715	/
1705644860	/mnt
1703348788	/mnt/Toshiba
1700873128	/mnt/Toshiba/dietpi_userdata
916151384	/mnt/Toshiba/dietpi_userdata/Video
602783060	/mnt/Toshiba/dietpi_userdata/Zapas
475674520	/mnt/Toshiba/dietpi_userdata/Video/Filmy
440476860	/mnt/Toshiba/dietpi_userdata/Video/Seriale
346885684	/mnt/Toshiba/dietpi_userdata/Zapas/Przemek-dell
147057092	/mnt/Toshiba/dietpi_userdata/Zapas/Elwirka-laptop
92962012	/mnt/Toshiba/dietpi_userdata/Pictures
92962008	/mnt/Toshiba/dietpi_userdata/Pictures/Zdjęcia
85089372	/mnt/Toshiba/dietpi_userdata/Zapas/Lusia-Laptop
47301960	/mnt/Toshiba/dietpi_userdata/nextcloud_data
39541024	/mnt/Toshiba/dietpi_userdata/nextcloud_data/admin
39335764	/mnt/Toshiba/dietpi_userdata/Music
36234428	/mnt/Toshiba/dietpi_userdata/nextcloud_data/admin/files
29280388	/mnt/Toshiba/dietpi_userdata/Pictures/Zdjęcia/Obrazy
24274336	/var
22384544	/mnt/Toshiba/dietpi_userdata/Zapas/Ciocia
after that I detach my HDD:

Code: Select all

root@DietPi:/# du -a / | sort -n -r | head -n 20
du: cannot read directory '/mnt/Toshiba': Input/output error
du: cannot access '/proc/1909/task/1909/fd/3': No such file or directory
du: cannot access '/proc/1909/task/1909/fdinfo/3': No such file or directory
du: cannot access '/proc/1909/fd/4': No such file or directory
du: cannot access '/proc/1909/fdinfo/4': No such file or directory
29206815	/
24274348	/var
22229820	/var/cache
22224200	/var/cache/lighttpd
22215756	/var/cache/lighttpd/uploads
2296076	/mnt
2296044	/mnt/dietpi-backup
2296036	/mnt/dietpi-backup/data
1401568	/home
1401564	/home/dietpi
1195136	/var/lib
1164416	/var/lib/emby
1074124	/home/dietpi/emby
788912	/mnt/dietpi-backup/data/var
744784	/mnt/dietpi-backup/data/var/www
744708	/mnt/dietpi-backup/data/var/www/nextcloud
713136	/var/www
712480	/var/www/nextcloud
644452	/var/lib/emby/metadata
644440	/var/lib/emby/metadata/library
Regards.
Top
User avatar
Joulinar
Posts: 5115
Joined: Sat Nov 16, 2019 12:49 am

Re: No space left on sd-card

Post by Joulinar »

majority of space is used on /var/cache/lighttpd/uploads

Code: Select all

22215756	/var/cache/lighttpd/uploads
Can you have a look which files are in
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
Top
przemko
Posts: 170
Joined: Sun Mar 15, 2020 5:40 pm

Re: No space left on sd-card

Post by przemko »

There're lot of files like this:

Code: Select all

lighttpd-upload-007uSV	lighttpd-upload-APUL95	lighttpd-upload-KcGCqg	lighttpd-upload-V1ZUah	lighttpd-upload-fVeH6I	lighttpd-upload-pjlzM6
lighttpd-upload-00H0Ru	lighttpd-upload-APiDoz	lighttpd-upload-KcHQpX	lighttpd-upload-V1cBWd	lighttpd-upload-fVpXHq	lighttpd-upload-pjwPOL
lighttpd-upload-00IOZZ	lighttpd-upload-AQV9Vl	lighttpd-upload-KcOdSD	lighttpd-upload-V1m6FC	lighttpd-upload-fVu5Kd	lighttpd-upload-pkDLWx
lighttpd-upload-00Lpnx	lighttpd-upload-AQWsQO	lighttpd-upload-KcXQoL	lighttpd-upload-V1tSjc	lighttpd-upload-fVv3GE	lighttpd-upload-pkHyoi
lighttpd-upload-00P9SY	lighttpd-upload-AQlr8k	lighttpd-upload-KcqR6q	lighttpd-upload-V1tWNq	lighttpd-upload-fWeeIv	lighttpd-upload-pkf8Ze
lighttpd-upload-00SxeD	lighttpd-upload-AQptDx	lighttpd-upload-Kd1pQM	lighttpd-upload-V1yeZq	lighttpd-upload-fWpVla	lighttpd-upload-pklQb7
lighttpd-upload-00W3Vf	lighttpd-upload-ARPDMT	lighttpd-upload-KdJMhb	lighttpd-upload-V22O5q	lighttpd-upload-fWppVw	lighttpd-upload-pldqkR
lighttpd-upload-00dhS3	lighttpd-upload-ARjvTU	lighttpd-upload-KdjYHP
To much to post it.
Top
User avatar
Joulinar
Posts: 5115
Joined: Sat Nov 16, 2019 12:49 am

Re: No space left on sd-card

Post by Joulinar »

can you check time stamp on these files? maybe some leftover from the past
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
Top
Post Reply

Return to “General Discussion”