Nextcloud slow upload and freezing when uploading big files

General Discussion
1

Hi, I would like some help from the community, I have a raspberry pi 4 4 gb ram, installed dietpi on a 16 gb card with currently 4 gb of free space, I have nextcloud in subfolder (reverse proxy) my web server is nginx, my dietpi folder -userdata is installed on an external 3TB hard drive…
the problem I have is uploading large files from the computer or cell phone, there are crashes and drops, in addition to the upload being slow, error 400 , error 504 and other errors occur, I’ve been reading about and found this link >>>https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html

I would like help to solve this problem, at first I think it may be my little free space in temporary files on the 16 gb ram sd (4 gb free space)and I need to do some commands according to the link, I would like to know if someone can guide me? thanks

2

Are those does larger than 512 MiB? For this to work you’d need to change the PHP settings as stated in the Nextcloud docs. You cane create and enable an own module for this, so default configs are not touched. Have a look on how those look like in /etc/php/7.4/mods-available. Create a new one with priority 99, e.g. named upload_size.ini, then enable it via:

phpenmod upload_size
systemctl restart php7.4-fpm

Settings would be for 2 GiB:

upload_max_filesize 2G
post_max_size 2G

2 GiB is max for 32-bit systems, on 64-bit it can be much larger, of course.

3

My system it’s aarch64 , my php it’s 7.3 , how update php 7.3 for 7.4?

4

how update php 7.3 for 7.4?

You would need to upgrade from Buster to Bullseye. DietPi - How to upgrade to "Bullseye" - DietPi blog

5

Edit:I did what you instructed me in php 7.3 and it is giving error 504 (error when assembling the parts)
(I typed 20gb into the upload_size file and uploaded a 4gb file from the computer…)

6

I had the same problem with large files and for me the php timeout limit was reached (it was set to 30 seconds, if I remember correctly), because the upload took so long. But I also always run out of RAM if the file was larger then the RAM limit for PHP…

To change the timeout settings, you just need to add some lines to the nextcloud/.user.ini file. (I’m not sure if this can also be set in the .ini file you created earlier, but if it can, then without the “php_value” at the beginning)

php_value max_input_time 600
php_value max_execution_time 600

The time is set in seconds, so this set’s it to 10 minutes.
Also restart php7.4-fpm again.

For more infos see https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html#configuring-your-web-server

7

I updated dietpi to debian 11 bullseye
so i went into /etc/php/7.4/mods-available/upload_size.ini and typed in the codes>>

; priority=99
upload_max_filesize 20G
post_max_size 20G
max_input_time 600
max_execution_time 600

I tested a 4 gig file upload and at the end it gave an error >> (Error assembling parts, status code 504) and the upload did not happen, but after a while the file appeared on my nextcloud like magic, weird. .

8

Still a timeout issue, you could increase values for the timeouts further, to 3600 maybe (1 hour).
Some threads in the official nextcloud forum referred to this value to solve this error message.

edit: I’m not even quite shure if this is the right syntax for php ini files? In every example/tutorial it’s like
" = " like memory_limit = 2G
When I look into my etc/php/7.4/fpm/php.ini and /var/www/nextcloud/.user.ini it’s that way.
Can somebody clearify this?

9

a question where is this file for me to test?

\

nextcloud/.user.ini

10

I edited my post above, there you can find the full path

11

On DietPi the webroot by default is /var/www and the Nextcloud dir hence /var/www/nextcloud/.user.ini.

Btw another aspect may be the temporary upload dir which is the tmpfs /tmp by default and hence bound to overall RAM + swap space. Probably it helps to move it to disk:

upload_tmp_dir=/var/tmp

But Nextcloud also has one internally, so I’m not 100% sure whether this has an effect. Also, writing all uploads once more to SD card wouldn’t be great, so e.g. with Nextcloud data on external drive it would make sense to create a www-data user writable dir on that external drive and use that for as PHP tmp upload dir.

12

Yeah it depends how much memory the system has available. At least on my testing, speed seems to be increasing if the Nextcloud user upload directory is placed into ram

root@DietPi4:~# df -h | grep nextcloud
tmpfs           6.0G  384M  5.7G   7% /mnt/usb/dietpi_userdata/nextcloud_data/admin/uploads
root@DietPi4:~#




root@DietPi4:~# cat /etc/fstab | grep nextcloud
tmpfs /mnt/dietpi_userdata/nextcloud_data/admin/uploads tmpfs size=6G,noatime,lazytime,nodev,nosuid,mode=1777
root@DietPi4:~#

This way I was able to upload a 4GB file in around 10,5 minutes. 6 Minutes are spend to upload the file, rest was to assemble the chunks and to store it on disk.

13

Strange is that according to the docs, by default the upload chunks are stored in the same dir of the final target file: https://docs.nextcloud.com/server/stable/admin_manual/configuration_server/config_sample_php_parameters.html
“part_file_in_storage”.

Ah although “same storage” may not mean same directory. So linking this to RAM sounds like a good idea then, as long as physical RAM is sufficient for the max expected upload sizes.

14

At least in past the PHP upload folder was used, but it seems behaviour changed. Nextcloud is now using /mnt/dietpi_userdata/nextcloud_data/<user_name>/uploads/. Means, each user will have it’s own upload folder.

15

That fits to the settings description. Setting it to false merges everything in one upload dir, but still one inside Nextcloud data. I guess issues with this being in tmpfs (with small RAM or default 512 MiB tmpfs size) or on small system drive caused to many user issue so that it was changed.

16

thank you all for your help, the problem was solved by creating the upload_size file

; priority=99
upload_max_filesize 10G
post_max_size 10G
max_input_time 3600
max_execution_time 3600
upload_tmp_dir=/var/tmp

and also putting

nginx client_max_body_size 10240M;

17

Does this really work on the RPi4 with only 4GB RAM? Mine always crashed when the files were bigger then my free memory.

19

I did some tests and it worked but i’m worried now do you think it’s better to put the size to 4096 (4G)?

20

but /var/tmp should be located on rootFS isn’t it? It’s not placed in memory.

21

I think so, my solution got kinda messed up, I’m not sure if what made it work was changing /var/tmp or increasing php and nginx memory