I’m currently running Nextcloud on a Raspberry Pi 4. I’m aware that a Pi isn’t the strongest computer, and I notice that cpu load goes up very fast if I’m trying to download big files (600+ mb). Peak load was about 7. This goes so far that I can’t access any other websites if the load is too high (I assume it’s because I’m running Pihole and Unbound on the same Pi, and no DNS request gets through). After a while, the download starts as expected and fast. I assume this is because nextcloud has to prepare the file to be downloaded somehow, which drains ressources.
Is there a way to reduce the load on the Pi while downloading big files? I tried limiting download speed, but it didn’t help. The data is stored on an external HDD with its own power source. I wouldn’t mind if it took longer to download, I just don’t want it to influence the Pihole/Unbound (if that’s the source of the error). Or is there a way to dedicate one core to Pihole/Unbound, so that that one does not get used by other programs?
Nextcloud is a web app, so it runs with lighttpd, mariadb/mysql and php. You could check with htop which process consumes the most process time while you copy large files and them just lower the process priotity in dietpi-services.
But keeps in mind, this also effects other services and applications which depend on lighttpd, mysql or php.
I tried loading a ~900mb video file while watching htop.
Most of the time, it looked like the attached “htop1”, with occasional spikes (about 1 frame, rarely 2 or 3) from php-fpm and lighttpd (htop2 and htop3). You can also see that the short-term load is >4.
While doing this, the connection to nextcloud crashed (timeout) (this usually happens when loading big files) and the other (probably related?) problem with connecting to websites appeared. After a while, the download picked up again.
I guess data are copied from HDD to some cache located on SD card to prepare the download. A faster SD card might reduce the effect but I guess it will still take the resources, even if it might be shorter. Another option is to have the entire system located on a USB drive. It should be faster compare to SD card. For testing, you could clone your SD card into a USB3 pen stick and boot from it. It might give some performance increase.
As another alternative, you could ask on the Nextcloud forum if there is a way to avoid these caching within the web browser/php stack.
Pi-hole has a higher priority assigned already, but Unbound not. You could try to set the nice level to e.g. -10 for Unbound via dietpi-services so that it has higher CPU and I/O priority than webserver, database and PHP.
Pi-hole and Unbound btw both require nearly no system resources. So it is really (as can be seen in htop) PHP and the database while the Nextcloud web interface is accessed, which peak so high. This can be also adjusted, e.g. by assigning only 3 of the 4 CPU cores to the php7.4-fpm service (CPU affinity) so that one core is always free for Pi-hole and Unbound. For I/O, apart of nice level, there are other options as well, but I don’t think that this is the major issue here.
I set unbounds nice level to -10, and assigned cores 1-3 to php7.4-fpm. Using the same ~900mb file as before, the load goes up to ~5.5 in the beginning, later it goes up to ~7, and visiting websites still crashes. The download itself runs smoother though, no crashes in the nextcloud client. The behaviour of htop is the same as in my other post (except core 0 is not spiking with the others). Overall, it seems to run smoother already, thank you!
Would that be “tempdirectory” in /var/www/nextcloud/config/config.php ? It’s not in there yet, but the nextcloud docs say that it’s the parameter to “Override where Nextcloud stores temporary files.”
I’d just add
'tempdirectory => '/mnt/hdd/tmp',
and then it should take my external hdd as the temp file location? (/mnt/hdd is the mount point for my hdd)
There is a tmp upload directory for PHP, used (AFAIK) when uploading files via web interface (not when done via Nextcloud clients). That is set to /tmp tmpfs, so moving that to disk should worsen things and has no effect when just browsing the Nextcloud interface.
hmm at least for me uploads are done to /mnt/dietpi_userdata/nextcloud_data/admin/uploads where they are stored until upload is complete. Afterwards they got assembled into the target file on target location.
Let me link myself as I’m to lazy to type thinks twice. This is basically similar to the other topic
It’s a Pi 4, htop lists it at 3.75G. Even when loading huge files, the whole memory isn’t used.
What is the largest file you are trying to upload? And is it a single user who is uploading large files? Or are their multiple user who will do this? Because an idea would be to move the upload folder into memory. Bu there you are limited due to the physical ram size. It might be an option if files you upload will not be bigger than 2GB.
Just to be clear, Uploading is no problem, just Downloading.
To be sure, I tried uploading a 3.46 GB file via the nextcloud desktop app for windows (I moved the file into a folder and waited). lighttpd and php-fpm shoot up from time to time, but internet still works fine, and the download goes smoothly, too. Load stays <3.00.
Downloading, I right-click the file in the folder and choose “Always keep on this device”. Doing this on a ~900mb file leads to the behaviour described in this thread. This happens on several devices, even if there are no other uploads/downloads at the same time.
.