hi i install dietpi + nextcloud on a pi4
i move, with dietpi drive manager the data dir to ext drive
now when i up a big file (about 50gb) from webui file is up in a temp dir somewhere on sd i think and when it is complete moved to ext drive data dir.
use of this tmp dir slow all and i need doubled space, 50gb for tmp file and 50gb for storaged file. i know tmp purge when process is complete but until then 100gb are used
2 question:
- can i disable tmp creation and up direct to data on ext drive? (best and fast solution, dont need to move file when cmpleted in tmp dir)
- if i cant poiint 1, can i move this tmp upload to ext data where i dont have space problem?
thanks for support, really
Hi,
many thanks for your request. I played a little bit with Nextcloud and your assumption on the upload process is not fully correct. During an updaload, files will be uploaded in smaller chunks. The file is not uploaded as a whole. These small files are stored on
/mnt/dietpi_userdata/nextcloud_data/admin/uploads/
For each upload a temp folder will be created containing the files. An example from my test system:
root@DietPiVM1:/mnt/dietpi_userdata/nextcloud_data/admin/uploads/web-file-upload-......# ls -la
total 61448
drwxr-xr-x 2 www-data www-data 4096 Aug 20 21:54 .
drwxr-xr-x 3 www-data www-data 4096 Aug 20 21:51 ..
-rw-r--r-- 1 www-data www-data 10485760 Aug 20 21:51 0
-rw-r--r-- 1 www-data www-data 10485760 Aug 20 21:52 10485760
-rw-r--r-- 1 www-data www-data 10485760 Aug 20 21:53 20971520
-rw-r--r-- 1 www-data www-data 10485760 Aug 20 21:53 31457280
-rw-r--r-- 1 www-data www-data 10485760 Aug 20 21:54 41943040
-rw-r--r-- 1 www-data www-data 10485760 Aug 20 21:54 52428800
As soon as the upload finished, these files will be assembled to the final file you have been uploaded and stored on correct location.
Means, if you moved dietpi_userdata to an external HDD, these temp files are already on your disk and not on the SD Card.
oh, very thanks for reply and support
i forgot to write im disabling the chunk creation for speed up upload process… speed go up from 10mb/s to 60mb/s on local upload with this command. (60mb/s is my USB pendrive write speed limit)
chunk process use lots cpu rss for pack and unpacking, when upload cpu go to 100%, with this command no cpu load on upload process
ncc config:app:set files max_chunk_size --value 0
the problem is: file is uploaded somewhere on system partition and then , when completed, moved to ext usb data folder.
my goal is to up direct to ext usb without move it at end
im sure upload is going to system partiton becouse use /dev/root free space
ok I set as well ncc config:app:set files max_chunk_size --value 0
root@DietPiVM1:~# ncc config:app:set files max_chunk_size --value 0
Config value max_chunk_size for app files set to 0
root@DietPiVM1:~#
Files are uploaded to /var/cache/lighttpd/uploads now
root@DietPiVM1:/var/cache/lighttpd/uploads# ls -la
total 37628
drwxr-x--- 2 www-data www-data 61440 Aug 21 10:46 .
drwxr-x--- 4 www-data www-data 4096 Aug 17 22:04 ..
-rw------- 1 www-data www-data 1076020 Aug 21 10:46 lighttpd-upload-2x6olV
-rw------- 1 www-data www-data 1052660 Aug 21 10:46 lighttpd-upload-46mDkB
-rw------- 1 www-data www-data 1077340 Aug 21 10:46 lighttpd-upload-4Nnpym
-rw------- 1 www-data www-data 1062880 Aug 21 10:46 lighttpd-upload-6TyJG8
-rw------- 1 www-data www-data 1071640 Aug 21 10:46 lighttpd-upload-6xeXcw
-rw------- 1 www-data www-data 1052660 Aug 21 10:46 lighttpd-upload-8KhMt1
-rw------- 1 www-data www-data 1068720 Aug 21 10:46 lighttpd-upload-af20HA
-rw------- 1 www-data www-data 1051200 Aug 21 10:46 lighttpd-upload-AniVnL
-rw------- 1 www-data www-data 1055366 Aug 21 10:46 lighttpd-upload-Ch2cgC
-rw------- 1 www-data www-data 1049740 Aug 21 10:46 lighttpd-upload-ciBG0r
-rw------- 1 www-data www-data 1074560 Aug 21 10:46 lighttpd-upload-CKlPio
-rw------- 1 www-data www-data 1056479 Aug 21 10:46 lighttpd-upload-eV4Yg3
-rw------- 1 www-data www-data 167900 Aug 21 10:46 lighttpd-upload-g2O72c
-rw------- 1 www-data www-data 1071398 Aug 21 10:46 lighttpd-upload-ICVpeO
-rw------- 1 www-data www-data 1060906 Aug 21 10:46 lighttpd-upload-iGkZnG
-rw------- 1 www-data www-data 1051200 Aug 21 10:46 lighttpd-upload-K8MWRX
-rw------- 1 www-data www-data 1073590 Aug 21 10:46 lighttpd-upload-kAY3Uj
-rw------- 1 www-data www-data 1052660 Aug 21 10:46 lighttpd-upload-KRFiJC
-rw------- 1 www-data www-data 1059476 Aug 21 10:46 lighttpd-upload-M68VpP
-rw------- 1 www-data www-data 1049740 Aug 21 10:46 lighttpd-upload-mJQ12g
-rw------- 1 www-data www-data 1051200 Aug 21 10:46 lighttpd-upload-MzOjSA
-rw------- 1 www-data www-data 1075306 Aug 21 10:46 lighttpd-upload-OdwxHV
-rw------- 1 www-data www-data 1074072 Aug 21 10:46 lighttpd-upload-QiwoVz
-rw------- 1 www-data www-data 1054120 Aug 21 10:46 lighttpd-upload-Qp9DWE
-rw------- 1 www-data www-data 1073100 Aug 21 10:46 lighttpd-upload-qRh5OV
-rw------- 1 www-data www-data 1064340 Aug 21 10:46 lighttpd-upload-S1gMal
-rw------- 1 www-data www-data 1099380 Aug 21 10:46 lighttpd-upload-SLNoNY
-rw------- 1 www-data www-data 1049740 Aug 21 10:46 lighttpd-upload-sQJX3f
-rw------- 1 www-data www-data 1068720 Aug 21 10:46 lighttpd-upload-uaLSPW
-rw------- 1 www-data www-data 1067260 Aug 21 10:46 lighttpd-upload-uEpZ0b
-rw------- 1 www-data www-data 1059030 Aug 21 10:46 lighttpd-upload-uKROWg
-rw------- 1 www-data www-data 1049740 Aug 21 10:46 lighttpd-upload-UP3Z5X
-rw------- 1 www-data www-data 1054120 Aug 21 10:46 lighttpd-upload-WGzV9f
-rw------- 1 www-data www-data 1060710 Aug 21 10:46 lighttpd-upload-wj9UBq
-rw------- 1 www-data www-data 1052660 Aug 21 10:46 lighttpd-upload-YlVzIi
-rw------- 1 www-data www-data 1057646 Aug 21 10:46 lighttpd-upload-yXzxO4
-rw------- 1 www-data www-data 1055580 Aug 21 10:46 lighttpd-upload-YyY2QH
root@DietPiVM1:/var/cache/lighttpd/uploads#
But still in chunks 
oh… that si funny 
anyways i dont know why, but speed is so high in my case and no cpuload with occ command, do u notice this too?
ok, how can i move that dir to external usb disk?, becosue my system partion cant manage big files, its 32gb only
want to try if i move httpd cache to ext usb ill lose the speed up boost with occ command set
setting is done on global lighttpd config file.
root@DietPiVM1:~# cat /etc/lighttpd/lighttpd.conf |grep server.upload-dirs
server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
root@DietPiVM1:~#
ok testing caching on same destination disk: upload speed is fast but moving when completed is a pain… disk need to read and write same time.
sorry if i ask yet, its clear that occ command speed up the upload but this time is ligghtpd that chunk files for caching.
my question is the same: can i up a files direct to dietpi data dir without chunk big files?
i have to disable lighttpd caching big uploaded files? can i do this?
sorry this I don’t know. Probably something to ask on lighttpd board https://redmine.lighttpd.net/projects/lighttpd/boards
ok thanks for all the reply, ill ask in lighttpd forum, i think its right place for the question.
anyways for to complete info about this problem, someone need a temp solution maybe.
my temp solution:
up a server samba, add this section to /etc/samba/smb.conf
[Cloud]
path = /mnt/dietpi_userdata/nextcloud_data/admin/files
writeable = yes
create mode = 0775
valid users = dietpi
comment = Nextcloud Share
max connections = 8
force group = www-data
force user = www-data
directory mode = 0775
add this 2 lines in /var/www/nextcloud/config/config.php
'check_data_directory_permissions' => false,
'filesystem_check_changes' => 1,
change permission of /mnt/dietpi_userdata/nextcloud_data/ and recursive to 777
exec in dietpi shell:
ncc config:app:set files max_chunk_size --value 0 && reboot
login and upload files with samba protocol at 110mb/s , full my write speed of ext disk. u can found new files on web interface in real time with a refresh page
note: this have a boost speed from 10mb/s to 65mb/s web upload too in my case
thats 
thx for sharing your solution. Uploading big data via samba is indeed a good solution.