Request: syncthing-inotify

[Sciamano]

Hello,
another request by me
I have noticed that using syncthing on arm devices, especially the development boards DietPi supports, can easily boost the cpu usage to 100% for long periods of time, when rescans of the shared folders take place.
I have read that syncthing-inotify sensibly reduces cpu usage by allowing longer times between rescans, simply by notifying the system of the files that have changed, instead of having it fully rescan the directories.
Could this be added to DietPi? I think it would make our boards more responsive.

More information here: https://github.com/syncthing/syncthing-inotify

Thanks!

[Sciamano]

In case anyone else is interested, here is how to install syncthing-inotify.

1. Make a directory and download latest version:

Code: Select all

mkdir syncthing-inotify
cd syncthing-inotify
wget https://github.com/syncthing/syncthing-inotify/releases/download/v0.8.4/syncthing-inotify-linux-arm-v0.8.4.tar.gz

2. Untar:

Code: Select all

tar zxvf syncthing-inotify-linux-arm-v0.8.4.tar.gz

3. Copy the binary to /usr/bin:

Code: Select all

cp syncthing-inotify /usr/bin

4. Remove the download folder:

Code: Select all

cd ..
rm -r syncthing-inotify

6. run it:

Code: Select all

/usr/bin/syncthing-inotify

[Sciamano]

Unfortunately though, I don't seem to be able to make it autostart or run as a service.
The syncthing-inotify project gives two scripts (one to run as users and one as system), and I have tried both without success. They make syncthing-inotify start but it does not work (and rescans of the directories happen as usual).
I have also tried adding it to /etc/rc.local but it happens the same.

BTW, here are the two scripts provided by the project... If anyone more capable than me could make them work I would be extremely grateful:

Run as system service: /etc/systemd/system/syncthing-inotify@.service

Code: Select all

[Unit]
Description=Syncthing Inotify File Watcher for %I
Documentation=https://github.com/syncthing/syncthing-inotify/blob/master/README.md
After=network.target syncthing@.service
Requires=syncthing@.service

[Service]
User=%i
ExecStart=/usr/bin/syncthing-inotify -logflags=0
SuccessExitStatus=2
RestartForceExitStatus=3
Restart=on-failure
ProtectSystem=full
ProtectHome=read-only

[Install]
WantedBy=multi-user.target

Run as user: /etc/systemd/user/syncthing-inotify.service

Code: Select all

[Unit]
Description=Syncthing Inotify File Watcher
Documentation=https://github.com/syncthing/syncthing-inotify/blob/master/README.md
After=syncthing.service
Requires=syncthing.service

[Service]
ExecStart=/usr/bin/syncthing-inotify -logflags=0
SuccessExitStatus=2
RestartForceExitStatus=3
Restart=on-failure
ProtectSystem=full
ProtectHome=read-only

[Install]
WantedBy=default.target

Thanks!!

[Sciamano]

Proposed solution (it seems to be working):

Code: Select all

crontab -e

add the following:

Code: Select all

@reboot /usr/bin/syncthing-inotify

[johnvick]

Have you noticed much reduction of CPU use? I have XU4/Cloudshell so am well aware of how SyncThing can tax the system. I'll try out your solution here.

[Sciamano]

I can notice a drastic decrease in CPU usage, as the system won't scan every shared directory every few minutes, but only checks and syncs those files that get changed.
It would be nice if I could get it to work as a service, though.

[Sciamano]

I have been discussing this issue with the guys from syncthing, as running syncthing-inotify from the crontab is not working properly.
There seems to be a problem with how syncthing is run in DietPi. Specifically, they are saying that the service file provided by DietPi is not correct and that this is causing the issue with syncthing-inotify.

This is what came out regarding the syncthing.service script in DietPi:

This file is plain wrong:

It starts syncthing is root; there is no possibility to overwrite it.
It uses some bash Foo in order to create a logfile
The type is oneshot (i guess because of the bash foo?)

I don't understand much about these things, but I think that our development boards would benefit a lot from the introduction of syncthing-inotify on DietPi, therefore I am calling out to Fourdee again for this to be introduced or at least fixed in order for us to install syncthing-inotify successfully. Thanks!

[johnvick]

I was going to give this a go this weekend but I'll hold off for now.

Before SyncThing was installable from the DietPi menu I had installed in manually but it used a lot of CPU that way also.

John

[Sciamano]

Yes, syncthing uses a lot of cpu during the rescan of the shared directories.
This would be avoided by switching from full rescans to partial ones by adopting the syncthing-inotify way.
The main problem seems to be that syncthing runs as root in DietPi, and this creates some kind of incompatibility with the syncthing-inotify.service script.
On the syncthing support forum they suggest to install syncthing manually on DietPi and avoid using the provided syncthing.service script, but I guess this defeats the purpose of having the logs kept in the ramdisk to avoid the frequent writes of the SD card...

[johnvick]

I've looked in the forum you refer to:

https://forum.syncthing.net/t/cant-make ... ot/8758/24

And have got to the same point as you in that syncthing-inotify works but I cannot autostart it as a service.

I'll keep an eye on that thread and see what develops.

John