Sonarr/Radarr services failed on v6.29.2

I meant just ssh into your pi as dietpi then run that command. I.e. don’t run as root. Just trying to make sure ownership stays as dietpi

Will do, so far so good. I would have thought lots of people would have been complaining if it was a general sonarr issue, but it seems to be only dietpi users (from what I can see).

FWIW, I’ve embarked down the path of rolling back mono (with the guidance of Taloth on the sonarr support discord) until I get to a version on which sonarr doesn’t crash. I just did 6.6 with no luck. Running 6.4 now and haven’t crashed yet. Here are the steps I’m doing for each version:

–Stop Sonarr: sudo service sonarr stop
–Remove mono & sonarr: apt remove mono-complete mono-runtime nzbdrone
–Auto remove: apt auto-remove
–Confirm mono removal: apt-mark showauto | grep mono
–Modify mono-xamarin.list to replace raspbianstretch with raspbianstretch/snapshots/6.x: nano /etc/apt/sources.list.d/mono-xamarin.list
–Run: apt update
–Run: apt install mono-complete nzbdrone
–Start Sonarr: sudo service sonarr start
–After crash, output log to file: journalctl -u sonarr -n100 > ~/crash-mono-6.6.log

is it running stable using mono 6.4 now?

Nope he gone all way back to 5.20… i was going to try the same to try an show its not tied to just one person issue.

but im stuck at it wont remove Mono lol

what is blocking to remove mono?

its ok. i removed via Dietpi-Software…

now running from a manual install of Mono and Sonarr

It will obviously wont have all the extra setup that are applied to the install done by Dietpi-software… eg permission hardenning
and whatever else it did.

so left it at its out of the box Get ready to use stage. just to see if it runs OK with nothing added to it.

Version 2.0.0.5344
Mono Version 5.20.1.34
AppData directory /root/.config/NzbDrone
Startup directory /opt/NzbDrone

Joulinar is there an update log file for the DietPi update to 6.29?

Not sure if the update is storing detailed logs. Something to ask MichaIng

@MichaIng any chance there’s a detailed update log for DietPi 6.29?

Yeah, this would help the Sonarr developers immensely!

Fwiw no crashes on the dockers for sonarr and radarr

thats good… out of interest what version of Mono does it run with… it tells you this in Sonarr System tab

ta

5.20.1.34

Hey guys, I’ll have a closer look tomorrow. For now:

Mono v6 indeed might be the problem, at least it causes a known issue when using a file system that does not natively support UNIX permissions: https://github.com/MichaIng/DietPi/issues/3179
So check that your media mount is not NTFS (at least without “permissions” mount option), FAT or exFAT, a Samba mount or something like this. I.e. chmod/chown must work on those mounts, otherwise Sonarr + Mono v6 fails expectedly.

Another thing to try is to move to Sonarr v3 (https://sonarr.tv/#downloads-v3-linux) which works around the above mono bug at least and is compared to v2 actively developed, but yeah in beta. I plan to migrate to Sonarr v3 with DietPi v6.31, probably it is released then officially as well.

Old Sonarr versions are btw available: https://apt.sonarr.tv/pool/main/n/nzbdrone/
Downgrade e.g. like this: apt-get install nzbdrone=2.0.0.5337
The dev branch versions are always one version number behind the master branch versions, but those two match nearly 100%. Not sure why we applied the dev branch instead of master, I never touched this, but should not matter due to internal updater anyway.

The Mono repo sadly does not allow downgrading easily. You would need to download the related packages manually. Here they can be found, but WARNING the list is VERY long and might crash your browser :wink:: https://download.mono-project.com/repo/debian/pool/main/m/mono/
Probably better to handle this via curl and grep to scrape out some specific packages/versions.

The mono --debug flag should not play a role. It increases mono runtime output, probably interesting for debugging like here indeed, but it must not fix anything by itself. No idea why Sonarr warns about this. We simply to not want to mess the system log :wink:.

Another thing to try is to disable the service file hardenings, if Sonarr was installed or reinstalled after DietPi v6.29 update. Edit /etc/systemd/system/sonarr.service and comment the few lines below “Hardenings”, then systemctl daemon-reload; systemctl restart sonarr.

Hi MichaIng

First many thanks for posting and reading through the posts to catch up.

you mentioned disabling the service file hardenings… thats actually interesting you say that,
i did this about 3 hours ago and Sonarr so far as been running (as a service) now for 3 hours… however i am now on Mono 5.20
so the true test is to get back to the latest mono and try this again.

Thank you so much! Wanted to pass along the link to the temp discord channel the sonarr devs have set up for this issue:

Hi,

small updated on my tests. I installed Sonarr v2 from DietPi software on my RPi3B+ and it was crashing as expected :face_with_raised_eyebrow: . Switching to Sonarr v3 seems to fix the issue. My installation is online since 10 hours now. I know it’s not that long but v2 was crashing after a hour already.

root@DietPi3:~# systemctl status sonarr.service
● sonarr.service - Sonarr Daemon
   Loaded: loaded (/lib/systemd/system/sonarr.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-05-25 00:40:51 CEST; 10h ago
 Main PID: 2288 (mono)
    Tasks: 10 (limit: 2077)



root@DietPi3:~# dpkg -l sonarr
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  sonarr         3.0.3        all          Internet PVR
root@DietPi3:~# mono -V
Mono JIT compiler version 6.8.0.123 (tarball Tue May 12 15:31:43 UTC 2020)

I need to say that I did not configured anything on Sonarr. It was the plain installation already crashing. Therefor I did not test if all configs and settings are taken over correctly into v3. Maybe someone with a real Sonarr installation could test the upgrade to v3 as well. If yes, pls do a backup before, to be able to role back if needed.
[hr]EDIT
Or maybe a side effect due to the new created service file without hardening :thinking:

# /lib/systemd/system/sonarr.service
[Unit]
Description=Sonarr Daemon
After=network.target

[Service]
User=sonarr
Group=sonarr
UMask=002

Type=simple
ExecStart=/usr/bin/mono --debug /usr/lib/sonarr/bin/Sonarr.exe -nobrowser -data=/var/lib/sonarr
TimeoutStopSec=20
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target
root@DietPi3:~#

Hi Joulinar,

I tried running without the hardening settings however this didnt work for me… it would crash nearly on or around the 1hr.

over on the Sonarr site Taloth did a few VM tests and found something to do with how TempFS as been setup
was causing crashing.

I’ve also unlinked the logs files and so far Sonarr as now been running for 6hr+

I am back to mono 6.8.0.123 and sonarr. 2.0.0.5344

Below from Taloth on the test he also done.

Taloth Today at 08:43
- 6.28 unmodified: still stable.
- 6.29.2 vmtouch disabled: dozen crashes on 61 min intervals. 
- 6.29.2 symlinks off: still stable.
- 6.29.2 running in tmpfs: on time crash with "System.Data.SQLite.SQLiteException (0x80004005): attempt to write a readonly database", then corrupt config.xml

an update from Taloth, hopefully that something that helps get this bug patched :slight_smile:

TalothToday at 16:50
I found it
I FOUND IT
Comparing 6aa872b0df37ea7d41acb8e45cc80c23b9b70a31...29ce2d51926f31ec0b439b35cae4e37c39a9aaca · MichaIng/DietPi · GitHub
the fucking hourly cleanup process truncates the .db-wal and .db-shm files
v6.28 · MichaIng/DietPi@e6c3a50 · GitHub
GitHub
v6.28 · MichaIng/DietPi@e6c3a50

  • DietPi-Logclear | Only process log file array, if it has any content

there’s a better link
that’s why it happens hourly on the next rsssync that happens.
btw. what triggered me to look is because twice in a row config.xml in my last VM (everything in tmpfs) was truncated. as well as shm, wal and the log file.
anyway, report that on the dietpi forums and they can get it fixed.