I tried running dietpi-software reinstall 65 - this is the result:
[ 7043.923205] netdata [25992]: 2021-10-19 02:05:53: netdata INFO : MAIN : CONFIG: cannot load cloud config ‘/var/lib/netdata/cloud.d/cloud.conf’. Running with internal defualts.
[ 7043.957675] netdata[25992]: 2021-10-19 02:05:53: netdata FATAL : netdata : Cannot cd to directory ‘/var/log/netdata’ # : No such file or directory
Sure enough, there is indeed no directory /var/log/netdata, and although /var/lib/netdata/cloud.d/ exists, there is no cloud.conf in it.
Hi,
The missing cloud.conf is just a warning and should not have any impact as system is failing back to default values.
Missing /var/log/netdata is the real error. Did you tried to create it?
No - but I’ll do that now. What permissions are needed?
OK - created it & tried to restart services - got a whole scrolling screen of errors! Service state went from Failed to Inactive in the dietpi services management utility
So - reinstalled netdata again and service started, but I see lots of fast scrolling error messages - many are permission denied errors for docker, resulting in docker container names not being resolved . netdata was running when checked from a browser on another machine.
I then stopped the service & restarted it - again a whole bunch of error messages in the console. Netdata runs while these are displaying, but if I press enter key & return to the services management app, it says Inactive & netdata is no longer running when checked from a browser on another machine.
While its running I also get error message /etc/netdata/.environment - no such file or directory. The /etc/netdata directory does exist & has some files & subdirectories in it.
Reinstalled netdata again from cli & it starts & stays started until II try & restart it in dietpi-services it then will not stay started once I restrun to the dietpi-services utility
once service is failing, you could have a look to journalctl -u netdata to check what the issue is on the service.
OK - I rebooted the system, and then found that /var/log/netdata had disappeared again.
QUESTION: Any reason this would happen?
Anyway, I recreated it & the service again failed. As I kept seeing a lot of permission denied messages, I then ran chmod 777 /var/log/netdata
Full permissions then allowed the service to start correctly (when created, the permissions were 755)
QUESTION: what should the permissions be for this directory?
I re-booted and this time /var/log/netdata was still on the machine & all works properly.
BTW - I have now got netdata to properly display the docker container names. To do so. I added the netdata user to the docker group. By granting permissions, netdata can now read all required docker information.
Hi,
correct permission on /var/log/netdata
User:Gorup > netdata:adm
permission: rwxr-s— > 2750
root@DietPi4:~# ls -la /var/log | grep net
drwxr-s--- 2 netdata adm 100 Oct 20 12:07 netdata
root@DietPi4:~#
This way Netdata should be able to write to the directory
Why this could happen? Because on DietPi, /var/log is a tempfs.
root@DietPi4:~# df -h | grep log
tmpfs 50M 104K 50M 1% /var/log
root@DietPi4:~#
Means, it is a temporary file system. Aim is to reduce r/w operation on SD cards. Usually this will system is located in memory and saved down to disk on every clean reboot/shutdown. If you install some software and you just simply power cycle without performing a reboot, newly created information in /var/log is lost. But as said, as long as you perform a regular reboot, this should not be the case.
And of cause, Netdata needs access to Docker data. Therefore adding the user to docker group is correct approach.
Thanks for this information.
I’ve updated the permissions etc as indicated above.
With regard to the tmpfs file system for /var/logs - does dietpi delete the persistent data immediately after copying to the ram-disk, or does it retain the data and just over-write it on shutdown? I ask, as I would like to know if a sudden power-off of the system is likely to result in loss of the netdata directory. If the data is not deleted, then it should just reload the netdata directory as it was at the last graceful shutdown.
I would also be interested to know how to have /var/logs be stored on the root partition, rather than on a tmpfs mount. (I run dietpi on a virtual machine, not from a flash drive, so not concerned with issues about flash storage).
then it should just reload the netdata directory as it was at the last graceful shutdown.
It is working like this. Data are available on disk, get written down in shutdown and restored on reboot.
But usually nothing to worry about even on a VM. By default log files will be cleared on hourly basis anyhow 
OK - thanks for that.
If I understand you correctly, my initial problem was likely to be because I had a system crash (hypervisor) after the initial installation of netdata, but before I was able to gracefully shut down the machine?
yup would explain it very much. To be on safe side, a reboot would be required after important installations. This is to ensure all tmpfs are stored (initially) on disk.