Looking for anything like /var/log/syslog when doing RAMlog #2

General Discussion
1

Intro:

/root/logfile_storage/ contains the accumulated /var/log/ file content, but not the “syslog”, if you mean that literally. E.g. /var/log/syslog is created by rsyslog from what it gets from systemd-journald, hence requires full logging option.

However, systemd-journald runs and logs anyway (to RAM) and instead of having another daemon that writes those logs to a plain text file, you can access them directly via: journalctl Use journalctl -u dietpi-ramlog to see logs for the dietpi-ramlog service, likewise for any other particular service/unit/target/mount, and you can sort and filter and format the output in many ways.

Issue:
@MichaIng In Python I add the system logger to the python logger by

handler = SysLogHandler(addresss = '/var/log/syslog')

I want to use RamLOG Option two, to have a log on disk.

Questions:
How do I have to link this in dietpi?? I mean as I do not have syslog in /var/syslog. I want to avoid using the rsyslog framework…Is there any symlink to this ram logging device?

Above piece of code is taken from the tutorial here.

2

It doesn’t really make sense to post the same question on 2 different platforms RAMlog #2 - where is the syslog? · Issue #4109 · MichaIng/DietPi · GitHub
It is creating overhead for us only, to manage both of them. Please try to stick to a single platform. Thanks for understanding.

4

I thought that github was less maintained. Sorry!

And on Github I reopened just an old case to add my question so maybe we could handle it here, please? I will delete on GitHub now…

5

@Joulinar Could I use instead of /var/sys/log maybe /var/log/journal or even /var/tmp/dietpi/logs/dietpi-ramlog_store as I followed your instructions here.

The latter is maybe the best choice for dynamic data but would it work?

I will try it out, it is maybe more up to the code of the python lib than to dietpi…

6

The other way around. GitHub is our development platform where we host our code and do all our development on.

@MichaIng
could you take this over pls. You are the expert for the logging system :wink:

7

Every file from /var/log, which is a tmpfs (= RAM disk), is synced/concateneted to disk and cleared by RAMlog #2, so no need to manually link during l something.

8

The idea was to log own messages to the system log via the binded python logger. Afterwards i wanted to transfer the complete log via fopen and the socket to the server… (on client runs my rsnapshot)…

9

You could transfer /root/logfile_storage/syslog then. But if it’s about the own messages, and if they must be up-to-date and persistent/complete, easiest is to log them to a file elsewhere. That way it is also assured that the file is not written to while being transferred to the server. It could be inside /tmp, if you prefer to keep it in RAM (not boot persistent then).

You could also log them to the actual syslog and also receive them from there, like:

echo myMessage | logger -t myLogs
journalctl -t myLogs
Jul 16 11:53:41 my.host.org myLogs[10532]: myMessage
1 Like
10

Thanks a lot! But honestly there is nothing like a syslog inside that directory! I am using ramlg#2 option. Should I install syslog to have this feature ??

11

Does the file exist in /var/log/syslog? Otherwise obviously your Python script did not log something yet, or it failed to do so? Note that root permissions are required to create new files in /var/log.

12

Hi MichaIng,

maybe we are talking differently here. I have no syslog (as a file or directory). I want to add a python logger to log to system logs, so that it appears there.

As far as I understood dietpi uses option ramlog #1 and #2 to reduce SSD usage. So it is a concept of two layered process. One is logging temporarily and for example in #2 you copy every hour to somewhere else. So in this case there is no syslog as a file or directory, that is what I understood (and see)?

So I have a process which definitely runs longer than 1 hour, possibly 7-8 hours. It runs once a week.

I have to first look into the ssd stored log and then into the ram file log. I have to put them virtually together to have one log for my complete process, I guess.

My questions are:

  1. Is there a one to one copy from /var/tmp/dietpi/* to /var/logs ? Or where else? I did not find a copy of dietpi-ramlog.log somewhere else…
  2. Or is dietpi-ramlog.log to somewhere else? Is there a mapping table for copying?
  3. Information any where on this?
  4. How can I configure the time when the log is copied from ram to ssd? Or when is it?
    The documentation on dietpi , I have read, but there is not such detailed information.
13

No,you use either ramlog #1 or #2. Or if you want permanent logging to syslog, you can avtivate this too.

As the name suggests, the log stays in RAM and is not written to disk.
If you want logging to disk, choose the third option

14

image
image801×573 61.9 KB

Maybe I misunderstood documentation here, but I wanted to use ram_log #2 as mentioned above, not #1 (only mentioned for completeness). this log is stored permanently on disk every hour. Am I wrong here?

15

What we do is quite simple. Copy everything that is available in /var/log to /root/logfile_storage. If there is no /var/log/syslog, it will not be copied. For this you might need to install rsyslog. The logs are moved once a hour. The execution time for all our jobs can be adjusted using dietpi-cron

1 Like
16

Simple is quite good. Things get complicated by its own… :smiley:

But if I use syslog I give up the advantages of having a ramlog based log, right? So I must live with the spread log files, but for most common message, diepi-.ramlog.log is used?

Or in other words, the two dietpi logs are under your maintenance and the rest is side applications. I have no clue how system log works really in depth, I only assume, that every message of Interest is somehow registered to be logged in syslog. What happens, if there is no syslog? Maybe the apps on report to their “own” owned log?

17

DietPi don’t has own logs. The base OS is using journald for majority of logs. And some application doing logs down to /var/log by it’s own. However we try to to configure most of them to use journald.

You could try to install rsyslog if you need /var/log/syslog. This should be possible on all logging options.

1 Like
18

Not sure why this topic popped up as notification, but some words about /var/log/syslog:

  • On most modern Linux distros, systemd is used as init system, service control and logging daemon. The first two imply the third, since systemd starts all services and hence can catch all their output. But it also explicitly provides the main syslog daemon which any other application can use, e.g. also from console via logger command.
  • systemd does not do classic file logging to /var/log, but all logs are instead stored in binary mode, viewable only to root or explicitly authorised users, via journalctl. For this reason there is no /var/log/syslog file, but of course nothing prevents you from creating it for own usage. Therefore also dietpi-ramlog does not cover system logs, since those are in RAM already, /run/systemd/journal/ to be precise.
  • However, you can enable the classic system file logging by installing rsyslog. It will be a secondary syslog daemon. systemd-journald catches and stores the log as it does now, but then additionally forwards them to rsyslog, which stores them additionally to respective /var/log plain files you know, which would then be covered by dietpi-ramlog as well (which doesn’t make sense, doubled logs in RAM, of course).
  • Another alternative is to store systemd binary logs on disk, so they stay available only via journalctl, but boot persistent. Nothing more than mkdir /var/log/journal is required for this. However, also in this case dietpi-ramlog should be disabled, logging mode None 0.

In your case, I suggest to log to a custom file, like /var/log/myapp.log. You can also store it anywhere else on disk, and have a symlink in /var/log only, so it is not cleared by dietpi-ramlog:

ln -s /root/myapp.log /var/log/myapp.log

How to log to systemd-journald I mentioned above already. This can be also done with the whole application:

python3 /path/to/myapp.py | logger -t myapp

journalctl -t myapp
1 Like
19

Thank you two for answering in that detailed exactness! I think I will now have the time to finish my app!