I’ve connected my Pi to a HDD and shifted the user data location to it. That’s been working fine with no problem.
Now, I used to use Dietpi-RAMlog option 1, but I need to save logs now.
I don’t want to use the Full Logging option on my SDcard due to the decreased lifespan, so I need to change the location of where my logs are stored to my hard disk just like swapfile. (Very easy to do, thank you for making the process painless for beginners)
Reading online, I’ve come to the conclusion that its best to just move the entire /var/log directory to the hard disk rather and journal and syslog, is this correct?
Changing the rootFS to the hard disk will move everything from the SDcard to the hard disk and also solve my problem right? (This will cause the Pi to boot off the hard disk rather than the SDcard?)
Lastly are there any disadvantages to changing the rootFS to the hard disk?
Can someone please advise and detail the best method to solve the issue.
There are some misunderstandings on your side. By default /var/log is an own mount point and not just a folder that can be moved. Furthermore /var/log is a tmpfs. Means, it’s temporary file system. And using LogMode #1 will clean your log files once an hour.
If you like to save your logs, you would need to set LogMode #2 or FULL. If you go for LogMode #2, your logs will be saved once an hour to disk. And this folder could be linked to your HDD if needed.
And if you like to boot from HDD, you would need to move BootFS as well as RootFS. Just moving BootFS will not make you system booting from HDD.
Maybe one of the developers can verify if log2ram is compatible on the dietpi setup.
It seems the Dietpi-RAMlog already does that for you…it keeps the logs in it’s own folder in ramspace, and writes to the drive occasionally rather than constantly in some cases.
If I understand correctly, you need persistent logs (hence no RAM logging) but don’t want to bear the SD card for this. Then indeed your idea would work, to move the logs to an external drive, e.g.:
dietpi-services stop # Stop services to assure no log file is locked anymore
mv /var/log /mnt/<external>/log
ln -s /mnt/<external>/log /var/log
dietpi-services start
And yes, moving the root file system completely to an external drive of course includes the logs. If you don’t mind a spinning HDD or have a space SSD, in both cases attached via dedicated power supply (!), then I’d recommend that setup generally. From RPi 3 on you could even flash the image to an external drive directly to even boot from there, i.e. no SD card required at all.
Thank you for the reply, I should have phrased the question in a better way, I understand that /var/log is a tmpfs that’s mounted to a specific location, initially, it was in RAM and cleared hourly, but I wanted to save logs, so I decided to enable Full Logging to mount and save logs to disk for later use, but instead of the SDcard, I wanted to mount them to my HDD instead, and was confused about doing so. I think the answer by MichaIng as correct as it matches my question.
Also, by your explanation it seems transferring RootFS would provide no additional benefits without transferring BootFS too. Crazy question, but will I get better performance distributing the data though the SDcard and HDD as I’ll have both their read/write speeds available (I’m thinking of something like a RAID 0 setup where I’ve got 2 disks), or having everything on 1 disk much better?
Thanks for responding. I was considering this option, but currently I need the logs to find out why my Pi would randomly crash after 72-80 hours, since it could happen anytime, the hourly save wasn’t going to suffice if the Pi crashed without saving the logs.
Thank you for the response. My Pi is out of sight anyways, so a spinning HDD isn’t an issue. Also your assumptions are correct, persistent logging is exactly what I need on my external drive.
I’ve seen a lot of posts about having a dedicated power supply for your external drive. Unfortunately, my WD Blue Hard Drive connect through a single USB 2.0 cable, so I have to rely on the Pi for power.
Logging full btw installs rsyslog + logrotate (aside of disabling RAMlog), which is the “classic” way of having persistent system logs as well. Just for completeness and as I aim to change this at some point in the future:
systemd-journald is active as syslog daemon already (and cannot really be disabled when using systemd as init system), so having rsyslog installed doubles it. Actually rsyslog pulls its logs from systemd-journald in this case and writes it to syslog, daemon.log, kernel.log, auth.log and such in /var/log.
Instead, a cleaner solution is to have systemd-journald writing to disk itself, which by default writes to RAM only. This can be achieved simply by creating mkdir /var/log/journal. There are not plain text log files created, but the logs are accessible persistently across boots via journalctl (as now already, but limited to current boot session). It might then make sense to adjust max size and lifetime of system logs via /etc/systemd/journald.conf.
logrotate still make sense to rotate logs of other applications like apt, dpkg and service logs.
Transferring RootFS only usually provides MUCH benefit for stability (life span of SD cards) and performance (usually SD cards are slow) . Since the /boot partition is not written to often, or at least no large files, it has nearly no impact. I made the suggestion to flash the whole image (both partitions) to an external drive only as it is easier and might feel more consequent instead of using both.
Since I cannot imagine any task that writes to /boot and / partitions concurrently, you’ll not get any benefit of using an SD card for /boot and an external drive root /, compared to both on external drive. A RAID setup where single files are explicitly written to multiple drives concurrently would allow such, but that is not possible and not reasonable to do across SD card + external drive . And thinking further, I don’t think that RPi can mount the root files system from RAID on two external drives, although never tested or saw anyone testing it. This is more an idea for separate data drives, e.g. a file (sharing) server that stores data on a dedicated data RAID.
About dedicated power supply for external drives. I would consider this as a general recommendation to assure no random drive locks, crashes and in worst case data loss happens, but there might be cases where it works through RPi USB power only. In the end you simply need to try it out. And if you have persistent system logs enabled, in case of a crash, you should be able to track it down being due to a voltage drop on RPi, HDD or both or a different reason. Note that all RPi models have an absolute maximum of 1.2A = 6W on all USB ports together, i.e. attaching a drive with 5W and a USB WiFi with 2W is not possible, as an example. And since this is the upper limit, random voltage fluctuations might lead to USB device crashes when reaching that limit on a regular basis. Then of course bad cables further decrease voltage. So if your external drive takes max 4W and you do not attach any other USB device, it could be fine, but if it takes any more, I would not risk it since random system crashes are nasty .
That’s some great points
I’ll be sure to edit the /etc/systemd/journald.conf so things don’t just keep piling up.
Ahh using different drives was an idea, thanks for clearing that up.
I’m using the official were supply, my testing hasn’t reveled any flaws so far. Hopefully nothing bad happens.
P.S. I don’t know if its just a spelling mistake or mine is different, my journal settings are stored under /etc/systemd/journald.conf. I’m just adding this if anyone else gets confused like me.
. Since the /boot partition is not written to often, or at least no large files, it has nearly no impact. I made the suggestion to flash the whole image (both partitions) to an external drive only as it is easier and might feel more consequent instead of using both.
.
.