Vaultwarden installation needs bigger swap file

Hi!

I installed vaultwarden using dietpi-software on my RPi 3 Model B (aarch64), Raspbian Bullseye and DietPi v7.6.2.

Just wanted to let ya know that I needed to set my swap file size to at least 2 GB to make the installation work. I found out by watching the output of “htop” while installing. The installation took 51 minutes in total.

Using less than 2 GB made the system freeze after while.

By default our installation script will check for the availability of at least 2GB total memory. If not, we will create a swap file

https://github.com/MichaIng/DietPi/blob/03bc475e5dfc2a06e105fbbb15d1b95f4277bb45/dietpi/dietpi-software#L6996-L7007

Means on your RPi3, a SWAP file should have been created automatically during install.

Yes, true. But the initially created swap file was too small for my system as I found out. It was 1 GB and prevented the installation from working. I tried to install vaultwarden three times and it worked once I increased the swap file size to 2 GB. Just wanted to let you know of what I found out. :slight_smile:

How many other services you had running? Maybe memory was already utilized?

Not many. It was a clean install from scratch in order to get Bullseye. I only had installed two packages before vaultwarden: unbound + pihole, both by using dietpi-software. That’s it.

Probably we need to raise the min memory a little to le at least 2.5 GiB or so, to be failsafe.

I did a test. Will post quite some information later the evening ones back

As promised my testing result. As I’m far away from my home test lab, I just tested on what was active. It was a RPi3B+ ARMv7 Bullseye. I installed as well PiHole + Unbound. Below the full log of the entire installation. I just cut download and compile sections to shorten the log. As well I disabled SWAP before installation to see how it get created.

root@DietPi3:~# /boot/dietpi/func/dietpi-set_swapfile 0

 DietPi-Set_swapfile
─────────────────────────────────────────────────────
 Mode: Applying 0 /var/swap

[ INFO ] DietPi-Set_swapfile | Disabling and deleting all existing swap files
[  OK  ] DietPi-Set_swapfile | swapoff -a
removed '/var/swap'
[  OK  ] DietPi-Set_swapfile | Setting in /boot/dietpi.txt adjusted: AUTO_SETUP_SWAPFILE_SIZE=0
[  OK  ] DietPi-Set_swapfile | Desired setting in /boot/dietpi.txt was already set: AUTO_SETUP_SWAPFILE_LOCATION=/var/swap
[ INFO ] DietPi-Set_swapfile | Setting /tmp tmpfs size: 486 MiB
[  OK  ] DietPi-Set_swapfile | mount -o remount /tmp
root@DietPi3:~# free -m
               total        used        free      shared  buff/cache   available
Mem:             972          60         383          17         528         832
Swap:              0           0           0
root@DietPi3:~# dietpi-software install 183
[  OK  ] DietPi-Software | Initialised database
[  OK  ] DietPi-Software | Reading database

 DietPi-Software
─────────────────────────────────────────────────────
 Mode: Automated install

[  OK  ] DietPi-Software | Installing vaultwarden: Unofficial Bitwarden password manager server written in Rust
[  OK  ] DietPi-Software | Free space check: path=/ | available=8049 MiB | required=500 MiB
[  OK  ] DietPi-Software | DietPi-Userdata validation: /mnt/dietpi_userdata
[  OK  ] DietPi-Software | Checking network connectivity
[  OK  ] DietPi-Software | Checking DNS resolver
[  OK  ] Network time sync | Completed
[ SUB1 ] DietPi-Services > unmask
[  OK  ] DietPi-Services | unmask : php7.4-fpm
[  OK  ] DietPi-Services | unmask : lighttpd
[  OK  ] DietPi-Services | unmask : cron
[ SUB1 ] DietPi-Services > stop
[  OK  ] DietPi-Services | stop : cron
[  OK  ] DietPi-Services | stop : lighttpd
[  OK  ] DietPi-Services | stop : php7.4-fpm
[  OK  ] DietPi-Software | mkdir -p /mnt/dietpi_userdata/Music /mnt/dietpi_userdata/Pictures /mnt/dietpi_userdata/Video /mnt/dietpi_userdata/downloads /var/www /opt /usr/local/bin
[  OK  ] DietPi-Software | chown dietpi:dietpi /mnt/dietpi_userdata/Music /mnt/dietpi_userdata/Pictures /mnt/dietpi_userdata/Video /mnt/dietpi_userdata/downloads
[  OK  ] DietPi-Software | chmod 0775 /mnt/dietpi_userdata/Music /mnt/dietpi_userdata/Pictures /mnt/dietpi_userdata/Video /mnt/dietpi_userdata/downloads
[ INFO ] DietPi-Software | APT update, please wait...
Hit:1 https://repos.influxdata.com/debian bullseye InRelease
Hit:2 https://deb.debian.org/debian bullseye InRelease
Hit:3 https://deb.debian.org/debian bullseye-updates InRelease
Hit:4 https://deb.debian.org/debian-security bullseye-security InRelease
Hit:5 https://deb.debian.org/debian bullseye-backports InRelease
Hit:6 https://archive.raspberrypi.org/debian bullseye InRelease
Reading package lists...
[  OK  ] DietPi-Software | APT update

 DietPi-Software
─────────────────────────────────────────────────────
 Step: Checking for prerequisite software

[ INFO ] DietPi-Software | Build-Essential will be installed
[ INFO ] DietPi-Software | SQLite will be reinstalled

 DietPi-Software
─────────────────────────────────────────────────────
 Step: Installing Build-Essential: GNU C/C++ compiler, development libraries and headers

[ INFO ] DietPi-Software | APT install for: g++ make automake, please wait...
...
[  OK  ] DietPi-Software | APT install for: g++ make automake

 DietPi-Software
─────────────────────────────────────────────────────
 Step: Installing SQLite: Persistent single-file database system

[ INFO ] DietPi-Software | APT install for: sqlite3 php7.4-sqlite3, please wait...
[  OK  ] DietPi-Software | APT install for: sqlite3 php7.4-sqlite3

 DietPi-Software
─────────────────────────────────────────────────────
 Step: Installing vaultwarden: Unofficial Bitwarden password manager server written in Rust

[  OK  ] DietPi-Software | Checking URL: https://github.com/dani-garcia/vaultwarden/archive/1.22.2.tar.gz
[  OK  ] DietPi-Software | cd /tmp/DietPi-Software
[ INFO ] DietPi-Software | G_THREAD_START_0 | curl -sSfL https://github.com/dani-garcia/vaultwarden/archive/1.22.2.tar.gz -o 1.22.2.tar.gz
[ INFO ] DietPi-Software | APT install for: pkg-config libssl-dev, please wait...
...
[  OK  ] DietPi-Software | APT install for: pkg-config libssl-dev
[  OK  ] DietPi-Software | G_THREAD: All threads finished
[  OK  ] DietPi-Software | tar xf 1.22.2.tar.gz
[  OK  ] DietPi-Software | rm 1.22.2.tar.gz
[  OK  ] DietPi-Software | mv vaultwarden-1.22.2 /opt/vaultwarden
[ INFO ] DietPi-Software | Vaultwarden build requires at least 2 GiB memory. We will now increase your swap size to satisfy this requirement.
[ SUB1 ] DietPi-Set_swapfile > Applying 1 /var/swap
[ INFO ] DietPi-Set_swapfile | Disabling and deleting all existing swap files
[  OK  ] DietPi-Set_swapfile | swapoff -a
[  OK  ] DietPi-Set_swapfile | Free space check: path=/var/ | available=7882 MiB | required=1076 MiB
[  OK  ] DietPi-Set_swapfile | Generating new swap space
[ INFO ] DietPi-Set_swapfile | Size = 1076 MiB
[ INFO ] DietPi-Set_swapfile | Path = /var/swap
[  OK  ] DietPi-Set_swapfile | fallocate -l 1076M /var/swap
[  OK  ] DietPi-Set_swapfile | mkswap /var/swap
[  OK  ] DietPi-Set_swapfile | swapon /var/swap
[  OK  ] DietPi-Set_swapfile | chmod 0600 /var/swap
[  OK  ] DietPi-Set_swapfile | eval echo '/var/swap none swap sw' >> /etc/fstab
[  OK  ] DietPi-Set_swapfile | Setting in /boot/dietpi.txt adjusted: AUTO_SETUP_SWAPFILE_SIZE=1076
[  OK  ] DietPi-Set_swapfile | Desired setting in /boot/dietpi.txt was already set: AUTO_SETUP_SWAPFILE_LOCATION=/var/swap
[ INFO ] DietPi-Set_swapfile | Setting /tmp tmpfs size: 1024 MiB
[  OK  ] DietPi-Set_swapfile | mount -o remount /tmp
[  OK  ] DietPi-Software | mount -o remount,size=1610612736 /tmp
[  OK  ] DietPi-Software | curl -sSfL https://sh.rustup.rs -o rustup-init.sh
[  OK  ] DietPi-Software | chmod +x rustup-init.sh
[  OK  ] DietPi-Software | ./rustup-init.sh -y --default-toolchain none
[  OK  ] DietPi-Software | rm rustup-init.sh
[  OK  ] DietPi-Software | . .cargo/env
[  OK  ] DietPi-Software | cd /opt/vaultwarden
[ INFO ] DietPi-Software | cargo build -j 2 --features sqlite --release, please wait...
info: syncing channel updates for 'nightly-2021-06-24-armv7-unknown-linux-gnueabihf'
info: latest update on 2021-06-24, rust version 1.55.0-nightly (5a7834050 2021-06-23)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-std'
info: installing component 'rustc'
info: installing component 'rustfmt'
    Updating git repository `https://github.com/servo/rust-url`
    Updating git repository `https://github.com/jjlin/job_scheduler`
    Updating git repository `https://github.com/SergioBenitez/Rocket`
    Updating crates.io index
    Updating git repository `https://github.com/SergioBenitez/Devise.git`
 Downloading crates ...
  Downloaded aho-corasick v0.7.18
  Downloaded bytes v0.4.12
  Downloaded futures-channel v0.3.16
...
   Compiling xml5ever v0.16.1
   Compiling rocket_contrib v0.5.0-dev (https://github.com/SergioBenitez/Rocket?rev=263e39b5b429de1913ce7e3036575a7b4d88b6d7#263e39b5)
   Compiling markup5ever_rcdom v0.1.0
    Finished release [optimized] target(s) in 76m 40s

All was working and Vaultwarden got compiled in 76 minutes. SWAP was sufficient and did not exhausted. Below a full monitoring view for the 90 minutes of installation. You see the SWAP off and how it was cretated. Max SWAP usage was short before the end at around 70%. Enjoy looking to the statistics. They are available for next 7 days.

https://snapshot.raintank.io/dashboard/snapshot/LdnUL2MJxkkgKMrwgFjgelJ2erT7yu8d

I need to say that my system was compiling with 2 processes as intended by our script. What I could think of, that for the OP installation was running with more prosseses ending up in a higher memory usage. :thinking:

Awesome report, many thanks :slight_smile:. More than 2 build jobs should only run when more than 3 GiB total memory were detected, so with 1 GiB swap on 1 GiB RPi3 this shouldn’t be the case. One difference is OP installed it on aarch64, you on armv7l, and I’m pretty sure, as always, 64-bit requires more memory, as related files are usually larger, so that may be the key here, aside of possibly running services which are not controlled by DietPi-Services, like Pi-hole/Unbound, VNC, VPN and such things.

Yeah I know. I could check 64bit only once I’m back home having physical access to my test lab. I could run it on RPi4B 8GB ARMv8 but result might be different. Time to use PXE boot on my RPi’s to be able to switch images more easily :rofl:

Hello!

Awesome stats. :slight_smile: Unfortunately I can’r provide you with data that soffisticatedly.

I used htop to monitor ‘real-time’ what was going on. In the first half hour only about 1 GB was used, then it increased to around 1.4 GB on average and only in the last step of the installation (like the last 10 - 5 minutes) it rose to 2.0 GB. That’s all I can tell.

Yes, I’m on aarch64, maybe that’s an issue here, too.

I wasn’t supposed to cause work on your part, I only wanted to report what I found out on my system. :slight_smile: Vaultwarden is by the way the only installation that needed that much swap space here. All other programs were satisfied with a lot less then 1 GB of swap space.

On the failed install (1GB swap), do you remember how man man CPU cores got utilized? 2 or more?

htop shows 4 cores for my system. And the bars looked pretty busy during installation, i.e. often at 100%. However, as far as I can remember the cores were only used sequentially, i.e. not all at the same time but one after another. Unfortunately I can’t remember for sure if only 1 core was used at a time or two. :thinking: I tend to say it was only one core at a time …

usually it should have been 2 cores at the same time on a standard setup on RPi3B+
Ok I will replicate next week and share my monitoring result afterwards.

ZRAM candidate??

MichaIng, huidbui25 as promised, I did some more testing.

Below the link to my monitoring data. These time I reduced the view to CPU, Memory and SWAP usage. Again it is valid for 7 days

https://snapshot.raintank.io/dashboard/snapshot/qLwB1H6QtUgOB3WNKivlWWUh3prSXUga

What I testes: RPi3B+ ARMv8 Bullseye

09:04 - 09:09

  • SWAP set to 1GB
  • 2 parallel jobs triggered
  • already whole Mem + SWAP used after minutes
  • I cancelled installation manually

09:13 - 10:32

  • SWAP set to 1.5GB
  • 2 parallel jobs triggered
  • SWAP usage increased over time until it reached max
  • installation killed by OOM reaper

10:34 - 12:02

  • SWAP set to 2GB
  • 2 parallel jobs triggered
  • SWAP usage increased over time until it reached 1.85GB usage short before installation finished
  • installation finished after 83 minutes

12:16 - 13:28

  • SWAP set to 3GB
  • max number parallel jobs triggered (4 jobs)
  • SWAP usage increased over time until it reached 2.00GB usage short before installation finished
  • installation finished after 69 minutes

And just to compare long running installation time vs docker

13:30 - 13:34

  • SWAP set to 3GB (still)
  • docker software + portainer + vaultwarden install
  • SWAP usage > none
  • installation finished after 4 minutes

Summary:

  • 64bit ARMv8 requires much more Memory/SWAP compare to 32bit ARMv7
  • even 2 GB SWAP (3GB total) seems to be close as it reaching the limits on an empty system already
  • 3 GB SWAP (4GB total) seems to be fine but this will trigger quite high CPU usage as it will utilize all 4 cores
  • using docker image is much faster and done in a couple of minutes without SWAP usage
  • downside on docker is the overhead of around 90-100 MB memory

Many thanks for testing. Just to be sure, you did every test on a clean system, respectively cleared /tmp before doing the next run? As every Rust build produces a /tmp entry that stays, at least when the build is cancelled, and a new entry is created on next attempt (sadly I didn’t find a way to resume a cancelled build).

So basically we need to to set the min memory 1 GiB higher for ARMv8.

Absolutely amazing testing you did! :open_mouth: :slight_smile: It confirms the observations I made on my system.

If I could choose, I’d opt for the docker-free installation. Though it consumes considerably more resources during installation, it runs with lower impact later on. However, maybe a note or so in dietpi-software about the duration of the installation would be helpful for users.

To avoid any left overs I did a reboot and performed another installation. To simulate some load I kept Docker + Telegraf running. As well I extended monitoring view and added /tmp. As you can see it was empty before install. Runtime was nearly same. SWAP usage was around 1.99 GB at the end. Even on a rebooted system it’s close if you count some apps running. No problem on an empty system but if you have something running already it might not fit.

There you go with the charts https://snapshot.raintank.io/dashboard/snapshot/yDu2Ca68nWexnfuATU5BJs3Oppszw03w

Happy Christmas to all of you!

Joulinar, after installing vaultwarden, is it possible to delete swap partition?

I am seeing that is almost full and physical RAM is at 30% and I’m afraid the SD wear out.

What do you think?