How do I run a Raspberry Pi 4 DietPi image using Virt-Manager or QEMU?

bvobart · May 28, 2024, 9:36pm

Hi, I’m developing a custom DietPi image that I want to be able to test by emulating a Raspberry Pi on my personal desktop PC and running the image as a VM before I flash it onto my real Raspberry Pi 4B (though 3B should also be supported).

However, I can’t figure out how to configure Virt-Manager or QEMU in such a way that it will emulate an ARM64 device that actually boots the current RPi 2/3/4 DietPi image. Most resources I find online emulate earlier 32-bit Raspberry Pis, give me other errors, or rely on dhruvvyas90/qemu-rpi-kernel for kernel and DTB files, which is no longer being maintained and does not contain what is necessary for Debian Bookworm (what DietPi is based on at the time of writing). I’ve tried extracting the kernel8.img and .dtb files from the boot partition of the image and using those with Virt-Manager / QEMU, but to no avail.

Since my end goal is to test my custom DietPi image, built on CI where I cannot also make an x64 version, I can’t just use an x64 DietPi image. To make matters more complicated, I also need to test the DE and DE settings I pre-install, so I need to have video emulated as well.

Could you help me setup a VM in Virt-Manager that emulates an ARM64 chip and runs DietPi?

This is the Virt-Manager configuration I’ve tried so far:

<domain type="qemu">
  <name>virt-dietpi</name>
  <uuid>2970c242-ab17-405c-a769-d8d8ab09cfaa</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://debian.org/debian/12"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">4194304</memory>
  <currentMemory unit="KiB">4194304</currentMemory>
  <vcpu placement="static">4</vcpu>
  <os firmware="efi">
    <type arch="aarch64" machine="virt-8.1">hvm</type>
    <firmware>
      <feature enabled="no" name="enrolled-keys"/>
      <feature enabled="no" name="secure-boot"/>
    </firmware>
    <loader readonly="yes" type="pflash" format="qcow2">/usr/share/edk2/aarch64/QEMU_EFI-silent-pflash.qcow2</loader>
    <nvram template="/usr/share/edk2/aarch64/vars-template-pflash.qcow2" format="qcow2">/var/lib/libvirt/qemu/nvram/virt-dietpi_VARS.qcow2</nvram>
    <kernel>/home/bart/projects/pi-img/kernel8.img</kernel>
    <cmdline>root=/dev/vda2 panic=1</cmdline>
    <boot dev="hd"/>
    <bootmenu enable="no"/>
  </os>
  <features>
    <acpi/>
    <gic version="2"/>
  </features>
  <cpu mode="custom" match="exact" check="none">
    <model fallback="allow">cortex-a72</model>
  </cpu>
  <clock offset="utc"/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-aarch64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="raw"/>
      <source file="/home/bart/Downloads/DietPi_RPi-ARMv8-Bookworm.img"/>
      <target dev="vda" bus="virtio"/>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x8"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x9"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0xa"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0xb"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0xc"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0xd"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0xe"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0xf"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="12" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="12" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="13" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="13" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="14" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="14" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="52:54:00:4e:59:1d"/>
      <source network="default"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="system-serial" port="0">
        <model name="pl011"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <tpm model="tpm-tis">
      <backend type="emulator" version="2.0"/>
    </tpm>
    <audio id="1" type="none"/>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </memballoon>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </rng>
  </devices>
</domain>

but that just prints:

EFI stub: Booting Linux Kernel...
EFI stub: Generating empty DTB
EFI stub: Exiting boot services...

and then hangs forever, constantly using 100% of 1 CPU core on my host machine, hopping back and forth between two different cores.

Virt-Manager version 4.1.0
libvirtd version 9.7.0
QEMU emulator version 8.1.3
Host OS: Nobara Linux 39, fully up-to-date as of 2024-05-28
DietPi version: DietPi_RPi-ARMv8-Bookworm downloaded on 2024-05-28

Joulinar · May 30, 2024, 10:03am

I’m not able to answer your question but maybe could give a hint. On DietPi we use GitHub actions to build all our images automatically. Probably you could have a look to the build script to get some ideas

bvobart · May 30, 2024, 2:30pm

Thanks for those links I looked through DietPi’s build process and I notice that you’re using mmdebstrap to create a chroot environment with a minimal base which you then build upon, using qemu for platform emulation. That’s a similar approach as what the pguyot/arm-runner-action that I’m using to build my custom DietPi image on CI also does, as described here:

This wiki page explains the required steps needed to be able to chroot into a root filesystem whose platform (e.g. aarch64) is different from the running system (e.g. amd64). Software can then be compiled within the chroot transparently using QEMU emulation.

That works fine for command-line compiling of ARM(64) programs and compiling DietPi images, but I don’t think this covers my use-case as I want see my DietPi(-based image) booting the graphical environment as well and without distro requirements or weird hacks on my host system.

As simple as it sounds: I just want to run the Raspberry Pi 2/3/4 DietPi image (and derivatives) as a VM on my PC.

It seems I’m going to have to try digging into dhruvvyas90/qemu-rpi-kernel’s tools directory in the hope that I can compile a versatile-pb Linux kernel and DTB file with it, that is compatible with current-day DietPi. Then I should be able to use those similarly to the various tutorials and blog posts that require those versatible-pb files.

Just a couple questions then:

What Linux kernel version does DietPi currently run?
Is that the same as what can be found on the raspberrypi/linux GitHub repo? If not, where can I find the kernel sources?

boxer · May 30, 2024, 4:27pm

I think you should rather check proper build framework. It provides endless hacking opportunities …

MichaIng · May 30, 2024, 7:23pm

The question is what you actually want to test? QEMU cannot emulate the exact boot stages of the Raspberry Pi, with its closed source bootloader, loading kernel, dtb and in case initramfs from FAT partition etc. If you want to test the userland only, i.e. everything from systemd init, then you could boot it as container, using e.g. systemd-nspawn -bD mountpoint (systemd-container package, and mounting the image via losetup as loop device).

If you really want to test the kernel as well, I guess it strictly requires the device tree as well, which I do not see in your config. But I do not know virt-manager and whether/how it is able to load device trees. Also, I see <os firmware="efi">, while our image, same as RPi OS, do not come with an EFI partition and bootloader.

It is in fact the kernel builds from the official Raspberry Pi OS repository. So yes, from this GitHub repository, and the actual stable builds from here: GitHub - raspberrypi/firmware at stable
Ah, though, the images from our download page still have the old kernel/firmware stack. Above builds is what you find on our “RPi1/2/…” images, resp. using the 1/2/4/5 hardware IDs with our build script.

bvobart · August 12, 2024, 11:23pm

Hi, sorry for the late response, but I finally got around to trying your suggestion

systemd-nspawn seems to do the trick for me, as I indeed only need to test the userland. I didn’t know about that tool yet, thanks a lot for the tip!

I wrote a quick script just to try systemd-nspawn with a DietPi image on my desktop PC running Nobara 39. Here’s the script:

Click to show script

#!/bin/bash -e

cd "$(dirname "$0")"

function assert_is_installed() {
  if ! command -v "$1" &> /dev/null; then
    echo "Error: $1 not found. Please install it."
    exit 1
  fi
}

function assert_is_defined() {
  if [ -z "$1" ]; then
    echo "Error: $1 is not defined."
    exit 1
  fi
}

function assert_file_exists() {
  if [ ! -f "$1" ]; then
    echo "Error: $1 not found."
    exit 1
  fi
}

function assert_directory_exists() {
  if [ ! -d "$1" ]; then
    echo "Error: $1 not found or is not a directory."
    exit 1
  fi
}

function mount_img() {
  local image_file="$1"
  local mount_point="$2"

  loop_device="$(sudo losetup -f)"
  sudo losetup -P "$loop_device" "$image_file"

  mkdir -p "$mount_point"
  sudo mount "${loop_device}p2" "$mount_point"
}

function unmount_img() {
  mount_point="$1"
  sudo umount "$mount_point"
  sudo losetup -d "$loop_device"
  # rmdir "$mount_point"
}

function main() {
  local image_file="$1"

  assert_is_installed systemd-nspawn
  assert_is_defined "$image_file"
  assert_file_exists "$image_file"

  # strip path and extension
  local image_name=$(basename "$image_file" .img)
  local mount_point="mount/$image_name"

  echo "> Mounting image: $image_name"
  mount_img "$image_file" "$mount_point"

  set +e # ignore errors, because we want to unmount the image even if nspawn fails

  echo "> Running systemd-nspawn"
  sudo systemd-nspawn -bD "$mount_point"

  echo "> Unmounting image: $image_name"
  unmount_img "$mount_point"
}

main "$@"

Running it, I see systemd-nspawn booting up DietPi until a login prompt, although there are are a couple of systemd services and DietPi services (dietpi-ramlog.service and dietpi-preboot.service) failing to start correctly. I also still need to configure display output (I wanna test the DE I’m including in my image) and networking. I haven’t debugged and tested much further yet, so I can’t give anymore updates for now, but I will definitely report back when I take some time to continue with this.

@boxer Thanks for your suggestion, I looked into the Armbian build framework that you mention and I think it would be a good choice if I wanted to give my users a locked-down image which never updates and contains exactly the components they need and nothing more, nothing less. Updating is then done through reflashing. However, I want my users to be able to tinker with it themselves if they like to and also be able to update the image as they go, without reinstalling. DietPi allows that and has ample opportunity for people to tinker and extend my pre-made image to their heart’s content.

MichaIng · August 28, 2024, 2:12pm

Looks somewhat similar to what I have written together : hacks/boot_img.bash at 3e6fb06fc0b383bf90d9e374ee2bc6001ff6835b · MichaIng/hacks · GitHub

system · February 24, 2025, 2:13pm

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.