Orange Pi 5 - Boot from NVME

helmer · 13 October 2024 13:33

Thank you for your effort!
I have done everything. Unfortunately, it still doesn’t work.

root@DietPi:~# /boot/dietpi/func/dietpi-set_software apt clean

DietPi-Set_software
─────────────────────────────────────────────────────
Mode: apt (clean)
[ OK ] DietPi-Set_software | apt-get clean
[ OK ] DietPi-Set_software | rm -Rf /tmp/apt /var/cache/apt /var/lib/apt/lists /var/lib/apt/mirrors /var/lib/apt/periodic /var/lib/apt/cdroms.list
[ OK ] DietPi-Set_software | mkdir -p /var/cache/apt/archives
[ OK ] apt clean | Completed
root@DietPi:~# apt update
Get:1 https://dietpi.com/apt bookworm InRelease [3529 B]
Get:2 https://dietpi.com/apt all InRelease [25.3 kB]
Get:3 https://deb.debian.org/debian bookworm InRelease [151 kB]
Get:4 https://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:5 https://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:6 https://deb.debian.org/debian bookworm-backports InRelease [59.0 kB]
Get:7 https://dietpi.com/apt bookworm/main arm64 Packages [2172 B]
Get:8 https://dietpi.com/apt all/orangepi5 arm64 Packages [1660 B]
Get:9 https://dietpi.com/apt all/orangepi5 all Packages [528 B]
Get:10 https://deb.debian.org/debian bookworm/main arm64 Packages [8689 kB]
Get:11 https://deb.debian.org/debian bookworm/non-free-firmware arm64 Packages [5832 B]
Get:12 https://deb.debian.org/debian bookworm/contrib arm64 Packages [45.7 kB]
Get:13 https://deb.debian.org/debian bookworm/non-free arm64 Packages [75.8 kB]
Get:14 https://deb.debian.org/debian bookworm-updates/non-free arm64 Packages [12.0 kB]
Get:15 https://deb.debian.org/debian bookworm-updates/contrib arm64 Packages [768 B]
Get:16 https://deb.debian.org/debian bookworm-updates/non-free-firmware arm64 Packages [616 B]
Get:17 https://deb.debian.org/debian bookworm-updates/main arm64 Packages [2468 B]
Get:18 https://deb.debian.org/debian-security bookworm-security/main arm64 Packages [185 kB]
Get:19 https://deb.debian.org/debian bookworm-backports/main arm64 Packages [249 kB]
Get:20 https://deb.debian.org/debian bookworm-backports/contrib arm64 Packages [4964 B]
Get:21 https://deb.debian.org/debian bookworm-backports/non-free arm64 Packages [6864 B]
Get:22 https://deb.debian.org/debian bookworm-backports/non-free-firmware arm64 Packages [3852 B]
Fetched 9628 kB in 4s (2202 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.

root@DietPi:~# cd /tmp

root@DietPi:/tmp# ls -la

total 4
drwxrwxrwt 6 root root 120 Oct 13 15:18 .
drwxr-xr-x 18 root root 4096 Aug 28 21:05 ..
drwxrwxrwt 2 root root 40 Oct 13 11:41 .ICE-unix
drwxrwxrwt 2 root root 40 Oct 13 11:41 .X11-unix
drwxrwxrwt 2 root root 40 Oct 13 11:41 .XIM-unix
drwxrwxrwt 2 root root 40 Oct 13 11:41 .font-unix

root@DietPi:/tmp# mount /dev/nvme0n1p1 /media
mount: (hint) your fstab has been modified, but systemd still uses
the old version; use 'systemctl daemon-reload' to reload.

root@DietPi:/tmp# systemctl daemon-reload

root@DietPi:/tmp# ls /media/
bin boot dev etc home lib lost+found media mnt opt proc root run sbin srv sys tmp usr var

root@DietPi:/tmp# apt download linux-{image,dtb}-vendor-rk35xx
**E:** Can't find a source to download version '24.11.0-trunk-dietpi1' of 'linux-dtb-vendor-rk35xx:arm64'
**E:** Can't find a source to download version '24.11.0-trunk-dietpi1' of 'linux-image-vendor-rk35xx:arm64'

root@DietPi:/tmp# ls /media/tmp/
linux-dtb-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb linux-image-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb

root@DietPi:/tmp# cp /media/tmp/*.deb .

root@DietPi:/tmp# ls -la
total 31896
drwxrwxrwt 6 root root 160 Oct 13 15:21 .
drwxr-xr-x 18 root root 4096 Aug 28 21:05 ..
drwxrwxrwt 2 root root 40 Oct 13 11:41 .ICE-unix
drwxrwxrwt 2 root root 40 Oct 13 11:41 .X11-unix
drwxrwxrwt 2 root root 40 Oct 13 11:41 .XIM-unix
drwxrwxrwt 2 root root 40 Oct 13 11:41 .font-unix
-rw-r--r-- 1 root root 2035636 Oct 13 15:21 linux-dtb-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb
-rw-r--r-- 1 root root 30620936 Oct 13 15:21 linux-image-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb

root@DietPi:/tmp# dpkg -i --root=/media linux-*.deb
**dpkg**: **unrecoverable fatal error, aborting:**
unknown system group 'messagebus' in statoverride file; the system group got removed
before the override, which is most probably a packaging bug, to recover you
can remove the override manually with dpkg-statoverride

root@DietPi:/tmp# cat /var/lib/dpkg/statoverride
root crontab 2755 /usr/bin/crontab

root@DietPi:/tmp# cat /media/var/lib/dpkg/statoverride
root crontab 2755 /usr/bin/crontab
root root 1733 /var/lib/php/sessions
root messagebus 4754 /usr/lib/dbus-1.0/dbus-daemon-launch-helper

root@DietPi:/tmp# grep messagebus /etc/passwd /media/etc/passwd
/media/etc/passwd:messagebus:x:100:107::/nonexistent:/usr/sbin/nologin

root@DietPi:/tmp# export DPKG_ROOT='/media'

root@DietPi:/tmp# dpkg -i --root=/media linux-*.deb
**dpkg**: **unrecoverable fatal error, aborting:**
unknown system group 'messagebus' in statoverride file; the system group got removed
before the override, which is most probably a packaging bug, to recover you
can remove the override manually with dpkg-statoverride

root@DietPi:/tmp# unset DPKG_ROOT

root@DietPi:/tmp# cp -a /var/lib/dpkg/statoverride /var/lib/dpkg/statoverride.bak
dpkg-statoverride --remove /usr/lib/dbus-1.0/dbus-daemon-launch-helper
dpkg -i --root=/media linux-*.deb
mv /var/lib/dpkg/statoverride.bak /var/lib/dpkg/statoverride
**dpkg-statoverride:** **warning:** no override present
**dpkg**: **unrecoverable fatal error, aborting:**
unknown system group 'messagebus' in statoverride file; the system group got removed
before the override, which is most probably a packaging bug, to recover you
can remove the override manually with dpkg-statoverride

helmer · 13 October 2024 13:52

One moment - the first command “dpkg -i --root=/media linux-*.deb” didn’t produce any output. Maybe it worked. I’m trying to boot from the NVME…

helmer · 13 October 2024 14:02

… didn’t help.

Joulinar · 13 October 2024 14:06

@MichaIng maybe still an issue with uboot?

Booting from NVMe gives following

▒DDR d5483af87d cym 23/11/23-16:15:24,fwver: v1.15
LPDDR4X, 2112MHz
channel[0] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
channel[1] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
channel[2] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
channel[3] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 Size=2048MB
Manufacturer ID:0x1
CH0 RX Vref:28.9%, TX Vref:22.8%,21.8%
CH1 RX Vref:26.7%, TX Vref:22.8%,22.8%
CH2 RX Vref:27.9%, TX Vref:20.8%,20.8%
CH3 RX Vref:28.9%, TX Vref:19.8%,19.8%
change to F1: 528MHz
change to F2: 1068MHz
change to F3: 1560MHz
change to F0: 2112MHz
out
U-Boot SPL board init
U-Boot SPL 2017.09 (Aug 01 2024 - 08:25:11)
unrecognized JEDEC id bytes: 00, 00, 00
unknown raw ID 0 0 0
Trying to boot from MMC2
MMC: no card present
mmc_init: -123, time 0
spl: mmc init failed with error: -123
Trying to boot from MMC1
Card did not respond to voltage select!
mmc_init: -95, time 14
spl: mmc init failed with error: -95
SPL: failed to boot from all boot devices
### ERROR ### Please RESET the board ###
# Reset the board to bootrom #

MichaIng · 13 October 2024 14:30

One after another. It broke with the kernel upgrade to 24.11, not with U-Boot. If latest U-Boot breaks it as well in its own way, that would be a dedicated issue.

@helmer
So the problem is with statsoverride in the root mount while the host does not have the user, i.e. the other way round. Also, weird about the APT update, maybe it just does not want to download an older version/downgrade:

cd /tmp
apt download --allow-downgrade linux-{image,dtb}-vendor-rk35xx
cp -a /media/var/lib/dpkg/statoverride /media/var/lib/dpkg/statoverride.bak
sed -i '/messagebus/d' /media/var/lib/dpkg/statoverride
dpkg -i --root=/media linux-*.deb
mv /media/var/lib/dpkg/statoverride.bak /media/var/lib/dpkg/statoverride

helmer · 13 October 2024 14:52

root@DietPi:/tmp# apt download --allow-downgrade linux-{image,dtb}-vendor-rk35xx
E: Command line option --allow-downgrade is not understood in combination with the other options

I had downloaded “linux-dtb-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb” and “linux-image-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb” via the July Dietpi release. So I continued working with that…

root@DietPi:/tmp# ls -la
total 31896
drwxrwxrwt 6 root root 160 Oct 13 16:36 .
drwxr-xr-x 18 root root 4096 Aug 28 21:05 ..
drwxrwxrwt 2 root root 40 Oct 13 16:33 .ICE-unix
drwxrwxrwt 2 root root 40 Oct 13 16:33 .X11-unix
drwxrwxrwt 2 root root 40 Oct 13 16:33 .XIM-unix
drwxrwxrwt 2 root root 40 Oct 13 16:33 .font-unix
-rw-r--r-- 1 root root 2035636 Oct 13 16:36 linux-dtb-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb
-rw-r--r-- 1 root root 30620936 Oct 13 16:36 linux-image-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb

root@DietPi:/tmp# cp -a /media/var/lib/dpkg/statoverride /media/var/lib/dpkg/statoverride.bak
root@DietPi:/tmp# sed -i '/messagebus/d' /media/var/lib/dpkg/statoverride
root@DietPi:/tmp# dpkg -i --root=/media linux-*.deb
**dpkg:** **warning:** downgrading linux-dtb-vendor-rk35xx from 24.11.0-trunk-dietpi1 to 24.8.0-trunk-dietpi1
(Reading database ... 51853 files and directories currently installed.)
Preparing to unpack linux-dtb-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb ...
Armbian 'linux-dtb-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'preinst' starting.
Armbian 'linux-dtb-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'preinst' finishing.
Unpacking linux-dtb-vendor-rk35xx (24.8.0-trunk-dietpi1) over (24.11.0-trunk-dietpi1) ...
**dpkg:** **warning:** downgrading linux-image-vendor-rk35xx from 24.11.0-trunk-dietpi1 to 24.8.0-trunk-dietpi1
Preparing to unpack linux-image-vendor-rk35xx_24.8.0-trunk-dietpi1_arm64.deb ...
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'prerm' starting.
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'prerm' finishing.
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'preinst' starting.
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'preinst' finishing.
Unpacking linux-image-vendor-rk35xx (24.8.0-trunk-dietpi1) over (24.11.0-trunk-dietpi1) ...
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'postrm' starting.
Removing obsolete initramfs images
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'postrm' finishing.
Setting up linux-dtb-vendor-rk35xx (24.8.0-trunk-dietpi1) ...
Armbian 'linux-dtb-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'postinst' starting.
Armbian: DTB: symlinking /boot/dtb to /boot/dtb-6.1.75-vendor-rk35xx...
'dtb' -> 'dtb-6.1.75-vendor-rk35xx'
Armbian 'linux-dtb-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'postinst' finishing.
Setting up linux-image-vendor-rk35xx (24.8.0-trunk-dietpi1) ...
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'postinst' starting.
Removing obsolete initramfs images
update-initramfs: Generating /boot/initrd.img-6.1.75-vendor-rk35xx
W: Couldn't identify type of root file system for fsck hook
update-initramfs: Converting to U-Boot format
Image Name: uInitrd
Created: Sun Oct 13 16:39:58 2024
Image Type: AArch64 Linux RAMDisk Image (gzip compressed)
Data Size: 7462990 Bytes = 7288.08 KiB = 7.12 MiB
Load Address: 00000000
Entry Point: 00000000
'/boot/uInitrd' -> 'uInitrd-6.1.75-vendor-rk35xx'
Armbian: update last-installed kernel symlink to 'Image'...
'/boot/Image' -> 'vmlinuz-6.1.75-vendor-rk35xx'
Armbian: Debian compat: linux-update-symlinks install 6.1.75-vendor-rk35xx boot/vmlinuz-6.1.75-vendor-rk35xx
Armbian 'linux-image-vendor-rk35xx' for '6.1.75-vendor-rk35xx': 'postinst' finishing.

root@DietPi:/tmp# mv /media/var/lib/dpkg/statoverride.bak /media/var/lib/dpkg/statoverride
root@DietPi:/tmp#

The system boots from the NVME again!

helmer@OrangePi5:~$ uname -a
Linux OrangePi5 6.1.75-vendor-rk35xx #1 SMP Thu Jul 25 14:36:35 UTC 2024 aarch64 GNU/Linux

Thank you so much!

I just ran “Update SPI bootloader” from the June DietPi version. Should I dare to do it again with the current version?

MichaIng · 13 October 2024 15:09

Oh, it is --allow-downgrades (missing “s”):

cd /tmp
apt download --allow-downgrades linux-{image,dtb}-vendor-rk35xx

Would be good to have this tested. In case, you can downgrade it again from the SD card system.

helmer · 13 October 2024 16:52

Okay - i will try ist…

helmer · 13 October 2024 17:06

I have now run:

root@OrangePi5:/home/helmer# apt install patch
Reading package lists... Done
Building dependency tree... Done
Reading status information... Done
Suggested packages:
ed diffutils-doc
The following NEW packages will be installed:
patch
0 updated, 1 newly installed, 0 to remove and 0 not updated.
121 kB of archives need to be downloaded.
After this operation, 250 kB of additional disk space will be used.
Fetching:1 https://deb.debian.org/debian bookworm/main arm64 patch arm64 2.7.6-7 [121 kB]
Fetched 121 kB in 0 s (334 kB/s).
debconf: delaying package configuration, since apt-utils is not installed
Previously unselected package patch is selected.
(Reading database... 51422 files and directories are currently installed.)
Preparing to unpack.../patch_2.7.6-7_arm64.deb...
Unpacking patch (2.7.6-7)...
Setting up patch (2.7.6-7)...
root@OrangePi5:/home/helmer# cd /boot
root@OrangePi5:/boot# curl -sSf 'https://github.com/MichaIng/DietPi/commit/64af7e9.patch' | patch -p1
patching file dietpi/dietpi-config
root@OrangePi5:/boot# dietpi-config
16732672 bytes (17 MB, 16 MiB) copied, 268 s, 62.4 kB/s
32768+0 records in
32768+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 268.741 s, 62.4 kB/s
root@OrangePi5:/boot#

The system no longer boots. No boot screen or anything else.
I’m now repairing it via the SD card. I have to create a new one first because an “apt upgrade” has already run on the current one.

helmer · 13 October 2024 17:25

The system is running again. With the versions:

-rw-rw-r-- 1 root root 301056 May 19th 21:17 idbloader.img
-rw-rw-r-- 1 root root 16777216 May 19th 21:17 rkspi_loader.img
-rw-rw-r-- 1 root root 16777216 May 19th 21:17 rkspi_loader_sata.img
-rw-rw-r-- 1 root root 1626624 May 19th 21:17 u-boot.itb

it works.

With the versions:

-rw-rw-r-- 1 root root 313344 Aug 1 10:25 idbloader.img 
-rw-rw-r-- 1 root root 16777216 Aug 1 10:25 rkspi_loader.img 
-rw-rw-r-- 1 root root 16777216 Aug 1 10:25 rks pi_loader_sata.img 
-rw-rw-r-- 1 root root 1648128 Aug 1 10:25 u-boot.itb

not!

MichaIng · 13 October 2024 17:27

Okay one thing I am actually confused about is that the bootloader package did not receive an update, but is still from August 1st. The only older one is legacy branch from June 20th, while both were based on same vendor sources . The switch from legacy to vendor was with last DietPi update, but the bootloader was not flashed.

I’ll check for changes in source and Armbian build system between those two dates, and trigger a new build, just in case it might fix it already

helmer · 13 October 2024 17:39

So far - thank you very much!
Please let me know if you want me to test something - assuming I (we) can always fix a potential problem.

MichaIng · 13 October 2024 17:41

Regarding the kernel: Armbian is freezing it at 24.08 as well for all Orange Pi 5 variants: Improved freezing mechanism with README · armbian/build@edbf678 · GitHub
Good that we have dedicated APT server components for each boards, so we can easily update the kernel for ROCK 5 and NanoPi 6 series, while keeping it at 24.08. for Orange Pi 5 variants.

And U-Boot has been fixed indeed after our latest package build: orangepi5: fix SPI flash boot by efectn · Pull Request #7053 · armbian/build · GitHub
I just pushed the new package to our APT server.

helmer · 13 October 2024 17:50

Okay - I’ll try it…

helmer · 13 October 2024 17:59

helmer@OrangePi5:~$ sudo apt list --upgradable
Listing... Done
linux-u-boot-orangepi5-vendor/all 24.11.0-trunk-dietpi1 arm64 [upgradable from: 24.8.0-trunk-dietpi1]
N: There is 1 additional version. Please use the "-a" option to view it.
helmer@OrangePi5:~$ sudo apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading status information... Done
Calculating package upgrade... Done
The following packages will be upgraded:
linux-u-boot-orangepi5-vendor
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
There are 1,762 kB of archives to download.
After this operation, 6,144 B of additional disk space will be used.
Do you want to continue? [Y/n]
Fetching:1 https://dietpi.com/apt all/orangepi5 arm64 linux-u-boot-orangepi5-vendor arm64 24.11.0-trunk-dietpi1 [1,762 kB]
1,762 kB were fetched in 1 s (2,812 kB/s).
debconf: delaying package configuration, since apt-utils is not installed
(Reading database... 51432 files and directories are currently installed.)
Preparing to unpack.../linux-u-boot-orangepi5-vendor_24.11.0-trunk-dietpi1_arm64.deb...
Unpacking linux-u-boot-orangepi5-vendor (24.11.0-trunk-dietpi1) via (24.8.0-trunk-dietpi1)...
Setting up linux-u-boot-orangepi5-vendor (24.11.0-trunk-dietpi1)...
Armbian 'uboot-orangepi5-vendor' for '2017.09-S303f-P2429-Hbbb4-Vab56-Bda0a-R448a': 'postinst' starting.
Armbian 'uboot-orangepi5-vendor' for '2017.09-S303f-P2429-Hbbb4-Vab56-Bda0a-R448a': 'postinst' finishing.
helmer@OrangePi5:~$ sudo dietpi-config 16740864 bytes (17 MB, 16 MiB) copied, 269 s, 62.2 kB/s 32768+0 records in 32768+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 269.602 s, 62.2

The OrangePi 5 boots after the update.

MichaIng · 13 October 2024 18:46

Okay great, so we finally found clear both. Sad that we hence cannot update the kernel for the Orange Pi 5’s now. I’ll keep an eye on the freeze at Armbian. Difficult to manage this vendor kernel, since it is from Radxa sources, but applied for NanoPi 6 and Orange Pi 5 as well. I can imagine that by times Radxa pushes out some change which works fine for their ROCK 5, but breaks things on the other vendor boards.

krisskross · 13 October 2024 19:57

After my update on Friday, October 11, 2024, my OrangePi5 no longer booted on NVME.

I have now run:

mount /dev/nvme0n1p1 /media
cd /tmp
apt download --allow-downgrades linux-{image,dtb}-vendor-rk35xx
cp -a /media/var/lib/dpkg/statoverride /media/var/lib/dpkg/statoverride.bak
sed -i '/messagebus/d' /media/var/lib/dpkg/statoverride
dpkg -i --root=/media linux-*.deb
mv /media/var/lib/dpkg/statoverride.bak /media/var/lib/dpkg/statoverride

Now the System boots on NVME.

THX!

MichaIng · 13 October 2024 22:00

Thanks for summing up the needed steps, including the workaround for statsoverrides, if they contain a username or group on the NVMe system which does not exist on the host system you run these commands from.

Renoria · 14 October 2024 13:48

I was trying to follow the convo but it got too technical what should I do since I am flashing dietpi to SD card as per How to install DietPi - DietPi.com Docs tonight, what are the next steps to boot from NVme?

Thanks in advance for the help!

Joulinar · 14 October 2024 15:18

The last days was about fixing an issue with a specific kernel + u-boot and not directly related to the possibility of booting from NVMe.

Basically for you it would be

boot from SD
flash u-boot to SPI
flash an image to NVMe
boot from NVMe