Hello everyone,
I’m currently trying to get the NPU on my Orange Pi 5 Plus working to run various AI-related Docker projects, with a specific focus on getting the rkllama project up and running. I’m hoping to get some guidance or share experiences with anyone who might have attempted something similar.
My Hardware:
-
Device: Orange Pi 5 Plus (32GB RAM)
-
Storage: NVMe SSD
-
OS: DietPi v9.15.2
My Goal:
My plan is to use DietPi as the base OS, install OpenMediaVault (OMV) on top of it, and then use Docker to run several different containers. A primary goal is to leverage the NPU for performance in projects like rkllama.
What I’ve Tried So Far:
Unfortunately, I’ve already had a few unsuccessful attempts. I’ve tried this with the official Orange Pi image and also with Armbian, but in every case, I’ve failed to get access to the NPU driver from within the operating system or the Docker container.
I’m relatively new to the world of single-board computers with NPUs, so I’m still learning. My current setup is a Raspberry Pi 4 running OMV with a few Docker containers, and I’m looking to migrate and expand on my new Orange Pi.
Has anyone had success with accessing the NPU on the Orange Pi 5 Plus with a recent DietPi build? I would be very grateful for any help, tips, or pointers on how to get the driver recognized and passed through to Docker containers.
Thanks in advance for your help!