RK3399 HDMI output and USB 3.0 not working with Linux 6.1

Troubleshooting
42

Hi all

I’ve tested with Armbian with older kernel

Welcome to Armbian 22.11.1 Jammy with Linux 5.19.17-media

Downloaded from here https://armbian.hosthatch.com/archive/nanopct4/archive/Armbian_22.11.1_Nanopct4_jammy_current_5.19.17_cinnamon_desktop.img.xz

Both HDMI and USB3.0 are working

uname -a
Linux nanopct4 5.19.17-media #22.11.1 SMP PREEMPT Wed Nov 30 11:08:24 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
2 Likes
43

I have the same issue on RockPro64 with current DietPi 8.21.1

Linux gateway 6.1.50-current-rockchip64 #3 SMP PREEMPT Wed Aug 30 14:11:13 UTC 2023 aarch64 GNU/Linux

and it is definitely a kernel problem. Because I was doing some kernel hacking on 5.15 and 5.19 I had those installed previously and no issues at all… I will try the overlay you mentioned @MichaIng and will report back…

PS: It “sometimes” works. So usually if I reboot it, it works - it seems to be an issue with the NAS I have connected coming up while the USB3.0 is initialized, and then it fails, but if the NAS is already up, then it works… still it got introduced with the move to the 6.x something kernel and was not there with 5.x at all…

PPS: Kernel Overlay did not solve the problem…

PPPS: Finally I indeed downloaded the kernels from DotSrc and installed them, hold them with

apt-mark hold linux-{image,dtb}-current-rockchip64

and it works again - thx @MichaIng

1 Like
44

According to my tests, These two commits are suspicious.
kernel/git/stable/linux.git - Linux kernel stable tree
kernel/git/stable/linux.git - Linux kernel stable tree

45

Same issue here with Rock Pi 4B with the top USB 3.0 port no matter what I select with the switch. Doesn’t even work in 2.0 mode.
Can’t try HDMI.
Everything upgraded.

46

any news?

47

kernel/git/stable/linux.git - Linux kernel stable tree
This patch may fix the issue, but I haven’t tested yet.

2 Likes
48

Hi, just to continue updating the thread and not let it die even with the latest release 8.23.3 it doesn’t work, greetings

I hope @MichaIng can help us :slight_smile:

root@DietPi:~# dmesg -l 0,1,2,3
[    2.414659] rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout!
[    2.503272] dw-apb-uart ff180000.serial: Failed to create device link (0x180) with 0-001b
[    2.649609] dw-apb-uart ff180000.serial: Failed to create device link (0x180) with vcc3v3-sys
[    2.681086] phy phy-ff7c0000.phy.6: phy poweron failed --> -110
[    2.681808] dwc3 fe800000.usb: error -ETIMEDOUT: failed to initialize core
[    2.724924] dw-apb-uart ff180000.serial: Failed to create device link (0x180) with 0-001b
[   10.564388] OF: graph: no port node found in /i2c@ff3d0000/typec-portc@22
root@DietPi:~#
49

We have reports on the Armbian forum as well in the meantime (strangely not many, and so late): usb port not working since last kernel update - ROCK Pi 4 series - Armbian Community Forums

Kernel and device tree development is sadly outside of my abilities. All I could do when I find time is building an unmodified mainline kernel to see whether it works there, and hence whether one of Armbian’s patches is the culprit.

50

I admittedly haven’t fully processed this thread. I’ll go back and read it in more detail when I’ve got a bit more time. For now just leaving myself a breadcrumb. I’ve got the Radxa Rock Pi 4, and have been attempting a large RSYNC between drives in a 2 bay adapter. They connect over USB 3.0 to my Pi and I believe I’ve had them plugged into the bottom (not top) USB 3.0 port. The speeds were great, but the RSYNC would enviably fail and I’d get input/output errors. I decided to try a 2.0 port with all other factors being the same. It’s been running the RSYNC without failure for over 24 hours now. I’m curious if the kernel support for the 3.0 ports on this board is lacking and if there’s more info I can help provide. Thanks!

51

Hi, bad story in armbian thread, you read… @MichaIng

52

Dammit, that topic at Armbian was closed. Probably we should make something clear whenever referring to the Armbian kernel or linking their forum: I (obviously) do not agree with most of what Igor says, about open source piracy (the word itself is in contradiction to the GPL licence, given that all packages are transparently installed, including their license and credits as shown in e.g. apt show linux-image-current-rockchip64 etc), and that he seems to see DietPi as business selling something, instead of as the little spare time project with no income ambitions it is, but he is right in the following points:

  • As the GPL states as well that “This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.”, be kind and careful when asking for support or having things fixed. As long as you do not have a support contract, it is a spare time invest open source developers do of good will, or own ambition to serve a good product. Some may react friendlier, but some react allergic on requests, if the wording sounds a little to requesting. Armbian actually does have a payed support forum and various donation options, if someone really needs a fix urgently.
  • Even when we are sure that it is a kernel problem, and of course hope that it gets fixed either upstream or by Armbian, if we are not able to fix it ourselves (and we are far away form being kernel developers, though I am learning it slowly working on RISC-V SBC kernels), this does not mean that we can expect anyone else to fix it, since we have no support contract either and can use Armbian only under the same GPL “no warranty” conditions.
  • When posting at the Armbian forum, reasonably, use an official, at best unmodified, Armbian image, replicate the issue and use the armbianmonitor tool to provide debug information, basically following the steps provided here: Bug reporting – Armbian
    Do not mention DietPi, this is only counter-productive, and it wouldn’t be relevant anyway, as of course Armbian cannot know our images, and whether it is a DietPi-specific issue, no matter how sure we are that it is a kernel issue. You might still get an unfriendly answer as long as you are not donating (or posting in the private support forum), but it definitely has better chances this way.
1 Like
53

If you don’t pay, you will be blamed. Unfortunately, the Armbian developers don’t like us and so pretty much every reference to DietPi is in the same direction. It is recommended to install plain Armbian and see if the problem is there too.

54

To be true, I see this ongoing blames form Igor only, everyone else is taking bug reports as what they are: A hint you might want to look into to make your product better, or leave if you do not have time, skipping lengthy posts, blaming the ones who report the bugs.

55

For now, the kernel could be downgraded by installing the image and dtb packages from here: mirrors.dotsrc.org
Download and install via dpkg -i, then set the packages on hold so that apt upgrade won’t upgrade them again:

apt-mark hold linux-{image,dtb}-current-rockchip64

I don’t have any experience downgrading kernels, but am interested in getting USB 3.0 working reliably again. My main goal is stability, and I can live with 2.0 speeds for a while, but if the likelihood of a fix is low, I’d ultimately be fine downgrading the kernel and holding updates. Can someone share a recommendation and more detail on how to downgrade (if I go that route)?

56

Let’s test it forwards one time. Here are builds with the very latest version of the Armbian build systemd: Index of /downloads/binaries/testing

cd /tmp
curl -fLO 'https://dietpi.com/downloads/binaries/testing/linux-image-current-rockchip64.deb'
curl -fLO 'https://dietpi.com/downloads/binaries/testing/linux-dtb-current-rockchip64.deb'
dpkg -i linux-{image,dtb}-current-rockchip64.deb
rm linux-{image,dtb}-current-rockchip64.deb
reboot
57

Thank you! I’ll give this upgrade a try once my slow USB 2.0 rsync is done (should be today).

1 Like
58

after launching the latest builds
The errors have changed.
I need to check the raid better

dietpi@DietPi:~$ dmesg -l 0,1,2,3
[    2.353298] rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout!
[    2.443927] dw-apb-uart ff180000.serial: Failed to create device link (0x180) with 0-001b
[    2.591936] dw-apb-uart ff180000.serial: Failed to create device link (0x180) with vcc3v3-sys
[    2.778021] dw-apb-uart ff180000.serial: Failed to create device link (0x180) with 0-001b
[   10.591427] OF: graph: no port node found in /i2c@ff3d0000/typec-portc@22
dietpi@DietPi:~$ uname -a
Linux DietPi 6.1.63-current-rockchip64 #1 SMP PREEMPT Mon Nov 20 10:52:19 UTC 2023 aarch64 GNU/Linux
dietpi@DietPi:~$ sudo mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Mar 12 14:31:12 2023
        Raid Level : raid1
        Array Size : 468719424 (447.01 GiB 479.97 GB)
     Used Dev Size : 468719424 (447.01 GiB 479.97 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Fri Nov 24 23:52:05 2023
             State : clean, degraded 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : bitmap

              Name : DietPi:0  (local to host DietPi)
              UUID : 58b4190f:4bf2b26d:7d4de636:c6513e6a
            Events : 2239322

    Number   Major   Minor   RaidDevice State
       2       8        0        0      active sync   /dev/sda
       -       0        0        1      removed
dietpi@DietPi:~$
59

I’ve also tested the kernel upgrade. I’m still having the same issue on the bottom USB 3.0 port (regular input/output errors during rsync between the USB devices). As soon as I move the drive back to the 2.0 port, the same rsync works perfectly.

root@rockpi:/tmp# pwd
/tmp
root@rockpi:/tmp# curl -fLO 'https://dietpi.com/downloads/binaries/testing/linux-image-current-rockchip64.deb'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 39.8M  100 39.8M    0     0  4955k      0  0:00:08  0:00:08 --:--:-- 5373k
root@rockpi:/tmp# curl -fLO 'https://dietpi.com/downloads/binaries/testing/linux-dtb-current-rockchip64.deb'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  309k  100  309k    0     0   650k      0 --:--:-- --:--:-- --:--:--  651k
root@rockpi:/tmp# dpkg -i linux-{image,dtb}-current-rockchip64.deb
(Reading database ... 40803 files and directories currently installed.)
Preparing to unpack linux-image-current-rockchip64.deb ...
Armbian 'linux-image-current-rockchip64' for '6.1.50-current-rockchip64': 'prerm' starting.
Armbian 'linux-image-current-rockchip64' for '6.1.50-current-rockchip64': 'prerm' finishing.
Armbian 'linux-image-current-rockchip64' for '6.1.63-current-rockchip64': 'preinst' starting.
Armbian 'linux-image-current-rockchip64' for '6.1.63-current-rockchip64': 'preinst' finishing.
Unpacking linux-image-current-rockchip64 (23.11.0-trunk) over (23.8.1) ...
Armbian 'linux-image-current-rockchip64' for '6.1.50-current-rockchip64': 'postrm' starting.
Removing obsolete initramfs images
Armbian 'linux-image-current-rockchip64' for '6.1.50-current-rockchip64': 'postrm' finishing.
Preparing to unpack linux-dtb-current-rockchip64.deb ...
Armbian 'linux-dtb-current-rockchip64' for '6.1.63-current-rockchip64': 'preinst' starting.
Armbian 'linux-dtb-current-rockchip64' for '6.1.63-current-rockchip64': 'preinst' finishing.
Unpacking linux-dtb-current-rockchip64 (23.11.0-trunk) over (23.8.1) ...
Setting up linux-image-current-rockchip64 (23.11.0-trunk) ...
Armbian 'linux-image-current-rockchip64' for '6.1.63-current-rockchip64': 'postinst' starting.
Removing obsolete initramfs images
removed '/boot/initrd.img-6.1.50-current-rockchip64'
removed '/boot/uInitrd-6.1.50-current-rockchip64'
update-initramfs: Generating /boot/initrd.img-6.1.63-current-rockchip64
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8156b-2.fw for module r8152
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8156a-2.fw for module r8152
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8153c-1.fw for module r8152
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8153a-4.fw for module r8152
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8153a-2.fw for module r8152
update-initramfs: Converting to U-Boot format
Image Name:   uInitrd
Created:      Fri Nov 24 16:57:59 2023
Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
Data Size:    12704439 Bytes = 12406.68 KiB = 12.12 MiB
Load Address: 00000000
Entry Point:  00000000
'/boot/uInitrd' -> 'uInitrd-6.1.63-current-rockchip64'
Armbian: update last-installed kernel symlink to 'Image'...
'/boot/Image' -> 'vmlinuz-6.1.63-current-rockchip64'
Armbian: Debian compat: linux-update-symlinks install 6.1.63-current-rockchip64 boot/vmlinuz-6.1.63-current-rockchip64
I: /vmlinuz.old is now a symlink to boot/vmlinuz-6.1.63-current-rockchip64
I: /initrd.img.old is now a symlink to boot/initrd.img-6.1.63-current-rockchip64
I: /vmlinuz is now a symlink to boot/vmlinuz-6.1.63-current-rockchip64
I: /initrd.img is now a symlink to boot/initrd.img-6.1.63-current-rockchip64
Armbian 'linux-image-current-rockchip64' for '6.1.63-current-rockchip64': 'postinst' finishing.
Setting up linux-dtb-current-rockchip64 (23.11.0-trunk) ...
Armbian 'linux-dtb-current-rockchip64' for '6.1.63-current-rockchip64': 'postinst' starting.
Armbian: DTB: symlinking /boot/dtb to /boot/dtb-6.1.63-current-rockchip64...
'dtb' -> 'dtb-6.1.63-current-rockchip64'
Armbian 'linux-dtb-current-rockchip64' for '6.1.63-current-rockchip64': 'postinst' finishing.

Rebooted, but couldn’t SSH into the RockPi.

Unplugged the RockPi, moved USB adapter to lower USB 3.0 plug

Plugged in the RockPi

SSH working again

root@rockpi:~# uname -a
Linux rockpi 6.1.63-current-rockchip64 #1 SMP PREEMPT Mon Nov 20 10:52:19 UTC 2023 aarch64 GNU/Linux

Both drives were automatically mounted.

Started rsync between 2 drives

Speeds were very slow (in the kilobytes instead of MBps)

Then got input/output errors like I was before when I tried using USB 3.0.

I’d be happy to try downgrading and see if the rsync issues go way on the 3.0 port. Otherwise I can hang tight on 2.0 until an upstream fix comes.

60

This Armbian thread seems relevant with mentions of USB host failing during file transfers, and the last post mentioning the regression is no longer observable in Linux Kernel 6.7rc2. If that does fix it, I still expect it could be a very long time until that kernel version lands in Armbian. Again, new to all of this kernel level stuff, so please correct me if I’m wrong here.

1 Like
61

That is great to read regarding Linux 6.7. So it is a mainline bug then. It will take a while until it reaches Armbian stable, but not so long until it reaches Armbian edge. I’ll have a closer look into the topic, probably the fix is even backported into Linux 6.6. I’ll keep an eye on the release notes.

1 Like