Why is compaction disabled on the Rockchip kernel?

Hello,

I am using dietpi with the kernel 6.1.115-vendor-rk35xx #1 SMP Mon Apr 6 11:36:06 UTC 2026 aarch64 GNU/Linux on Radxa Rock 5B.

Recently I’ve been observing that some of my containers don’t see the full “available memory”, especially if I enable zram. ChatGPT suggested that it may be an issue with memory compaction.

And indeed the kernel config says:
# CONFIG_COMPACTION is not set
Is this a known issue on Rockchip 6.1?

Maybe it wouldn’t hurt to enable this?

Best to my knowledge this is a vendor kernel. But something @MichaIng can confirm.

Yes, this is a vendor kernel (dietpi default). But this doesn’t necessarily mean compaction can’t be enabled, does it?

I’m not 100% sure but usually vendor kernel means it’s maintained by vendor, not by us. But I could be wrong. @MichaIng knows for sure.

No, everyone can edit the defconfig of the vendor kernel. The question is whether this option is disabled for a reason (for example, because it is bugged in the vendor kernel) - but if not, and it’s just incidental, it helps with memory management.

You can always build the kernel by yourself with CONFIG_COMPACTION=y and test if it’s working then.

This is expected since zram needs some of your system memory.

How much memory do you have and how much is the OS showing?

I would but I don’t know where the kernel source for dietpi lives and what scripts are used to generate the debs… Another issue is that it is quite risky and I can’t really test it on the system I want it to run on. So I am asking if anyone else could.

The amount that the containers see is, for example, 5.5 GB when the available memory is, say 11 GB (I have 16 GB of total memory, and just 1 GB of zram). The caches appear as to be unavailable for programs, while they should always be available to reallocate. I can paste some of the output that the LLM gave me to understand this but it’s quite verbose. The issue is that RAM is fragmented. This can apparently be monitored using buddyinfo:

cat /proc/buddyinfo

And when the higher levels are 0 it means fragmentation occurred.

The text generator also suggested that disabling zram only delays the issue but ultimately doesn’t solve it (and this is what I observe indeed).

Basically the issue is that I have lots of RAM but it can’t be used because it is not managed properly by the kernel.

And also to answer the question - how much the OS showing? See above: 11 GB available (with caches) but docker images can only “see” 5 because the caches can’t be evicted.
Here’s some non-LLM info on memory fragmentation: Memory Fragmentation in Linux: Causes, Fixes & Tools

I don’t think this is a fragmentation issue, I think it is more likely related to memory accounting and cgroup limits.
The memory shown as “available” at system level is not fully visible to containers due to isolation.
Also, zram and kernel reservations can change how memory is reported without actually reducing usable RAM.

Can you check the following to get a picture

free -h
cat /sys/fs/cgroup/memory.max
cat /proc/meminfo | grep -i cma

Before I do,
Do I need to somehow create conditions for this (it happens very sporadically, I’d need somehow do things to make the system behave this way).

Can’t do the second command:

ls /sys/fs/cgroup/memory.* |cat
/sys/fs/cgroup/memory.pressure
/sys/fs/cgroup/memory.reclaim
/sys/fs/cgroup/memory.stat

The others currently:

free -h
               total        used        free      shared  buff/cache   available
Mem:            15Gi        10Gi       1.6Gi       167Mi       3.7Gi       4.9Gi
Swap:             0B          0B          0B

I disabled swap.

cat /proc/meminfo | grep -i cma
CmaTotal:         262144 kB
CmaAllocated:      14516 kB
CmaReleased:      247628 kB
CmaFree:          247628 kB

I don’t know either why that kernel is built without CONFIG_COMPACTION. We build those with a fork of the Armbian build system. It is explicitly unset for the vendor kernel: build/config/kernel/linux-rk35xx-vendor.config at f13bc6b8225b165c87d8c18267f7096d48538083 · armbian/build · GitHub

Tracing back things: That config was taken from the older rk35xx-legacy config, which originally that CONFIG_COMPACTION=y, but got it disabled when the other older rk3588-legacy config was merged: merge rockchip-rk3588-legacy and rk35xx-legacy · armbian/build@5ee6d8b · GitHub
And that rk3588-legacy did have it explicitly disabled.

This was originally taken from Radxa: arm64: configs: add Rockchip linux default configure file · radxa/kernel@6fd9def · GitHub
They took it most likely from Rockchip themselves.

We could create a build with this enabled, but there might be a reason why it was explicitly disabled before (as it is enabled by default).

I would instead suggest to switch to mainline Linux. I plan this for all/most RK3588 images soon anyway. The only left reason one might want to keep the vendor kernel is the RKMPP VPU driver, while mainline Linux ships with the less capable Hantro VPU mainline driver. But otherwise mainline has just tons of enhancements, security fixes, a lot less to zero “expected” kernel errors on boot etc:

sudo apt install linux-image-current-rockchip64 linux-u-boot-rock-5b-current
sudo apt autopurge linux-{image,dtb}-vendor-rk35xx
[[ -d '/boot/dtb' ]] || sudo dpkg-reconfigure -f noninteractive linux-image-current-rockchip64
/boot/dietpi/func/dietpi-set_hardware flash-u-boot-mmc

I have also never heard of containers seeling lower available memory on vendor Linux (due to missing compaction).

There is docker_optimizations=on possible in /boot/dietpiEnv.txt to enable the memory cgroup, but that is actually enabled by default, good reminder to remove that option.

This statement is no longer correct as it stands: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/drivers/media/platform/rockchip/rkvdec

@MichaIng thanks for the reply. I would love to switch to the mainline kernel, but I use jellyfin w/ transcoding and the mainline implementation only provides decoding, and only for a few formats out of the myriad that RK3588 supports natively.

Docker optimisations are on.

Right, but on Linux 6.18, it does not support RK3588 yet: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit?id=e5aa698
And it is a decoder only, no encoder, right?

And uroster’s example shows that there is still software supporting RKMPP only. But that alone is no reason to keep shipping images with vendor kernel by default. Jellyfin needs to start supporting mainline drivers, resp. the respective FFmpeg methods, to remain relevant on ARM. They even ship packages with an own FFmpeg build as dependency, but instead of adding the pending V4L2 Request API, they explicitly disable a bunch of mainline codecs, so that RKMPP remains as only supported hardware accelerated codec on ARM, rendering it a very inefficient streamer on all but Rockchip SBCs with vendor kernel :roll_eyes:.

If the support has not already landed in 6.18-rc1, the 6.18 line will never officially receive this support. New features are never officially back-ported, but there is nothing against building a kernel with the corresponding patches applied. I build my kernels with a bunch of such WIP and in-flight patches and have been able to use these functionalities for quite some time. This strategy has the advantage that with each landed feature, applying its corresponding patch-sets becomes unnecessary.
Whereas porting forward out-of-tree hacks to hold on to proprietary interfaces is becoming more difficult with every mainline update, as development in the mainline steadily progresses. There is a reason why vendors do not offer updates for their BSPs, as their development of the out-of-tree hacks ends with the release of the BSP. The BSP is used to demonstrate that all advertised device functionalities are usable. Users can use the BSP as is if it meets all their requirements. Otherwise, they are on their own if they want changes. But fortunately, there are many people who are eager to help such users for free :wink:
The attempt to combine the use of proprietary implementations with mainline concepts will ultimately always fail. But it is certainly a legitimate strategy to implement container solutions where no one worries too much about the actuality of the container content in relation to a specific application.

Since the VDPU381 IP only has decoder capabilities, it is of no help for encoder tasks.
For encoding you have to look e.g. for VEPU580 IP. But as the encoder framework is not
jet finalized in mainline there exist only some WIP.

I have also read somewhere that Rockchip are planning to release another BSP which is more current (not sure but I think 6.12, 6.6 was in the works but never ready).

I’ll build and check the vendor kernel with compaction on on another Rockchip board, maybe it is safe

This is what I meant. So rkvdec on RK3588 is relevant for us only from next LTS on. And it does not provide hardware accelerated streaming (encoding), for which hantro_vpu is still needed, which provides worse performance (on RK3588), compared to RKMPP.

I actually did not check whether Armbian patched RK3588 support into Rockchip Linux 6.18. … actually they do: build/patch/kernel/archive/rockchip64-6.18/media-0001-Add-rkvdec-Support-v5.patch at 3ce7dec74b313211c969afa04ee07b55df180102 · armbian/build · GitHub
Okay, I have not tested it, but that looks good. But still does not help with Jellyfin or any other streaming server.

Rockchip does rebase/update their kernel every few LTS versions, and some (SBC) vendors rebase their sources onto this as well. But not all of them, and in any case it takes a long time, and will always be way behind mainline, with all those vendor-only drivers and APIs anyway. So I totally agree, that going with pending mainline patchsets, is usually the better approach, but it takes years after a SoC release, before (mostly volunteers) mainlined all/most features properly. RK3588 boards became available in 2022, and now in 2026 we have a proper decoder in mainline, not yet LTS, and a working but not great encoder. And an NPU driver just arrived with last LTS.

Sure, but once that one gets released, adopted by vendors, and in case by Armbian’s Rockchip vendor kernel repo, 6.12 is as outdated as 6.1 is now. And it will still contain a bunch of vendor-only drivers, in case new/additional ones, bad code/configuration, and all the issues we have with this kernel. This is something which will never change, as long as there is any need for a vendor kernel in the first place. The only sane “update” would be if Rockchip started to mainline support for their SoCs immediately, using their own repo only for staging patches/commits to mainline from. Of course this means more work, requires a certain code quality, and updating/rebasing things with reviews and upstream changes. But it would raise the quality and make new Rockchip SoCs competitive among distros years earlier. That would speed up U-Boot support as well, which usually relies on Linux support, pulling device trees and in case drivers from there.

StarFive partly showed how it can work, with a consequent mainlining plan from day 0. But to be honest, mainline support for the JH7110 still lacks some essential features. At least their sources are much closer to mainline, with vendor-only drivers/methods/diff decreasing with every LTS.

Anyway, it is as it is, and the community will need to keep playing this vendor vs mainline kernel/driver backwards/forwards porting balance, with every new Rockchip SoC/family, to provide images that enable at least most of the hardware features.

Hi @MichaIng

Sorry for reviving this topic, but it seems that enabling compaction is quite low-risk and would be beneficial not just to me. I haven’t found a good solution yet and I am not ready to move to mainline.

I am now using an overlay to mask bad RAM (just 2 x 1 MB out of 16 GB) and this plays very badly with no compaction. This is the actual reason why I was getting these OOM errors.

Can I ask you for a build of the Rockchip kernel 6.1 with this option enabled?:

CONFIG_COMPACTION=y

I asked an LLM why Rockchip might have disabled it and it seems that it stems from config inertia and media-focus. Here is some of its response.

The config tells the story: CONFIG_MIGRATION=y and CONFIG_CMA=y are already on, while CONFIG_COMPACTION and CONFIG_TRANSPARENT_HUGEPAGE are both off. That pattern is a deliberate Rockchip BSP choice, and it also means recompiling with compaction enabled is low-risk. Here’s the reasoning:

Why Rockchip likely disabled it
Latency determinism for media pipelines. Compaction runs in two ways: a background kcompactd thread and direct compaction, where an allocating task stalls while pages get migrated around. Rockchip’s core market is video boxes, cameras, and NPU workloads where an allocation stalling for tens of milliseconds means a dropped frame. Disabling compaction (and THP, which they also turned off — same motivation, THP allocations trigger compaction) trades throughput robustness for predictable latency.
They solve contiguity a different way. In the BSP worldview, anything needing large physically-contiguous buffers (VPU, ISP, GPU) goes through CMA or behind an IOMMU — that’s why CMA is fully enabled with 7 areas. From their perspective, well-behaved drivers never make large-order buddy allocations, so compaction is pure overhead for their reference designs.

Config inertia.
These options were set in much older BSP trees (4.4/4.19 era, when compaction also had more real bugs) and get carried forward from defconfig to defconfig without re-evaluation. Vendor kernels accumulate these fossils.
What they didn’t account for is a general-purpose Linux workload with a long uptime, where fragmentation accumulates and eventually some order-2/order-3 allocation (network stack, ext4, fork stacks) fails despite gigabytes free — exactly your symptom. You can confirm this is what’s happening from an OOM trace in dmesg: look for order=2 or higher in the allocation failure line.

Recompiling is a reasonable option
The dangerous-sounding part of compaction is migrating pages out from under drivers — but CONFIG_MIGRATION=y means that machinery is already compiled in and already exercised every time CMA reclaims its region. Enabling CONFIG_COMPACTION=y adds the compaction scanner and kcompactd on top of infrastructure the vendor tree already ships. It has no other dependencies beyond MMU. So flipping it in the vendor 6.1 tree should build cleanly and is unlikely to destabilize anything; the theoretical cost is the latency jitter described above, which for your use case is a fine trade.

If anyone is interested, I can share the script to find the physical locations of bad memory sectors from memtester output, as well as the device tree overlay.

I wouldn’t trust an LLM at all in this regards. It can tell you what compaction is, and can guess into the blue why it is not enabled. But it has no way of knowing a potential particular, probably critical reason, whether or why Rockchip BSP kernel devs made the decision to disable it. Unless it links the particular issue/PR/commit where it is explained.

Just checked: arm64: configs: add Rockchip linux default configure file · rockchip-linux/kernel@6fd9def · GitHub
So they pulled it in from arch/arm/configs/rockchip_linux_defconfig (32-bit).
Checking that one: ARM: config: add the rockchip_linux_defconfig for rockchip · rockchip-linux/kernel@45cee0c · GitHub
So their first 32-bit defconfig had it disabled already, without explanation, from which they pulled it in into the 64-bit defconfig. Even if it was needed for 32-bit, our vendor kernel builds are or 64-bit only. Safe to just enable it and see. I’m anyway testing the monthly kernel updates currently.

Hang on …

EDIT: rk35xx-vendor: enable compaction · MichaIng/build@e6557e2 · GitHub
Build running: Armbian · MichaIng/DietPi@f9a7b9b · GitHub

Thanks, I’ll test whenever it builds

The dates on the website Index of /downloads/binaries/testing are wrong, but the files are correct. I installed them and compaction is enabled. It will take a few days/weeks before I can say if the situation improved, but the board booted and so far no issues.