Possibly lightest solution:
sudo modprobe vc_sm_cma
Check whether this normalizes average CPU load as seen in e.g. uptime. If so, to make it permanent:
sudo rm /etc/modprobe.d/dietpi-disable_vcsm.conf
echo vc_sm_cma | sudo tee /etc/modules-load.d/vc_sm_cma.conf
If less than 32 MiB GPU memory are applied (default on DietPi is 16 MiB, which is also the possible minimum), this kernel driver fails to load. You will see the related errors it in dmesg:
[ 847.480096] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[ 847.481448] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
[ 847.481491] [vc_sm_connected_init]: start
[ 847.484262] vc_sm_cma_vchi_init: failed to open VCHI service (-22)
[ 847.484297] [vc_sm_connected_init]: failed to initialize shared memory service
Hence we blacklist in that case, and allow it to load only if 32 MiB or higher GPU memory is applied. However, the driver contacts this VCHI communication node, and that way releases the kernel thread that is waiting for such, causing this higher load average. At least if KMS/DRM is disabled, the driver does not load by default. Hence, removing the blacklist is not enough in that case, but it needs to be loaded actively, which is assured with the second command.