Dietpi-config on Buster // Installing rpisurv on Bullseye

MichaIng · 22 April 2023 22:04

That is a GPU memory split: You split of 512 MiB from system memory to GPU memory . However, if you did remove the gpu_mem_nnn settings, the one you manually set does the same. The others are overrides for 256, 512 and >=1024 MiB RAM models.

512 MiB is however much more than you’ll ever need. I’d set it max to 96 MiB to not leave the precious RAM unused.

The vc_sm_cma driver is loaded?

modprobe vc_sm_cma

If you set the GPU memory via dietpi-config, it creates or removes /etc/modprobe.d/dietpi-disable_vcsm.conf to prevent this driver from loading, which fails with >32 MiB GPU memory.

MINERAL-115 · 24 April 2023 07:15

Good morning, Michalng!

I wasn’t aware it counted as a split if it was explicitly defined; thank you for for the clarification.

rpisurv does seem to be quite fussy about gpu memory, I did try to set 96mb on my stable Buster RPI and it can only manage one video stream (out of two) with that, as all streams are loaded concurrently. 128mb seems to be about the sweet spot. I actually had set 512mb based on SvenVD’s (the author of rpisurv)'s README.md on the rpisurv github:

If you are going to have multiple streams, add gpu_mem=512 to your /boot/config.txt

I do wonder if this is now outdated, however, as it also mentions manually defining framebuffer resolutions, which I don’t think I’ve ever had to (regardless of if it was on Raspian/RPOS or dietpi). Perhaps this is only an issue on certain hardware configurations?

If you installed the “Lite” OS version you will need to edit /boot/config.txt and uncomment framebuffer_width=1280' and 'framebuffer_height=720 and make them 1920 and 1080 respectively or you will encounter the following error pygame.error: No video mode large enough for 1920x1080

Interestingly enough, running modprobe vc_sm_cma seems to solve the issue:

Apr 24 08:03:12 EOD-Office rpisurv[582]: [f3400da8] mmal_vout vout display error: VCSM init fail
Apr 24 08:03:12 EOD-Office rpisurv[582]: [f3623788] main video output error: video output creation failed
Apr 24 08:03:12 EOD-Office rpisurv[582]: [f4215450] main decoder error: failed to create video output
Apr 24 08:03:12 EOD-Office rpisurv[582]: [h264 @ 0xf320e2f0] get_buffer() failed
Apr 24 08:03:12 EOD-Office rpisurv[582]: [h264 @ 0xf320e2f0] thread_get_buffer() failed
Apr 24 08:03:12 EOD-Office rpisurv[582]: [h264 @ 0xf320e2f0] decode_slice_header error
Apr 24 08:03:12 EOD-Office rpisurv[582]: [h264 @ 0xf320e2f0] no frame!
<modprobe vc_sm_cma executed>
Apr 24 08:03:12 EOD-Office kernel: vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
Apr 24 08:03:12 EOD-Office kernel: bcm2835_vc_sm_cma_probe: Videocore shared memory driver
Apr 24 08:03:12 EOD-Office kernel: [vc_sm_connected_init]: start
Apr 24 08:03:12 EOD-Office kernel: [vc_sm_connected_init]: installed successfully

As you predicted, /etc/modprobe.d/dietpi-disable_vcsm.conf had been modified:

blacklist vc_sm_cma

To clarify, this was after manually setting /boot/config.txt to gpu_mem=128 (for 2 rtsp streams).

I commented out that line, rebooted, and the issue seems to have been resolved! There are still errors in the system log, but… well, it seems to be working, so I’m unsure how important these actually are:

-- Journal begins at Mon 2023-04-24 08:06:47 BST. --
Apr 24 08:07:09 EOD-Office rpisurv[536]: [f4515448] avcodec decoder: Using DRM Video Accel for hardware decoding
Apr 24 08:07:09 EOD-Office rpisurv[536]: [h264_v4l2m2m @ 0xf3501450] Could not find a valid device
Apr 24 08:07:09 EOD-Office rpisurv[536]: [h264_v4l2m2m @ 0xf3501450] can't configure decoder
Apr 24 08:07:09 EOD-Office rpisurv[536]: [f4515448] avcodec decoder error: cannot start codec (h264_v4l2m2m)
Apr 24 08:07:09 EOD-Office rpisurv[543]: [f4115588] avcodec decoder: Using DRM Video Accel for hardware decoding
Apr 24 08:07:09 EOD-Office rpisurv[543]: [h264_v4l2m2m @ 0xf3002020] Could not find a valid device
Apr 24 08:07:09 EOD-Office rpisurv[543]: [h264_v4l2m2m @ 0xf3002020] can't configure decoder
Apr 24 08:07:09 EOD-Office rpisurv[543]: [f4115588] avcodec decoder error: cannot start codec (h264_v4l2m2m)
Apr 24 08:07:09 EOD-Office rpisurv[556]: [f4015428] main decoder error: Timestamp conversion failed (delay 500000, buffering 0, bound 9000000)
Apr 24 08:07:09 EOD-Office rpisurv[556]: [f4015428] main decoder error: Could not convert timestamp 1679979085086692 for FFmpeg

I’ll run it for today and see if there are any issues with stability, and if it’s still working this time tomorrow, I’ll get in contact with SvenVD and point him to this thread (as well as popping a link to it in the ’ VLC issue on raspbian bullseye #141’ thread).

@MichaIng, I don’t know if you realise just how long standing an issue this is - I have been trying to solve it on and off since November 2011. Thank you so much for taking the time to help me with this!

MichaIng · 24 April 2023 20:23

Okay great. The vc_sm_cma module came with a kernel version only available on Bullseye, and that one (CMA itself) is also the reason why since Bullseye one usually does not need that much GPU memory anymore. CMA (Contiguous Memory Allocator) basically means allocation from system memory (instead of using dedicated GPU memory). However, in the end it depends on what applications do/how they use it, so quote possible that rpisurv still requires more.

You can safely remove the config file btw. It will be automatically generated when selecting <32 MiB GPU memory in dietpi-config and removed when selecting >= 32 MiB:

rm /etc/modprobe.d/dietpi-disable_vcsm.conf

Also keeping it removed with <32 MiB GPU memory does not really hurt, just means 2 additional errors at boot when the driver tries to load (and fails to do so) and still the additional non-functional kernel module being loaded and consuming some memory.

About the h264 error, is the related codec enabled?

/boot/dietpi/func/dietpi-set_hardware rpi-codec enable

MINERAL-115 · 25 April 2023 09:42

Thank you for your continued support, you have been invaluable!

I’m pleased to report that the rpisurv streams have been rock solid since my last post; it no longer does the ‘stream freezes every 1-3 hours’ thing that it was doing back on Buster, and the streams themselves even seem to load noticeably quicker.

However, in the end it depends on what applications do/how they use it, so quote possible that rpisurv still requires more.

It would seem that 256mb is about right for two streams (one ‘main’ and one ‘sub’ rtsp stream), so I do wonder if SvenVD wrote that README.md early on in development when it wasn’t quite so refined, or if it was a result of a hardware complication.

You were exactly right about that configuration file; it looks like I caused that issue myself by editing the memory splits myself rather than using dietpi-config. I did a quick test by setting a 16mb GPU memory split in dietpi-config, rebooting, checking the file, and then confirming that it removed the .conf file after setting memory back to 128mb (via dietpi-config):

Set dietpi-config … to ‘Server’ (16MB).
Rebooted.
dietpi-disable_vcsm.conf created and populated as expected.
Set dietpi-config … to ‘Video de/encoding’ (256MB).
Rebooted.
dietpi-disable_vcsm.conf removed, as expected.
Streams started normally!

Also keeping it removed with <32 MiB GPU memory does not really hurt, just means 2 additional errors at boot when the driver tries to load (and fails to do so) and still the additional non-functional kernel module being loaded and consuming some memory.

Ahh, but that’s the point of dietpi, is it not? To trim things down to run as smoothly and streamlined as possible! Every mb of unnecessary memory that is freed up is a victory in that end!

I love this distro for that very reason, otherwise I’d not have been trying so hard to try and get rpisurv working correctly on it for so long; I’ve 5 Raspberry Pis of my own and a handful that have been deployed as CUPS print servers for my customers, and every single one of them is running DietPi!

About the h264 error, is the related codec enabled?

Nope, it wasn’t! Another modprobe.d .conf file situation:

DietPi-Set_hardware
─────────────────────────────────────────────────────
 Mode: rpi-codec (enable)

[  OK  ] DietPi-Set_hardware | rm /etc/modprobe.d/dietpi-disable_rpi_codec.conf
[  OK  ] DietPi-Set_hardware | Added setting dtoverlay=rpivid-v4l2 to end of file /boot/config.txt
[  OK  ] rpi-codec enable | Completed

What would I normally have to do to enable this codec? Is it related to the GPU memory splits, or something else?

I was still getting errors related to the codec with a 256mb GPU memory split:

-- Journal begins at Tue 2023-04-25 10:11:11 BST. --
Apr 25 10:11:33 EOD-Office rpisurv[583]: MultiFramedRTPSource::doGetNextFrame1(): The total received frame size exceeds the client's buffer size (250000).  61389 bytes of trailing data will be dropped!
Apr 25 10:11:33 EOD-Office rpisurv[590]: [f44154a8] main decoder error: buffer deadlock prevented
Apr 25 10:11:33 EOD-Office rpisurv[583]: [f4715580] main decoder error: buffer deadlock prevented
Apr 25 10:11:33 EOD-Office rpisurv[603]: [f46154a8] main decoder error: buffer deadlock prevented
Apr 25 10:11:38 EOD-Office rpisurv[583]: [f4715580]
Apr 25 10:11:38 EOD-Office rpisurv[590]: [f44154a8] avcodec decoder:
Apr 25 10:11:38 EOD-Office rpisurv[583]: avcodec decoder: Using DRM Video Accel for hardware decoding
Apr 25 10:11:38 EOD-Office rpisurv[590]: Using DRM Video Accel for hardware decoding
Apr 25 10:11:38 EOD-Office rpisurv[603]: [f46154a8] avcodec decoder: Using DRM Video Accel for hardware decoding
Apr 25 10:11:38 EOD-Office rpisurv[603]: [f46154a8] main decoder error: buffer deadlock prevented

I tried 512mb and still got the errors (log below), so I don’t think it’s anything to do with the GPU memory available. I do wonder if this is an rpisurv bug or misconfiguration?

-- Journal begins at Tue 2023-04-25 10:21:46 BST. --
Apr 25 10:22:00 EOD-Office rpisurv[581]: unspecified video dimensions
Apr 25 10:22:00 EOD-Office rpisurv[601]: [f4715500] avcodec decoder error: unspecified video dimensions
Apr 25 10:22:00 EOD-Office rpisurv[581]: MultiFramedRTPSource::doGetNextFrame1(): The total received frame size exceeds the client's buffer size (250000).  63878 bytes of trailing data will be dropped!
Apr 25 10:22:00 EOD-Office rpisurv[588]: [f4115450]
Apr 25 10:22:00 EOD-Office rpisurv[601]: [f4715500]
Apr 25 10:22:00 EOD-Office rpisurv[588]: avcodec decoder: Using DRM Video Accel for hardware decoding
Apr 25 10:22:00 EOD-Office rpisurv[601]: avcodec decoder: Using DRM Video Accel for hardware decoding
Apr 25 10:22:00 EOD-Office rpisurv[588]: [f4115450] main decoder error: buffer deadlock prevented
Apr 25 10:22:00 EOD-Office rpisurv[581]: [f4715450] main decoder error: buffer deadlock prevented
Apr 25 10:22:05 EOD-Office rpisurv[581]: [f4715450] avcodec decoder: Using DRM Video Accel for hardware decoding

I’ve set the GPU memory split to 256mb for the time being, as that seems to be the sweet spot; if I go for 128mb I occasionally get a timestamp error during the startup of rpisurv that does not seem to appear when the memory split is set 256mb:

rpisurv[554]: [f4115710] main decoder error: Could not convert timestamp 1682372383562739 for FFmpeg

MINERAL-115 · 26 April 2023 08:35

I’m pleased to report that the rpisurv streams have been rock solid since my last post; it no longer does the ‘stream freezes every 1-3 hours’ thing that it was doing back on Buster, and the streams themselves even seem to load noticeably quicker.

It seems I spoke too soon, although it did take much, much longer for the streams to freeze on Bullseye; over 24 hours!

Apr 25 21:47:15 EOD-Office rpisurv[566]: 2023/04/25 21:47:15 - l_default - INFO - Sending stats is disabled, not sending stats
Apr 25 22:17:01 EOD-Office CRON[11151]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Apr 25 22:17:01 EOD-Office CRON[11152]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
Apr 25 22:17:01 EOD-Office CRON[11151]: pam_unix(cron:session): session closed for user root
Apr 25 22:25:03 EOD-Office rpisurv[24048]: [f47155b0] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
Apr 25 22:25:06 EOD-Office rpisurv[24012]: [f4215520] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
Apr 25 22:25:17 EOD-Office rpisurv[24048]: [h264_v4l2m2m @ 0xf3801a00] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:18 EOD-Office rpisurv[23999]: [h264_v4l2m2m @ 0xf3301320] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:18 EOD-Office rpisurv[24012]: [h264_v4l2m2m @ 0xf3201250] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:20 EOD-Office rpisurv[24048]: [h264_v4l2m2m @ 0xf3801a00] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:21 EOD-Office rpisurv[23999]: [h264_v4l2m2m @ 0xf3301320] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:21 EOD-Office rpisurv[24012]: [h264_v4l2m2m @ 0xf3201250] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:23 EOD-Office rpisurv[24048]: [h264_v4l2m2m @ 0xf3801a00] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:24 EOD-Office rpisurv[23999]: [h264_v4l2m2m @ 0xf3301320] V4L2 capture poll unexpected timeout: events=0x145
Apr 25 22:25:24 EOD-Office rpisurv[23999]: [f4315528] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
# repeated ad nauseum

Interestingly enough, the stream has frozen at 22:24:41; there isn’t actually any log entries at the time of the freeze, just the decoder error entry about 40 seconds after the freeze occurred.

Here’s rpisurv’s unit status log:

root@EOD-Office:~# systemctl status rpisurv
● rpisurv.service - Rpisurv Raspberry Pi Surveillance
     Loaded: loaded (/etc/systemd/system/rpisurv.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-04-25 10:42:00 BST; 22h ago
   Main PID: 564 (rpisurv)
      Tasks: 57 (limit: 261)
        CPU: 14h 16.601s
     CGroup: /system.slice/rpisurv.service
             ├─  564 /bin/bash /usr/bin/rpisurv
             ├─  566 python3 surveillance.py
             ├─12893 [vcgencmd]
             ├─23998 python3 surveillance.py
             ├─23999 /usr/bin/vlc -I dummy --aspect-ratio=1920:1080 --vout mmal_vout --network-caching 500 --no-video-title-show --mmal-display=hdmi-1 --input-timeshift-granularity=0 --repeat --mmal-vout-transparent --mmal-vout-window 1920x1080+0+0 --mmal-layer 1999999991 --rtsp-tcp --no-audio rtsp://[CameraURL]
             ├─24011 python3 surveillance.py
             ├─24012 /usr/bin/vlc -I dummy --aspect-ratio=620:340 --vout mmal_vout --network-caching 500 --no-video-title-show --mmal-display=hdmi-1 --input-timeshift-granularity=0 --repeat --mmal-vout-transparent --mmal-vout-window 620x340+1280+720 --mmal-layer 1999999992 --rtsp-tcp --no-audio rtsp://[CameraURL]
             ├─24046 bin/pngview -b 0 -l 1999999990 -d 2 -x 0 -y 0 images/blackbackground_3840_2160.png
             ├─24047 python3 surveillance.py
             └─24048 /usr/bin/vlc -I dummy --aspect-ratio=1920:1080 --vout mmal_vout --network-caching 500 --no-video-title-show --mmal-display=hdmi-1 --input-timeshift-granularity=0 --repeat --mmal-vout-transparent --mmal-vout-window 1920x1080+0+0 --mmal-layer 1999999988 --rtsp-tcp --no-audio rtsp://[CameraURL]

Apr 26 00:48:59 EOD-Office rpisurv[23999]: [h264_v4l2m2m @ 0xf3301320]
Apr 26 00:48:59 EOD-Office rpisurv[24048]: [h264_v4l2m2m @ 0xf3801a00]
Apr 26 00:48:59 EOD-Office rpisurv[23999]: V4L2 capture poll unexpected timeout: events=0x145
Apr 26 00:48:59 EOD-Office rpisurv[24048]: VIDIOC_STREAMON on output context: Timer expired
Apr 26 00:49:02 EOD-Office rpisurv[24048]: [h264_v4l2m2m @ 0xf3801a00] output set status 1074026002 (ON) failed: err=62
Apr 26 00:49:02 EOD-Office rpisurv[24012]: [h264_v4l2m2m @ 0xf3201250] V4L2 capture poll unexpected timeout: events=0x145
Apr 26 00:49:02 EOD-Office rpisurv[24048]: [h264_v4l2m2m @ 0xf3801a00] VIDIOC_STREAMON on output context: Timer expired
Apr 26 00:49:24 EOD-Office rpisurv[24048]: [f4b006e0] main input error: ES_OUT_SET_(GROUP_)PCR  is called too late (pts_delay increased to 1074 ms)
Apr 26 04:05:52 EOD-Office rpisurv[24012]: [f46006e0] main input error: ES_OUT_SET_(GROUP_)PCR  is called too late (pts_delay increased to 500 ms)
Apr 26 06:40:26 EOD-Office rpisurv[23999]: [f47006f0] main input error: ES_OUT_SET_(GROUP_)PCR  is called too late (pts_delay increased to 500 ms)

([CameraURLs] added by me.)

I’ve confirmed that the RTSP streams themselves were live and transmitting correctly at the time of the freeze. I’m already using reduced quality streams (medium quality for the main stream, low quality for the ‘picture in picture’ sub stream), so I doubt that’s the issue. It’s also worth noting that although I sent the reboot command via SSH, the output on the monitor still showed the frozen streams, and I was unable to reconnect to try further commands.

The Raspberry Pi is well encapsulated in a heatsink case, so I’m going to pop a ‘medium GPU’ (1700 Mhz ARM / 600 MHz core w/2 overvolt) overclock on for the time being and see if it helps mitigate this issue.

If it freezes again within 48 hours, I’ll try increasing the GPU Memory split to 512mb. The system memory used in normal operation never seems to peak above around 50%, so I suspect that the larger GPU memory allocation won’t be detrimental to performance.

MichaIng · 27 April 2023 11:06

Exactly, but in this case it caused some trouble due to disabled kernel modules which are usually enabled. Two-handed sword .

It’s a dedicated setting in dietpi-config video options.

Hmm, best to ask rpisurv devs about this. Not sure which buffer it is and where it can be adjusted, or whether rpisurv is expected to negotiate this with the client in the first place. It is not the “framebuffer” which can be configured via /boot/config.txt, as (fake) KMS is used.

Two full HD streams and one lower res. Should be actually no problem. But probably keep watching cpu temperature and dmesg -l 0,1,2,3 for kernel errors.

No rpisurv log or no system log at all, i.e. jouenalctl?

MINERAL-115 · 27 April 2023 11:58

Found it! It was already enabled via the /boot/dietpi/func/dietpi-set_hardware rpi-codec enable command, but I toggled it on and off via dietpi-config as well. You know, for luck.

root@EOD-Office:~# dietpi-config
[ SUB1 ] DietPi-Set_hardware > rpi-codec (disable)
[  OK  ] DietPi-Set_hardware | eval echo 'blacklist bcm2835_codec' > /etc/modprobe.d/dietpi-disable_rpi_codec.conf
[  OK  ] DietPi-Set_hardware | sed -i /^[[:blank:]]*dtoverlay=rpivid-v4l2$/d /boot/config.txt
[  OK  ] rpi-codec disable | Completed
[ SUB1 ] DietPi-Set_hardware > rpi-codec (enable)
[  OK  ] DietPi-Set_hardware | rm /etc/modprobe.d/dietpi-disable_rpi_codec.conf
[  OK  ] DietPi-Set_hardware | Added setting dtoverlay=rpivid-v4l2 to end of file /boot/config.txt
[  OK  ] rpi-codec enable | Completed

Will do! I’m planning to make a ‘how to set up DietPi Bullseye for rpisurv’ guide to post on the rpisurv support forum, as a handful of people were having the same issues as me, and went through all of manner of different potential fixes before (seemingly) giving up. I’ll cite this post and tag SvenVD (the author of rpisurv) into it as well.

Temperature is looking good; I haven’t seen it peak above 48’c since applying the overclock.
dmesg -l 0,1,2,3 didn’t seem to actually do anything:

root@EOD-Office:~# dmesg -l 0,1,2,3
root@EOD-Office:~#

Literally nothing. journalctl seems to be working correctly as there are logs on each side of the freeze.

Perhaps SvenVD will be able to shed some light on it? As it stands, since overclocking - and still maintaining 256mb GPU memory - it hasn’t frozen, but I’d like to give it a couple of days of uptime before I’d consider the overclock to be a fix for the freezing.

I suppose it would make sense if Bullseye is more efficient in some aspects than Buster (due to the use of armhf), it could potentially help avoid the freeze for longer if the issue is related to the Raspberry Pi taking too long to respond to some command or request. If that’s the case, the overclock could well be helping to mitigate the problem, right? I should reiterate that I don’t have a deep understanding of armhf, so my interpretation could be off.