I need some help investigating.
DietPi 8.0.2
Pi Zero W use as a simple camera stream
I am experiencing a resource lock.
The evidence is lack of RTSP stream and unresponsive htop in Terminal.
And a log event of ‘systemd-udevd.service: Watchdog timeout (limit 3min)!’ which can keep repeating, or stop logging, until I take an action.
I have spent the last few weeks getting the camera stream running and suspecting various issues, (voltage drop, wifi strength, stream as a service, Motioneye on the other machine)
Current suspects are device conflicts of some sort, wifi or camera, or my camrun service.
I see DietPi-Wifi_Monitor checks wifi frequently, and I browsed through a lot of brmc firmware issues over the past 2 years.
The RTSP stream runs as a service (camrun.service, raspivid piped to cvlc.)
The problem is the stream stops and htop command is unresponsive.
The stream is seen stopping in Motioneye (running on a separate machine).
Most of the time journalctl and systemctl still run, and ping works from and to.
Restarting the camrun.service seems to fix the issue for a while - and of course a reboot.
The system stops responding after ‘some’ time.
It has just occurred within 5 minutes of a reboot.
Quite often the log entry immediately before the ‘Watchdog timeout’ is a command line input from me, often a journalctl. What a coincidence I can input just before the log entry!
Unusual logs:
- no logs for over an hour this evening (that was supper time) and logging re-triggered by htop
- log with a jump back in time of an hour or so, after complete lock out and dropped wifi (no ping)
Thanks in advance for help.
Some info …
system details
Clean install of 7.9 mid Dec, upgraded to 8.0.2 , motion package removed manually
Linux version 5.10.63+ (dom@buildbot) (arm-linux-gnueabihf-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1488 Thu Nov 18 16:14:04 GMT 2021
CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
Machine model: Raspberry Pi Zero W Rev 1.1
camrun.sh
#!/bin/bash
raspivid -ih -stm -fl -n -o - -t 0 -md 5 -fps 24 -qp 30 -b 0 |
cvlc -I dummy stream:///dev/stdin --h264-fps 24 --sout '#rtp{sdp=rtsp://:8554/stream}' :demux=h264
camrun.service
[Unit]
Description = acyanic RTSP cam stream
After = network.target
[Service]
Type=simple
ExecStart = /home/dietpi/camrun.sh
User=dietpi
WorkingDirectory=/home/dietpi
Restart=always
[Install]
WantedBy = default.target
log showing a gap … logging triggered by terminal input of htop
Jan 12 17:46:45 DietPi19 sudo[6259]: dietpi : TTY=pts/0 ; PWD=/home/dietpi ; USER=root ; COMMAND=/usr/bin/journalctl
Jan 12 17:46:45 DietPi19 sudo[6259]: pam_unix(sudo:session): session opened for user root(uid=0) by dietpi(uid=1000)
Jan 12 17:46:45 DietPi19 sudo[6259]: pam_unix(sudo:session): session closed for user root
Jan 12 19:14:05 DietPi19 sudo[6897]: dietpi : TTY=pts/0 ; PWD=/home/dietpi ; USER=root ; COMMAND=/usr/bin/journalctl
Jan 12 19:14:05 DietPi19 sudo[6897]: pam_unix(sudo:session): session opened for user root(uid=0) by dietpi(uid=1000)
Jan 12 19:14:05 DietPi19 systemd[1]: Starting Clean php session files...
Jan 12 19:14:05 DietPi19 systemd[1]: systemd-udevd.service: Watchdog timeout (limit 3min)!
Jan 12 19:14:05 DietPi19 systemd[1]: systemd-udevd.service: Killing process 5597 (systemd-udevd) with signal SIGABRT.
Jan 12 19:14:07 DietPi19 systemd[1]: phpsessionclean.service: Succeeded.
Jan 12 19:14:07 DietPi19 systemd[1]: Finished Clean php session files.
Jan 12 19:14:07 DietPi19 systemd[1]: systemd-udevd.service: Main process exited, code=killed, status=6/ABRT
Jan 12 19:14:07 DietPi19 systemd[1]: systemd-udevd.service: Failed with result 'watchdog'.
log showing backward jump in timestamp - no manual action
Jan 12 19:14:07 DietPi19 systemd-udevd[6944]: Network interface NamePolicy= disabled on kernel command line, ignoring.
Jan 12 19:14:08 DietPi19 systemd[1]: Started Rule-based Manager for Device Events and Files.
Jan 12 19:14:08 DietPi19 sudo[6897]: pam_unix(sudo:session): session closed for user root
Jan 12 18:09:01 DietPi19 CRON[7121]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Jan 12 18:09:01 DietPi19 CRON[7122]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
Jan 12 18:09:01 DietPi19 CRON[7121]: pam_unix(cron:session): session closed for user root