Problem with serial-getty@ttyAMA0.service on an NTP server

Somewhere on the todo is to implement a GPS-based time sync mode to DietPi as option naively. So figuring out some potential issues, GPS hardware or UART device behavior, even if it is in rare circumstances, is no waste time. And it is interesting in any case :smile:.

Maybe you can swap the GPS receiver between the two systems, so see whether the same hardware behaves the same way on both systems, or either hardware behaves the same on the same system (indicating a difference in setup/software).

You mean that even when having the console/serial-getty@ttyAMA0.service enabled at boot, it stops working when removing console=ttyAMA0 from /boot/cmdline.txt, so it requires both, the cmdline entry as well as the console service starting at boot? So weird, as if the HAT requires quite some activity on the UART pins before it (its GPS receiver) becomes active or so.

One last check: In a state where gpsmon does not work, does:

systemctl status dev-ttyAMA0.device

always show it being inactive. And when gpsmon works, does above always show the device active? I am still not 100% sure what such device status means (need to check systemd docs), but maybe it shows whether there is actual hardware attached/answering on that device node. When it is inactive, that would indicate that the HAT/GPS receiver is somehow disabled, not answering or showing any signal on the UART pins :thinking:.

2 Likes

I could theoretically change the hardware. But that would be a lot of effort, as the system, which now works as expected with your help, is installed in a closed metal housing and the GPS module is attached to the housing and not inside.

I will most likely build a 3rd system for a different location, possibly even with the same HAT.
If the same problem occurs again, I will contact you again in confidence, if I may.
If contrary to expectations, it works without trickery, then I will also contact you.

Maybe something is bent in the software through my fault.

I hadn’t mentioned that an adsb-feeder is also running on the system with the HAT, so I wouldn’t want to set it up again for the time being.

By the way, please keep the GPS-based synchronization mode for DietPi on your to-do list!
It would make me and maybe others happy! :+1:

2 Likes

I’ll run the test tomorrow if gpsmon shows no output and post it. :sign_of_the_horns:

Check_01

  • gps-setup.service disable (workaround service)
  • ttyAMA0 console deactivated
  • /boot/cmdline.txt checks whether the entry console=ttyAMA0,115000 is still present (no)
  • reboot performed

Status afterwards:

  • gpsmon no output
  • sytemctl Status dev-ttyAMA0.device active
  • gpsd.service active, without error messages

Check_02

  • gps-setup.service disable (workaround service)
  • ttyAMA0 console activated
  • /boot/cmdline.txt the entry console=ttyAMA0,115200 exists
  • reboot performed

Status afterwards:

  • gpsmon no output
  • sytemctl status dev-ttyAMA0.device active
  • gpsd.service active, but with error messages
    gpsd:ERROR: SER: device open of /dev/ttyAMA0 failed: Permission denied - retrying read-only
    gpsd:ERROR: SER: read-only device open of /dev/ttyAMA0 failed: Permission denied
    gpsd:ERROR: SER: /dev/ttyAMA0: device activation failed, freeing device.

Check_03

  • gps-setup.service disable (workaround service)
  • ttyAMA0 console activated
  • removed the entry console=ttyAMA0,115200 from /boot/cmdline.txt
  • reboot performed

Status afterwards:

  • gpsmon no output
  • sytemctl status dev-ttyAMA0.device active
  • gpsd.service active, but with error messages
    gpsd:ERROR: SER: device open of /dev/ttyAMA0 failed: Permission denied - retrying read-only
    gpsd:ERROR: SER: read-only device open of /dev/ttyAMA0 failed: Permission denied
    gpsd:ERROR: SER: /dev/ttyAMA0: device activation failed, freeing device.

Check_04

  • ttyAMA0 console activated
  • /boot/cmdline.txt the entry console=ttyAMA0,115200 exists
  • gps-setup.service enable (workaround)
  • gps-setup.service started
  • reboot performed

Status afterwards:

  • gpsmon has output
  • sytemctl status dev-ttyAMA0.device active
  • gpsd.service active, without error messages
  • everything works fine

By chance I found a setting for the gpsd configuration on another GPS-HAT.

In my case it looks like this:

DEVICES=“/dev/ttyAMA0 /dev/pps0”
GPSD_OPTIONS=”-n -G”
USBAUTO="false”

The other setting says:

DEVICES=“/dev/ttyS0”
GPSD_OPTIONS=”/dev/ttyUSB0”
USSBAUTO="false”

In my case, the DEVICE /dev/ttySO is the only DEVICE under Serial/Uart that I can activate or deactivate (and apparently must activate for correct function), a DEVICE /dev/ttyAMA0 is not there.

What if I leave the console /dev/ttyAMA0 disabled, but adjust the gpsd configuration as follows:

DEVICE=“/dev/ttyS0 /dev/pps0”
GPSD_OPTIONS=”/dev/ttyAMA0 -n -G”
USBAUTO="false”

One more thing, what do the gpsd error messages regarding permission denied, read-only and activation failed mean under Check_02 and Check_03?
Are there perhaps missing rights? Can I check this on the other system here?

1 Like

Okay, so the dev-ttyAMA0.device is just always active, not indicating whether something is attached or answering on related pins. Good to know, actually also matches my tests, where all UART devices are active which are known to function on a certain board.

Since you want to use the full hardware UART ttyAMA0 (and PPS), your config is correct. The other one would use the software mini UART, and ttyUSB0 would be the name of a USB-UART adapter attached to the RPi’s USB port, that does not exist in your case. Whether that is added to GPSD_OPTIONS or DEVICES does not make a difference in this case. Both variables are just added one after another, so as long as there is no other option after a device node in GPSD_OPTIONS, it is used as source device. Hence adding it to GPSD_OPTIONS like that is also misleading, incorrect in a way, and can lead to invalid arguments (when other options are added).

As said, other than ttyS0, ttyAMA0 is always active and cannot be disabled on RPi 3 and 4 (via config.txt entry), so there is consequently no option to enable or disable it in dietpi-config.

That would be the invalid configuration I mentioned above: source devices like /dev/ttyAMA0 need to be the last arguments. Adding other options like -n -G after a device node renders the args invalid, unless argument parsing is done very generously. Also, you cannot get GPS input on both, ttyS0 and ttyAMA0 at the same time. If Bluetooth is disabled, ttyAMA0 is using the related GPIO pins (GPIO 14+15, pins 8+10 respectively). If Bluetooth is enabled, and the ttyS0 UART device is enabled, that one will use those pins. If Bluetooth is enabled and the ttyS0 device is disabled, there is no UART active on those pins at all.

This is most likely due to the serial-getty@ttyAMA0.service using and blocking the device, which is also the reason you need to stop it after boot, before gpsd can use it. But, now I remember that also the service start changes permissions of the device, allowing group dialout to write to it, which is not always the default (depending on kernel). Can you compare permissions

ls -l /dev/ttyAMA0

with and without the serial console enabled in dietpi-config? gpsd starts as root, but maybe it drops permissions at some point.

1 Like

Wow, that was another detailed explanation that shed a lot of light on the subject for me, thanks for that! :+1:

Here are the outputs of the command on both systems.

System 1 - OK

ls -l /dev/ttyAMA0
crw-rw---- 1 root dialout 204, 64  5. Jun 14:17 /dev/ttyAMA0

System 2 - OK with workaround and activated serial console

ls -l /dev/ttyAMA0
crw-rw---- 1 root tty 204, 64  5. Jun 15:14 /dev/ttyAMA0

System 2 - Condition as system 1 without activated serial console

ls -l /dev/ttyAMA0
crw-rw---- 1 root dialout 204, 64 29. Mai 19:04 /dev/ttyAMA0

At the weekend I will most probably set up another “small” system as a test.
It will consist of an RPi Zero 2 WH, a GT-U7 GPS module and a USB 2.0 - Ethernet adapter, I still have the hardware lying around here.
I’ll be using DietPi, of course, so we’ll see how that works and how the little Pi performs.
If I look at the utilization of the other two systems, the Zero 2 should be sufficient, the USB-Ethernet connection could possibly cause problems, but it’s certainly better than WiFi.

As announced, a short update on the new installation on an RPi Zero 2 WH.

The installation went smoothly and the Zero 2 has no problem running as a time server. It is connected via a USB Ethernet adapter and a GT-U7 GPS module provides the GPS data.

I did not have to use a workaraound. This suggests that I either made a mistake when installing the Waveshare LC29H (AA) GPS-HAT, or the HAT is behaving differently than expected. The instructions at Waveshare assume that ttyUSB0, i.e. the USB port, is used.

Anyway, if I reinstall the Pi 4 once, I’ll see if the same problem exists or not.

I hadn’t mentioned one thing before, I only noticed it during the reinstallation. Since I still shy away from IPv6 like the plague, I had to comment it out in /lib/systemd/system/gpsd.socket ListenStream=[::1]:2947 and set BindIPv6Only=no.

However, this should not have been the cause of my problem with the HAT.

Now that all ambiguities have been resolved, I would like to thank you once again for your great support. :sign_of_the_horns:

Best regards, Carsten

1 Like

As said, a serial device would appear as /dev/ttyUSB0 if it is a UART adapter attached to the USB port of the RPi, like the USB-UART adapters you use to connect to the serial console of the RPi from a PC, but the other way round. This HAT however sits on the GPIO header, and is not connected to any USB port of the RPi, is it?

I mean, theoretically, it seems to be possible to program the GPIO pins to act like a USB port, and have the HAT act not only as GPS device, but as USB-UART adapter as well. But that would be pretty much nonsense as there is a perfectly functional UART adapter available on the GPIO header the HAT is sitting on.

But you can just check whether a /dev/ttyUSB0 exists.

Can you link the instructions you mean? I found this one: https://www.waveshare.com/wiki/LC29H(XX)_GPS/RTK_HAT
There it mentions ttyS0 (which is correct as long as Bluetooth is enabled, which is by default on RPi OS, but not on DietPi), and falsefully ttyAMA0 on RPi 5, but AFAIK it is ttyAMA10 there.

It also correctly mentions ttyUSB0 in case the USB port needs to be used. I guess it is possible to connect the mini USB port of header with a USB-A port of the RPi for this purpose. E.g. if Bluetooth is needed, and the mini UART is not wanted enabled, because it limits the core frequency to 250 MHz.

Now I see the example settings further down the wiki. Indeed a bit weird, probably aimed to be most compatible, regardless whether USB is used or not. Funny that it does not take into account the possibility to disable Bluetooth at all, which just makes perfectly sense in 95% of cases. And RPi models without onboard WiFi/BT use ttyAMA0 by default as UART device.

No, the HAT is actually plugged into the GPIO HEADER and is not connected via USB.
It’s good that you also find the instructions or the wiki a little unusual with regard to use with a Pi.

But as I said, the other two are inexpensive GPS modules that also do their job.
If I ever feel like working on more accuracy (RTK), using an even higher quality module or a HAT and a Pi 5 for PTP, then I’ll report on my success or failure here.