ntpd setup

Gord_W · 28 November 2015 13:40

Hi,

I would like to get the clock on the rpi set so that it doesn’t drift. I’m using the rpi for a timing application over a wide temperature range and just having the clock updated once per day with a cron job is not going to cut it as the drift is significant.

So I would like to have ntp running all the time.

If I run nptd from the command line, it starts and runs.

I’ve edited the ntp.config file to allow for logs to be created

/etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntp/ntp.drift

Enable this if you want statistics to be logged.

statsdir /var/log/ntpstats/

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

My rpi is headless and I’m using ssh.

So my problems and questions and remember that I am a beginner here:

What is the best way to start ntp at boot and set the clock?
Can I just remove the ntp lines in the daily cron job and not break anything else?
When i run ntp from the command line the drift file does not get updated. ntp is running under root and the drift file is owned by ntp. I don’t know if that is the problem or not with my ignorance.
Is there a place that errors (like file permissions) would show up? I have logging turned on, but there is little that I see in the /var/log area.
loopstats and peerstats start to show in the /var/log/ntpstats dir, but they get deleted every few minutes by something so that I only have 0 to 5 last entries. I was expecting a full day per file of data that I could look at.
There is no clockstats file - should there be in ntp client mode?

Any other pointers that I may be missing?

Thanks,
Gordon Williams

Fourdee · 28 November 2015 13:59

Hi Gordon,

DietPi runs ntpd during boot and once daily. We use the ntpd -gq flag to quit ntpd when its obtained the time, saving resources.

I’ll need to update DietPi code to allow for a constant running ntpd. I will create a option in dietpi.txt that will allow for both options.

ntpd runs as root, so there is no chance of file permission issues.

DietPi-Ramlog will clear log files every hour. If you need to keep your logfiles, you can change the logging mode to Ramlog #2 or Full:https://dietpi.com/forum/t/dietpi-survey-information/32/1

Git Ticket: https://github.com/Fourdee/DietPi/issues/124
It will be available in v102.

Fourdee · 28 November 2015 14:14

Gordon,

Would an hourly NTPD update be sufficient?

Fourdee · 28 November 2015 18:35

I’ve added:

DietPi-Config > Advanced > Time sync (ntpd)

Available in V102

Gord_W · 28 November 2015 20:14

Hi,

I was not asking for you to change the setup as my needs are not like most users. It was more how to make changes myself to make it run as I need it to. Some advice please

I want ntpd to run continuously to keep the clock disciplined and synchronized to real time from the servers. The current uncorrected clock has about a 40 ppm error slow (at room temp) and if ntpd runs it will increase the clock frequency to remove this error. This error gets stored in the drift file when everything is working properly (but it is not currently being stored, maybe due to permissions as root doesn’t own that file, user ntp does).

As I will be eventually running this not connected to the internet and ntp servers, I’m expecting a clock error in excess of 150 ppm at -20 deg. For clock discipline then I will be using a temperature corrected rtc with a pulse per second output with a max. error of 3 ppm. That is a struggle for another day! Right now I just want ntp working normally.

So I’m looking for:

ntpd started at boot with the time of day set and then left running to continuously nudge the rpi clock to correct for errors.
No hourly or daily ntp correction through cron.
Drift file update working. It is currently at -6.366 ppm with ntp owner on your V101 wheezy version
I’ll have another look at the log files to try and figure out why they are being deleted - it doesn’t seem like it hourly.

Thanks,
Gordon

Hi, I see that you have already implemented something while I was writing the reply! It looks like 1) and 2) is covered. Could you have a look at the drift file? /var/lib/ntp/ntp.drift Does this get updated regularly? or is it sitting with the Apr 1 2015 last modified date?

Fourdee · 28 November 2015 20:29

Hi Gordon,

So you are looking for a way to obtain the slew (drift) without applying the current time to the system clock?

Drift file update working. It is currently at -6.366 ppm with ntp owner on your V101 wheezy version

Seems you are right, appears this file is not being updated.

Gord_W · 28 November 2015 20:49

Hi,

So you are looking for a way to obtain the slew (drift) without applying the current time to the system clock?

On boot I want the system time to jump to the real time and then start making corrections based on the errors found from the ntp servers.

Once ntpd is working it will automatically correct for the drift and clock offset to slowly move the rpi clock to real time from the servers. The drift file is used on boot to set the initial expected drift and then it adjusts from there. If the file is not present the drift is assumed to be zero and therefore takes longer to synchronize the rpi clock to real time.

Gordon

Gord_W · 29 November 2015 01:19

Drift file update working. It is currently at -6.366 ppm with ntp owner on your V101 wheezy version

Seems you are right, appears this file is not being updated.

I changed the ownership of the drift file to root from ntp. After a couple of hours of running and the frequency offset determined and stabilized it wrote to the file. So every thing seems to be working properly.

I will wait until I see V102 to get ntp started automatically at boot.

Thanks,

Gordon Williams

Fourdee · 29 November 2015 11:44

Great to hear.

I tried no end of options to try and get the drift file working. Maybe I didn’t wait long enough during testing.

Are you running ntpd with command line options (eg: -f /etc/ntp.drift), or just ntpd?

Gord_W · 29 November 2015 20:37

Are you running ntpd with command line options (eg: -f /etc/ntp.drift), or just ntpd?

Just running ntpd from the command line with no arguments. It does take a while to stabilize before it writes the file. If you want to see the log files and track convergence you need to uncomment the section in ntp.conf.

Gordon Williams

Gord_W · 30 November 2015 00:50

─────────────────────────────────────────────────────
DietPi CPU Info
Use dietpi-config to change CPU / performance options
─────────────────────────────────────────────────────
Architecture | armv6l
Temp | 4’c | Who put me in the freezer!
Governor | ondemand
Throttle up | 60% CPU usage

Very funny!

Gordon Williams

Fourdee · 30 November 2015 11:38

hehe, got to have a little fun now and then

Ok to get NTPD running from boot and constantly on your current system:
use ctrl+w to find…

nano /etc/cron.daily/dietpi
#disable (comment) the following lines:

#killall ntpd &> /dev/null
#ntpd -gq &> /dev/null

nano /DietPi/dietpi/boot
#Change the following line

#Update NTP (+thread)
ntpd

For v102:
I will also add an option to dietpi-config that will run NTPD from boot and keep the daemon running.

Gord_W · 30 November 2015 16:18

Hi,

I’m not sure your boot code is going to do it.

I think there is a time stored in a file somewhere that the system time get written to occasionally. I think that is used to help set the system clock on boot. If that is far off true time or non-existent then just a straight ntpd command will fail based on my reading ntpd - Network Time Protocol (NTP) daemon. It looks like ntpd will exit if sys time error exceeds 1000s from real time.

-g
Normally, ntpd exits if the offset exceeds the sanity limit, which is 1000 s by default. If the sanity limit is set to zero, no sanity checking is performed and any offset is acceptable. This option overrides the limit and allows the time to be set to any value without restriction; however, this can happen only once. After that, ntpd will exit if the limit is exceeded. This option can be used with the -q option.

I therefore think that we should have
ntpd -g
as the command to start ntp in the boot file.

Do you come to the same conclusion? I haven’t tried stopping the rpi for a while and then restarting it to confirm that ntp will fail and exit due to the time discrepancy > 1000s

Gordon Williams

Fourdee · 30 November 2015 19:46

Yep, the -g flag will do it, allows for excessive time differences. We use that at the moment with -q which quits when done.

Gord_W · 30 November 2015 21:55

I’m not sure what the problem is with
ntpd -g
but ntp gets running but the PLL doesn’t get synced and the drift frequency doesn’t get corrected. It just uses the drift value in the drift file.

After about 15 minutes I was still getting this (peers were good with reach=377 (all data received) long ago):

root@DietPi:~# ntptime
ntp_gettime() returns code 5 (ERROR)
time da073f69.7c60e000 Mon, Nov 30 2015 16:34:33.485, (.485853),
maximum error 16000000 us, estimated error 16000000 us, TAI offset 0
ntp_adjtime() returns code 5 (ERROR)
modes 0x0 (),
offset 0.000 us, frequency -47.137 ppm, interval 1 s,
maximum error 16000000 us, estimated error 16000000 us,
status 0x41 (PLL,UNSYNC),
time constant 7, precision 1.000 us, tolerance 500 ppm,

If I run:
ntpd -gq
ntpd

then it magically jumps to this
root@DietPi:~# ntptime
ntp_gettime() returns code 0 (OK)
time da073fbd.1686726c Mon, Nov 30 2015 16:35:57.087, (.087989690),
maximum error 2016 us, estimated error 16 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 0.000 us, frequency -43.938 ppm, interval 1 s,
maximum error 2016 us, estimated error 16 us,
status 0x2001 (PLL,NANO),
time constant 3, precision 0.001 us, tolerance 500 ppm,

and a few minutes later everything seems to be working OK with reasonable errors identified, pll offset is now non-zero too:

root@DietPi:~# ntptime
ntp_gettime() returns code 0 (OK)
time da07433f.914e1a08 Mon, Nov 30 2015 16:50:55.567, (.567598512),
maximum error 328335 us, estimated error 2759 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 487.965 us, frequency -43.321 ppm, interval 1 s,
maximum error 328335 us, estimated error 2759 us,
status 0x2001 (PLL,NANO),
time constant 6, precision 0.001 us, tolerance 500 ppm,

Will change the boot file and see the effect
Gordon Williams

Gord_W · 1 December 2015 00:52

This is what works and syncs up fast!

If I modify the boot file as shown:
#----------------------------------------------------------------
#Update NTP (+thread) gw modified
ntpd -gq
ntpd
#----------------------------------------------------------------

Then as soon as the wifi comes up and I ssh in (about 50 sec after boot):

First I get:

root@DietPi:~# ntptime
ntp_gettime() returns code 0 (OK)
time da07695b.9090c000 Mon, Nov 30 2015 19:33:31.564, (.564709),
maximum error 1016 us, estimated error 16 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 0.000 us, frequency -41.414 ppm, interval 1 s,
maximum error 1016 us, estimated error 16 us,
status 0x1 (PLL),
time constant 7, precision 1.000 us, tolerance 500 ppm,

The a couple of seconds later:

root@DietPi:~# ntptime
ntp_gettime() returns code 0 (OK)
time da076961.b7ee8038 Mon, Nov 30 2015 19:33:37.718, (.718483614),
maximum error 1042781 us, estimated error 738 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 2081.837 us, frequency -41.400 ppm, interval 1 s,
maximum error 1042781 us, estimated error 738 us,
status 0x2001 (PLL,NANO),
time constant 6, precision 0.001 us, tolerance 500 ppm,

So no error messages! Both show returns code 0 (OK) rather than error 5.

Regards,
Gordon Williams

Fourdee · 1 December 2015 17:36

Hi Gordon,

Thanks for the update and info. I’ve updated the v102 code.

When DietPi v102 is released, you can do the following:
dietpi-config > advanced options > Time Sync (NTPD)
Then select option 4 and reboot

ps. If you run the following, it will execute ntpd -gq then the daemon in that order, whilst creating a separate thread for it. This should prevent any delay during the boot script execution as the system won’t wait for it to complete, before it carries on with other things.

ntpd -gq && ntpd -g -l /var/log/ntpd.log &

pps. although, if you want the time to be correct before other software runs, the above isn’t recommended.

Gord_W · 2 December 2015 17:37

Hi,

You have -l /var/log/ntpd.log

The log file dir is already identified in the ntp.config file so you may not want to override that here. As well, the stats are commented out in the ntp.conf file by default so normally you wouldn’t get any stats anyway.

The hourly ram log copies the ntp log to my root home directory, but it leaves a lot of empty log files behind in the /var/log/ntpstats/ dir. An example

root@DietPi:/var/log/ntpstats# ls -l
total 24
-rwxrwxr-x 2 root root 119 Dec 2 12:29 loopstats
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1893C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1907C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1912C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1918C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1919C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1920C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1923C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1924C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1925C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1928C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.1932C1
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.20151128
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.20151129
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.20151130
-rwxrwxr-x 1 root root 1 Dec 1 17:17 loopstats.20151201
-rwxrwxr-x 2 root root 119 Dec 2 12:29 loopstats.20151202
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.2270C0
-rwxrwxr-x 1 root root 0 Dec 1 14:17 loopstats.2704C1
-rwxrwxr-x 2 root root 418 Dec 2 12:29 peerstats
-rwxrwxr-x 1 root root 0 Dec 1 14:17 peerstats.1893C0
-rwxrwxr-x 1 root root 0 Dec 1 14:17 peerstats.1907C0
-rwxrwxr-x 1 root root 0 Dec 1 14:17 peerstats.1912C0
-rwxrwxr-x 1 root root 0 Dec 1 14:17 peerstats.1918C0
… and so on …

Gordon Williams

Fourdee · 2 December 2015 18:53

This is intentional.
I wanted to override the filepath. This will ensure the file gets included (/var/log/*) when dietpi-bugreport is sent.

Thanks for the report.

DietPi-Ramlog clears the contents of the logfile but keeps the original file in place. I originally designed the program to delete the file itself, however, this caused certain programs (eg: dnsmasq) to stop writing to the log files after that point.
I also tested restarting the services after deleting the logfile, although it fixed the issue above, it wasn’t practical to create a downtime.

Although the size is reported at 0, each of those empty files will use 4096bytes of RAM (tmpfs allocation size).
You can run the following to delete all log files:

dietpi-logclear 2

Then i’d suggest you reboot.

Gord_W · 2 December 2015 21:32

Hi,

As I’ve now had a chance to play with ntp a bit and monitor it’s clock performance, I’m quite impressed.

A suggestion that might be useful to people who want to use their rpi not connected to the internet:
If their rpi during setting up is connected to the internet, they can let ntp run for a few hours to calibrate the drift in their system clock and that drift will be written to the drift file. When ntpd is started again it will apply that drift to the oscillator and the correction will be continued to be applied even if ntpd is stopped again.

Therefore on reboot when the rpi is not connected to the internet and ntpd -gq is run, the time will have to be set manually, but with the drift offset the rpi will keep time pretty well.

My rpi has a drift of about 45 ppm at room temp and using the calibration it brings it down to 2 ppm or so.

To my surprise, putting the rpi in the deep freeze didn’t cause it to drift very much. From my initial test it seems less than 5ppm, which is amazing as I was expecting 100 ppm. I need to confirm that is true, but it looks that way.

Gordon Williams