Tag: ntp

leap-seconds.list SHA1 checksum

For fun I wrote a simple program that computes the SHA1 checksum for leap-seconds.list.

It turns out there's quirky convention of writing out the 40-character SHA1 checksum in 5 groups of 8 hex characters - whith the special undocumented rule that leading zeros are suppressed. This means the SHA1 check fails for some files where we happen to have a leading zero in one of the 8-character groups - unless you happen to know about the undocumented rule...

The output looks like this. "New" is the checksum computed by the program, "Old" is the checksum contained in the published file.

https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list
read  117  lines
New:  1e2613791c4627c2d0a34c872ece0ae428dfb714
Old:  1e2613791c4627c2d0a34c872ece0ae428dfb714
Identical ?  True

ftp://tycho.usno.navy.mil/pub/ntp/leap-seconds.list
read  220  lines
New:  3f00425591f969f7252361e527aa6754eb6b7c72
Old:  3f00425591f969f7252361e527aa6754eb6b7c72
Identical ?  True

https://www.ietf.org/timezones/data/leap-seconds.list
read  250  lines
New:  5101445a69948b5109153e2b2086e3d8d54561a3
Old:  5101445a69948b519153e2b2086e3d8d54561a3
Identical ?  False

https://data.iana.org/time-zones/code/leap-seconds.list
read  250  lines
New:  5101445a69948b5109153e2b2086e3d8d54561a3
Old:  5101445a69948b519153e2b2086e3d8d54561a3
Identical ?  False

ftp://ftp.nist.gov/pub/time/leap-seconds.list
read  250  lines
New:  5101445a69948b5109153e2b2086e3d8d54561a3
Old:  5101445a69948b519153e2b2086e3d8d54561a3
Identical ?  False

There's also another simple script for authoring a leap-seconds.list file. It might be used for adding an artificial leap-second, generating a leap-seocnds.list file, and testing how different devices react to a leap-second - without having to wait for a real leap-second event.

In other news IERS recently announced there will be no leap-second in the summer of 2018.

See also time-stamp for leap-seconds.list.

NTP Timestamp

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import datetime
import pytz
 
 
def generate_ntp_timestamp(year,month,day,hour,minute,second):
    t = datetime.datetime(year,month,day,hour,minute,second,tzinfo=pytz.utc)
    #NTP leap-seconds list wants seconds since 1900
    epoch_start = datetime.datetime(1900,1,1,0,0,0,tzinfo=pytz.utc)
    delta = t-epoch_start
    delta_s = delta.total_seconds()
    return int(delta_s)
 
 
# test that this generates OK values.
# from existing leap-seconds.list
# 2918937600    27    # 1 Jul 1992
# 2950473600    28    # 1 Jul 1993
# 2982009600    29    # 1 Jul 1994
# 3029443200    30    # 1 Jan 1996
# 3076704000    31    # 1 Jul 1997
# 3124137600    32    # 1 Jan 1999
# 3345062400    33    # 1 Jan 2006
# 3439756800    34    # 1 Jan 2009
print "1 Jan 2009: ", generate_ntp_timestamp(2009,1,1,0,0,0)
print "1 Jan 2006: ", generate_ntp_timestamp(2006,1,1,0,0,0)
print "1 Jul 1997: ", generate_ntp_timestamp(1997,7,1,0,0,0)

Meinberg Lantime M600

Here's a peek inside a Meinberg Lantime M600 PTP/NTP server. It follows the same modular design as the Lantime M300 with the GPS receiver on the right and the computer-board to the left. The new thing is a Time Stamping Unit (TSU) in the middle.

The TSU (by Toradex) seems to be built in a memory-stick form factor (SODIMM?) around a Marvell 88ap270m chip (PXA270 processor?). Maybe it's a Toradex Colibri PXZ270?

Meinberg Lantime M300

Following Dave's advice here: "Don't turn it on - take it apart!"

Meinberg Lantime M300 - Multi Reference NTP server. This server works as stratum-1 using a number of different input signals such as GPS, 1PPS, 10MHz, and IRIG/Timecode.

The loop time-constants seem to be very long, since the ADEV plot does not change much (at least not immediately) when connecting or disconnecting e.g. the 1PPS or 10MHz inputs. If I would be more patient I would measure it free-running and with all the different stratum-1 input signals for a few days each - that would maybe show some interesting things about how the internal OCXO is locked to the reference inputs. My assumption would be that IRIG would produce the worst ADEV, GPS second worst, 1PPS quite good, and the 10MHz input should produce an excellent ADEV with the shortest lock time-constant.

NTP stratum-1 on Ubuntu 14.04LTS

Some (incomplete) notes on setting up a stratum-1 NTP server on Ubuntu 14.04LTS

To handle the upcoming leap-second we want a leapfile, from: http://www.ietf.org/timezones/data/leap-seconds.list

The path of the leapfile goes into /etc/ntp.conf
leapfile /etc/leap-seconds.list
But Ubuntu uses apparmor, so we must grant permission for the ntp service to read this file in /etc/apparmor.d/usr.sbin.ntpd by adding:
/etc/leap-seconds.list r,
To make apparmor parse and apply the new rules we do:
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.ntpd
when the ntp service starts it is useful to look at /var/log/syslog where ntp will complain if it doesn't have permission to read the leapfile or if it is badly formatted.

Now let's edit the default options for the ntp service in /etc/default/ntp by adding:
NTPD_OPTS='-gN'
(-N runs ntpd at highest priority, -g makes it more robust agaist large time offsets, see man ntpd)

To get time in NMEA-format and a pulse-per-second (PPS) from gpsd we add two shared-memory (type 28) refclock drivers to /etc/ntp.conf

# GPS Serial data reference
server 127.127.28.0 maxpoll 3
fudge 127.127.28.0 time1 -0.230 refid GPS

# GPS PPS reference
server 127.127.28.1 prefer maxpoll 2
fudge 127.127.28.1 refid PPS

(the time1 adjustment number needs to be calibrated somehow..).

Finally we let ntp distribute time to the outside world by adding this line to /etc/ntp.conf (this is usually at the end of the file).
restrict default noquery

Now let's set up gpsd. The service configuration file is /etc/default/gpsd, and as suggested in the file we edit it with the utility:
sudo dpkg-reconfigure gpsd
The options that worked for me are device /dev/ttyS0 and options -n (don't wait for clients to connect). After running the utility /etc/default/gpsd should look something like:

START_DEAMON="true"
GPSD_OPTIONS="-n"
DEVICES="/dev/ttyS0"
USBAUTO="false"
GPSD_SOCKET="/var/run/gpsd.sock"

You can verify that gpsd is working with cgps, xgps, or gpsmon.

This should result in Ubuntu automagically starting gpsd and ntpd in the correct order at bootup, and ntpq -p should show something along the lines of:
Screenshot - 06262015 - 10:18:31 AM

If you want to manually restart (or just start or stop) the services, required e.g. after any changes are made to /etc/ntp.conf, it is done with
sudo service ntp restart
sudo service gpsd restart

NTP failure detection

Update: Here's how the graph looks like when NTP traffic is enabled again:
ntp

A computer that doesn't receive NTP traffic for a while will have its system time drift quite a lot.

In the absence of NTP (UDP port 123) traffic, we can try to roughly ask for the current time over HTTP using wget from google with this shell script:

#!/bin/sh
/usr/bin/wget --no-cache -S -O /dev/null google.com 2>&1 | \
    /bin/sed -n -e '/ *Date: */ {' -e s///p -e q -e '}'

This outputs a string such as "Sat, 23 Nov 2013 09:14:45 GMT"

Now we can put together a python script that calls this shell script, converts the text-format time-stamp into UTC seconds, and compares against the system time. For plotting we then store the time-error in an RRDTool database. This script is called once per minute using cron.

import subprocess 
import time
import datetime
import rrdtool
import syslog
 
args = ['googletime.sh']
datetimestring = subprocess.check_output(args)
syslog.syslog( "googletime {0}".format(datetimestring))
 
# input: Sat, 23 Nov 2013 09:18:02 GMT
# output: 1385191082.0  (seconds since 1.1.1970)
timestamp = time.mktime( time.strptime(datetimestring, '%a, %d %b %Y %H:%M:%S GMT\n'))
 
# system time, e.g.: 1385191082.0
stime = time.mktime( time.gmtime() )
 
# should be zero, if all is well
terror = stime-timestamp
 
# store the measured error in a database
datastring = 'N:{0}'.format(str(terror)) # 'N:1234'
syslog.syslog( "rrd update: {0}".format(datastring) )
ret = rrdtool.updatev( "time_error.rrd" ,datastring)
syslog.syslog( "rrd updatev: {0}".format(ret) )

Once we have all the values in our time_error.rrd database we can plot them with rrdtool graph. This is what I get:
system_vs_google_time
There is about -4 seconds of drift during 24 hours, or 46 us/s (46 ppm). If the drift is steady we can guess that the computer was on time 14/4 = ~4 days ago.

The script for creating the rrdtool database is this:

import rrdtool
import time
 
# DS:ds-name:GAUGE | COUNTER | DERIVE | ABSOLUTE:heartbeat:min:max
data_sources=[ 'DS:TERROR:GAUGE:70:U:U']
# RRA:AVERAGE | MIN | MAX | LAST:xff:steps:rows
 
utcsecs = int( time.time() )
pts_day= 24*60
primary = 'RRA:AVERAGE:0.5:1:{0}'.format(pts_day) # 2016 points
rrdtool.create( 'time_error.rrd',           # filename
                 '--start', str(utcsecs),   # when to start
                 '--step', '60',            # step between datapoints
                 data_sources,
                 primary)

And the graph is created by this script. I am using the simple python-rrdtool python bindings - the object-oriented python-pyrrd may have neater syntax and be more pythonic.

import rrdtool
import time
 
graphname = '/var/www/test.png'
day = 24*3600
span = 1*day
starttime = int(time.time()-span)
endtime = int( time.time() + 0.1*span)
updatestamp = time.strftime("%Y-%m-%d %H:%M:%S UTC", time.gmtime(time.time()))
graphtitle = '192.168.1.55 System time - google.com time upated: '+updatestamp
rrdtool.graph(  graphname,
               '--start', str(starttime),
               '--end',str(endtime),
               '--title',graphtitle,
               '--width',str(1024),
               '--height',str(600),
               '--full-size-mode',
               '--upper-limit',str(20),
               '--lower-limit',str(-20),
               '--vertical-label','Error (s)', 
               '--right-axis', '1:0',
               'DEF:terror=time_error.rrd:TERROR:AVERAGE',
               'LINE2:terror#FF0000')

NTP shared-memory refclock driver for White Rabbit SPEC

ntp_graph

I've been playing with White Rabbit hardware at work. White Rabbit uses a combination of SyncE and PTP to perform very precise (<1 ns) time-distribution.

The standard low-precision way to distribute time is NTP. I hacked together a very experimental "Type 28" NTP refclock driver that reads the WR-time from SPEC shared memory and writes it to another location in shared memory where NTP expects it. Code over here: https://github.com/aewallin/ptp2ntpd

The graph shows system clock variations compared to WR-time (which we assume is very accurate) for a computer with the WR-refclock driver enabled (blue trace, minpoll 16 s), and another computer where the system clock is kept on time using standard NTP (I just added some servers to ntpd.conf, no other settings changed from default Ubuntu 12.04LTS). The WR-disciplined clock stays within maybe 50 microseconds with no net drift during the ~9 hour measurement, but the trace is quite jumpy. The NTP-disciplined clock wanders around much more (300 us) but the trace is smoother.

NTP time measurement

NTP_time_2013jul26

Here's a plot of the time error between the standard unix system-time, kept on time using NTP, and a much more accurate PTP-server based on White Rabbit that runs on a fancy FPGA-based network-card.

Note that without NTP a typical computer clock will be off by 10 ppm (parts-per-million) or more. This particular one measured about 40 ppm error in free-running mode (no NTP). That means during the duration of this 16e4 s measurement we'd be off by about 640 milliseconds (way off the chart) without NTP. With NTP the error seems to stay within 3 milliseconds or so. The offset of -16 milliseconds is not that accurately measured and could be caused by a number of things.

freerunning_vs_ntp_2013jul26