Tag: ntp

leap-seconds.list SHA1 checksum

Post author By admin
Post date January 13, 2018
No Comments on leap-seconds.list SHA1 checksum

For fun I wrote a simple program that computes the SHA1 checksum for leap-seconds.list.

It turns out there's quirky convention of writing out the 40-character SHA1 checksum in 5 groups of 8 hex characters - whith the special undocumented rule that leading zeros are suppressed. This means the SHA1 check fails for some files where we happen to have a leading zero in one of the 8-character groups - unless you happen to know about the undocumented rule...

The output looks like this. "New" is the checksum computed by the program, "Old" is the checksum contained in the published file.

https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list
read  117  lines
New:  1e2613791c4627c2d0a34c872ece0ae428dfb714
Old:  1e2613791c4627c2d0a34c872ece0ae428dfb714
Identical ?  True

ftp://tycho.usno.navy.mil/pub/ntp/leap-seconds.list
read  220  lines
New:  3f00425591f969f7252361e527aa6754eb6b7c72
Old:  3f00425591f969f7252361e527aa6754eb6b7c72
Identical ?  True

https://www.ietf.org/timezones/data/leap-seconds.list
read  250  lines
New:  5101445a69948b5109153e2b2086e3d8d54561a3
Old:  5101445a69948b519153e2b2086e3d8d54561a3
Identical ?  False

https://data.iana.org/time-zones/code/leap-seconds.list
read  250  lines
New:  5101445a69948b5109153e2b2086e3d8d54561a3
Old:  5101445a69948b519153e2b2086e3d8d54561a3
Identical ?  False

ftp://ftp.nist.gov/pub/time/leap-seconds.list
read  250  lines
New:  5101445a69948b5109153e2b2086e3d8d54561a3
Old:  5101445a69948b519153e2b2086e3d8d54561a3
Identical ?  False

https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list

read 117 lines

New: 1e2613791c4627c2d0a34c872ece0ae428dfb714

Old: 1e2613791c4627c2d0a34c872ece0ae428dfb714

Identical ? True

ftp://tycho.usno.navy.mil/pub/ntp/leap-seconds.list

read 220 lines

New: 3f00425591f969f7252361e527aa6754eb6b7c72

Old: 3f00425591f969f7252361e527aa6754eb6b7c72

Identical ? True

https://www.ietf.org/timezones/data/leap-seconds.list

read 250 lines

New: 5101445a69948b5109153e2b2086e3d8d54561a3

Old: 5101445a69948b519153e2b2086e3d8d54561a3

Identical ? False

https://data.iana.org/time-zones/code/leap-seconds.list

read 250 lines

New: 5101445a69948b5109153e2b2086e3d8d54561a3

Old: 5101445a69948b519153e2b2086e3d8d54561a3

Identical ? False

ftp://ftp.nist.gov/pub/time/leap-seconds.list

read 250 lines

New: 5101445a69948b5109153e2b2086e3d8d54561a3

Old: 5101445a69948b519153e2b2086e3d8d54561a3

Identical ? False

There's also another simple script for authoring a leap-seconds.list file. It might be used for adding an artificial leap-second, generating a leap-seocnds.list file, and testing how different devices react to a leap-second - without having to wait for a real leap-second event.

In other news IERS recently announced there will be no leap-second in the summer of 2018.

Tags leap-second, leap-seconds.list, ntp

Time and Frequency

Leap second 2016-12-31

Insertion of the 37th leap second on 2016-12-31. Note how 23:59:59 is held for 2 seconds on the displays. The beeps come from our modified-DCF77 AM-coded 25 MHz radio transmitter.

More analysis to come later based on NTP logs. Some stratum-2 NTP servers seemed to take up to 10 minutes to apply the leap second.

Tags leapsecond, ntp

Programming

NTP Timestamp

import datetime
import pytz


def generate_ntp_timestamp(year,month,day,hour,minute,second):
    t = datetime.datetime(year,month,day,hour,minute,second,tzinfo=pytz.utc)
    #NTP leap-seconds list wants seconds since 1900
    epoch_start = datetime.datetime(1900,1,1,0,0,0,tzinfo=pytz.utc)
    delta = t-epoch_start
    delta_s = delta.total_seconds()
    return int(delta_s)


# test that this generates OK values.
# from existing leap-seconds.list
# 2918937600    27    # 1 Jul 1992
# 2950473600    28    # 1 Jul 1993
# 2982009600    29    # 1 Jul 1994
# 3029443200    30    # 1 Jan 1996
# 3076704000    31    # 1 Jul 1997
# 3124137600    32    # 1 Jan 1999
# 3345062400    33    # 1 Jan 2006
# 3439756800    34    # 1 Jan 2009
print "1 Jan 2009: ", generate_ntp_timestamp(2009,1,1,0,0,0)
print "1 Jan 2006: ", generate_ntp_timestamp(2006,1,1,0,0,0)
print "1 Jul 1997: ", generate_ntp_timestamp(1997,7,1,0,0,0)

import datetime

import pytz

def generate_ntp_timestamp(year,month,day,hour,minute,second):

t = datetime.datetime(year,month,day,hour,minute,second,tzinfo=pytz.utc)

#NTP leap-seconds list wants seconds since 1900

epoch_start = datetime.datetime(1900,1,1,0,0,0,tzinfo=pytz.utc)

delta = t-epoch_start

delta_s = delta.total_seconds()

return int(delta_s)

# test that this generates OK values.

# from existing leap-seconds.list

# 2918937600 27 # 1 Jul 1992

# 2950473600 28 # 1 Jul 1993

# 2982009600 29 # 1 Jul 1994

# 3029443200 30 # 1 Jan 1996

# 3076704000 31 # 1 Jul 1997

# 3124137600 32 # 1 Jan 1999

# 3345062400 33 # 1 Jan 2006

# 3439756800 34 # 1 Jan 2009

print "1 Jan 2009: ", generate_ntp_timestamp(2009,1,1,0,0,0)

print "1 Jan 2006: ", generate_ntp_timestamp(2006,1,1,0,0,0)

print "1 Jul 1997: ", generate_ntp_timestamp(1997,7,1,0,0,0)

Tags ntp

Time and Frequency

Meinberg Lantime M600

Here's a peek inside a Meinberg Lantime M600 PTP/NTP server. It follows the same modular design as the Lantime M300 with the GPS receiver on the right and the computer-board to the left. The new thing is a Time Stamping Unit (TSU) in the middle.

The TSU (by Toradex) seems to be built in a memory-stick form factor (SODIMM?) around a Marvell 88ap270m chip (PXA270 processor?). Maybe it's a Toradex Colibri PXZ270?

ADEV of GPS-disciplined MV85 OCXO. Shown with 53230A RCON-mode noise floor.

TDEV of GPS-disciplined MV85 OCXO suggesting a phase measurement resolution of a few nanoseconds.

Morion MV85 OCXO on the GPS receiver board.

Modular design from left to right: Computer, Time Stamping Unit, GPS-reicever and OCXO, Power Supply.

Tags m600, meinberg, ntp, ptp

Time and Frequency

Meinberg Lantime M300

Following Dave's advice here: "Don't turn it on - take it apart!"

Meinberg Lantime M300 - Multi Reference NTP server. This server works as stratum-1 using a number of different input signals such as GPS, 1PPS, 10MHz, and IRIG/Timecode.

The loop time-constants seem to be very long, since the ADEV plot does not change much (at least not immediately) when connecting or disconnecting e.g. the 1PPS or 10MHz inputs. If I would be more patient I would measure it free-running and with all the different stratum-1 input signals for a few days each - that would maybe show some interesting things about how the internal OCXO is locked to the reference inputs. My assumption would be that IRIG would produce the worst ADEV, GPS second worst, 1PPS quite good, and the 10MHz input should produce an excellent ADEV with the shortest lock time-constant.

Lantime M300 10MHz output, free running ADEV.

Overview of internal components. When did you last see a 1Gb CompactFlash card? 😉

Tags meinberg, ntp, ocxo

Computer

NTP stratum-1 on Ubuntu 14.04LTS

Post author By admin
Post date June 26, 2015
No Comments on NTP stratum-1 on Ubuntu 14.04LTS

Some (incomplete) notes on setting up a stratum-1 NTP server on Ubuntu 14.04LTS

To handle the upcoming leap-second we want a leapfile, from: http://www.ietf.org/timezones/data/leap-seconds.list

The path of the leapfile goes into /etc/ntp.conf
leapfile /etc/leap-seconds.list
But Ubuntu uses apparmor, so we must grant permission for the ntp service to read this file in /etc/apparmor.d/usr.sbin.ntpd by adding:
/etc/leap-seconds.list r,
To make apparmor parse and apply the new rules we do:
sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.ntpd
when the ntp service starts it is useful to look at /var/log/syslog where ntp will complain if it doesn't have permission to read the leapfile or if it is badly formatted.

Now let's edit the default options for the ntp service in /etc/default/ntp by adding:
NTPD_OPTS='-gN'
(-N runs ntpd at highest priority, -g makes it more robust agaist large time offsets, see man ntpd)

To get time in NMEA-format and a pulse-per-second (PPS) from gpsd we add two shared-memory (type 28) refclock drivers to /etc/ntp.conf
# GPS Serial data reference server 127.127.28.0 maxpoll 3 fudge 127.127.28.0 time1 -0.230 refid GPS

# GPS PPS reference server 127.127.28.1 prefer maxpoll 2 fudge 127.127.28.1 refid PPS
(the time1 adjustment number needs to be calibrated somehow..).

Finally we let ntp distribute time to the outside world by adding this line to /etc/ntp.conf (this is usually at the end of the file).
restrict default noquery

Now let's set up gpsd. The service configuration file is /etc/default/gpsd, and as suggested in the file we edit it with the utility:
sudo dpkg-reconfigure gpsd
The options that worked for me are device /dev/ttyS0 and options -n (don't wait for clients to connect). After running the utility /etc/default/gpsd should look something like:
START_DEAMON="true" GPSD_OPTIONS="-n" DEVICES="/dev/ttyS0" USBAUTO="false" GPSD_SOCKET="/var/run/gpsd.sock"

You can verify that gpsd is working with cgps, xgps, or gpsmon.

This should result in Ubuntu automagically starting gpsd and ntpd in the correct order at bootup, and ntpq -p should show something along the lines of:

If you want to manually restart (or just start or stop) the services, required e.g. after any changes are made to /etc/ntp.conf, it is done with
sudo service ntp restart sudo service gpsd restart

Tags ntp, ubuntu

Computer

NTP failure detection

Update: Here's how the graph looks like when NTP traffic is enabled again:

A computer that doesn't receive NTP traffic for a while will have its system time drift quite a lot.

In the absence of NTP (UDP port 123) traffic, we can try to roughly ask for the current time over HTTP using wget from google with this shell script:

#!/bin/sh
/usr/bin/wget --no-cache -S -O /dev/null google.com 2&gt;&amp;1 | \
    /bin/sed -n -e '/ *Date: */ {' -e s///p -e q -e '}'

#!/bin/sh

/usr/bin/wget --no-cache -S -O /dev/null google.com 2>&1 | \

/bin/sed -n -e '/ *Date: */ {' -e s///p -e q -e '}'

This outputs a string such as "Sat, 23 Nov 2013 09:14:45 GMT"

Now we can put together a python script that calls this shell script, converts the text-format time-stamp into UTC seconds, and compares against the system time. For plotting we then store the time-error in an RRDTool database. This script is called once per minute using cron.

import subprocess 
import time
import datetime
import rrdtool
import syslog

args = ['googletime.sh']
datetimestring = subprocess.check_output(args)
syslog.syslog( "googletime {0}".format(datetimestring))

# input: Sat, 23 Nov 2013 09:18:02 GMT
# output: 1385191082.0  (seconds since 1.1.1970)
timestamp = time.mktime( time.strptime(datetimestring, '%a, %d %b %Y %H:%M:%S GMT\n'))

# system time, e.g.: 1385191082.0
stime = time.mktime( time.gmtime() )

# should be zero, if all is well
terror = stime-timestamp

# store the measured error in a database
datastring = 'N:{0}'.format(str(terror)) # 'N:1234'
syslog.syslog( "rrd update: {0}".format(datastring) )
ret = rrdtool.updatev( "time_error.rrd" ,datastring)
syslog.syslog( "rrd updatev: {0}".format(ret) )

import subprocess

import time

import datetime

import rrdtool

import syslog

args = ['googletime.sh']

datetimestring = subprocess.check_output(args)

syslog.syslog( "googletime {0}".format(datetimestring))

# input: Sat, 23 Nov 2013 09:18:02 GMT

# output: 1385191082.0 (seconds since 1.1.1970)

timestamp = time.mktime( time.strptime(datetimestring, '%a, %d %b %Y %H:%M:%S GMT\n'))

# system time, e.g.: 1385191082.0

stime = time.mktime( time.gmtime() )

# should be zero, if all is well

terror = stime-timestamp

# store the measured error in a database

datastring = 'N:{0}'.format(str(terror)) # 'N:1234'

syslog.syslog( "rrd update: {0}".format(datastring) )

ret = rrdtool.updatev( "time_error.rrd" ,datastring)

syslog.syslog( "rrd updatev: {0}".format(ret) )

Once we have all the values in our time_error.rrd database we can plot them with rrdtool graph. This is what I get:

There is about -4 seconds of drift during 24 hours, or 46 us/s (46 ppm). If the drift is steady we can guess that the computer was on time 14/4 = ~4 days ago.

The script for creating the rrdtool database is this:

import rrdtool
import time

# DS:ds-name:GAUGE | COUNTER | DERIVE | ABSOLUTE:heartbeat:min:max
data_sources=[ 'DS:TERROR:GAUGE:70:U:U']
# RRA:AVERAGE | MIN | MAX | LAST:xff:steps:rows

utcsecs = int( time.time() )
pts_day= 24*60
primary = 'RRA:AVERAGE:0.5:1:{0}'.format(pts_day) # 2016 points
rrdtool.create( 'time_error.rrd',           # filename
                 '--start', str(utcsecs),   # when to start
                 '--step', '60',            # step between datapoints
                 data_sources,
                 primary)

import rrdtool

import time

# DS:ds-name:GAUGE | COUNTER | DERIVE | ABSOLUTE:heartbeat:min:max

data_sources=[ 'DS:TERROR:GAUGE:70:U:U']

# RRA:AVERAGE | MIN | MAX | LAST:xff:steps:rows

utcsecs = int( time.time() )

pts_day= 24*60

primary = 'RRA:AVERAGE:0.5:1:{0}'.format(pts_day) # 2016 points

rrdtool.create( 'time_error.rrd', # filename

'--start', str(utcsecs), # when to start

'--step', '60', # step between datapoints

data_sources,

primary)

And the graph is created by this script. I am using the simple python-rrdtool python bindings - the object-oriented python-pyrrd may have neater syntax and be more pythonic.

import rrdtool
import time

graphname = '/var/www/test.png'
day = 24*3600
span = 1*day
starttime = int(time.time()-span)
endtime = int( time.time() + 0.1*span)
updatestamp = time.strftime("%Y-%m-%d %H:%M:%S UTC", time.gmtime(time.time()))
graphtitle = '192.168.1.55 System time - google.com time upated: '+updatestamp
rrdtool.graph(  graphname,
               '--start', str(starttime),
               '--end',str(endtime),
               '--title',graphtitle,
               '--width',str(1024),
               '--height',str(600),
               '--full-size-mode',
               '--upper-limit',str(20),
               '--lower-limit',str(-20),
               '--vertical-label','Error (s)', 
               '--right-axis', '1:0',
               'DEF:terror=time_error.rrd:TERROR:AVERAGE',
               'LINE2:terror#FF0000')

import rrdtool

import time

graphname = '/var/www/test.png'

day = 24*3600

span = 1*day

starttime = int(time.time()-span)

endtime = int( time.time() + 0.1*span)

updatestamp = time.strftime("%Y-%m-%d %H:%M:%S UTC", time.gmtime(time.time()))

graphtitle = '192.168.1.55 System time - google.com time upated: '+updatestamp

rrdtool.graph( graphname,

'--start', str(starttime),

'--end',str(endtime),

'--title',graphtitle,

'--width',str(1024),

'--height',str(600),

'--full-size-mode',

'--upper-limit',str(20),

'--lower-limit',str(-20),

'--vertical-label','Error (s)',

'--right-axis', '1:0',

'DEF:terror=time_error.rrd:TERROR:AVERAGE',

'LINE2:terror#FF0000')

Tags ntp, python

Time and Frequency

NTP shared-memory refclock driver for White Rabbit SPEC

Post author By admin
Post date September 27, 2013
No Comments on NTP shared-memory refclock driver for White Rabbit SPEC

I've been playing with White Rabbit hardware at work. White Rabbit uses a combination of SyncE and PTP to perform very precise (<1 ns) time-distribution.

The standard low-precision way to distribute time is NTP. I hacked together a very experimental "Type 28" NTP refclock driver that reads the WR-time from SPEC shared memory and writes it to another location in shared memory where NTP expects it. Code over here: https://github.com/aewallin/ptp2ntpd

The graph shows system clock variations compared to WR-time (which we assume is very accurate) for a computer with the WR-refclock driver enabled (blue trace, minpoll 16 s), and another computer where the system clock is kept on time using standard NTP (I just added some servers to ntpd.conf, no other settings changed from default Ubuntu 12.04LTS). The WR-disciplined clock stays within maybe 50 microseconds with no net drift during the ~9 hour measurement, but the trace is quite jumpy. The NTP-disciplined clock wanders around much more (300 us) but the trace is smoother.

Tags ntp, white rabbit

Time and Frequency

NTP time measurement

Here's a plot of the time error between the standard unix system-time, kept on time using NTP, and a much more accurate PTP-server based on White Rabbit that runs on a fancy FPGA-based network-card.

Note that without NTP a typical computer clock will be off by 10 ppm (parts-per-million) or more. This particular one measured about 40 ppm error in free-running mode (no NTP). That means during the duration of this 16e4 s measurement we'd be off by about 640 milliseconds (way off the chart) without NTP. With NTP the error seems to stay within 3 milliseconds or so. The offset of -16 milliseconds is not that accurately measured and could be caused by a number of things.

Tags ntp, time, white rabbit