Date: Wed, 24 Aug 2016 18:55:37 -0700 From: Cy Schubert <Cy.Schubert@komquats.com> To: Ian Lepore <ian@freebsd.org> Cc: Andreas Ott <andreas@naund.org>, Kevin Oberman <rkoberman@gmail.com>, Randy Bush <randy@psg.com>, FreeBSD Stable <freebsd-stable@freebsd.org>, Cy Schubert <cy@FreeBSD.org>, pfg@freebsd.org Subject: Re: leapsecond file Message-ID: <201608250155.u7P1tbda010990@slippy.cwsent.com> In-Reply-To: Message from Ian Lepore <ian@freebsd.org> of "Wed, 24 Aug 2016 14:21:14 -0600." <1472070074.1430.39.camel@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multipart MIME message.
--==_Exmh_1472089702_29840
Content-Type: text/plain; charset=us-ascii
In message <1472070074.1430.39.camel@freebsd.org>, Ian Lepore writes:
> On Tue, 2016-08-23 at 22:40 -0700, Andreas Ott wrote:
> > On Sun, Aug 07, 2016 at 09:44:55PM -0700, Kevin Oberman wrote:
> > > On 11.0-BETA4 I have:
> > > > grep expires /var/db/ntpd.leap-seconds.list
> > > # File expires on: 1 Jun 2017
> > >
> > > But I see what you do on 10.3-RELEASE. Looks like the update has
> > > not made
> > > it into 10 (an I would guess 9).
> >
> > The flaw is in the internal versioning of files, it does fetch the
> > newer
> > file from IETF and puts it into /var/run/ntpd.leap-seconds.list, but
> > although the expiry is newer, the FreeBSD onboard source version
> > claims
> > to be newer in the 'last update date in NTP timestamp format' line
> > ...
> >
> > FreeBSD 10.3-RELEASE-p7 onboard source is in /etc/ntp/leap-seconds,
> > which gets copied to /var/db/ntpd.leap-seconds.list. The fetch
> > IETF file downloads to /var/run/ntpd.leap-seconds.list .
> >
> >
> > [root@mon ~]# egrep -e Update\|expires\|^'#\$' /etc/ntp/leap-seconds
> > # Last Update of leap second values: 31 Dec 2015
> > #$ 3660508800
> > # Updated through IERS Bulletin C 50
> > # File expires on: 1 Jun 2016
> > [root@mon ~]# egrep -e Update\|expires\|^'#\$' /var/db/ntpd.leap
> > -seconds.list
> > # Last Update of leap second values: 31 Dec 2015
> > #$ 3660508800
> > # Updated through IERS Bulletin C 50
> > # File expires on: 1 Jun 2016
> > [root@mon ~]# egrep -e Update\|expires\|^'#\$' /var/run/ntpd.leap
> > -seconds.list
> > # Last Update of leap second values: 5 January 2015
> > #$ 3629404800
> > # Updated through IERS Bulletin C51
> > # File expires on: 28 December 2016
> > [root@mon ~]#
> >
> > with the end result that the file from /var/run/ never gets moved to
> > /var/db/ .
> > The 480.leapfile-ntpd used by periodic calls the same rc file fetch
> > and
> > comparison.
> >
> > -andreas
>
> The Last Update value reflects only additions of leap seconds to the
> file, and it is specifically documented that Last Update does NOT
> change when the file's expiration date is extended without changing the
> other contents.
>
> It looks like part of the problem here is that the Last Update value IS
> changing when the leap data itself is not. Our commit logs say the
> files have been obtained from USNO. Either USNO is violating the
> standard in their files, or the value is getting modified before it's
> commited to the freebsd repos. (I'm adding Cy to the CC list, since he
> committed them.)
The file was obtained from USNO. Except for a $FreeBSD$ and a minor
spelling fix that was brought forward from r298087, the file was not
altered in any way. One of the problems is that the minor spelling fix
invalidated the secure hash at the end of the file, ntpd ignores the file.
r298087 needs to be reverted. (cc'd pfg@) I'll revert r298087.
Fetching and comparing shows the files being totally different because of
comments and whitespace differences. Comparing the version numbers (update
dates) we have the following:
slippy$ grep '#\$' leap-seconds.*
leap-seconds.iers:#$ 3676752000
leap-seconds.ietf:#$ 3629404800
leap-seconds.usno:#$ 3676752000
(Slippy BTW is the name of a dead pet. I name my computers after dead pets.)
The IETF file hasn't been updated for a while:
slippy$ date -r $((3629404800-2208988800))
Sun Jan 4 16:00:00 PST 2015
The other two files were update in July:
slippy$ date -r $((3676752000-2208988800))
Tue Jul 5 17:00:00 PDT 2016
Looking at expiry dates:
slippy$ grep '#@' leap-seconds.*
leap-seconds.iers:#@ 3707596800
leap-seconds.ietf:#@ 3691872000
leap-seconds.usno:#@ 3705264000
The IETF file expires at:
slippy$ date -r $((3691872000-2208988800))
Tue Dec 27 16:00:00 PST 2016
slippy$
The IERS file expires at:
slippy$ date -r $((3707596800-2208988800))
Tue Jun 27 17:00:00 PDT 2017
slippy$
The USNO file expires at:
slippy$ date -r $((3705264000-2208988800))
Wed May 31 17:00:00 PDT 2017
slippy$
>
> It looks like the fetch/install decisions in rc.d/ntpd are not quite
> right either. Both Last Update and Expiration date have to be taken
> into account. To allow for these broken files that incorrectly change
> the Last Update, workable logic would be to keep the file with the
> highest Expiration date, and if the expirations are equal, then keep
> the one with the highest Last Update. (I think it would be better to
> test Last Update first, then use Expiration as the tie-breaker, but
> that fails with these broken files.) Testing both Expiration and Last
> Update will allow for a corrected file to be published after
> accidentally publishing bad data, and we'd take the corrected file.
The attached patch should address this last issue.
I'll revert pfg's spelling fixup which I had brought forward and I'll need
to remove $FreeBSD$ as well, validating the hash again. Additional code
will need to be added to rc.d/ntpd to replace the copy in /var/db if
$FreeBSD$ exists.
--==_Exmh_1472089702_29840
Content-Type: text/plain ; name="ntpd.diff"; charset=us-ascii
Content-Description: ntpd.diff
Content-Disposition: attachment; filename="ntpd.diff"
Index: ntpd
===================================================================
--- ntpd (revision 304600)
+++ ntpd (working copy)
@@ -67,11 +67,13 @@
}
get_ntp_leapfile_ver() {
+ # Leapfile update date (version number).
expr "$(awk '$1 == "#$" { print $2 }' "$1" 2>/dev/null)" : \
'^\([1-9][0-9]*\)$' \| 0
}
get_ntp_leapfile_expiry() {
+ # Leapfile expiry date.
expr "$(awk '$1 == "#@" { print $2 }' "$1" 2>/dev/null)" : \
'^\([1-9][0-9]*\)$' \| 0
}
@@ -88,11 +90,15 @@
ntp_tmp_leapfile="/var/run/ntpd.leap-seconds.list"
ntp_ver_no_src=$(get_ntp_leapfile_ver $ntp_src_leapfile)
+ ntp_expiry_src=$(get_ntp_leapfile_expiry $ntp_src_leapfile)
ntp_ver_no_db=$(get_ntp_leapfile_ver $ntp_db_leapfile)
+ ntp_expiry_db=$(get_ntp_leapfile_expiry $ntp_db_leapfile)
$verbose ntp_src_leapfile version is $ntp_ver_no_src
$verbose ntp_db_leapfile version is $ntp_ver_no_db
- if [ "$ntp_ver_no_src" -gt "$ntp_ver_no_db" ]; then
+ if [ "$ntp_ver_no_src" -gt "$ntp_ver_no_db" -o \
+ "$ntp_ver_no_src" -eq "$ntp_ver_no_db" -a \
+ "$ntp_expiry_src" -gt "$ntp_expiry_db" ]; then
$verbose replacing $ntp_db_leapfile with $ntp_src_leapfile
cp -p $ntp_src_leapfile $ntp_db_leapfile
ntp_ver_no_db=$ntp_ver_no_src
@@ -99,8 +105,8 @@
else
$verbose not replacing $ntp_db_leapfile with $ntp_src_leapfile
fi
+ ntp_leapfile_expiry_seconds=$((ntp_leapfile_expiry_days*86400))
ntp_leap_expiry=$(get_ntp_leapfile_expiry $ntp_db_leapfile)
- ntp_leapfile_expiry_seconds=$((ntp_leapfile_expiry_days*86400))
ntp_leap_fetch_date=$((ntp_leap_expiry-ntp_leapfile_expiry_seconds))
if [ $(current_ntp_ts) -ge $ntp_leap_fetch_date ]; then
$verbose Within ntp leapfile expiry limit, initiating fetch
@@ -108,8 +114,11 @@
$verbose fetching $url
fetch $ntp_leapfile_fetch_opts -o $ntp_tmp_leapfile $url && break
done
+ ntp_ver_no_tmp=$(get_ntp_leapfile_ver $ntp_tmp_leapfile)
ntp_expiry_tmp=$(get_ntp_leapfile_expiry $ntp_tmp_leapfile)
- if [ "$ntp_expiry_tmp" -gt "$ntp_leap_expiry" ]; then
+ if [ "$ntp_ver_no_tmp" -gt "$ntp_ver_no_db" -o \
+ "$ntp_ver_no_tmp" -eq "$ntp_ver_no_db" -a \
+ "$ntp_expiry_tmp" -gt "$ntp_expiry_db" ]; then
$verbose using $url as $ntp_db_leapfile
mv $ntp_tmp_leapfile $ntp_db_leapfile
else
--==_Exmh_1472089702_29840
Content-Type: text/plain; charset=us-ascii
Cheers,
Cy Schubert <Cy.Schubert@cschubert.com>
FreeBSD UNIX: <cy@FreeBSD.org> Web: http://www.FreeBSD.org
The need of the many outweighs the greed of the few.
--==_Exmh_1472089702_29840--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201608250155.u7P1tbda010990>