From nobody Fri Dec 8 05:10:35 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SmfNl6qffz53KDX for ; Fri, 8 Dec 2023 05:10:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SmfNl1hMyz3CN5 for ; Fri, 8 Dec 2023 05:10:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=pQyfi1C3; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::62a) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-a1ceae92ab6so223891466b.0 for ; Thu, 07 Dec 2023 21:10:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1702012243; x=1702617043; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=iCBBMx96O5Axgd6dc27jv5zfwR7+RTHjThUm8L9MCKU=; b=pQyfi1C3nQHXdaSzveRy+flu8KgEjz+wdcRlGYRRP1a+rZjNF1ftHMXlbFbSklFjDt AamVNhhzZnMjLgpU8cLbYSKC1ofOHFT7sRkFpc4a48wFzOoOymRrvFAtOK1UFOM+n1YJ ydhxQPjJdc/SO50/3m8poawemP7pFhsUDWEB7NHqDE/V8rOTTuneWVXeFzq6DzHccNc5 V9j0woZHCnm9vTlXtB/J1paUfDBTN0ZMoVImcFfnWmPR4dg/rnHbRAgS8JD+lz2qTzIj G4F8DH8R3aD2FtUjZ4u4D6SR78KyOrGZjxbwl8HlfDAADPVQmm+b30+LNf3TdkC11KM3 ZmwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702012243; x=1702617043; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iCBBMx96O5Axgd6dc27jv5zfwR7+RTHjThUm8L9MCKU=; b=gfsZ99h2Pdw5Y9X4JOHRIcFEYAjAWLvkbxFbVPQSUjAODgSB1yt7xiO+xaHro8yKSt cl9qnMdjVsiZ7oEqqU1Yy+G+z/jgOafiJmoEG1N9TrlQ7uzsjz115Q7W8LnN5z0usBP4 fSDVFP0sXjBHloomX6fOncK4Vc35dXj1HGSFcxXLfeT4V9nFrfDafrt/zMi2M9hQdDqX qcNj0s6BjJCipz101pMAmQr9i0D62JyekMXt4A/wMJPVnM4id839uw51oTbJ5m/3yrc5 cnujtI30YQ/ro2TBr2+RSKeqo8swCz6jH1esSSKQhUkorD7PYIZq2o8KPV3KTGBUC9T6 i+zw== X-Gm-Message-State: AOJu0YyQrBX9sfCvivryQJetg10zShsWwsJQN19PFaV2eQfzUQuxHOg2 2wpW2OkHbtHbDur5UxqaoaRAyYbw0VDzOJwC80bLzw== X-Google-Smtp-Source: AGHT+IFug4GrkVUcJT1/30ZRC+Y3egSSqvNfoVPyrvhXT1htBVxWteOXPfRbo6mCKUfq7baeHvNvp6R3y/M47VSTLrI= X-Received: by 2002:a17:906:101:b0:a1d:9d6c:57d4 with SMTP id 1-20020a170906010100b00a1d9d6c57d4mr2331141eje.55.1702012242982; Thu, 07 Dec 2023 21:10:42 -0800 (PST) List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 References: <202312070550.3B75o8WV066387@gitrepo.freebsd.org> <389AB29C-D5C0-4091-91ED-219F33351B35@freebsd.org> <20231207222716.obSthG6r@steffen%sdaoden.eu> <20231208010731.3hijmSTL@steffen%sdaoden.eu> In-Reply-To: <20231208010731.3hijmSTL@steffen%sdaoden.eu> From: Warner Losh Date: Thu, 7 Dec 2023 22:10:35 -0700 Message-ID: Subject: Re: git: b1c95af45488 - main - rc.conf: correct $ntp_leapfile_sources To: Warner Losh , Xin Li , Philip Paeps , src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Content-Type: multipart/alternative; boundary="000000000000258860060bf89b7f" X-Spamd-Result: default: False [0.00 / 15.00]; INTRODUCTION(2.00)[]; SUBJECT_HAS_CURRENCY(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[dev-commits-src-main@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_SPF_NA(0.00)[no SPF record]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; PREVIOUSLY_DELIVERED(0.00)[dev-commits-src-main@freebsd.org]; DMARC_NA(0.00)[bsdimp.com] X-Rspamd-Queue-Id: 4SmfNl1hMyz3CN5 X-Spamd-Bar: / --000000000000258860060bf89b7f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Dec 7, 2023 at 6:07=E2=80=AFPM Steffen Nurpmeso wrote: > Warner Losh wrote in > = : > |On Thu, Dec 7, 2023 at 3:27=E2=80=AFPM Steffen Nurpmeso > wrote: > |> Xin Li wrote in > |> : > |>|On 2023-12-06 22:34, Philip Paeps wrote: > |>|> On 2023-12-07 14:26:05 (+0800), Warner Losh wrote: > |>|>> We should point to bipm > |>|>> https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list since > |> they > |>|>> are > |>|>> the source of truth, no? > |>|> > |>|> I went for the IANA copy because data.iana.org is a much shorter an= d > |>|> trustworthy looking URL. And it's also where other operating syste= ms > |>|> get their copies. > |>| > |>|My understanding is that IANA's copy is part of tzdata and it's only > |>|updated when a new set of zone data is released, so it's sometimes > |>|outdated. It is actually going to be outdated really soon by the way= . > |> > |> But nothing will change. > |> It is only about the included end-of-life tag why there is > |> discussion at all. > |> The IANA TZ data is always updated as necessary, "early enough". > | > |Yes. TZ data updates multiple times a year. The lead time on NIST/BIPM > |updating the file usually is within days or weeks after the new leap is > |announced. > |But ntpd can't possibly use it for about 5 months. TZ updates are plent= y > |fast. > | > |The bigger problem is that we have to do a EN to get a new set of zone > |files. If we had a way to fetch them, we could just copy this file from > the > |updated > |zone files. > > I never spoke against fetching the plain file (who is my role in > this project in the end?), i only spoke against using the server > of the french institute directly. > The French institute is the source of truth. The BIPM defines what UTC is, based on atomic clock measurements from all over the world. A subagency, the IERS, measures the delta between UTC and the earth's orientation and makes the determination of when a leap second is scheduled. There's no cryptographic signature of this file. There is a hash that ensures it's not corrupted, but it can't be verified as authoritative since it's just a SHA hash. By grabbing it from BIPM, the source of truth for time, we at least get their TLS certs to back up the file. Grabbing it from anywhere else means our users have to trust the other places. While the IETF/IANA are trustworthy, it's one level removed. Then again, given this file, in this context, is only used when ntpd can't otherwise determine the leap seconds, so maybe that high level of trust isn't strictly needed. The lack of easy verification of this file has been discussed in the time community on and off for the last 25 or more years. > Ie one could poll this once a day or so, and upload it somewhere > else. Or ping the github URL of the raw IANA TZ file, which is > placed in Eggert's repo quite fast, even without release. > (Just in case the FreeBSD project want to lock out all the > countries that github blocks, i think Iran, North Korea, Cuba > even, who knows which otherwise, i will not look up that political > war thing.) > Polling any more frequently than every other month is overkill. This file is used, at most, twice a year. It changes 6 months before the leap second, give or take. Polling it every 60 days is given how often it is used. Ideally, our scripts would spread out the polling over those 60 days... > |> Also the beasts are about to get rid of leap seconds until 2035 > ... > |> earlier). Bets can be placed whether it will happen before > |> a possible occurring leap second, or not. (My bet is that they > |> run everything against the wall, and then run away yelling about > |> the evil leap second, after having missed to create an appropriate > ... > |Yea, we're years away from the next leap second. And there's still > |a good chance we'll have at least one more. And there's also rumblings > |that leaps will stop before 2035 (that's the current absolute last date= , > |and there's several folks that want to pull that in). What will happen > |for sure isn't well known... > | > |ntpv5 discussions have, at times, assumed there will be no more leap > |seconds and so ntpv5 needn't have anything to accommodate them. > > I have not looked into current WG outcome (there was just recently > some IETF post regarding NTP, i have not looked). > I would only wish they distribute TAI and the offset to UTC, but > i think, i would hope, they will doing the opposite regulary. > I have no time, and i am not the right person to do anything about > NTP, let alone in the IETF. All i could ever say i did say, and > that was that it was always wrong to go for "civil time" aka UTC, > at least without a permanent indication to a constant reliable > time scale. And a longer possibility to detect leaps, as i knew > secretaries who turn off their computer at Friday afternoon, not > to turn it on again until Monday morning. And my impression in > the past, before i came a little bit involved in international > email etc communication, was that the engineers simply could not > imagine such a situation. Or, *at best*, decided that then > silence is the best communication on such a switch. > The need for leap seconds, or its lack, has complicated history. > |>|The IERS one is more up-to-date because they publish the bulletin. > |> > |> In general i think distribution of load is a good thing, and > |> i find it very unfriendly to put all the load onto some jealous > |> institute (if it is one) and its single server. > |> The FreeBSD project has an established set of mirrors, and, the > ... > |We can skew the load in time by spreading all our users out over > |the few months we have if there's a load issue. > > I have recognized even the nice name of this mechanism in the > FreeBSD rc system, Warner Losh. > Yea, I worked for a timing company in the 2000s. I couldn't recall if I'd written this or Ian had. I think my name is on it, but Ian fixed all the mistakes I made. :) > |> Btw PHK had a thrilling idea of DNS distributing leap ticks some > |> years ago, and he even started to host it. As it unfortunately > |> did not fly i did not track it further. > |> Would also be an idea for the FreeBSD project: simply download the > |> file ones, then place a DNS record that FreeBSD installations then > |> can query. DNSSEC is in place i think. > | > |Yea, that's not a thing that's happening. It was an interesting idea, > |but hasn't been standardized and there's little to apetite to distribut= e > |this way. > > Unfortunately. And then there is the fully blown tzdist protocol > that the IETF hammered through with XML and what not formats, > unfortunately not CBOR ("later"), this is what Meinberg says on > the state of that: > > https://kb.meinbergglobal.com/kb/time_sync/tzdist > Yea... > |>|The bundled version was from NIST ftp, but fetching from ftp for ever= y > |>|FreeBSD system out there was too scary for me. > |>| > |>|There may be some security / privacy concerns if we direct users to a > |>|place that we do not have control, by the way. > |> > |> Interesting aspect! > | > |There might be, but this sounds somewhat speculative. What's the antici= p\ > |ated > |concerns? > > Maybe Xin Li has stumbled over the same thread as i after that > publicsuffix CVE of cURL (first sentence of the quoted message): > > https://lists.gnu.org/archive/html/bug-wget/2014-03/msg00113.html > > What i mean is, the FreeBSD project and its pkg database, isn't > this a natural place for such a thing? With guaranteed / > controlled availability. > The ntp leap stuff does pre-date the pkg by a decade. Having a package for it might be a natural evolution, Warner > --steffen > | > |Der Kragenbaer, The moon bear, > |der holt sich munter he cheerfully and one by one > |einen nach dem anderen runter wa.ks himself off > |(By Robert Gernhardt) > | > | Only in December: lightful Dubai COP28 Narendra Modi quote: > | A small part of humanity has ruthlessly exploited nature. > | But the entire humanity is bearing the cost of it, > | especially the inhabitants of the Global South. > | The selfishness of a few will lead the world into darkness, > | not just for themselves but for the entire world. > --000000000000258860060bf89b7f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Dec 7, 2023 at 6:07=E2=80=AFP= M Steffen Nurpmeso <steffen@sdaode= n.eu> wrote:
Warner Losh wrote in
=C2=A0<CANCZdfpHWRECi=3DDyhxJAW4MkA-CyPLK=3DOSdSwBdKQJ57MyPwNA@mail.gmail.co= m>:
=C2=A0|On Thu, Dec 7, 2023 at 3:27=E2=80=AFPM Steffen Nurpmeso <steffen@sdaoden.eu>= wrote:
=C2=A0|> Xin Li wrote in
=C2=A0|>=C2=A0 <d75b041f-05f8-44c1-8de6-1fef89b7e537@delph= ij.net>:
=C2=A0|>|On 2023-12-06 22:34, Philip Paeps wrote:
=C2=A0|>|> On 2023-12-07 14:26:05 (+0800), Warner Losh wrote:
=C2=A0|>|>> We should point to bipm
=C2=A0|>|>> https://hpiers.obspm= .fr/iers/bul/bulc/ntp/leap-seconds.list since
=C2=A0|> they
=C2=A0|>|>> are
=C2=A0|>|>> the source of truth, no?
=C2=A0|>|>
=C2=A0|>|> I went for the IANA copy because data.iana.org is a much sh= orter and
=C2=A0|>|> trustworthy looking URL.=C2=A0 And it's also where oth= er operating systems
=C2=A0|>|> get their copies.
=C2=A0|>|
=C2=A0|>|My understanding is that IANA's copy is part of tzdata and = it's only
=C2=A0|>|updated when a new set of zone data is released, so it's so= metimes
=C2=A0|>|outdated.=C2=A0 It is actually going to be outdated really soon= by the way.
=C2=A0|>
=C2=A0|> But nothing will change.
=C2=A0|> It is only about the included end-of-life tag why there is
=C2=A0|> discussion at all.
=C2=A0|> The IANA TZ data is always updated as necessary, "early en= ough".
=C2=A0|
=C2=A0|Yes. TZ data updates multiple times a year. The lead time on NIST/BI= PM
=C2=A0|updating the file usually is within days or weeks after the new leap= is
=C2=A0|announced.
=C2=A0|But ntpd can't possibly use it for about 5 months. TZ updates ar= e plenty
=C2=A0|fast.
=C2=A0|
=C2=A0|The bigger problem is that we have to do a EN to get a new set of zo= ne
=C2=A0|files. If we had a way to fetch them, we could just copy this file f= rom the
=C2=A0|updated
=C2=A0|zone files.

I never spoke against fetching the plain file (who is my role in
this project in the end?), i only spoke against using the server
of the french institute directly.

The F= rench institute is the source of truth. The BIPM defines what
UTC= is, based on atomic clock measurements from all over the world.
= A subagency, the IERS, measures the delta between UTC and
the ear= th's orientation and makes the determination of when a
leap s= econd is scheduled.

There's no cryptographic s= ignature of this file. There is a hash
that ensures it's not = corrupted, but it can't be verified as authoritative
since it= 's just a SHA hash. By grabbing it from BIPM, the source
of t= ruth for time, we at least get their TLS certs to back up the file.
Grabbing it from anywhere else means our users have to trust the
other places. While the IETF/IANA are trustworthy, it's one level
removed.

Then again, given this file, in t= his context, is only used when ntpd
can't otherwise determine= the leap seconds, so maybe that high
level of trust isn't st= rictly needed. The lack of easy verification
of this file has bee= n discussed in the time community on and off
for the last 25 or m= ore years.
=C2=A0
Ie one could poll this once a day or so, and upload it somewhere
else.=C2=A0 Or ping the github URL of the raw IANA TZ file, which is
placed in Eggert's repo quite fast, even without release.
(Just in case the FreeBSD project want to lock out all the
countries that github blocks, i think Iran, North Korea, Cuba
even, who knows which otherwise, i will not look up that political
war thing.)

Polling any more frequently= than every other month is overkill. This file is used,
at most, = twice a year. It changes 6 months before the leap second,
give or= take. Polling it every 60 days is given how often it is used.
Id= eally, our scripts would spread out the polling over those 60 days...

=C2=A0
=C2=A0|> Also the beasts are about to get rid of leap seconds until 2035=
=C2=A0...
=C2=A0|> earlier).=C2=A0 Bets can be placed whether it will happen befor= e
=C2=A0|> a possible occurring leap second, or not.=C2=A0 (My bet is that= they
=C2=A0|> run everything against the wall, and then run away yelling abou= t
=C2=A0|> the evil leap second, after having missed to create an appropri= ate
=C2=A0...
=C2=A0|Yea, we're years away from the next leap second. And there's= still
=C2=A0|a good chance we'll have at least one more. And there's also= rumblings
=C2=A0|that leaps will stop before 2035 (that's the current absolute la= st date,
=C2=A0|and there's several folks that want to pull that in). What will = happen
=C2=A0|for sure isn't well known...
=C2=A0|
=C2=A0|ntpv5 discussions have, at times, assumed there will be no more leap=
=C2=A0|seconds and so ntpv5 needn't have anything to accommodate them.<= br>
I have not looked into current WG outcome (there was just recently
some IETF post regarding NTP, i have not looked).
I would only wish they distribute TAI and the offset to UTC, but
i think, i would hope, they will doing the opposite regulary.
I have no time, and i am not the right person to do anything about
NTP, let alone in the IETF.=C2=A0 All i could ever say i did say, and
that was that it was always wrong to go for "civil time" aka UTC,=
at least without a permanent indication to a constant reliable
time scale.=C2=A0 And a longer possibility to detect leaps, as i knew
secretaries who turn off their computer at Friday afternoon, not
to turn it on again until Monday morning.=C2=A0 And my impression in
the past, before i came a little bit involved in international
email etc communication, was that the engineers simply could not
imagine such a situation.=C2=A0 Or, *at best*, decided that then
silence is the best communication on such a switch.
The need for leap seconds, or its lack, has complicated histor= y.
=C2=A0
=C2=A0|>|The IERS one is more up-to-date because they publish the bullet= in.
=C2=A0|>
=C2=A0|> In general i think distribution of load is a good thing, and =C2=A0|> i find it very unfriendly to put all the load onto some jealous=
=C2=A0|> institute (if it is one) and its single server.
=C2=A0|> The FreeBSD project has an established set of mirrors, and, the=
=C2=A0...
=C2=A0|We can skew the load in time by spreading all our users out over
=C2=A0|the few months we have if there's a load issue.

I have recognized even the nice name of this mechanism in the
FreeBSD rc system, Warner Losh.

Yea, I = worked for a timing company in the 2000s. I couldn't recall
i= f I'd written this or Ian had. I think my name is on it, but Ian fixed<= /div>
all the mistakes I made. :)
=C2=A0
=C2=A0|> Btw PHK had a thrilling idea of DNS distributing leap ticks som= e
=C2=A0|> years ago, and he even started to host it.=C2=A0 As it unfortun= ately
=C2=A0|> did not fly i did not track it further.
=C2=A0|> Would also be an idea for the FreeBSD project: simply download = the
=C2=A0|> file ones, then place a DNS record that FreeBSD installations t= hen
=C2=A0|> can query.=C2=A0 DNSSEC is in place i think.
=C2=A0|
=C2=A0|Yea, that's not a thing that's happening. It was an interest= ing idea,
=C2=A0|but hasn't been standardized and there's little to apetite t= o distribute
=C2=A0|this way.

Unfortunately.=C2=A0 And then there is the fully blown tzdist protocol
that the IETF hammered through with XML and what not formats,
unfortunately not CBOR ("later"), this is what Meinberg says on the state of that:

=C2=A0 https://kb.meinbergglobal.com/kb/time_sync/t= zdist

Yea...
=C2=A0<= /div>
=C2=A0|>|The bundled version was from NIST ftp, but fetching from ftp fo= r every
=C2=A0|>|FreeBSD system out there was too scary for me.
=C2=A0|>|
=C2=A0|>|There may be some security / privacy concerns if we direct user= s to a
=C2=A0|>|place that we do not have control, by the way.
=C2=A0|>
=C2=A0|> Interesting aspect!
=C2=A0|
=C2=A0|There might be, but this sounds somewhat speculative. What's the= anticip\
=C2=A0|ated
=C2=A0|concerns?

Maybe Xin Li has stumbled over the same thread as i after that
publicsuffix CVE of cURL (first sentence of the quoted message):

=C2=A0 https://lists.gnu.org/archiv= e/html/bug-wget/2014-03/msg00113.html

What i mean is, the FreeBSD project and its pkg database, isn't
this a natural place for such a thing?=C2=A0 With guaranteed /
controlled availability.

The ntp leap stuff = does pre-date the pkg by a decade. Having a package
for it might be a natural evolution,

Warner
=C2=A0
--steffen
|
|Der Kragenbaer,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The= moon bear,
|der holt sich munter=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0he cheerfully= and one by one
|einen nach dem anderen runter=C2=A0 wa.ks himself off
|(By Robert Gernhardt)
|
| Only in December: lightful Dubai COP28 Narendra Modi quote:
|=C2=A0 A small part of humanity has ruthlessly exploited nature.
|=C2=A0 But the entire humanity is bearing the cost of it,
|=C2=A0 especially the inhabitants of the Global South.
|=C2=A0 The selfishness of a few will lead the world into darkness,
|=C2=A0 not just for themselves but for the entire world.
--000000000000258860060bf89b7f--