Date: Thu, 7 Dec 2023 22:10:35 -0700 From: Warner Losh <imp@bsdimp.com> To: Warner Losh <imp@bsdimp.com>, Xin Li <delphij@delphij.net>, Philip Paeps <philip@freebsd.org>, src-committers <src-committers@freebsd.org>, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: b1c95af45488 - main - rc.conf: correct $ntp_leapfile_sources Message-ID: <CANCZdfrQU-7yXbzMT8fCmBp=w=CqTVWCgwdXJNXXcHBiOb7vvw@mail.gmail.com> In-Reply-To: <20231208010731.3hijmSTL@steffen%sdaoden.eu> References: <202312070550.3B75o8WV066387@gitrepo.freebsd.org> <CANCZdfrSitY2W%2B7EVaa_yX=KhsJNq_FZqyOLnBeTZX_-6YGxpg@mail.gmail.com> <389AB29C-D5C0-4091-91ED-219F33351B35@freebsd.org> <d75b041f-05f8-44c1-8de6-1fef89b7e537@delphij.net> <20231207222716.obSthG6r@steffen%sdaoden.eu> <CANCZdfpHWRECi=DyhxJAW4MkA-CyPLK=OSdSwBdKQJ57MyPwNA@mail.gmail.com> <20231208010731.3hijmSTL@steffen%sdaoden.eu>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000258860060bf89b7f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Dec 7, 2023 at 6:07=E2=80=AFPM Steffen Nurpmeso <steffen@sdaoden.eu= > wrote: > Warner Losh wrote in > <CANCZdfpHWRECi=3DDyhxJAW4MkA-CyPLK=3DOSdSwBdKQJ57MyPwNA@mail.gmail.com>= : > |On Thu, Dec 7, 2023 at 3:27=E2=80=AFPM Steffen Nurpmeso <steffen@sdaode= n.eu> > wrote: > |> Xin Li wrote in > |> <d75b041f-05f8-44c1-8de6-1fef89b7e537@delphij.net>: > |>|On 2023-12-06 22:34, Philip Paeps wrote: > |>|> On 2023-12-07 14:26:05 (+0800), Warner Losh wrote: > |>|>> We should point to bipm > |>|>> https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list since > |> they > |>|>> are > |>|>> the source of truth, no? > |>|> > |>|> I went for the IANA copy because data.iana.org is a much shorter an= d > |>|> trustworthy looking URL. And it's also where other operating syste= ms > |>|> get their copies. > |>| > |>|My understanding is that IANA's copy is part of tzdata and it's only > |>|updated when a new set of zone data is released, so it's sometimes > |>|outdated. It is actually going to be outdated really soon by the way= . > |> > |> But nothing will change. > |> It is only about the included end-of-life tag why there is > |> discussion at all. > |> The IANA TZ data is always updated as necessary, "early enough". > | > |Yes. TZ data updates multiple times a year. The lead time on NIST/BIPM > |updating the file usually is within days or weeks after the new leap is > |announced. > |But ntpd can't possibly use it for about 5 months. TZ updates are plent= y > |fast. > | > |The bigger problem is that we have to do a EN to get a new set of zone > |files. If we had a way to fetch them, we could just copy this file from > the > |updated > |zone files. > > I never spoke against fetching the plain file (who is my role in > this project in the end?), i only spoke against using the server > of the french institute directly. > The French institute is the source of truth. The BIPM defines what UTC is, based on atomic clock measurements from all over the world. A subagency, the IERS, measures the delta between UTC and the earth's orientation and makes the determination of when a leap second is scheduled. There's no cryptographic signature of this file. There is a hash that ensures it's not corrupted, but it can't be verified as authoritative since it's just a SHA hash. By grabbing it from BIPM, the source of truth for time, we at least get their TLS certs to back up the file. Grabbing it from anywhere else means our users have to trust the other places. While the IETF/IANA are trustworthy, it's one level removed. Then again, given this file, in this context, is only used when ntpd can't otherwise determine the leap seconds, so maybe that high level of trust isn't strictly needed. The lack of easy verification of this file has been discussed in the time community on and off for the last 25 or more years. > Ie one could poll this once a day or so, and upload it somewhere > else. Or ping the github URL of the raw IANA TZ file, which is > placed in Eggert's repo quite fast, even without release. > (Just in case the FreeBSD project want to lock out all the > countries that github blocks, i think Iran, North Korea, Cuba > even, who knows which otherwise, i will not look up that political > war thing.) > Polling any more frequently than every other month is overkill. This file is used, at most, twice a year. It changes 6 months before the leap second, give or take. Polling it every 60 days is given how often it is used. Ideally, our scripts would spread out the polling over those 60 days... > |> Also the beasts are about to get rid of leap seconds until 2035 > ... > |> earlier). Bets can be placed whether it will happen before > |> a possible occurring leap second, or not. (My bet is that they > |> run everything against the wall, and then run away yelling about > |> the evil leap second, after having missed to create an appropriate > ... > |Yea, we're years away from the next leap second. And there's still > |a good chance we'll have at least one more. And there's also rumblings > |that leaps will stop before 2035 (that's the current absolute last date= , > |and there's several folks that want to pull that in). What will happen > |for sure isn't well known... > | > |ntpv5 discussions have, at times, assumed there will be no more leap > |seconds and so ntpv5 needn't have anything to accommodate them. > > I have not looked into current WG outcome (there was just recently > some IETF post regarding NTP, i have not looked). > I would only wish they distribute TAI and the offset to UTC, but > i think, i would hope, they will doing the opposite regulary. > I have no time, and i am not the right person to do anything about > NTP, let alone in the IETF. All i could ever say i did say, and > that was that it was always wrong to go for "civil time" aka UTC, > at least without a permanent indication to a constant reliable > time scale. And a longer possibility to detect leaps, as i knew > secretaries who turn off their computer at Friday afternoon, not > to turn it on again until Monday morning. And my impression in > the past, before i came a little bit involved in international > email etc communication, was that the engineers simply could not > imagine such a situation. Or, *at best*, decided that then > silence is the best communication on such a switch. > The need for leap seconds, or its lack, has complicated history. > |>|The IERS one is more up-to-date because they publish the bulletin. > |> > |> In general i think distribution of load is a good thing, and > |> i find it very unfriendly to put all the load onto some jealous > |> institute (if it is one) and its single server. > |> The FreeBSD project has an established set of mirrors, and, the > ... > |We can skew the load in time by spreading all our users out over > |the few months we have if there's a load issue. > > I have recognized even the nice name of this mechanism in the > FreeBSD rc system, Warner Losh. > Yea, I worked for a timing company in the 2000s. I couldn't recall if I'd written this or Ian had. I think my name is on it, but Ian fixed all the mistakes I made. :) > |> Btw PHK had a thrilling idea of DNS distributing leap ticks some > |> years ago, and he even started to host it. As it unfortunately > |> did not fly i did not track it further. > |> Would also be an idea for the FreeBSD project: simply download the > |> file ones, then place a DNS record that FreeBSD installations then > |> can query. DNSSEC is in place i think. > | > |Yea, that's not a thing that's happening. It was an interesting idea, > |but hasn't been standardized and there's little to apetite to distribut= e > |this way. > > Unfortunately. And then there is the fully blown tzdist protocol > that the IETF hammered through with XML and what not formats, > unfortunately not CBOR ("later"), this is what Meinberg says on > the state of that: > > https://kb.meinbergglobal.com/kb/time_sync/tzdist > Yea... > |>|The bundled version was from NIST ftp, but fetching from ftp for ever= y > |>|FreeBSD system out there was too scary for me. > |>| > |>|There may be some security / privacy concerns if we direct users to a > |>|place that we do not have control, by the way. > |> > |> Interesting aspect! > | > |There might be, but this sounds somewhat speculative. What's the antici= p\ > |ated > |concerns? > > Maybe Xin Li has stumbled over the same thread as i after that > publicsuffix CVE of cURL (first sentence of the quoted message): > > https://lists.gnu.org/archive/html/bug-wget/2014-03/msg00113.html > > What i mean is, the FreeBSD project and its pkg database, isn't > this a natural place for such a thing? With guaranteed / > controlled availability. > The ntp leap stuff does pre-date the pkg by a decade. Having a package for it might be a natural evolution, Warner > --steffen > | > |Der Kragenbaer, The moon bear, > |der holt sich munter he cheerfully and one by one > |einen nach dem anderen runter wa.ks himself off > |(By Robert Gernhardt) > | > | Only in December: lightful Dubai COP28 Narendra Modi quote: > | A small part of humanity has ruthlessly exploited nature. > | But the entire humanity is bearing the cost of it, > | especially the inhabitants of the Global South. > | The selfishness of a few will lead the world into darkness, > | not just for themselves but for the entire world. > --000000000000258860060bf89b7f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Thu, Dec 7, 2023 at 6:07=E2=80=AFP= M Steffen Nurpmeso <<a href=3D"mailto:steffen@sdaoden.eu">steffen@sdaode= n.eu</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"mar= gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1= ex">Warner Losh wrote in<br> =C2=A0<CANCZdfpHWRECi=3DDyhxJAW4MkA-CyPLK=3D<a href=3D"mailto:OSdSwBdKQJ= 57MyPwNA@mail.gmail.com" target=3D"_blank">OSdSwBdKQJ57MyPwNA@mail.gmail.co= m</a>>:<br> =C2=A0|On Thu, Dec 7, 2023 at 3:27=E2=80=AFPM Steffen Nurpmeso <<a href= =3D"mailto:steffen@sdaoden.eu" target=3D"_blank">steffen@sdaoden.eu</a>>= wrote:<br> =C2=A0|> Xin Li wrote in<br> =C2=A0|>=C2=A0 <<a href=3D"mailto:d75b041f-05f8-44c1-8de6-1fef89b7e53= 7@delphij.net" target=3D"_blank">d75b041f-05f8-44c1-8de6-1fef89b7e537@delph= ij.net</a>>:<br> =C2=A0|>|On 2023-12-06 22:34, Philip Paeps wrote:<br> =C2=A0|>|> On 2023-12-07 14:26:05 (+0800), Warner Losh wrote:<br> =C2=A0|>|>> We should point to bipm<br> =C2=A0|>|>> <a href=3D"https://hpiers.obspm.fr/iers/bul/bulc/ntp/l= eap-seconds.list" rel=3D"noreferrer" target=3D"_blank">https://hpiers.obspm= .fr/iers/bul/bulc/ntp/leap-seconds.list</a> since<br> =C2=A0|> they<br> =C2=A0|>|>> are<br> =C2=A0|>|>> the source of truth, no?<br> =C2=A0|>|><br> =C2=A0|>|> I went for the IANA copy because <a href=3D"http://data.ia= na.org" rel=3D"noreferrer" target=3D"_blank">data.iana.org</a> is a much sh= orter and<br> =C2=A0|>|> trustworthy looking URL.=C2=A0 And it's also where oth= er operating systems<br> =C2=A0|>|> get their copies.<br> =C2=A0|>|<br> =C2=A0|>|My understanding is that IANA's copy is part of tzdata and = it's only<br> =C2=A0|>|updated when a new set of zone data is released, so it's so= metimes<br> =C2=A0|>|outdated.=C2=A0 It is actually going to be outdated really soon= by the way.<br> =C2=A0|><br> =C2=A0|> But nothing will change.<br> =C2=A0|> It is only about the included end-of-life tag why there is<br> =C2=A0|> discussion at all.<br> =C2=A0|> The IANA TZ data is always updated as necessary, "early en= ough".<br> =C2=A0|<br> =C2=A0|Yes. TZ data updates multiple times a year. The lead time on NIST/BI= PM<br> =C2=A0|updating the file usually is within days or weeks after the new leap= is<br> =C2=A0|announced.<br> =C2=A0|But ntpd can't possibly use it for about 5 months. TZ updates ar= e plenty<br> =C2=A0|fast.<br> =C2=A0|<br> =C2=A0|The bigger problem is that we have to do a EN to get a new set of zo= ne<br> =C2=A0|files. If we had a way to fetch them, we could just copy this file f= rom the<br> =C2=A0|updated<br> =C2=A0|zone files.<br> <br> I never spoke against fetching the plain file (who is my role in<br> this project in the end?), i only spoke against using the server<br> of the french institute directly.<br></blockquote><div><br></div><div>The F= rench institute is the source of truth. The BIPM defines what</div><div>UTC= is, based on atomic clock measurements from all over the world.</div><div>= A subagency, the IERS, measures the delta between UTC and</div><div>the ear= th's orientation and makes the determination of when a</div><div>leap s= econd is scheduled.</div><div><br></div><div>There's no cryptographic s= ignature of this file. There is a hash</div><div>that ensures it's not = corrupted, but it can't be verified as authoritative</div><div>since it= 's just a SHA hash. By grabbing it from BIPM, the source</div><div>of t= ruth for time, we at least get their TLS certs to back up the file.</div><d= iv>Grabbing it from anywhere else means our users have to trust the</div><d= iv>other places. While the IETF/IANA are trustworthy, it's one level</d= iv><div>removed.</div><div><br></div><div>Then again, given this file, in t= his context, is only used when ntpd</div><div>can't otherwise determine= the leap seconds, so maybe that high</div><div>level of trust isn't st= rictly needed. The lack of easy verification</div><div>of this file has bee= n discussed in the time community on and off</div><div>for the last 25 or m= ore years.<br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" styl= e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin= g-left:1ex"> Ie one could poll this once a day or so, and upload it somewhere<br> else.=C2=A0 Or ping the github URL of the raw IANA TZ file, which is<br> placed in Eggert's repo quite fast, even without release.<br> (Just in case the FreeBSD project want to lock out all the<br> countries that github blocks, i think Iran, North Korea, Cuba<br> even, who knows which otherwise, i will not look up that political<br> war thing.)<br></blockquote><div><br></div><div>Polling any more frequently= than every other month is overkill. This file is used,</div><div>at most, = twice a year. It changes 6 months before the leap second,</div><div>give or= take. Polling it every 60 days is given how often it is used.</div><div>Id= eally, our scripts would spread out the polling over those 60 days...<br></= div><div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style= =3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding= -left:1ex"> =C2=A0|> Also the beasts are about to get rid of leap seconds until 2035= <br> =C2=A0...<br> =C2=A0|> earlier).=C2=A0 Bets can be placed whether it will happen befor= e<br> =C2=A0|> a possible occurring leap second, or not.=C2=A0 (My bet is that= they<br> =C2=A0|> run everything against the wall, and then run away yelling abou= t<br> =C2=A0|> the evil leap second, after having missed to create an appropri= ate<br> =C2=A0...<br> =C2=A0|Yea, we're years away from the next leap second. And there's= still<br> =C2=A0|a good chance we'll have at least one more. And there's also= rumblings<br> =C2=A0|that leaps will stop before 2035 (that's the current absolute la= st date,<br> =C2=A0|and there's several folks that want to pull that in). What will = happen<br> =C2=A0|for sure isn't well known...<br> =C2=A0|<br> =C2=A0|ntpv5 discussions have, at times, assumed there will be no more leap= <br> =C2=A0|seconds and so ntpv5 needn't have anything to accommodate them.<= br> <br> I have not looked into current WG outcome (there was just recently<br> some IETF post regarding NTP, i have not looked).<br> I would only wish they distribute TAI and the offset to UTC, but<br> i think, i would hope, they will doing the opposite regulary.<br> I have no time, and i am not the right person to do anything about<br> NTP, let alone in the IETF.=C2=A0 All i could ever say i did say, and<br> that was that it was always wrong to go for "civil time" aka UTC,= <br> at least without a permanent indication to a constant reliable<br> time scale.=C2=A0 And a longer possibility to detect leaps, as i knew<br> secretaries who turn off their computer at Friday afternoon, not<br> to turn it on again until Monday morning.=C2=A0 And my impression in<br> the past, before i came a little bit involved in international<br> email etc communication, was that the engineers simply could not<br> imagine such a situation.=C2=A0 Or, *at best*, decided that then<br> silence is the best communication on such a switch.<br></blockquote><div><b= r></div><div>The need for leap seconds, or its lack, has complicated histor= y.<br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"mar= gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1= ex"> =C2=A0|>|The IERS one is more up-to-date because they publish the bullet= in.<br> =C2=A0|><br> =C2=A0|> In general i think distribution of load is a good thing, and<br= > =C2=A0|> i find it very unfriendly to put all the load onto some jealous= <br> =C2=A0|> institute (if it is one) and its single server.<br> =C2=A0|> The FreeBSD project has an established set of mirrors, and, the= <br> =C2=A0...<br> =C2=A0|We can skew the load in time by spreading all our users out over<br> =C2=A0|the few months we have if there's a load issue.<br> <br> I have recognized even the nice name of this mechanism in the<br> FreeBSD rc system, Warner Losh.<br></blockquote><div><br></div><div>Yea, I = worked for a timing company in the 2000s. I couldn't recall</div><div>i= f I'd written this or Ian had. I think my name is on it, but Ian fixed<= /div><div>all the mistakes I made. :)<br></div><div>=C2=A0</div><blockquote= class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so= lid rgb(204,204,204);padding-left:1ex"> =C2=A0|> Btw PHK had a thrilling idea of DNS distributing leap ticks som= e<br> =C2=A0|> years ago, and he even started to host it.=C2=A0 As it unfortun= ately<br> =C2=A0|> did not fly i did not track it further.<br> =C2=A0|> Would also be an idea for the FreeBSD project: simply download = the<br> =C2=A0|> file ones, then place a DNS record that FreeBSD installations t= hen<br> =C2=A0|> can query.=C2=A0 DNSSEC is in place i think.<br> =C2=A0|<br> =C2=A0|Yea, that's not a thing that's happening. It was an interest= ing idea,<br> =C2=A0|but hasn't been standardized and there's little to apetite t= o distribute<br> =C2=A0|this way.<br> <br> Unfortunately.=C2=A0 And then there is the fully blown tzdist protocol<br> that the IETF hammered through with XML and what not formats,<br> unfortunately not CBOR ("later"), this is what Meinberg says on<b= r> the state of that:<br> <br> =C2=A0 <a href=3D"https://kb.meinbergglobal.com/kb/time_sync/tzdist" rel=3D= "noreferrer" target=3D"_blank">https://kb.meinbergglobal.com/kb/time_sync/t= zdist</a><br></blockquote><div><br></div><div>Yea... <br></div><div>=C2=A0<= /div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo= rder-left:1px solid rgb(204,204,204);padding-left:1ex"> =C2=A0|>|The bundled version was from NIST ftp, but fetching from ftp fo= r every<br> =C2=A0|>|FreeBSD system out there was too scary for me.<br> =C2=A0|>|<br> =C2=A0|>|There may be some security / privacy concerns if we direct user= s to a<br> =C2=A0|>|place that we do not have control, by the way.<br> =C2=A0|><br> =C2=A0|> Interesting aspect!<br> =C2=A0|<br> =C2=A0|There might be, but this sounds somewhat speculative. What's the= anticip\<br> =C2=A0|ated<br> =C2=A0|concerns?<br> <br> Maybe Xin Li has stumbled over the same thread as i after that<br> publicsuffix CVE of cURL (first sentence of the quoted message):<br> <br> =C2=A0 <a href=3D"https://lists.gnu.org/archive/html/bug-wget/2014-03/msg00= 113.html" rel=3D"noreferrer" target=3D"_blank">https://lists.gnu.org/archiv= e/html/bug-wget/2014-03/msg00113.html</a><br> <br> What i mean is, the FreeBSD project and its pkg database, isn't<br> this a natural place for such a thing?=C2=A0 With guaranteed /<br> controlled availability.<br></blockquote><div><br></div>The ntp leap stuff = does pre-date the pkg by a decade. Having a package</div><div class=3D"gmai= l_quote">for it might be a natural evolution,</div><div class=3D"gmail_quot= e"><br></div><div class=3D"gmail_quote">Warner<br></div><div class=3D"gmail= _quote"><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:= 0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> --steffen<br> |<br> |Der Kragenbaer,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The= moon bear,<br> |der holt sich munter=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0he cheerfully= and one by one<br> |einen nach dem anderen runter=C2=A0 wa.ks himself off<br> |(By Robert Gernhardt)<br> |<br> | Only in December: lightful Dubai COP28 Narendra Modi quote:<br> |=C2=A0 A small part of humanity has ruthlessly exploited nature.<br> |=C2=A0 But the entire humanity is bearing the cost of it,<br> |=C2=A0 especially the inhabitants of the Global South.<br> |=C2=A0 The selfishness of a few will lead the world into darkness,<br> |=C2=A0 not just for themselves but for the entire world.<br> </blockquote></div></div> --000000000000258860060bf89b7f--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrQU-7yXbzMT8fCmBp=w=CqTVWCgwdXJNXXcHBiOb7vvw>