From owner-freebsd-hackers@freebsd.org Tue Jan 5 02:36:49 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D3E464D5912 for ; Tue, 5 Jan 2021 02:36:49 +0000 (UTC) (envelope-from nc@freebsd.org) Received: from rainpuddle.neelc.org (rainpuddle.neelc.org [IPv6:2001:19f0:8001:fed:5400:2ff:fe73:c622]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8xTY36hkz4rPd; Tue, 5 Jan 2021 02:36:49 +0000 (UTC) (envelope-from nc@freebsd.org) Received: from mail.neelc.org (rainpuddle.neelc.org [IPv6:2001:19f0:8001:fed:5400:2ff:fe73:c622]) by rainpuddle.neelc.org (Postfix) with ESMTPSA id 9CD9BEB2A5; Mon, 4 Jan 2021 18:36:44 -0800 (PST) MIME-Version: 1.0 Date: Mon, 04 Jan 2021 18:36:44 -0800 From: Neel Chauhan To: Doug Ambrisko Cc: Mark Johnston , freebsd-hackers@freebsd.org, ambrisko@freebsd.org Subject: Re: Debugging a WIP PCI/ACPI patch: Bad tailq NEXT(0xffffffff81cde660->tqh_last) != NULL In-Reply-To: <20210102190644.GB87535@ambrisko.com> References: <44528336fa9168966d121bf771e1e229@neelc.org> <3c9ff844e527daacd04c51f48836b57d@neelc.org> <20201231200744.GA95383@ambrisko.com> <4f3f6a02a452f766063ae2acb060dc64@neelc.org> <7cda3be6594d5ad5bdc69019f72b03d3@neelc.org> <20210102190644.GB87535@ambrisko.com> User-Agent: Roundcube Webmail/1.4.9 Message-ID: <1aa63edbf7abb4f6b3f0d8c18bd948e0@freebsd.org> X-Sender: nc@freebsd.org Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=_4445c4a291a95b8c663417445d53c4d7"; micalg=pgp-sha256 X-Rspamd-Queue-Id: 4D8xTY36hkz4rPd X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:20473, ipnet:2001:19f0:8000::/38, country:US] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 02:36:49 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_4445c4a291a95b8c663417445d53c4d7 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Unrelated to VMD, but I have recently gotten a Ports commit bit and now have a @FreeBSD.org email, in case you wanted to know who to give credit to. It's still me, and it forwards to ${FIRSTNAME}@neelc.org. My handle is nc@. -Neel On 2021-01-02 11:06, Doug Ambrisko wrote: > With VMD, the PCI "root" is hidden behind it. To access devices behind > the VMD device, a new domain is created and when PCI config. space > is accessed, it is indexed via the VMD device via the offset. Intel > seems to have reduced the available bus space on some HW. So for a bus > access less then what they implemented in HW we have to return error > that nothing is there. Then when we get to the starting bus device, > we need to offset that to 0 based in the HW. The PCI probe will run > look for busses from 0 to 255. From the Linux driver your HW only > works > from 224 to 255. So we need to fail anything under 224 and for bus > requests 224 and higher then subtract 224. Thus the b - > sc->vmd_bus_start > part. I'm not sure if we could do it the other way in which we allow > 0-12 bus requests to pass and fail if it is over. I'm not sure if > there is any specific reason why that wouldn't work. Linux didn't > do that but that doesn't mean it wouldn't work. It would be good to > start with the Linux method and then test 0-n, where n is the max. > busses that HW allows. Anything n or more would have to return a fail. > > Doug A. > > On Sat, Jan 02, 2021 at 09:20:20AM -0800, Neel Chauhan wrote: > | Just to ping you in case you may have missed my reply (I understand, > New > | Years Day). > | > | Is there a reason why "b = pci_get_bus(dev);" return 0 even when the > bus > | number is shifted (as it is on Linux)? > | > | -Neel > | > | On 2020-12-31 21:49, Neel Chauhan wrote: > | > Hi Doug, > | > > | > Thank you so much for this information. > | > > | > On 2020-12-31 12:07, Doug Ambrisko wrote: > | >> FYI, looks like this needs to be ported over from Linux: > | >> static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct > pci_bus > | >> *bus, > | >> unsigned int devfn, int reg, int > | >> len) > | >> { > | >> char __iomem *addr = vmd->cfgbar + > | >> ((bus->number - vmd->busn_start) << > 20) + > | >> (devfn << 12) + reg; > | >> > | >> to > | >> vmd_read_config > | >> offset = (b << 20) + (s << 15) + (f << 12) + reg; > | >> > | >> vmd_write_config(device_t dev, u_int b, u_int s, u_int f, u_int > reg, > | >> offset = (b << 20) + (s << 15) + (f << 12) + reg; > | >> > | >> ie. > | >> offset = ((b - sc->vmd_bus_start) << 20) + (s << 15) + (f << 12) > + > | >> reg; > | >> > | >> vmd_bus_start should be added to the softc as a uint8_t type and > needs > | >> to > | >> be set via attach. We need range checks to make sure > | >> vmd_write_config/vmd_read_config doesn't read something out of > range > | >> since it has been reduced. > | > > | > One thing I noticed is that the "b" variable (which corresponds to > the > | > Linux bus->number) is 0 (thanks to printf). This should be the bus > | > number if we want to attach. > | > > | > If I use: "b = pci_get_bus(dev);" in the attach, b is still 0. > | > > | > And that leads to a kernel panic. > | > > | >> Not sure what the shadow registers do. These both seem to be new > | >> Intel > | >> features and Intel doc's have been minimal. Looks like Intel is > doing > | >> a sparse map now on newer devices. > | > > | > I guess Linux is our best hope. Unless the new Intel docs is the > Linux > | > kernel source. > | > > | >> I'm concerned about the Linux comment of: > | >> * Certain VMD devices may have a root port configuration > | >> option which > | >> * limits the bus range to between 0-127, 128-255, or > 224-255 > | >> > | >> since I don't see anything to limit it between 0-127 only starting > | >> at 0, 128 or 224, Maybe there is max of 128 busses overall? > | > > | > I could be wrong, but I guess that's a typo. > | > > | >> I don't have this type of HW to test things. > | > > | > I can use my hardware for testing. In the worse case scenario, I > can > | > donate an entry-level 11th Gen/TigerLake system if I have the funds > | > and/or can get a tax credit. > | > > | >> Doug A. > | > > | > -Neel > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" --=_4445c4a291a95b8c663417445d53c4d7 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=488 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEFpeUj+sDItoNIly9vzSRBRPfYX0FAl/z0LwACgkQvzSRBRPf YX0PPwf8D8yQGlQG8SwVvT+H+joEPHN2T2EGsxn71R18eRzEZ36jmrMxKRGRvVLa G0FMwvBvXYxp1l4nGXSIpod4IWW9aibsXZby7acbOgfNoH0JLLsK5t3w369lzb/2 xc6oYy01+C6dqsWHMSGtaRA4HV4OdrDzwq4oyuML95e1qP9ifwtGGsu9ClTzAiyA u75PB5l1wvja0gy/HVXF6A8ln+Ctnb/a0NIBJiCXLb8uic9ilB8cbOND5JN2sHUZ mlozemDauWUQoU1BDCLdRmXOlTPQjXxYC5DWMFuQcoHmAa3fwWpLw8KLp8dtOS+H xUv5thKRbfOqx3hj0NZUCsipN09spQ== =90/K -----END PGP SIGNATURE----- --=_4445c4a291a95b8c663417445d53c4d7-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 03:37:49 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2DD684D7294 for ; Tue, 5 Jan 2021 03:37:49 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net (static-24-113-41-81.wavecable.com [24.113.41.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "ultimatedns.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8yqw5HpTz4trH for ; Tue, 5 Jan 2021 03:37:48 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.16.1/8.16.1) with ESMTP id 1053bixK025556 for ; Mon, 4 Jan 2021 19:37:50 -0800 (PST) (envelope-from bsd-lists@bsdforge.com) MIME-Version: 1.0 Date: Mon, 04 Jan 2021 19:37:44 -0800 From: Chris To: freebsd-hackers@freebsd.org Subject: Re: Debugging a WIP PCI/ACPI patch: Bad tailq NEXT(0xffffffff81cde660->tqh_last) != NULL In-Reply-To: <1aa63edbf7abb4f6b3f0d8c18bd948e0@freebsd.org> References: <44528336fa9168966d121bf771e1e229@neelc.org> <3c9ff844e527daacd04c51f48836b57d@neelc.org> <20201231200744.GA95383@ambrisko.com> <4f3f6a02a452f766063ae2acb060dc64@neelc.org> <7cda3be6594d5ad5bdc69019f72b03d3@neelc.org> <20210102190644.GB87535@ambrisko.com> <1aa63edbf7abb4f6b3f0d8c18bd948e0@freebsd.org> User-Agent: UDNSMS/17.0 Message-ID: <2c658d59208356049021003886c2df4c@bsdforge.com> X-Sender: bsd-lists@bsdforge.com Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4D8yqw5HpTz4trH X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:11404, ipnet:24.113.0.0/16, country:US]; local_wl_ip(0.00)[24.113.41.81] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 03:37:49 -0000 On 2021-01-04 18:36, Neel Chauhan wrote: > Unrelated to VMD, but I have recently gotten a Ports commit bit and now have > a > @FreeBSD.org email, That commit bit should be revoked, for top posting. ;-) > in case you wanted to know who to give credit to. It's still > me, and it forwards to ${FIRSTNAME}@neelc.org. > > My handle is nc@. > > -Neel > > On 2021-01-02 11:06, Doug Ambrisko wrote: >> With VMD, the PCI "root" is hidden behind it. To access devices behind >> the VMD device, a new domain is created and when PCI config. space >> is accessed, it is indexed via the VMD device via the offset. Intel >> seems to have reduced the available bus space on some HW. So for a bus >> access less then what they implemented in HW we have to return error >> that nothing is there. Then when we get to the starting bus device, >> we need to offset that to 0 based in the HW. The PCI probe will run >> look for busses from 0 to 255. From the Linux driver your HW only works >> from 224 to 255. So we need to fail anything under 224 and for bus >> requests 224 and higher then subtract 224. Thus the b - sc->vmd_bus_start >> part. I'm not sure if we could do it the other way in which we allow >> 0-12 bus requests to pass and fail if it is over. I'm not sure if >> there is any specific reason why that wouldn't work. Linux didn't >> do that but that doesn't mean it wouldn't work. It would be good to >> start with the Linux method and then test 0-n, where n is the max. >> busses that HW allows. Anything n or more would have to return a fail. >> >> Doug A. >> >> On Sat, Jan 02, 2021 at 09:20:20AM -0800, Neel Chauhan wrote: >> | Just to ping you in case you may have missed my reply (I understand, New >> | Years Day). >> | >> | Is there a reason why "b = pci_get_bus(dev);" return 0 even when the bus >> | number is shifted (as it is on Linux)? >> | >> | -Neel >> | >> | On 2020-12-31 21:49, Neel Chauhan wrote: >> | > Hi Doug, >> | > >> | > Thank you so much for this information. >> | > >> | > On 2020-12-31 12:07, Doug Ambrisko wrote: >> | >> FYI, looks like this needs to be ported over from Linux: >> | >> static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus >> | >> *bus, >> | >> unsigned int devfn, int reg, int >> | >> len) >> | >> { >> | >> char __iomem *addr = vmd->cfgbar + >> | >> ((bus->number - vmd->busn_start) << 20) + >> | >> (devfn << 12) + reg; >> | >> >> | >> to >> | >> vmd_read_config >> | >> offset = (b << 20) + (s << 15) + (f << 12) + reg; >> | >> >> | >> vmd_write_config(device_t dev, u_int b, u_int s, u_int f, u_int reg, >> | >> offset = (b << 20) + (s << 15) + (f << 12) + reg; >> | >> >> | >> ie. >> | >> offset = ((b - sc->vmd_bus_start) << 20) + (s << 15) + (f << 12) + >> | >> reg; >> | >> >> | >> vmd_bus_start should be added to the softc as a uint8_t type and needs >> | >> to >> | >> be set via attach. We need range checks to make sure >> | >> vmd_write_config/vmd_read_config doesn't read something out of range >> | >> since it has been reduced. >> | > >> | > One thing I noticed is that the "b" variable (which corresponds to the >> | > Linux bus->number) is 0 (thanks to printf). This should be the bus >> | > number if we want to attach. >> | > >> | > If I use: "b = pci_get_bus(dev);" in the attach, b is still 0. >> | > >> | > And that leads to a kernel panic. >> | > >> | >> Not sure what the shadow registers do. These both seem to be new >> | >> Intel >> | >> features and Intel doc's have been minimal. Looks like Intel is doing >> | >> a sparse map now on newer devices. >> | > >> | > I guess Linux is our best hope. Unless the new Intel docs is the Linux >> | > kernel source. >> | > >> | >> I'm concerned about the Linux comment of: >> | >> * Certain VMD devices may have a root port configuration >> | >> option which >> | >> * limits the bus range to between 0-127, 128-255, or 224-255 >> | >> >> | >> since I don't see anything to limit it between 0-127 only starting >> | >> at 0, 128 or 224, Maybe there is max of 128 busses overall? >> | > >> | > I could be wrong, but I guess that's a typo. >> | > >> | >> I don't have this type of HW to test things. >> | > >> | > I can use my hardware for testing. In the worse case scenario, I can >> | > donate an entry-level 11th Gen/TigerLake system if I have the funds >> | > and/or can get a tax credit. >> | > >> | >> Doug A. >> | > >> | > -Neel >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Jan 5 05:25:12 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 086B54D8F6D for ; Tue, 5 Jan 2021 05:25:12 +0000 (UTC) (envelope-from nc@freebsd.org) Received: from rainpuddle.neelc.org (rainpuddle.neelc.org [66.42.69.219]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D91Cq4xCmz3GM7 for ; Tue, 5 Jan 2021 05:25:11 +0000 (UTC) (envelope-from nc@freebsd.org) Received: from mail.neelc.org (rainpuddle.neelc.org [IPv6:2001:19f0:8001:fed:5400:2ff:fe73:c622]) by rainpuddle.neelc.org (Postfix) with ESMTPSA id A4453EB2A5; Mon, 4 Jan 2021 21:25:08 -0800 (PST) MIME-Version: 1.0 Date: Mon, 04 Jan 2021 21:25:08 -0800 From: Neel Chauhan To: Chris Cc: freebsd-hackers@freebsd.org Subject: Re: Debugging a WIP PCI/ACPI patch: Bad tailq NEXT(0xffffffff81cde660->tqh_last) != NULL In-Reply-To: <2c658d59208356049021003886c2df4c@bsdforge.com> References: <44528336fa9168966d121bf771e1e229@neelc.org> <3c9ff844e527daacd04c51f48836b57d@neelc.org> <20201231200744.GA95383@ambrisko.com> <4f3f6a02a452f766063ae2acb060dc64@neelc.org> <7cda3be6594d5ad5bdc69019f72b03d3@neelc.org> <20210102190644.GB87535@ambrisko.com> <1aa63edbf7abb4f6b3f0d8c18bd948e0@freebsd.org> <2c658d59208356049021003886c2df4c@bsdforge.com> User-Agent: Roundcube Webmail/1.4.9 Message-ID: <79acb8e9ca295f138d1f5c53f4c30d63@freebsd.org> X-Sender: nc@freebsd.org Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=_1ea4f53105b599d4e57b1d708b8599fc"; micalg=pgp-sha256 X-Rspamd-Queue-Id: 4D91Cq4xCmz3GM7 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:20473, ipnet:66.42.64.0/20, country:US] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 05:25:12 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_1ea4f53105b599d4e57b1d708b8599fc Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed On 2021-01-04 19:37, Chris wrote: > On 2021-01-04 18:36, Neel Chauhan wrote: >> Unrelated to VMD, but I have recently gotten a Ports commit bit and >> now have a >> @FreeBSD.org email, > That commit bit should be revoked, for top posting. ;-) LOL. In all seriousness, I didn't mean to top post, sorry. I just wanted to give an up-to-date reference for the "Reported By" section if/when this gets finished. -Neel --=_1ea4f53105b599d4e57b1d708b8599fc Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=488 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEFpeUj+sDItoNIly9vzSRBRPfYX0FAl/z+DQACgkQvzSRBRPf YX3CfggApJwBaMxlO+5I8/Ofnj/dhn7zw5Exdjddl91AAK/VNfdgdjO2GQk7M3Nj 0wzVXhUjWKHL9O4SbQT5QnkwTqVRcBBO2i5/4SxihqA9qcKfHMiaLVM7rJY3UlaS Ebza5Say+PeuTRyHcAlEcb9QPHGQrKKvWECgjQPMpuA6gvlGVhWCu4FZZ6MgsIXI fFqa55WDRNJjtSKm4rY4+npwGVJ9q0quBCb7Ox38iWrP3Pjrslah3Sr/3wOsgYWH muL4ZgqAiXQNq+Sg+fURIW8FIic6S5D9WwPsyXW8WpgUXwoWNLOw4fbTRLSn0RxN hfm1V+ObW4MC426H443u+nUBIoXwCQ== =pULX -----END PGP SIGNATURE----- --=_1ea4f53105b599d4e57b1d708b8599fc-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 07:51:18 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4055C4DBC27 for ; Tue, 5 Jan 2021 07:51:18 +0000 (UTC) (envelope-from hmurray@megapathdsl.net) Received: from ip-64-139-1-69.sjc.megapath.net (ip-64-139-1-69.sjc.megapath.net [64.139.1.69]) by mx1.freebsd.org (Postfix) with ESMTP id 4D94SP20MMz3N3k for ; Tue, 5 Jan 2021 07:51:16 +0000 (UTC) (envelope-from hmurray@megapathdsl.net) Received: from shuksan (localhost [127.0.0.1]) by ip-64-139-1-69.sjc.megapath.net (Postfix) with ESMTP id A412C40605C; Mon, 4 Jan 2021 23:51:09 -0800 (PST) X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 To: freebsd-hackers@freebsd.org cc: Hal Murray From: Hal Murray Subject: How does the time zone info get updated? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 04 Jan 2021 23:51:09 -0800 Message-Id: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> X-Rspamd-Queue-Id: 4D94SP20MMz3N3k X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of hmurray@megapathdsl.net has no SPF policy when checking 64.139.1.69) smtp.mailfrom=hmurray@megapathdsl.net X-Spamd-Result: default: False [1.48 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[64.139.1.69:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[megapathdsl.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.98)[0.977]; SPAMHAUS_ZRD(0.00)[64.139.1.69:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:4565, ipnet:64.139.0.0/18, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 07:51:18 -0000 It gets distributed with the base packages, but I haven't figured out how to keep it up to date. The info is distributed via IANA. https://www.iana.org/time-zones It gets updated several times a year. There were 6 during 2020. When is unpredictable, often shortly after some government passes a law to change something with short notice. I'd expect there to be some mechanism to track the updates but I haven't found it. The tz database includes the leap-seconds.list file. That gets updated every 6 months. I'd like to get whatever FreeBSD does for time zones to include that file so ntpd can use it without having to be concerned with keeping it up to date. That only works if the zone info is kept up to date. -- These are my opinions. I hate spam. From owner-freebsd-hackers@freebsd.org Tue Jan 5 11:24:28 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9FA8B4E1E2D for ; Tue, 5 Jan 2021 11:24:28 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D99BN41Lgz3qq4; Tue, 5 Jan 2021 11:24:28 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f110800e50b83eaa3464b63.dip0.t-ipconnect.de [IPv6:2003:cd:5f11:800:e50b:83ea:a346:4b63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 158E3359CA; Tue, 5 Jan 2021 11:24:27 +0000 (UTC) (envelope-from se@freebsd.org) To: freebsd-hackers@freebsd.org References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> From: Stefan Esser Cc: Hal Murray Subject: Re: How does the time zone info get updated? Message-ID: <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> Date: Tue, 5 Jan 2021 12:24:24 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="g8vxm87iYqCYVoC0Smm1D7NhV1jVBGPTu" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 11:24:28 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --g8vxm87iYqCYVoC0Smm1D7NhV1jVBGPTu Content-Type: multipart/mixed; boundary="ykIOctd0HLewIaPLaUFocxHgbszkNaivB"; protected-headers="v1" From: Stefan Esser To: freebsd-hackers@freebsd.org Cc: Hal Murray Message-ID: <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> Subject: Re: How does the time zone info get updated? References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> In-Reply-To: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> --ykIOctd0HLewIaPLaUFocxHgbszkNaivB Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable Am 05.01.21 um 08:51 schrieb Hal Murray:> It gets distributed with the=20 base packages, but I haven't figured out how to > keep it up to date. >=20 > The info is distributed via IANA. > https://www.iana.org/time-zones >=20 > It gets updated several times a year. There were 6 during 2020. When = is > unpredictable, often shortly after some government passes a law to chan= ge > something with short notice. >=20 > I'd expect there to be some mechanism to track the updates but I haven'= t found > it. >=20 > The tz database includes the leap-seconds.list file. That gets updated= every > 6 months. I'd like to get whatever FreeBSD does for time zones to incl= ude > that file so ntpd can use it without having to be concerned with keepin= g it up > to date. That only works if the zone info is kept up to date. We have recently moved the calendar files out of -CURRENT and into a=20 port, and I could imagine providing the timezone files in the same way (without removal from base, but from there only used as a fall-back if the port supplied files are not available). That way -STABLE users could update the definitions with base, but if you strictly follow a release, you could install the port and decouple timezone file updates from base updates ... I'd be willing to support this change and maintain the data files, but I can imagine that the current maintainer of those files in base might want to do it himself. --ykIOctd0HLewIaPLaUFocxHgbszkNaivB-- --g8vxm87iYqCYVoC0Smm1D7NhV1jVBGPTu Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl/0TGgFAwAAAAAACgkQR+u171r99USs VAf8Cj3U65CHn+fW4S0sevyutblrOxcugZkDr/sVuJhGWAFldpSqVwC5fLKNwcf/DjLX1GeezszA zYspd7ze1g/nGjlpJqgItgkM52bIvH/M6Ozg3q3xcdrvkJrmR9DfGGxEVuhC5karK2kC/p8MifhG mSWRN18E01+FYmuY/NyEhKS5uMh6/76K1gebbTeKa4QT7K7kQ2JuPFNns/o4k3XecdezaTmXtcBu APFXC6mvjM3CYz0zkyOPnqrEiND8oXUCk7dT3S/uzrTz4OxA09BXc9dzE8hCp0Qc5u008P0O5o+V VK7F4XsTZ7Bx5/gJ54Yfe7Md7C29axn/EAkmVhW8+g== =Pep8 -----END PGP SIGNATURE----- --g8vxm87iYqCYVoC0Smm1D7NhV1jVBGPTu-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 15:41:30 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CFE184C019A for ; Tue, 5 Jan 2021 15:41:30 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9Gtx5XMrz4crq for ; Tue, 5 Jan 2021 15:41:29 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 105FfQI0008108; Tue, 5 Jan 2021 07:41:26 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 105FfQq6008107; Tue, 5 Jan 2021 07:41:26 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202101051541.105FfQq6008107@gndrsh.dnsmgr.net> Subject: Re: How does the time zone info get updated? In-Reply-To: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> To: Hal Murray Date: Tue, 5 Jan 2021 07:41:25 -0800 (PST) CC: freebsd-hackers@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4D9Gtx5XMrz4crq X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [-1.10 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[69.59.192.140:from]; AUTH_NA(1.00)[]; DMARC_NA(0.00)[dnsmgr.net]; SPAMHAUS_ZRD(0.00)[69.59.192.140:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 15:41:30 -0000 > > It gets distributed with the base packages, but I haven't figured out how to > keep it up to date. > > The info is distributed via IANA. > https://www.iana.org/time-zones > > It gets updated several times a year. There were 6 during 2020. When is > unpredictable, often shortly after some government passes a law to change > something with short notice. > > I'd expect there to be some mechanism to track the updates but I haven't found > it. > > The tz database includes the leap-seconds.list file. That gets updated every > 6 months. I'd like to get whatever FreeBSD does for time zones to include > that file so ntpd can use it without having to be concerned with keeping it up > to date. That only works if the zone info is kept up to date. This only addresses the leap-seconds file, not the whole zoneinfo data: /etc/periodic/daily/480.leapfile-ntp calls "service ntpd onefetch", which is /etc/rc.d/ntpd, the onefetch command attempts to update the leap-second file. This works great if you have https support avaliable via the ca_root_nss port, if you do not then you can fix this to fetch via http by overriding ntp_leapfile_sources in /etc/rc.conf like so: ntp_leapfile_sources="http://www.ietf.org/timezones/data/leap-seconds.list" ^ remove s here This is the only file you need to keep up to date to keep ntpd happy. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Tue Jan 5 15:49:55 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 525434E7EC9 for ; Tue, 5 Jan 2021 15:49:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9H4g1RDhz4dLQ for ; Tue, 5 Jan 2021 15:49:54 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x82c.google.com with SMTP id z3so21071507qtw.9 for ; Tue, 05 Jan 2021 07:49:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VNfza6JJ+vTDuZPJ8P/B0lin3M3VUmr/Jtlu2pE/gKo=; b=a6jymouinz67pS5EOoSLWOWL3W431hGRKwc36QfQYKXjVvMJOoErY+gxo+0PbEmDNf Yq5UKJi5Mi4qS7YosZdv7nVPZl/6ps+Iz0gF91DYgqeDeX/bxF/SZJiZEyox6rXqfGoX 7JyYpZNfLfePpGYxyyIGVs5j8L2m0RO4RsNdL1cB1LJnZOXAW8/g487+tMK/0DlxymtZ VZA1/IX0GzmD3h7eaXapnkt8IoL8Ms/eSROlyLfWDNgNNdBNEK/dd57GgegqqTnn6AAu mfFkt+14K3x1Ystacxjw/t6FmahZfaKaa2VYwUyNYYiPqX4FyFCl/YvATykGfIKOBMvB AyJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VNfza6JJ+vTDuZPJ8P/B0lin3M3VUmr/Jtlu2pE/gKo=; b=k+AVBEbKGKbG/hVQ5Hihtom+ofKnC6BQh/nt7UT3jXFwJj3k1p6jiksaaqUlQZY2gD AAgn/tTyRCUCGi4URP20HGJKsono3pM2cDxoo7RApsxF+37Q/H1m4fYyosXG5OJUCrUN wzdqXjK5we8z168qJX6pEZHGtNO7Q9mTA5t21wJwQ157MJbiUEK+exWPo5jY52C/rlI2 u/DT3iYWpdJ1+N/uImqj+DDgokHUnpc2P36FNapI9pn8j1x5hwGABc0wPzViDPyZJdb3 oXbmrZYmF3rQ682OAc15M5H2QRV73CZoDbMPI4KAQBwLXUHXWN/esfnWsjqP8HwgwTDJ xJ8g== X-Gm-Message-State: AOAM532TAM5US8mNQKNwEmRYleyPwmrcA7GOIxIQ8E68ZiHvm5egUr5T 13Fs/6LmIPKsjw/WSZTN9z0FYxcgLopzMIcVu9EOeQ== X-Google-Smtp-Source: ABdhPJzUsKAbhOa5tmAqD+LJTqsbm2y3cH88eJHD83W1YbJmwCwCnxk0xW/t8dh/Vj03kqPddB5JSbS15/4Us4EWX9c= X-Received: by 2002:ac8:51d8:: with SMTP id d24mr102830qtn.73.1609861793774; Tue, 05 Jan 2021 07:49:53 -0800 (PST) MIME-Version: 1.0 References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> In-Reply-To: <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> From: Warner Losh Date: Tue, 5 Jan 2021 08:49:42 -0700 Message-ID: Subject: Re: How does the time zone info get updated? To: Stefan Esser Cc: FreeBSD Hackers , Hal Murray X-Rspamd-Queue-Id: 4D9H4g1RDhz4dLQ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 15:49:55 -0000 On Tue, Jan 5, 2021, 4:24 AM Stefan Esser wrote: > Am 05.01.21 um 08:51 schrieb Hal Murray:> It gets distributed with the > base packages, but I haven't figured out how to > > keep it up to date. > > > > The info is distributed via IANA. > > https://www.iana.org/time-zones > > > > It gets updated several times a year. There were 6 during 2020. When is > > unpredictable, often shortly after some government passes a law to change > > something with short notice. > > > > I'd expect there to be some mechanism to track the updates but I haven't > found > > it. > > > > The tz database includes the leap-seconds.list file. That gets updated > every > > 6 months. I'd like to get whatever FreeBSD does for time zones to > include > > that file so ntpd can use it without having to be concerned with keeping > it up > > to date. That only works if the zone info is kept up to date. > > We have recently moved the calendar files out of -CURRENT and into a > port, and I could imagine providing the timezone files in the same > way (without removal from base, but from there only used as a fall-back > if the port supplied files are not available). > > That way -STABLE users could update the definitions with base, but if > you strictly follow a release, you could install the port and decouple > timezone file updates from base updates ... > > I'd be willing to support this change and maintain the data files, but > I can imagine that the current maintainer of those files in base might > want to do it himself. > The calendar files aren't what he's talking about. These are the timezone files he's asking about. Those get updated very shortly after upstream releases. But, three things. First, we update leapseconds.list as soon as it is announce. Second, there is no way to get just these files updated: they come for free with a rebuild or and update using the beta pkgbase, but there is no automatic way to just get these. Third, there is an daily cron job that fetches leap-seconds.list if you have ntpd enabled and you are on the internet. So no TZ update outside an os upgrade, but the leap seconds file updates at the speed of NIST... Warner > From owner-freebsd-hackers@freebsd.org Tue Jan 5 18:59:55 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62FF14C6214 for ; Tue, 5 Jan 2021 18:59:55 +0000 (UTC) (envelope-from reichert@numachi.com) Received: from away.numachi.com (away.numachi.com [66.228.38.138]) by mx1.freebsd.org (Postfix) with SMTP id 4D9MHt47Rpz4syr for ; Tue, 5 Jan 2021 18:59:54 +0000 (UTC) (envelope-from reichert@numachi.com) Received: (qmail 11941 invoked from network); 5 Jan 2021 18:59:48 -0000 Received: from unknown (HELO meisai.numachi.com) (72.71.251.201) by away.numachi.com with SMTP; 5 Jan 2021 18:59:48 -0000 Received: (qmail 16656 invoked by uid 1001); 5 Jan 2021 18:59:48 -0000 Date: Tue, 5 Jan 2021 13:59:48 -0500 From: Brian Reichert To: Hal Murray Cc: freebsd-hackers@freebsd.org Subject: Re: How does the time zone info get updated? Message-ID: <20210105185948.GJ47402@numachi.com> References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> User-Agent: Mutt/1.5.9i X-Rspamd-Queue-Id: 4D9MHt47Rpz4syr X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of reichert@numachi.com has no SPF policy when checking 66.228.38.138) smtp.mailfrom=reichert@numachi.com X-Spamd-Result: default: False [-0.50 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[numachi.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.228.38.138:from]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[66.228.38.138:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:63949, ipnet:66.228.32.0/20, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 18:59:55 -0000 On Mon, Jan 04, 2021 at 11:51:09PM -0800, Hal Murray wrote: > > It gets distributed with the base packages, but I haven't figured out how to > keep it up to date. Don't forget: java comes with it's own set of timezone files, in case you have any webapps to wrassle with. -- Brian Reichert BSD admin/developer at large From owner-freebsd-hackers@freebsd.org Tue Jan 5 19:51:52 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3AC694C7FAF for ; Tue, 5 Jan 2021 19:51:52 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9NRr13X6z3DZr; Tue, 5 Jan 2021 19:51:52 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f110800e50b83eaa3464b63.dip0.t-ipconnect.de [IPv6:2003:cd:5f11:800:e50b:83ea:a346:4b63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 8CCA52235; Tue, 5 Jan 2021 19:51:51 +0000 (UTC) (envelope-from se@freebsd.org) To: Warner Losh Cc: FreeBSD Hackers , Hal Murray References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> From: Stefan Esser Subject: Re: How does the time zone info get updated? Message-ID: <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> Date: Tue, 5 Jan 2021 20:51:49 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AL4cD8n6iueTMTSMNWy8MB1DMuokSp0xN" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 19:51:52 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --AL4cD8n6iueTMTSMNWy8MB1DMuokSp0xN Content-Type: multipart/mixed; boundary="8q6jLyrg1n30rU5AxHYsr1X8pPaupEqWG"; protected-headers="v1" From: Stefan Esser To: Warner Losh Cc: FreeBSD Hackers , Hal Murray Message-ID: <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> Subject: Re: How does the time zone info get updated? References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> In-Reply-To: --8q6jLyrg1n30rU5AxHYsr1X8pPaupEqWG Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: de-DE Content-Transfer-Encoding: quoted-printable Am 05.01.21 um 16:49 schrieb Warner Losh: >=20 >=20 > On Tue, Jan 5, 2021, 4:24 AM Stefan Esser > wrote: >=20 > Am 05.01.21 um 08:51 schrieb Hal Murray:> It gets distributed with = the > base packages, but I haven't figured out how to > > keep it up to date. > > > > The info is distributed via IANA. > > https://www.iana.org/time-zones > > > > It gets updated several times a year.=C2=A0 There were 6 during = 2020.=20 > When is > > unpredictable, often shortly after some government passes a law > to change > > something with short notice. > > > > I'd expect there to be some mechanism to track the updates but I= > haven't found > >=C2=A0 =C2=A0it. > > > > The tz database includes the leap-seconds.list file.=C2=A0 That = gets > updated every > > 6 months.=C2=A0 I'd like to get whatever FreeBSD does for time z= ones > to include > > that file so ntpd can use it without having to be concerned with= > keeping it up > > to date.=C2=A0 That only works if the zone info is kept up to da= te. >=20 > We have recently moved the calendar files out of -CURRENT and into = a > port, and I could imagine providing the timezone files in the same > way (without removal from base, but from there only used as a fall-= back > if the port supplied files are not available). >=20 > That way -STABLE users could update the definitions with base, but = if > you strictly follow a release, you could install the port and decou= ple > timezone file updates from base updates ... >=20 > I'd be willing to support this change and maintain the data files, = but > I can imagine that the current maintainer of those files in base mi= ght > want to do it himself. >=20 >=20 > The calendar files aren't what he's talking about. Yes, I'm very well aware of that fact. > These are the timezone files he's asking about. Those get updated very = > shortly after upstream releases. Yes, and they are not considered critical security fixes and thus are not available as patches for those that install binary releases. By providing the timezone files by a port/package in the same way as the calendar data files (and preferring them if present), they could be kept up-to-date with other packages on system that run only release versions of FreeBSD. (And as written before, I do not suggest to remove the timezone files from base at all.) > But, three things. First, we update leapseconds.list as soon as it is=20 > announce. Second, there is no way to get just these files updated: they= =20 > come for free with a rebuild or and update using the beta pkgbase, but = > there is no automatic way to just get these. Third, there is an daily=20 > cron job that fetches leap-seconds.list if you have ntpd enabled and yo= u=20 > are on the internet. Yes, if only the leap-seconds are required, this helps get the time synchronized - but is quite different from timezone file updates. > So no TZ update outside an os upgrade, but the leap seconds file update= s=20 > at the speed of NIST... The OP did not ask for leap-second updates but for timezone files, if I understand his mail correctly. Regards, STefan --8q6jLyrg1n30rU5AxHYsr1X8pPaupEqWG-- --AL4cD8n6iueTMTSMNWy8MB1DMuokSp0xN Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl/0w1YFAwAAAAAACgkQR+u171r99USu 6wf+IlnrRsCwedIOoejRsLoiOdY0X3ThhfhDd1f5o1e+TJ6SpVfj8ClMVYKCWAZ2Ls8QhlIE65eN uZYj+vU/n1yBcdvt3F9SXHHKe6CQCHUrzx1wrjbeetK3TfqydkmkQhLQZe/dRplt+BEz9Cd6cqDn WJH4CH6NdCR1hCPbs25m/yb3LGs/7thoDEogAukEhEuq4X2aObfLJWgzpJxZnAMyMs5/6fc4yFdx AyRoeDWnc74Gg5OqB9h8EtyvRXPcOFKusrp/16n2Llqi6eb4J/DquUgHxyHPDWJnOdur46zygRQk rTmRI5sy2mbUTKOxos1fzBhkxZH7ux0EJH6GILMouw== =wwTv -----END PGP SIGNATURE----- --AL4cD8n6iueTMTSMNWy8MB1DMuokSp0xN-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 20:25:48 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1633D4C98BB for ; Tue, 5 Jan 2021 20:25:48 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9PC004F7z3H4v; Tue, 5 Jan 2021 20:25:47 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f182.google.com with SMTP id w12so893731ilm.12; Tue, 05 Jan 2021 12:25:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XT7PGgwwusyi9/8XXXLvuAT1n1SP1vNwYudMZoIX65k=; b=JuzKSNkk+XM6J00AwQ7NxZefDalUhU6nbJS1wHSUhA0jQmmzbJXWUSQXfNJU7TV9Qs BleFoqWURLuNtUYlCNPjsrfTR93LjXlSRUeKdD0VDh7ceibEnHq/r2k39OgLA87Q+NFq 9RVAnPwCwufgp4n1xvkL8gDdiMJygcA4y3MG0t0wUAkDKQylnTcR0Q8uwBQxwaV2Sp1X FWGfxlDB02herqq03dGD+a4HzFP3gxzeGYqR2zZ12XhMtCZ3L5B4q1zH5HybEYFxpnxZ j2ns8ui1jaRi8EyIPMsF+BIpU00NU9jUgV+7kr/fY/CfKVxLKM+QsJ12EkQwKvuEPwqX hmtA== X-Gm-Message-State: AOAM531qt3D/4pXN4l8wZguPj5TwOgSLcXjHNjH/qKxDaAZvBQxcyZ4R Cqn0Xxwdj7VU8qDrSKqoFnJCobwVKgLku13VprCIkS0JTa0EkQ== X-Google-Smtp-Source: ABdhPJx1FT1cHlK3ZjvvKNd+x3wjlw4l/goPTj+CQkzkJM2nFe1YxmxQFAKQmOx17GkNFokGCOt+ceYk3bn8ewn5sGY= X-Received: by 2002:a92:4a12:: with SMTP id m18mr1333140ilf.98.1609878346432; Tue, 05 Jan 2021 12:25:46 -0800 (PST) MIME-Version: 1.0 References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> In-Reply-To: <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> From: Ed Maste Date: Tue, 5 Jan 2021 15:25:34 -0500 Message-ID: Subject: Re: How does the time zone info get updated? To: Stefan Esser Cc: Warner Losh , FreeBSD Hackers , Hal Murray Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4D9PC004F7z3H4v X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 20:25:48 -0000 On Tue, 5 Jan 2021 at 14:51, Stefan Esser wrote: > > Yes, and they are not considered critical security fixes and thus > are not available as patches for those that install binary releases. In fact they are released as errata updates through FreeBSD-update. See for example FreeBSD-EN-20:20.tzdata https://www.freebsd.org/security/advisories/FreeBSD-EN-20:20.tzdata.asc From owner-freebsd-hackers@freebsd.org Tue Jan 5 20:36:45 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7556F4C9CF9 for ; Tue, 5 Jan 2021 20:36:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9PRd2Lsmz3J1D for ; Tue, 5 Jan 2021 20:36:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x735.google.com with SMTP id w79so656136qkb.5 for ; Tue, 05 Jan 2021 12:36:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HjY8fibpXVeFccScUfziSxCtSJyHEtCy6+imaQCq2Ko=; b=DkyvQUmoP9qFateiPpLFtRWSAJsa8dY3wRPYFpFSOs2Kod1OfBXr0fhzfu2oohCbkb MSOgj3roUt3RARpvgwknesmzolQ3a/qXsUnAZn2BzpfJPMDSzMjxqqAuclj+WkE+0Y8z 2OzkpkG+UKjd9WpBe2HxrilBU1TEymGBHl+lop+ped9hW0T5XQ0Gulk57TdYY99aBpuf CUaHfmsO8vqSJ0spbXiiFdsYebTo9ub8uHfkQ4b9e7darl2w578+c8jW9wRkYNyv1ytj JQjQYuG292IA+6/2aP3hko4ZRvHzvUXb+DNS7lfh/UX1vXG9VWE/lP0acXA1HEf5fNk9 JGEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HjY8fibpXVeFccScUfziSxCtSJyHEtCy6+imaQCq2Ko=; b=kKy4iNvZBa1T9Y4EehbwxwR97dmoDy3u1FSjMV3dy1YH8Y+d/XkDBfWgJckQ5th+ZO TPokVIUGGksb2kWGDsByG2/3q+NC2Yvp8dhhxI+4p8Ldh1vFVgGeVhSlm1p9CaPUSEfU G1VCWzRRJ8ek4Nqn8xQYk+hEHT82WqOO4W+aXZhJBRva6u8V9CzkG+xEIG9XB7HWWwFh ceeu+WArn9EDTCz/NvQrOzZ/uifQO/qPDbw4sYIzmGt+0F8F1j5oIWjFXZnINabM0jgj 09FLxFwPSyBdV0ugwkfp/Vyq4+LdKgorpKseAdzdd2lwZKcPbEKKqnvQB1hx1Ypm3Agw PsqQ== X-Gm-Message-State: AOAM532OIVIf9FTSEfb/JovoIN0U3NEIT0yDN9Khr25Q9kDGN4+kJbpC TaKeagBDyXsFIplW0ooZSHvmMMG83o6/7KNl8dj2iquO98X6qQ== X-Google-Smtp-Source: ABdhPJx9pNReLAFpbQEg3/tf8TmpVhAxCehphUXxMOgpoDmmI5ltbO3QZNgKyKL5PcKZ0LQYZBOjBouDnPRKnArmKLQ= X-Received: by 2002:a37:8505:: with SMTP id h5mr1342833qkd.44.1609879004278; Tue, 05 Jan 2021 12:36:44 -0800 (PST) MIME-Version: 1.0 References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> In-Reply-To: <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> From: Warner Losh Date: Tue, 5 Jan 2021 13:36:33 -0700 Message-ID: Subject: Re: How does the time zone info get updated? To: Stefan Esser Cc: FreeBSD Hackers , Hal Murray X-Rspamd-Queue-Id: 4D9PRd2Lsmz3J1D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 20:36:45 -0000 On Tue, Jan 5, 2021 at 12:51 PM Stefan Esser wrote: > Am 05.01.21 um 16:49 schrieb Warner Losh: > > > > > > On Tue, Jan 5, 2021, 4:24 AM Stefan Esser > > wrote: > > > > Am 05.01.21 um 08:51 schrieb Hal Murray:> It gets distributed with > the > > base packages, but I haven't figured out how to > > > keep it up to date. > > > > > > The info is distributed via IANA. > > > https://www.iana.org/time-zones > > > > > > It gets updated several times a year. There were 6 during 2020. > > When is > > > unpredictable, often shortly after some government passes a law > > to change > > > something with short notice. > > > > > > I'd expect there to be some mechanism to track the updates but I > > haven't found > > > it. > > > > > > The tz database includes the leap-seconds.list file. That gets > > updated every > > > 6 months. I'd like to get whatever FreeBSD does for time zones > > to include > > > that file so ntpd can use it without having to be concerned with > > keeping it up > > > to date. That only works if the zone info is kept up to date. > > > > We have recently moved the calendar files out of -CURRENT and into a > > port, and I could imagine providing the timezone files in the same > > way (without removal from base, but from there only used as a > fall-back > > if the port supplied files are not available). > > > > That way -STABLE users could update the definitions with base, but if > > you strictly follow a release, you could install the port and > decouple > > timezone file updates from base updates ... > > > > I'd be willing to support this change and maintain the data files, > but > > I can imagine that the current maintainer of those files in base > might > > want to do it himself. > > > > > > The calendar files aren't what he's talking about. > > Yes, I'm very well aware of that fact. > > > These are the timezone files he's asking about. Those get updated very > > shortly after upstream releases. > > Yes, and they are not considered critical security fixes and thus > are not available as patches for those that install binary releases. > > By providing the timezone files by a port/package in the same way > as the calendar data files (and preferring them if present), they > could be kept up-to-date with other packages on system that run > only release versions of FreeBSD. (And as written before, I do not > suggest to remove the timezone files from base at all.) > > > But, three things. First, we update leapseconds.list as soon as it is > > announce. Second, there is no way to get just these files updated: they > > come for free with a rebuild or and update using the beta pkgbase, but > > there is no automatic way to just get these. Third, there is an daily > > cron job that fetches leap-seconds.list if you have ntpd enabled and you > > are on the internet. > > Yes, if only the leap-seconds are required, this helps get the time > synchronized - but is quite different from timezone file updates. > > > So no TZ update outside an os upgrade, but the leap seconds file updates > > at the speed of NIST... > > The OP did not ask for leap-second updates but for timezone files, > if I understand his mail correctly. > Yes. But I know him from another list I'm on and knew the back story. How does FreeBSD know about leap seconds and what mechanism does it use to keep up to date was the topic there. Warner From owner-freebsd-hackers@freebsd.org Tue Jan 5 22:14:51 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D15144CC97D for ; Tue, 5 Jan 2021 22:14:51 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9Rcq5QWlz3Psh; Tue, 5 Jan 2021 22:14:51 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f110800e50b83eaa3464b63.dip0.t-ipconnect.de [IPv6:2003:cd:5f11:800:e50b:83ea:a346:4b63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 7370A344B; Tue, 5 Jan 2021 22:14:50 +0000 (UTC) (envelope-from se@freebsd.org) To: Ed Maste Cc: Warner Losh , FreeBSD Hackers , Hal Murray References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> From: Stefan Esser Subject: Re: How does the time zone info get updated? Message-ID: <646caaef-9a61-c34e-e820-536f6be6e07a@freebsd.org> Date: Tue, 5 Jan 2021 23:14:48 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="9RKdRI3itr9KqQoBRBeyDeM42lWPU6jz5" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 22:14:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9RKdRI3itr9KqQoBRBeyDeM42lWPU6jz5 Content-Type: multipart/mixed; boundary="PMcPZYV2fXbOaHiwnpCeS52tNNeY4TDl4"; protected-headers="v1" From: Stefan Esser To: Ed Maste Cc: Warner Losh , FreeBSD Hackers , Hal Murray Message-ID: <646caaef-9a61-c34e-e820-536f6be6e07a@freebsd.org> Subject: Re: How does the time zone info get updated? References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> In-Reply-To: --PMcPZYV2fXbOaHiwnpCeS52tNNeY4TDl4 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable Am 05.01.21 um 21:25 schrieb Ed Maste: > On Tue, 5 Jan 2021 at 14:51, Stefan Esser wrote: >> >> Yes, and they are not considered critical security fixes and thus >> are not available as patches for those that install binary releases. >=20 > In fact they are released as errata updates through FreeBSD-update. > See for example FreeBSD-EN-20:20.tzdata > https://www.freebsd.org/security/advisories/FreeBSD-EN-20:20.tzdata.asc= Thank you for pointing this out! I have not been aware that binary updates are offered via freebsd-update = (and apparently the OP did not know about this, too). Maybe this could be better documented (I do not know, since I always update my system from sources). Regards, Stefan --PMcPZYV2fXbOaHiwnpCeS52tNNeY4TDl4-- --9RKdRI3itr9KqQoBRBeyDeM42lWPU6jz5 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl/05NgFAwAAAAAACgkQR+u171r99UTF mgf+P/XrEZZHuBGl1wZdjvdNbc8arUl3nF3vslPUJsCi70/EItseVYkuvNpnPWcg2qM54PV2jUtY tZkymiaIY+9KtjdPO4t7vtDrnvJ0q9P4fIOE7UrJLJStDoJadVpaPwp1zyPLadBDG1EMX9Jb/onm bVYV5Ous4jlbpSUbhSdP++5nnxqFu1MiYQuk50KhHU4MzTelqHWtsGPX4mdVdo4L+NiULBzQwRsV ucy3A5jn/ZFKo2w1fOjCRdFzD4RNcLzJSC+zFRDc17AtNSyjCvyGtnLspx6uMH79RJr40fxM3HZ4 rLaabdTnj+0tmdybAmhyVsf4gP4Cc71i4/9UrCjmRA== =Kb8D -----END PGP SIGNATURE----- --9RKdRI3itr9KqQoBRBeyDeM42lWPU6jz5-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 22:21:44 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B61DE4CDDA3 for ; Tue, 5 Jan 2021 22:21:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9Rmm4dYPz3QY7; Tue, 5 Jan 2021 22:21:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f110800e50b83eaa3464b63.dip0.t-ipconnect.de [IPv6:2003:cd:5f11:800:e50b:83ea:a346:4b63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id E857C2BAC; Tue, 5 Jan 2021 22:21:43 +0000 (UTC) (envelope-from se@freebsd.org) To: Warner Losh Cc: FreeBSD Hackers , Hal Murray References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> From: Stefan Esser Subject: Re: How does the time zone info get updated? Message-ID: <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> Date: Tue, 5 Jan 2021 23:21:42 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="m3NxzA46hr9WDhHEyA2DiAy5T5gtO34iz" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 22:21:44 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --m3NxzA46hr9WDhHEyA2DiAy5T5gtO34iz Content-Type: multipart/mixed; boundary="5G05ioXAqikIrfWoJqbjgYDbXZSrpJnFa"; protected-headers="v1" From: Stefan Esser To: Warner Losh Cc: FreeBSD Hackers , Hal Murray Message-ID: <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> Subject: Re: How does the time zone info get updated? References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> In-Reply-To: --5G05ioXAqikIrfWoJqbjgYDbXZSrpJnFa Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable Am 05.01.21 um 21:36 schrieb Warner Losh: > On Tue, Jan 5, 2021 at 12:51 PM Stefan Esser > wrote: [...] > Yes, if only the leap-seconds are required, this helps get the time= > synchronized - but is quite different from timezone file updates. >=20 > > So no TZ update outside an os upgrade, but the leap seconds file= > updates > > at the speed of NIST... >=20 > The OP did not ask for leap-second updates but for timezone files, > if I understand his mail correctly. >=20 > Yes. But I know him from another list I'm on and knew the back story.=20 > How does FreeBSD know about leap seconds and what mechanism does it use= =20 > to keep up to date was the topic there. Well, then you answered the question sent to the other list, which I have not seen and did not know about this context ... But Ed Maste has meanwhile answered the question he asked on this list (as far as I understand it): -> "freebsd-update" applies the timezone changes and EN notes are sent out to notify users of the availability of this update. And release users should follow-up on EN notes anyway and thus should not miss the timezone updates. Regards, STefan --5G05ioXAqikIrfWoJqbjgYDbXZSrpJnFa-- --m3NxzA46hr9WDhHEyA2DiAy5T5gtO34iz Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl/05nYFAwAAAAAACgkQR+u171r99URg Wgf+KlxjRQBoCOsZ+ljkB7KBwzOcTAMkssd4rgnX0yda1zvtqvib09+6923upZylM3iNxJKTIxSG BIV3iZYoixDzUAkexIjZ+24/lUH+BKi9/wR5QNmbeQAJrpJpPluhTEKSUt2/2ikPi5Ti0IYsD2pX aPJXHN0awPbE8JM9fGboiDSjWPUz0Xq3l9Yaw0qtClIUu3RIBtiSVCNpwH69ixJyyWSIXpvLbMxg YqMz/hEgpM7gsKBvo3OXYUaK2T78A4ZJLRtB3PJtqylKDY2WZVC0o9y4FjjAK+lwugwKYkPE+TDi p+aji52HLJzjFqyOL3TCC5UDXMV3BnvBOpq5SNJpgA== =slDA -----END PGP SIGNATURE----- --m3NxzA46hr9WDhHEyA2DiAy5T5gtO34iz-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 22:30:57 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 149C64CEE3E for ; Tue, 5 Jan 2021 22:30:57 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9RzN73Kqz3h9V; Tue, 5 Jan 2021 22:30:56 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-pl1-f175.google.com with SMTP id y8so515281plp.8; Tue, 05 Jan 2021 14:30:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=7XaO7W8JX1Ux1lwWdBsIT/hVz8lvdAMrGid9YeAuSoM=; b=m+JVTZsN+NgZTLZH+ICgccWdkke5NR5Z0wF3nAxvNaGTI6rX8/zpxeYNDDFlt99sut 9yhNErDA4B4xFXT/UtyJRo7sQgWhVlnL+yCAtbwoqEWs1KdEZl0toGDKhhmaAsxuaWEB HIAa/uyOEoaoWP2si13MnHNuGWPfEE2RM6JU8ivQFSn9U+gfD0dT3n6oI3I1OmRpNUVz OgVkMZJCnuTFCG5l6Fw7Q5CTBiF8ytA7VVEKk/LW3wx2wyYz8qZuJShZo8j4XzvGMZFa Eh/z4Hocz8xlnir4Kw1CSBTyuaXuxQgcS0yr8wDJ5MtNOEhfLilMIdD9vW6oGxh2HOih IaNA== X-Gm-Message-State: AOAM531Av38LF1beFq7qsWxOBJijNtz6uPJ6yIFS3KEOhQq7HgyIQQXS 3U5k9tFVCpRo84PwCTEX26dg4h0X/Z5L1Q== X-Google-Smtp-Source: ABdhPJxYYKsA7rvjyoKB9dzaNV+HLh1tm1XW0xIj0ZP/1ug51T+avnAsD/U8KOZffJl2+VM5q+chww== X-Received: by 2002:a17:90a:394f:: with SMTP id n15mr1258989pjf.121.1609885855468; Tue, 05 Jan 2021 14:30:55 -0800 (PST) Received: from [192.168.0.24] ([195.64.148.76]) by smtp.googlemail.com with ESMTPSA id p3sm201552pjg.53.2021.01.05.14.30.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Jan 2021 14:30:54 -0800 (PST) Subject: Re: How does the time zone info get updated? To: Stefan Esser , Warner Losh Cc: FreeBSD Hackers , Hal Murray References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> From: Andriy Gapon Message-ID: <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> Date: Wed, 6 Jan 2021 00:30:47 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4D9RzN73Kqz3h9V X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 22:30:57 -0000 On 2021-01-06 00:21, Stefan Esser wrote: > Am 05.01.21 um 21:36 schrieb Warner Losh: >> On Tue, Jan 5, 2021 at 12:51 PM Stefan Esser > > wrote: > [...] >>     Yes, if only the leap-seconds are required, this helps get the time >>     synchronized - but is quite different from timezone file updates. >> >>      > So no TZ update outside an os upgrade, but the leap seconds file >>     updates >>      > at the speed of NIST... >> >>     The OP did not ask for leap-second updates but for timezone files, >>     if I understand his mail correctly. >> >> Yes. But I know him from another list I'm on and knew the back story. >> How does FreeBSD know about leap seconds and what mechanism does it >> use to keep up to date was the topic there. > > Well, then you answered the question sent to the other list, > which I have not seen and did not know about this context ... > > But Ed Maste has meanwhile answered the question he asked on > this list (as far as I understand it): > > -> "freebsd-update" applies the timezone changes and EN notes > are sent out to notify users of the availability of this update. > > And release users should follow-up on EN notes anyway and thus > should not miss the timezone updates. Also, another option (at least for me) is to replace timezone files from base with those from tzdata package. Port / package updates are more flexible / frequent. -- Andriy Gapon From owner-freebsd-hackers@freebsd.org Tue Jan 5 23:05:19 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CB1ED4D0C4F for ; Tue, 5 Jan 2021 23:05:19 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9Sl357WPz3m2M; Tue, 5 Jan 2021 23:05:19 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f110800e50b83eaa3464b63.dip0.t-ipconnect.de [IPv6:2003:cd:5f11:800:e50b:83ea:a346:4b63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 07F663AC6; Tue, 5 Jan 2021 23:05:18 +0000 (UTC) (envelope-from se@freebsd.org) To: Andriy Gapon , Warner Losh Cc: FreeBSD Hackers , Hal Murray References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> From: Stefan Esser Subject: Re: How does the time zone info get updated? Message-ID: Date: Wed, 6 Jan 2021 00:05:17 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="CKcm3bam4q4YcFtYoZSBNCyj9HhptbRQv" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 23:05:19 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --CKcm3bam4q4YcFtYoZSBNCyj9HhptbRQv Content-Type: multipart/mixed; boundary="yTYoK9id1GobnYQRru9RJCmAPJ4fPGung"; protected-headers="v1" From: Stefan Esser To: Andriy Gapon , Warner Losh Cc: FreeBSD Hackers , Hal Murray Message-ID: Subject: Re: How does the time zone info get updated? References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> In-Reply-To: <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> --yTYoK9id1GobnYQRru9RJCmAPJ4fPGung Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable Am 05.01.21 um 23:30 schrieb Andriy Gapon: > On 2021-01-06 00:21, Stefan Esser wrote: >> Am 05.01.21 um 21:36 schrieb Warner Losh: >>> On Tue, Jan 5, 2021 at 12:51 PM Stefan Esser >> > wrote: >> [...] >>> =C2=A0=C2=A0=C2=A0 Yes, if only the leap-seconds are required, this = helps get the time >>> =C2=A0=C2=A0=C2=A0 synchronized - but is quite different from timezo= ne file updates. >>> >>> =C2=A0=C2=A0=C2=A0=C2=A0 > So no TZ update outside an os upgrade, bu= t the leap seconds file >>> =C2=A0=C2=A0=C2=A0 updates >>> =C2=A0=C2=A0=C2=A0=C2=A0 > at the speed of NIST... >>> >>> =C2=A0=C2=A0=C2=A0 The OP did not ask for leap-second updates but fo= r timezone files, >>> =C2=A0=C2=A0=C2=A0 if I understand his mail correctly. >>> >>> Yes. But I know him from another list I'm on and knew the back story.= >>> How does FreeBSD know about leap seconds and what mechanism does it >>> use to keep up to date was the topic there. >> >> Well, then you answered the question sent to the other list, >> which I have not seen and did not know about this context ... >> >> But Ed Maste has meanwhile answered the question he asked on >> this list (as far as I understand it): >> >> -> "freebsd-update" applies the timezone changes and EN notes >> are sent out to notify users of the availability of this update. >> >> And release users should follow-up on EN notes anyway and thus >> should not miss the timezone updates. >=20 > Also, another option (at least for me) is to replace timezone files fro= m > base with those from tzdata package. Port / package updates are more > flexible / frequent. I had looked for a port but had not found it at first. It is misc/zoneinfo and the zoneinfo package. But this port is special since it installs files outside $PREFIX and requires WITHOUT_ZONEINFO=3Dyes to be defined in src.conf. I'd rather think we should have the port install files into $PREFIX/share/zoneinfo and modify the file lookup to search at that location first, before falling back to the base system location. This might require changes to zic, zdump, tzsetup, etcupdate, and bsdconfig, which only expect a single path in TZDIR. This might be considered worse than the port that overwrites files in base. In case that the alternate location for the port provided files is considered, I'd be willing to prepare patches for review that add the additional lookup in LOCALBASE. Regards, STefan --yTYoK9id1GobnYQRru9RJCmAPJ4fPGung-- --CKcm3bam4q4YcFtYoZSBNCyj9HhptbRQv Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAl/08K0FAwAAAAAACgkQR+u171r99UQC GAgAhqEXjugYE8abXfKTyehRvR/FkYy1gGmfjeQdtFrh9dV47VdkD/3MLAgiv94BCEONfrCNJjTv K14GnyeIdCRWEtU2OgE7EaZlUoUJsMBIa/0OVLsGRPC/5Zs1qLwceyPOYcGimVzCnS804TwNq0l6 dKSJnzlnYtZUZiL3+LYBW4HKSXqZBfhzxgsNB6GI9oWfTCiBuDL5h/mpOVyHq3KEeXIomPT1RF6M zEjdqe9r3O5BBlfUhoBl4uNbIba0IZYiO590GIagZ5nDzyj8SzxTjgcHofWbh3R8/VpNdvA6t8Hw pK67S9OWILKlYyhKm2YSKwF+9H/D8LXH/XGIwgKffA== =cB0N -----END PGP SIGNATURE----- --CKcm3bam4q4YcFtYoZSBNCyj9HhptbRQv-- From owner-freebsd-hackers@freebsd.org Tue Jan 5 23:19:48 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1AD064D4762 for ; Tue, 5 Jan 2021 23:19:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9T3l5fkzz3qGF; Tue, 5 Jan 2021 23:19:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 105NJbng055039 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 6 Jan 2021 01:19:40 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 105NJbng055039 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 105NJbkm055038; Wed, 6 Jan 2021 01:19:37 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 6 Jan 2021 01:19:37 +0200 From: Konstantin Belousov To: Stefan Esser Cc: Andriy Gapon , Warner Losh , FreeBSD Hackers , Hal Murray Subject: Re: How does the time zone info get updated? Message-ID: References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4D9T3l5fkzz3qGF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 23:19:48 -0000 On Wed, Jan 06, 2021 at 12:05:17AM +0100, Stefan Esser wrote: > I'd rather think we should have the port install files into > $PREFIX/share/zoneinfo and modify the file lookup to search at > that location first, before falling back to the base system > location. > > This might require changes to zic, zdump, tzsetup, etcupdate, > and bsdconfig, which only expect a single path in TZDIR. > > This might be considered worse than the port that overwrites > files in base. This really requires changes to libc, which implies changed behavior for all applications. From owner-freebsd-hackers@freebsd.org Tue Jan 5 23:39:44 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EFA694D5BFB for ; Tue, 5 Jan 2021 23:39:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9TVm6NRRz3rvl; Tue, 5 Jan 2021 23:39:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-WLAN.fritz.box (p200300cd5f110800e50b83eaa3464b63.dip0.t-ipconnect.de [IPv6:2003:cd:5f11:800:e50b:83ea:a346:4b63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 1D23B3FA1; Tue, 5 Jan 2021 23:39:44 +0000 (UTC) (envelope-from se@freebsd.org) From: Stefan Esser Subject: Re: How does the time zone info get updated? To: Konstantin Belousov , Stefan Esser Cc: Andriy Gapon , Warner Losh , FreeBSD Hackers , Hal Murray References: <20210105075109.A412C40605C@ip-64-139-1-69.sjc.megapath.net> <4e6721bc-e593-76b6-90f8-03d96486dc28@freebsd.org> <47c37f75-e082-7a34-7eb6-3ee2a4ade395@freebsd.org> <02ebd386-a7fd-1816-72b6-3211f4bc69fb@freebsd.org> <358b8939-5ec9-e340-8b06-f362cae38fa0@FreeBSD.org> Message-ID: <0e0c7dbb-2036-ef7c-762a-f599cd8c18bc@freebsd.org> Date: Wed, 6 Jan 2021 00:39:43 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 23:39:45 -0000 Am 06.01.21 um 00:19 schrieb Konstantin Belousov: > On Wed, Jan 06, 2021 at 12:05:17AM +0100, Stefan Esser wrote: >> I'd rather think we should have the port install files into >> $PREFIX/share/zoneinfo and modify the file lookup to search at >> that location first, before falling back to the base system >> location. >> >> This might require changes to zic, zdump, tzsetup, etcupdate, >> and bsdconfig, which only expect a single path in TZDIR. >> >> This might be considered worse than the port that overwrites >> files in base. > This really requires changes to libc, which implies changed behavior > for all applications. A library routine can easily implement the logic to lookup the zone definitions in a specified sequence in several places. The biggest problem I see is that the /etc/localtime link would point to either a definition in /usr/share/zoneinfo or $LOCALBASE/share/zoneinfo and if a port did update the files below $LOCALBASE when previously files from base had been used, the link would need to be adjusted. I do not consider such a change of a link in /etc by a port to be acceptable behavior, though, and thus the port that overwrites files in /usr/share/zoneinfo actually appears to be the lesser evil ... Regards, STefan PS: Resent from freebsd mail address to have the message be accepted by the list server ... From owner-freebsd-hackers@freebsd.org Wed Jan 6 19:53:26 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8E04D4DC25D for ; Wed, 6 Jan 2021 19:53:26 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB0R94yTMz4pRh for ; Wed, 6 Jan 2021 19:53:25 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: by mail-qk1-f179.google.com with SMTP id v126so3564568qkd.11 for ; Wed, 06 Jan 2021 11:53:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4WTa5mn9lUirwp2+Gj+zuy+hw05yi4+svSY5SQHzm/o=; b=KE6D1hPqDhQfPfLhqNZcxqaGHIeL1ifLe0u+BtrPNaVM6XLOaHBte2a8D757R84+OJ Mjz1huwX9ZMriRJoXirDvdjhWPwWA2UnQ4F+gGr75IyaDgQc+rjhf0tXpvSyzY4u6s5E Xtkb2iFLLrZDl4AuyeBAW7ZgoB58yzq752xfOoygWK9QZyVEfrV9cupN11UWGwxUfc5d /nH0nWlHxIRdJqk7F2fs1lcuKigBy8sfxQV9QyqJ54LwaDjxPqLFq4kwnppr+n+iOzkf Rk1GfxJLBddThG65jko0YLQ51L2wBvirhi+vElPRahOWxBYvoeo4rZ7hvc6SWkeonUUc Nw6Q== X-Gm-Message-State: AOAM531XYVRRGkH4FIkiRb6V+iHanSAD+5nFO0fylFnfP3hgnC3/EiuP xLY3AR1mT9P0HOOAfcxJ1NHIwlTJLN8= X-Google-Smtp-Source: ABdhPJykVO6TDnOEsHfp+HpU2elcyjZw3DsWf5G1Rs5laC5Y3WbUL6+H+Wr/3b3V8EZqoCeUcSlRnA== X-Received: by 2002:a37:a8c4:: with SMTP id r187mr5803873qke.481.1609962804205; Wed, 06 Jan 2021 11:53:24 -0800 (PST) Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com. [209.85.222.175]) by smtp.gmail.com with ESMTPSA id k141sm1983476qke.38.2021.01.06.11.53.23 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Jan 2021 11:53:23 -0800 (PST) Received: by mail-qk1-f175.google.com with SMTP id 143so3567659qke.10 for ; Wed, 06 Jan 2021 11:53:23 -0800 (PST) X-Received: by 2002:a05:620a:4047:: with SMTP id i7mr5866473qko.3.1609962803670; Wed, 06 Jan 2021 11:53:23 -0800 (PST) MIME-Version: 1.0 From: Ryan Libby Date: Wed, 6 Jan 2021 11:53:13 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: clang vs gcc warning flags To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DB0R94yTMz4pRh X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rlibby@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=rlibby@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[rlibby@freebsd.org,rlibby@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.222.179:from]; R_DKIM_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[rlibby@freebsd.org,rlibby@gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DOM_EQ_FROM_DOM(0.00)[]; FREEFALL_USER(0.00)[rlibby]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[209.85.222.179:from:127.0.2.255]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[209.85.222.179:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.222.179:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 19:53:26 -0000 One of the more annoying things about keeping the gcc build going is the set of warnings that gcc acts on but clang only recognizes for compatibility. As a common example, -Wredundant-decls has no effect in clang, but will break the gcc build. There are a couple dozen such flags [1][2], and a few of them are in our default set of warnings, such as in sys/conf/kern.mk, these ones in CWARNFLAGS: - Wredundant-decls - Wnested-externs - Wmissing-include-dirs additionally some warnings are explicitly disabled for clang, but not for gcc in CWARNEXTRA: - Wempty-body - Wunused-function Similarly, in share/mk/bsd.sys.mk: - Winline (although, Wno-error'd) - Wnested-externs - Wredundant-decls - Wold-style-definition So I suggest we harmonize these somewhat. - Wnested-externs I just do not understand. We have specified this warning flag for some 25 years but to me it seems completely without value. I suggest we just delete it. - Wredundant-decls, I'm not sure about. I have never seen this detect anything that will cause misbehavior, but most of the time that it fires it does indicate some kind of genuine--but harmless--mistake. - Wmissing-include-dirs doesn't seem to occur often and usually indicates a genuine (but again harmless) mistake in a makefile. I think we should keep it. - Wempty-body, Wunused-function. I'm not sure. These are proscriptive about things that are not necessarily problems. We are apparently already clean for them in the kernel gcc build, so perhaps we should enable them for the kernel clang build. In any case, I think we should bring these into agreement between clang and gcc, one way or the other. Another sticking point may be contrib software. I think we generally don't want to fail builds of contrib software for things that are ultimately harmless. For bsd.sys.mk this could be accomplished by enabling such warnings only at WARNS >= 6. For the kernel, we could come up with some other mechanism. I'll put up a review soon for deleting -Wnested-externs unless there are objections. If there is agreement about the others, I'll include those too. Ryan [1] https://clang.llvm.org/docs/DiagnosticsReference.html [2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html From owner-freebsd-hackers@freebsd.org Wed Jan 6 20:06:11 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9423B4DCB2C for ; Wed, 6 Jan 2021 20:06:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB0jt5wlCz4qTW; Wed, 6 Jan 2021 20:06:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 106K637g053658 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 6 Jan 2021 22:06:06 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 106K637g053658 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 106K63XL053657; Wed, 6 Jan 2021 22:06:03 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 6 Jan 2021 22:06:03 +0200 From: Konstantin Belousov To: Ryan Libby Cc: FreeBSD Hackers Subject: Re: clang vs gcc warning flags Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DB0jt5wlCz4qTW X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-0.99 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; R_SPF_SOFTFAIL(0.00)[~all:c]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.994]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(1.00)[1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 20:06:11 -0000 On Wed, Jan 06, 2021 at 11:53:13AM -0800, Ryan Libby wrote: > - Wnested-externs I just do not understand. We have specified this > warning flag for some 25 years but to me it seems completely without > value. I suggest we just delete it. I suspect this warning is to flag style(9) violations. From owner-freebsd-hackers@freebsd.org Wed Jan 6 20:38:45 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5CAF04DE5D8 for ; Wed, 6 Jan 2021 20:38:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB1RS58mRz3GMH for ; Wed, 6 Jan 2021 20:38:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x730.google.com with SMTP id c7so3786478qke.1 for ; Wed, 06 Jan 2021 12:38:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=QswnFcUKTzjYSZNRusFAIoysK/3tk1DOWVNyHQD3beE=; b=d/thcAXoR9O+wA6vi1rvq/sLQpMZ3NquIP6ZL3AnNHwoI5YfIpcBtsVqkfzlmT/IZQ 1/Ox0tD9937yliJvpTeFV1Zo4/JPLsPN0YAEqe2+31h/U79CBNEawFxYFkWXL5M7LCGN dOYbqM7uBvd5joC14ftLEaAFdZR7LU+/h6NZSiVV/6ohZzO7KETk2CMU4gyheMVwzS1z uuL/uMij1YwSrc+FPaIxK5v7UjKiJEOXSw+JJEscfSo9BZeyVhpMaw3wmNbDGT182Dmw +i2ebq4sMCSemoi/KhJnn2V8a/vfR7UT9Et19y6+z+8sGS+sIzhuIyJ4d5BzHA1P3DsY bxeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QswnFcUKTzjYSZNRusFAIoysK/3tk1DOWVNyHQD3beE=; b=G5lM6gbVF6w/8hIdCIUj4Op+SMaHY8T3M4S7SmNz0P0mzs4ASrQdlHaWFsxrsjFVGw TjqSNktjAcBqWDsYqwoSOuJbN+VspJ98mu35j/YsBvud0y0pyjP/30xW6ENQNFw+53ol HuDFSHn2k5Dd75kZM4yeU3yUFJHI6nVuWNeMUjlz+nxcz0NHao3DoRwmQpbIZ3OT58XB AXQQcE4946s0W+TF1BTGGorAzYzsopL7ZVvgGG9Bjof5fT+V7zIv+Z5DnnbFtrY4in/L N4BQdPvzXKaR6ZRCz89YVw6WP43PQL/KK0F9l1O7XirCEHOshSRrgyvJQ8ho9e2EkvoW w19w== X-Gm-Message-State: AOAM5301vIG9zTtLcfva9LMMldjPF44eQUhylImzPH3/gRiaE2mcdiMG 5v95zHltAQSYhe6l99gqRRfAXCwf3+N8S5DCS1JqXw== X-Google-Smtp-Source: ABdhPJwekF7c3JrnP2JkayBaMvc7MDgQi8y3LJT+A6oXffT/OnDyg6ZU9ifv0zqPw6HR51CIiQU+k5VIoUdmKFg1DdU= X-Received: by 2002:ae9:ebd5:: with SMTP id b204mr6000223qkg.195.1609965523552; Wed, 06 Jan 2021 12:38:43 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 6 Jan 2021 13:38:31 -0700 Message-ID: Subject: Re: clang vs gcc warning flags To: Ryan Libby Cc: FreeBSD Hackers X-Rspamd-Queue-Id: 4DB1RS58mRz3GMH X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=d/thcAXo; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::730) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.00 / 15.00]; ARC_NA(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::730:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::730:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_LONG(1.00)[1.000]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::730:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 20:38:45 -0000 On Wed, Jan 6, 2021 at 12:53 PM Ryan Libby wrote: > One of the more annoying things about keeping the gcc build going is the > set of warnings that gcc acts on but clang only recognizes for > compatibility. As a common example, -Wredundant-decls has no effect > in clang, but will break the gcc build. There are a couple dozen such > flags [1][2], and a few of them are in our default set of warnings, such > as in sys/conf/kern.mk, these ones in CWARNFLAGS: > > - Wredundant-decls > - Wnested-externs > - Wmissing-include-dirs > > additionally some warnings are explicitly disabled for clang, but not > for gcc in CWARNEXTRA: > > - Wempty-body > - Wunused-function > > Similarly, in share/mk/bsd.sys.mk: > > - Winline (although, Wno-error'd) > - Wnested-externs > - Wredundant-decls > - Wold-style-definition > > So I suggest we harmonize these somewhat. > > - Wnested-externs I just do not understand. We have specified this > warning flag for some 25 years but to me it seems completely without > value. I suggest we just delete it. > It's to prevent declaring functions inside of functions. why is that bad? It only has scope of the function, and we've decided that's not something we want in the tree... I'd say that the fact we don't have any in the tree shows that it's useful at its intended purpose. What code is causing issues here? > - Wredundant-decls, I'm not sure about. I have never seen this detect > anything that will cause misbehavior, but most of the time that it > fires it does indicate some kind of genuine--but harmless--mistake. > It also tends to prevent declarations that are the same on some platforms, but different on others. > - Wmissing-include-dirs doesn't seem to occur often and usually > indicates a genuine (but again harmless) mistake in a makefile. I > think we should keep it. > I do too. > - Wempty-body, Wunused-function. I'm not sure. These are proscriptive > about things that are not necessarily problems. We are apparently > already clean for them in the kernel gcc build, so perhaps we should > enable them for the kernel clang build. In any case, I think we > should bring these into agreement between clang and gcc, one way or > the other. > We do not remove it. We include the warning, but don't make it fatal for clang. Most of the clang neutering was due to bugs in the early days of clang. I'm not sure anybody has systematically reviewed them since then. Another sticking point may be contrib software. I think we generally > don't want to fail builds of contrib software for things that are > ultimately harmless. For bsd.sys.mk this could be accomplished by > enabling such warnings only at WARNS >= 6. For the kernel, we could > come up with some other mechanism. > The kernel is different. 'Ultimately harmless' is a value judgement. > I'll put up a review soon for deleting -Wnested-externs unless there are > objections. If there is agreement about the others, I'll include those > too. > > Ryan > > [1] https://clang.llvm.org/docs/DiagnosticsReference.html > [2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Wed Jan 6 21:34:43 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2DFFF4DFD0A for ; Wed, 6 Jan 2021 21:34:43 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB2h30gWkz3K8T for ; Wed, 6 Jan 2021 21:34:42 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: by mail-qt1-f179.google.com with SMTP id h19so3051678qtq.13 for ; Wed, 06 Jan 2021 13:34:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ECHEJM8W2nF/l50jAlwcNTfUki89l8587a96yvY0nnk=; b=nl4Oy1z16JKxZrPqmA1YRTQzaLWOzcTD9AdHL83mkOC7XWQRc6T9q7YpaVslj+TAau 1IR8CYJg2DVbM9Sqnn3Y3rPjj7HHGqsHFsgB6mDNC+HAwXm8qiI+xFZVKgQ/ardE6eN3 nuJyTf9gppAOPvG19Nz982EVuo1yfvOqs1owkcwDwQLvTunqYaWty4eMMnFLRVbDJ560 suqF3S3zyojaec3/znPup9RJXjwYJ2yptRW5pAsGwY1+Z88ijIxUAqB6PiDNHE3hK3ek HHANB0YrRfLgazDgCUDwmp0aHDd6l7Z6sbEObBAd1CY846g6hsyfE44vwc+rkPz2wtUA cVEQ== X-Gm-Message-State: AOAM532paRaaO/n0msWUJihTZpnCMEo6L8qaAWW9iu9LK1O3lK1qbyqW iBb5SCbPp6Nnl4OGZjeiuCBQxkjPtyU= X-Google-Smtp-Source: ABdhPJxnaKprmBAWQeKeDepmyZXSTPh/JK5a+UslS06U4HhAWIgsNcREZm/PUnShjwKzOTCHrG2SOQ== X-Received: by 2002:ac8:4692:: with SMTP id g18mr5850854qto.255.1609968881564; Wed, 06 Jan 2021 13:34:41 -0800 (PST) Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com. [209.85.222.170]) by smtp.gmail.com with ESMTPSA id b6sm2011340qkc.128.2021.01.06.13.34.41 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Jan 2021 13:34:41 -0800 (PST) Received: by mail-qk1-f170.google.com with SMTP id c7so3946037qke.1 for ; Wed, 06 Jan 2021 13:34:41 -0800 (PST) X-Received: by 2002:a05:620a:4047:: with SMTP id i7mr6244796qko.3.1609968881062; Wed, 06 Jan 2021 13:34:41 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Ryan Libby Date: Wed, 6 Jan 2021 13:34:30 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: clang vs gcc warning flags To: Warner Losh Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4DB2h30gWkz3K8T X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 21:34:43 -0000 On Wed, Jan 6, 2021 at 12:38 PM Warner Losh wrote: > > > > On Wed, Jan 6, 2021 at 12:53 PM Ryan Libby wrote: >> >> One of the more annoying things about keeping the gcc build going is the >> set of warnings that gcc acts on but clang only recognizes for >> compatibility. As a common example, -Wredundant-decls has no effect >> in clang, but will break the gcc build. There are a couple dozen such >> flags [1][2], and a few of them are in our default set of warnings, such >> as in sys/conf/kern.mk, these ones in CWARNFLAGS: >> >> - Wredundant-decls >> - Wnested-externs >> - Wmissing-include-dirs >> >> additionally some warnings are explicitly disabled for clang, but not >> for gcc in CWARNEXTRA: >> >> - Wempty-body >> - Wunused-function >> >> Similarly, in share/mk/bsd.sys.mk: >> >> - Winline (although, Wno-error'd) >> - Wnested-externs >> - Wredundant-decls >> - Wold-style-definition >> >> So I suggest we harmonize these somewhat. >> >> - Wnested-externs I just do not understand. We have specified this >> warning flag for some 25 years but to me it seems completely without >> value. I suggest we just delete it. > > > > It's to prevent declaring functions inside of functions. why is that bad?= It only has scope of the function, and we've decided that's not something = we want in the tree... I'd say that the fact we don't have any in the tree = shows that it's useful at its intended purpose. > > What code is causing issues here? > Well, but do we really care specifically about declaring extern functions (or variables) inside of functions? Is that worse than the usual fix, moving them just outside the function to file scope? Regarding _defining_ functions inside of functions, I agree we don't want that. But -Wnested-externs doesn't warn about that; gcc produces no warning for this with -Wnested-externs: void f(void) { void g(void){}; g(); } (Gcc produces a warning for that with -pedantic, which we don't use; and clang produces an error even with no warning flags.) How I've actually seen "nested externs" used is usually to make variables visible without including headers, for one reason or another. Here are a couple examples: https://cgit.freebsd.org/src/commit/?id=3Dd576ccdf01a4c6f8f02e8ed7e72290c72= 9d68de6 https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/module/zfs/arc.c#n472= 1 As far as I understand, this actually does limit the scope of the visibility of the extern declaration. I am not an expert in the standard, I could be mistaken. I don't want to overstate the problem though. As I said, these are annoyances, the solutions aren't hard. >> >> - Wredundant-decls, I'm not sure about. I have never seen this detect >> anything that will cause misbehavior, but most of the time that it >> fires it does indicate some kind of genuine--but harmless--mistake. > > > It also tends to prevent declarations that are the same on some platforms= , but different on others. > >> >> - Wmissing-include-dirs doesn't seem to occur often and usually >> indicates a genuine (but again harmless) mistake in a makefile. I >> think we should keep it. > > > I do too. > >> >> - Wempty-body, Wunused-function. I'm not sure. These are proscriptive >> about things that are not necessarily problems. We are apparently >> already clean for them in the kernel gcc build, so perhaps we should >> enable them for the kernel clang build. In any case, I think we >> should bring these into agreement between clang and gcc, one way or >> the other. > > > We do not remove it. We include the warning, but don't make it fatal for = clang. > Yes, "clean" was maybe not the right word. The end result is build errors with gcc, no build errors with clang. The suggestion is just that we do the same thing with both compilers, whether that be an error, or a warning without an error. > Most of the clang neutering was due to bugs in the early days of clang. I= 'm not sure anybody has systematically reviewed them since then. > >> Another sticking point may be contrib software. I think we generally >> don't want to fail builds of contrib software for things that are >> ultimately harmless. For bsd.sys.mk this could be accomplished by >> enabling such warnings only at WARNS >=3D 6. For the kernel, we could >> come up with some other mechanism. > > > The kernel is different. 'Ultimately harmless' is a value judgement. > I largely agree. What I'm hoping to break out of is situations where we import contrib software and it breaks the gcc build for reasons that don't affect the functionality of the built code, and then it's a pain to fix because it's contrib software. >> >> I'll put up a review soon for deleting -Wnested-externs unless there are >> objections. If there is agreement about the others, I'll include those >> too. >> >> Ryan >> >> [1] https://clang.llvm.org/docs/DiagnosticsReference.html >> [2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g" From owner-freebsd-hackers@freebsd.org Wed Jan 6 21:38:05 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BB52D4DFDB1 for ; Wed, 6 Jan 2021 21:38:05 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB2lx2N1cz3KwF for ; Wed, 6 Jan 2021 21:38:05 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: by mail-qv1-f45.google.com with SMTP id s6so1942251qvn.6 for ; Wed, 06 Jan 2021 13:38:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T+z0uG7iFasqHX4TU0UkJsn30/BLoBpndYg26nkoR34=; b=hZ4aYRmLUh7nn8G0Nunw69wnxm+IX+SwXvjuaNMPZSaniLebelTGoGtVYMI+iLcIal s3gMLAcg5O6aw/NWf4TNceuzaG7ZhYYieHMAo2CljWWNMW2/0GHvNFrP1uT8vg+duR55 001aIlIqqj7SnZYXcUFv3flnt5qfbPDEm8SPvMpvOokR7Fa68kikKeqbbtN3IFA6Pcy0 dw/jvjd/S6h2yegHQ4qW1c6b3DG1oN+MrAukunkoACWyKpCvetAAbqsndZYcaQkzhUPO vK4HTMwKjsIVaDkWduH3ZVcrRoGx98YN1sRmA0HHBtou2kJqGpV1JaaAuqOOM+2O3hxa jFzA== X-Gm-Message-State: AOAM532FI2NeDcoMtq4KY1Qqq71jDT1Z8cEtMDNK9NJpbF006BghUOLc 7wKrBcNJU0U3pf4kYgxs5KuAQvanQio= X-Google-Smtp-Source: ABdhPJxrBgN7CoiuJSxxj1KknEMuVYULul6h/aBfQXOQvPrvrMMD2yRIFwOor03+nPB7ZkDlvmp/0w== X-Received: by 2002:a05:6214:d05:: with SMTP id 5mr5837478qvh.54.1609969084236; Wed, 06 Jan 2021 13:38:04 -0800 (PST) Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com. [209.85.160.178]) by smtp.gmail.com with ESMTPSA id a35sm1918101qtk.82.2021.01.06.13.38.03 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Jan 2021 13:38:04 -0800 (PST) Received: by mail-qt1-f178.google.com with SMTP id 7so3127604qtp.1 for ; Wed, 06 Jan 2021 13:38:03 -0800 (PST) X-Received: by 2002:aed:2088:: with SMTP id 8mr6025795qtb.174.1609969083747; Wed, 06 Jan 2021 13:38:03 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Ryan Libby Date: Wed, 6 Jan 2021 13:37:53 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: clang vs gcc warning flags To: Konstantin Belousov Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DB2lx2N1cz3KwF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 21:38:05 -0000 On Wed, Jan 6, 2021 at 12:06 PM Konstantin Belousov wrote: > > On Wed, Jan 06, 2021 at 11:53:13AM -0800, Ryan Libby wrote: > > - Wnested-externs I just do not understand. We have specified this > > warning flag for some 25 years but to me it seems completely without > > value. I suggest we just delete it. > I suspect this warning is to flag style(9) violations. Which aspect do you mean? I don't see it mentioned explicitly, but I know some of style(9) is reading between the lines. If it's that objects with external linkage should be declared in headers, I think that doesn't address most of the cases where extern is actually used in our .c files. Maybe the answer should be different for contrib software (where we don't generally enforce style)? From owner-freebsd-hackers@freebsd.org Wed Jan 6 21:43:03 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DA62F4E023A for ; Wed, 6 Jan 2021 21:43:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB2sg0B4bz3LTQ for ; Wed, 6 Jan 2021 21:43:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x730.google.com with SMTP id v126so3890333qkd.11 for ; Wed, 06 Jan 2021 13:43:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vOAxcEMyZpPRFcBUdvIL3jUHcIjJSRVF8PH5NDzLeTM=; b=YcOpeETtS5Z5QpMbt/zbIsw6rCXRTSeZVeLOXuV24nxxhl1XrQApR9CjzPtsABhWFk VNNuCujloslYcd9EYusNakiNid3G3lMQ3DF9tRzu0Sz62b/1OaDHW9xlotOnFWeDf37J P6aeDSI0EkKXlojv+4fCZgbSKuCVUpBrW0MAePBjECMJ/vvOFT266kwwntSOW6/8ribV MYcQH6+nljpL5e0yQNTWTutZPRwfDSQHVsV01wiOFyt2gNb7ju5UVFMguGabWcGR6gnG thz0AJxTp10Ac5XL/kZR928YgAHkXB+Oaya2DPXciy2u634g2ssvU/tdDz8ktLPk3JYr e9fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vOAxcEMyZpPRFcBUdvIL3jUHcIjJSRVF8PH5NDzLeTM=; b=DDx/Jlgzcw1SGlg22x9QekCbK5gIn7eogLL5fCoPDwgqbfr7AtJ3YEUByHbRCWM1iI ezBlvXd9YiQW/4dmFZ4NFYQEmHyn/KEgiXyU2bGatPqZTzKms4hq7mSaegCn7TTko31u xgHPN0Kody6S19mnAuX9dach2RbaAnXDFhy2IxrfaaCVoaYLbPBeL62eSKwvA40KgMW2 nzFfwTfv26ADwbBTdRcTwT2OUWpwGF+qymP9dK6MuZPg6KqiN8gtMCBWUmCG4nRaMqy0 uxA831pd2C3Uup/0dIRrsgbRYNPwt94e2L1mOhvGGJHQT61N4UH+iVOGj+q/3HMyAr8p Y94A== X-Gm-Message-State: AOAM531kWgNeMmdCq4OLwOyklSRsSc+E2pfgHxn/YUpzCkG6FrqGZ9uZ 0zjUwaoCf1THwwRFdA9nbfEc/o+cmpgKFR46KX6mXQ== X-Google-Smtp-Source: ABdhPJyMdjnWsrubKpFaIKtfO2M7JHy3spPmrxtuIjVtKfx5zh8D2/79k2DDGLyyMPMh1Qw2UbjQ9dR9a43C7e7naW4= X-Received: by 2002:a37:a614:: with SMTP id p20mr6315302qke.359.1609969382100; Wed, 06 Jan 2021 13:43:02 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 6 Jan 2021 14:42:50 -0700 Message-ID: Subject: Re: clang vs gcc warning flags To: Ryan Libby Cc: FreeBSD Hackers X-Rspamd-Queue-Id: 4DB2sg0B4bz3LTQ X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=YcOpeETt; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::730) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::730:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::730:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_LONG(1.00)[1.000]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::730:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 21:43:03 -0000 On Wed, Jan 6, 2021, 2:34 PM Ryan Libby wrote: > On Wed, Jan 6, 2021 at 12:38 PM Warner Losh wrote: > > > > > > > > On Wed, Jan 6, 2021 at 12:53 PM Ryan Libby wrote: > >> > >> One of the more annoying things about keeping the gcc build going is the > >> set of warnings that gcc acts on but clang only recognizes for > >> compatibility. As a common example, -Wredundant-decls has no effect > >> in clang, but will break the gcc build. There are a couple dozen such > >> flags [1][2], and a few of them are in our default set of warnings, such > >> as in sys/conf/kern.mk, these ones in CWARNFLAGS: > >> > >> - Wredundant-decls > >> - Wnested-externs > >> - Wmissing-include-dirs > >> > >> additionally some warnings are explicitly disabled for clang, but not > >> for gcc in CWARNEXTRA: > >> > >> - Wempty-body > >> - Wunused-function > >> > >> Similarly, in share/mk/bsd.sys.mk: > >> > >> - Winline (although, Wno-error'd) > >> - Wnested-externs > >> - Wredundant-decls > >> - Wold-style-definition > >> > >> So I suggest we harmonize these somewhat. > >> > >> - Wnested-externs I just do not understand. We have specified this > >> warning flag for some 25 years but to me it seems completely without > >> value. I suggest we just delete it. > > > > > > > > It's to prevent declaring functions inside of functions. why is that > bad? It only has scope of the function, and we've decided that's not > something we want in the tree... I'd say that the fact we don't have any in > the tree shows that it's useful at its intended purpose. > > > > What code is causing issues here? > > > > Well, but do we really care specifically about declaring extern > functions (or variables) inside of functions? Is that worse than the > usual fix, moving them just outside the function to file scope? > Yes. We have in the past. It's something we've judged to be wrong. Functions should be declared in headers, lest they be declared differently in different places. That is the harm this guards against. That is far from benign. Regarding _defining_ functions inside of functions, I agree we don't > want that. But -Wnested-externs doesn't warn about that; gcc produces > no warning for this with -Wnested-externs: > > void f(void) { void g(void){}; g(); } > > (Gcc produces a warning for that with -pedantic, which we don't use; and > clang produces an error even with no warning flags.) > > How I've actually seen "nested externs" used is usually to make > variables visible without including headers, for one reason or another. > Generally that's really poor design. Here are a couple examples: > > https://cgit.freebsd.org/src/commit/?id=d576ccdf01a4c6f8f02e8ed7e72290c729d68de6 > > https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/module/zfs/arc.c#n4721 > > As far as I understand, this actually does limit the scope of the > visibility of the extern declaration. I am not an expert in the > standard, I could be mistaken. > The problem comes when the signatures of these functions change.... I don't want to overstate the problem though. As I said, these are > annoyances, the solutions aren't hard. > They are annoyances that prevent worse problems. >> > >> - Wredundant-decls, I'm not sure about. I have never seen this detect > >> anything that will cause misbehavior, but most of the time that it > >> fires it does indicate some kind of genuine--but harmless--mistake. > > > > > > It also tends to prevent declarations that are the same on some > platforms, but different on others. > > > >> > >> - Wmissing-include-dirs doesn't seem to occur often and usually > >> indicates a genuine (but again harmless) mistake in a makefile. I > >> think we should keep it. > > > > > > I do too. > > > >> > >> - Wempty-body, Wunused-function. I'm not sure. These are proscriptive > >> about things that are not necessarily problems. We are apparently > >> already clean for them in the kernel gcc build, so perhaps we should > >> enable them for the kernel clang build. In any case, I think we > >> should bring these into agreement between clang and gcc, one way or > >> the other. > > > > > > We do not remove it. We include the warning, but don't make it fatal for > clang. > > > > Yes, "clean" was maybe not the right word. The end result is build > errors with gcc, no build errors with clang. The suggestion is just > that we do the same thing with both compilers, whether that be an error, > or a warning without an error. > We should clean that part up. I think we just disagree over how to do that. > Most of the clang neutering was due to bugs in the early days of clang. > I'm not sure anybody has systematically reviewed them since then. > > > >> Another sticking point may be contrib software. I think we generally > >> don't want to fail builds of contrib software for things that are > >> ultimately harmless. For bsd.sys.mk this could be accomplished by > >> enabling such warnings only at WARNS >= 6. For the kernel, we could > >> come up with some other mechanism. > > > > > > The kernel is different. 'Ultimately harmless' is a value judgement. > > > > I largely agree. What I'm hoping to break out of is situations where we > import contrib software and it breaks the gcc build for reasons that > don't affect the functionality of the built code, and then it's a pain > to fix because it's contrib software. > Yes. I agree the differences are a problem and need a solution. I'm not sure I agree with the specific suggestions is all. Warner >> > >> I'll put up a review soon for deleting -Wnested-externs unless there are > >> objections. If there is agreement about the others, I'll include those > >> too. > >> > >> Ryan > >> > >> [1] https://clang.llvm.org/docs/DiagnosticsReference.html > >> [2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Wed Jan 6 21:55:53 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 136B44E099F for ; Wed, 6 Jan 2021 21:55:53 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB38S70GDz3MSG for ; Wed, 6 Jan 2021 21:55:52 +0000 (UTC) (envelope-from rlibby@gmail.com) Received: by mail-qv1-f42.google.com with SMTP id j18so1971849qvu.3 for ; Wed, 06 Jan 2021 13:55:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=9ecAUeO9ppXptFMog4j9w2kp4JhgUAHxNWtHmExFa0I=; b=fbaO6lpAH7kZDutmTc55tzkzkOlLWM1CgIIECQgacBmTUbetMPJmWGlYdiLfGxdYPA kc/5mVSMgRA5fttJ4I7QSgGrObEyHLvQb63nUEpktqhmVCj6kdZIi69QjfV2UWa2msqS vbinCJvxYmRcmRAgC5+GLGvI/LoD+1vMLojDJMm3xGcCKkx9ZRs7IgdaZvZJG9SmdTTG lll66V99QtA+XReYbtmiX25hxgFTNYARlMIIYZNJZ9fjB1XxkYkFcnmk91GvFjOrhw2r PNlNoJQH0c/8FFzEtCuhYsrNQn8AiK2a/sXodqspnLoGKSHIwho1EwKpFzUPtsvv4EBH GW9w== X-Gm-Message-State: AOAM53285QdlTSaI7QZ4Pn+dq2n+gXphlz1UaONyGevwY9e2ZVY2ynWL /u5ZNDqCOdURwQ2p5RwAKcracbU6P3w= X-Google-Smtp-Source: ABdhPJzMvYcQxogN+Ll7opYujD85i22APdUjriSaKLkws885sRpwkK4W8JFrcZ7/4asVoGGLz23DIQ== X-Received: by 2002:a0c:a366:: with SMTP id u93mr6115369qvu.53.1609970151713; Wed, 06 Jan 2021 13:55:51 -0800 (PST) Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com. [209.85.222.181]) by smtp.gmail.com with ESMTPSA id c7sm2222648qkm.99.2021.01.06.13.55.51 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Jan 2021 13:55:51 -0800 (PST) Received: by mail-qk1-f181.google.com with SMTP id 19so3935701qkm.8 for ; Wed, 06 Jan 2021 13:55:51 -0800 (PST) X-Received: by 2002:a05:620a:12e5:: with SMTP id f5mr6272211qkl.331.1609970151175; Wed, 06 Jan 2021 13:55:51 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Ryan Libby Date: Wed, 6 Jan 2021 13:55:40 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: clang vs gcc warning flags To: Warner Losh Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4DB38S70GDz3MSG X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 21:55:53 -0000 On Wed, Jan 6, 2021 at 1:43 PM Warner Losh wrote: > > > > On Wed, Jan 6, 2021, 2:34 PM Ryan Libby wrote: >> >> On Wed, Jan 6, 2021 at 12:38 PM Warner Losh wrote: >> > >> > >> > >> > On Wed, Jan 6, 2021 at 12:53 PM Ryan Libby wrote: >> >> >> >> One of the more annoying things about keeping the gcc build going is = the >> >> set of warnings that gcc acts on but clang only recognizes for >> >> compatibility. As a common example, -Wredundant-decls has no effect >> >> in clang, but will break the gcc build. There are a couple dozen suc= h >> >> flags [1][2], and a few of them are in our default set of warnings, s= uch >> >> as in sys/conf/kern.mk, these ones in CWARNFLAGS: >> >> >> >> - Wredundant-decls >> >> - Wnested-externs >> >> - Wmissing-include-dirs >> >> >> >> additionally some warnings are explicitly disabled for clang, but not >> >> for gcc in CWARNEXTRA: >> >> >> >> - Wempty-body >> >> - Wunused-function >> >> >> >> Similarly, in share/mk/bsd.sys.mk: >> >> >> >> - Winline (although, Wno-error'd) >> >> - Wnested-externs >> >> - Wredundant-decls >> >> - Wold-style-definition >> >> >> >> So I suggest we harmonize these somewhat. >> >> >> >> - Wnested-externs I just do not understand. We have specified this >> >> warning flag for some 25 years but to me it seems completely witho= ut >> >> value. I suggest we just delete it. >> > >> > >> > >> > It's to prevent declaring functions inside of functions. why is that b= ad? It only has scope of the function, and we've decided that's not somethi= ng we want in the tree... I'd say that the fact we don't have any in the tr= ee shows that it's useful at its intended purpose. >> > >> > What code is causing issues here? >> > >> >> Well, but do we really care specifically about declaring extern >> functions (or variables) inside of functions? Is that worse than the >> usual fix, moving them just outside the function to file scope? > > > Yes. We have in the past. It's something we've judged to be wrong. Functi= ons should be declared in headers, lest they be declared differently in dif= ferent places. That is the harm this guards against. That is far from benig= n. > >> Regarding _defining_ functions inside of functions, I agree we don't >> want that. But -Wnested-externs doesn't warn about that; gcc produces >> no warning for this with -Wnested-externs: >> >> void f(void) { void g(void){}; g(); } >> >> (Gcc produces a warning for that with -pedantic, which we don't use; and >> clang produces an error even with no warning flags.) >> >> How I've actually seen "nested externs" used is usually to make >> variables visible without including headers, for one reason or another. > > > Generally that's really poor design. > >> Here are a couple examples: >> https://cgit.freebsd.org/src/commit/?id=3Dd576ccdf01a4c6f8f02e8ed7e72290= c729d68de6 >> https://cgit.freebsd.org/src/tree/sys/contrib/openzfs/module/zfs/arc.c#n= 4721 >> >> As far as I understand, this actually does limit the scope of the >> visibility of the extern declaration. I am not an expert in the >> standard, I could be mistaken. > > > The problem comes when the signatures of these functions change.... > Okay, I take your points and I will leave -Wnested-externs alone. I guess as a final point, I would just suggest that non-nested externs at file scope in .c files seems to be just as harmful as nested externs. However, neither compiler seems to provide us a way to construct a warning about that. Would you have any general advice for dealing with this in contrib software, such as the openzfs example? What I ended up doing was disabling the warning for code that uses OPENZFS_CFLAGS. >> I don't want to overstate the problem though. As I said, these are >> annoyances, the solutions aren't hard. > > > They are annoyances that prevent worse problems. > >> >> >> >> - Wredundant-decls, I'm not sure about. I have never seen this dete= ct >> >> anything that will cause misbehavior, but most of the time that it >> >> fires it does indicate some kind of genuine--but harmless--mistake= . >> > >> > >> > It also tends to prevent declarations that are the same on some platfo= rms, but different on others. >> > >> >> >> >> - Wmissing-include-dirs doesn't seem to occur often and usually >> >> indicates a genuine (but again harmless) mistake in a makefile. I >> >> think we should keep it. >> > >> > >> > I do too. >> > >> >> >> >> - Wempty-body, Wunused-function. I'm not sure. These are proscript= ive >> >> about things that are not necessarily problems. We are apparently >> >> already clean for them in the kernel gcc build, so perhaps we shou= ld >> >> enable them for the kernel clang build. In any case, I think we >> >> should bring these into agreement between clang and gcc, one way o= r >> >> the other. >> > >> > >> > We do not remove it. We include the warning, but don't make it fatal f= or clang. >> > >> >> Yes, "clean" was maybe not the right word. The end result is build >> errors with gcc, no build errors with clang. The suggestion is just >> that we do the same thing with both compilers, whether that be an error, >> or a warning without an error. > > > We should clean that part up. I think we just disagree over how to do tha= t. > >> > Most of the clang neutering was due to bugs in the early days of clang= . I'm not sure anybody has systematically reviewed them since then. >> > >> >> Another sticking point may be contrib software. I think we generally >> >> don't want to fail builds of contrib software for things that are >> >> ultimately harmless. For bsd.sys.mk this could be accomplished by >> >> enabling such warnings only at WARNS >=3D 6. For the kernel, we coul= d >> >> come up with some other mechanism. >> > >> > >> > The kernel is different. 'Ultimately harmless' is a value judgement. >> > >> >> I largely agree. What I'm hoping to break out of is situations where we >> import contrib software and it breaks the gcc build for reasons that >> don't affect the functionality of the built code, and then it's a pain >> to fix because it's contrib software. > > > Yes. I agree the differences are a problem and need a solution. I'm not s= ure I agree with the specific suggestions is all. > > Warner > Great. I am not attached to the specific suggestions. >> >> >> >> I'll put up a review soon for deleting -Wnested-externs unless there = are >> >> objections. If there is agreement about the others, I'll include tho= se >> >> too. >> >> >> >> Ryan >> >> >> >> [1] https://clang.llvm.org/docs/DiagnosticsReference.html >> >> [2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html >> >> _______________________________________________ >> >> freebsd-hackers@freebsd.org mailing list >> >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd= .org" From owner-freebsd-hackers@freebsd.org Wed Jan 6 22:51:53 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8D9754E2813 for ; Wed, 6 Jan 2021 22:51:53 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "m5p.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB4P44zDRz3gyl for ; Wed, 6 Jan 2021 22:51:52 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::26] (court.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:26]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id 106MpewU085463 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 6 Jan 2021 17:51:45 -0500 (EST) (envelope-from george+freebsd@m5p.com) Subject: Re: clang vs gcc warning flags (off-topic anecdote) To: freebsd-hackers@freebsd.org References: From: George Mitchell Message-ID: Date: Wed, 6 Jan 2021 17:51:39 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="x09t1S8ak2I65BKJFV6E7yx9V9v7dArhZ" X-Spam-Status: No, score=-0.3 required=10.0 tests=HELO_MISC_IP, HELO_NO_DOMAIN, NICE_REPLY_A autolearn=unavailable autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mattapan.m5p.com X-Rspamd-Queue-Id: 4DB4P44zDRz3gyl X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of george@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=george@m5p.com X-Spamd-Result: default: False [-3.40 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.999]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[74.104.188.4:from]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[freebsd]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[m5p.com]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[74.104.188.4:from:127.0.2.255]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 22:51:53 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --x09t1S8ak2I65BKJFV6E7yx9V9v7dArhZ Content-Type: multipart/mixed; boundary="yzYODdsOtE9aTAX6MlBfQMsfGws0ya0R0"; protected-headers="v1" From: George Mitchell To: freebsd-hackers@freebsd.org Message-ID: Subject: Re: clang vs gcc warning flags (off-topic anecdote) References: In-Reply-To: --yzYODdsOtE9aTAX6MlBfQMsfGws0ya0R0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 1/6/21 4:34 PM, Ryan Libby wrote: > [...] > How I've actually seen "nested externs" used is usually to make > variables visible without including headers, for one reason or another.= > [...] My daughter, who knew quite a bit about programming before going to college, had an instructor who *insisted* that his students declare all their external functions inside the function calling them, even in C code. It appears he didn't really know C and had only taught Pascal up to that point. -- George --yzYODdsOtE9aTAX6MlBfQMsfGws0ya0R0-- --x09t1S8ak2I65BKJFV6E7yx9V9v7dArhZ Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAl/2PvsFAwAAAAAACgkQwRES3m+p4fmM wBAAiaE93scP13L12JKb8GVNKvE7WXphAbJ7r65I9mTcxbeIy0gkzpr9huz0zP/i6YfSEyVh00nR Q3GdI5aYiqXs/qVcp1yYkMJ4JPRszXhXTOq0Dgnlcy8MVem8MmPgerCQZtGucoZTEF7DCgov7ES/ kYTgWB0Ndx7w5BbyS76rykwmGt3jAJE7saQllJ+vJ6jlbHIC90OSsuPOqIQlp9I4V+5mXsqqZ5Hb o6K7YvU+JK6VpivLiJyZS+sqkNQgTG1AW55cEfD+gbzLR+AVm3cKo4euc+9nZaEaUmJda2XwTCF8 nUjXVW5xaYvkJQO3RBjvNBSgx/4V8cVafgjhPKIW3gzHuxFFHkhoo9wwBaMARnqhz68re+4sF31u bABRaWOb0nVvTRp7JECIrAdXLWMEwITJGgZdPD5Z7LWjiw816LwBMK77I3mZEipXaG5beSQKk5I2 ruX/YPlbQsHrJLI2lALHFF42feDpHtuaT6BaHu9KWd81Mh74Kobxl5K++Plvf0LoFi/7LQFtrlNc XVisi9yROXj5pFOiJ7Tn3phS2cqX8E4P+hezlonUH0xIs2SevtFabbKM8U8g2JsykD23lleUmI+h yMKv9lb9NTH9eH0OQ5EhAuiSZuIV1kzfMrqdY/wVK58dGdPPwxkrqaKe30xOqewLzwYdX6zAgCTL km0= =eQ5H -----END PGP SIGNATURE----- --x09t1S8ak2I65BKJFV6E7yx9V9v7dArhZ-- From owner-freebsd-hackers@freebsd.org Wed Jan 6 23:19:13 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 003074E48D6 for ; Wed, 6 Jan 2021 23:19:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB50Z47klz3jxL; Wed, 6 Jan 2021 23:19:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 106NJ20M099857 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 7 Jan 2021 01:19:05 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 106NJ20M099857 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 106NJ2mv099856; Thu, 7 Jan 2021 01:19:02 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 7 Jan 2021 01:19:02 +0200 From: Konstantin Belousov To: Ryan Libby Cc: FreeBSD Hackers Subject: Re: clang vs gcc warning flags Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4DB50Z47klz3jxL X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_SPF_SOFTFAIL(0.00)[~all:c]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.999]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(1.00)[1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-hackers]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 23:19:13 -0000 On Wed, Jan 06, 2021 at 01:37:53PM -0800, Ryan Libby wrote: > On Wed, Jan 6, 2021 at 12:06 PM Konstantin Belousov wrote: > > > > On Wed, Jan 06, 2021 at 11:53:13AM -0800, Ryan Libby wrote: > > > - Wnested-externs I just do not understand. We have specified this > > > warning flag for some 25 years but to me it seems completely without > > > value. I suggest we just delete it. > > I suspect this warning is to flag style(9) violations. > > Which aspect do you mean? I don't see it mentioned explicitly, but I > know some of style(9) is reading between the lines. > > If it's that objects with external linkage should be declared in > headers, I think that doesn't address most of the cases where extern is > actually used in our .c files. It is more of agreement to put declarations with external linkage at top level, not neccessary in headers. But yes, people usually get follow-ups to the commits when something is declared in .c (or did got follow-ups until recent times). And I think this is a reasonable rule for uniformity. I do not believe we have any instance of use of nested function definition in base, and I do not see why would we need this extension. For the same reason, for not being needed, Apple' blocks did not survived (but did damaged our ABI somewhat). > > Maybe the answer should be different for contrib software (where we > don't generally enforce style)? Of course we cannot enforce style(9) on contrib, it is not applicable by any means. From owner-freebsd-hackers@freebsd.org Thu Jan 7 19:42:32 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B44DF4DA1A0 for ; Thu, 7 Jan 2021 19:42:32 +0000 (UTC) (envelope-from me+freebsd@igalic.co) Received: from mail2.protonmail.ch (mail2.protonmail.ch [185.70.40.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "SwissSign Server Gold CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DBc8619W8z3lGF for ; Thu, 7 Jan 2021 19:42:29 +0000 (UTC) (envelope-from me+freebsd@igalic.co) Date: Thu, 07 Jan 2021 19:42:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igalic.co; s=protonmail2; t=1610048541; bh=kOCZtaP87SSo6KVN909SxiVloHfWoIAC8LER0vMWn1I=; h=Date:To:From:Reply-To:Subject:From; b=lv7usb8w/sVu/MwRncX8B2yRnO9HKJyMSczXNkQlFdq5X5quB6v761lFfSLimsPLQ xTXJ4gK0b4IDS2WEG1j/1LPZVUOMR6o+joMY6hTqSnSQ1zPclP4INWSyw/l6HkZ38b vbOH/yDQetluHgbECbtcUeLVTjfAWriWVepfedfhK1tOAzEQkBlZsKxMKBcLHTWIh/ DHxuilfu0Cs4BYFqIneJwkYd6ph5sBWqPHLmIdlA6ZD3raFK7eBg59uGHFFMNOx478 j7xW0LTtPk6K00ksydQKqovFc21OVunBuA3raTlx3W6Z0VLp8N34rzGmoCO3BLe+vG NP8eO9G0+A8EQ== To: freebsd-hackers From: =?utf-8?Q?Mina_Gali=C4=87?= Reply-To: =?utf-8?Q?Mina_Gali=C4=87?= Subject: A Box API without the Sand Message-ID: MIME-Version: 1.0 X-Spam-Status: No, score=1.8 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HTML_MESSAGE,NEW_DOMAIN_28D, URIBL_FRESH_28D_SURBL shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4DBc8619W8z3lGF X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=igalic.co header.s=protonmail2 header.b=lv7usb8w; dmarc=none; spf=pass (mx1.freebsd.org: domain of me@igalic.co designates 185.70.40.22 as permitted sender) smtp.mailfrom=me@igalic.co X-Spamd-Result: default: False [-0.60 / 15.00]; HAS_REPLYTO(0.00)[me+freebsd@igalic.co]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[igalic.co:+]; MIME_BASE64_TEXT(0.10)[]; NEURAL_HAM_SHORT(-0.83)[-0.827]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_MIXED_CHARSET(0.62)[subject]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[freebsd]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[igalic.co:s=protonmail2]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[me]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_PHPMAILER_SIG(0.00)[]; DMARC_NA(0.00)[igalic.co]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_LONG(1.00)[1.000]; RWL_MAILSPIKE_VERYGOOD(0.00)[185.70.40.22:from]; MAILMAN_DEST(0.00)[freebsd-hackers] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2021 19:42:32 -0000 SGVsbG8gRnJlZUJTRCBkZXZlbG9wZXJzCgpJ4oCZbSB0cnlpbmcgdG8gZmluZCBhbiAoZXhpc3Rp bmcpIEFQSSBvbiBGcmVlQlNEIHdoaWNoIGFsbG93cyBtZSB0byBib3ggaW4gYSBwcm9jZXNzIGFu ZCBhbGwgb2YgaXRzIGRlc2NlbmRhbnRzLiBUaGF0IGJveCBzaG91bGQgYmUgZWFzeSB0byBpZGVu dGlmeSwgYW5kIGFsbG93IHByb2Nlc3MgbWFuYWdlbWVudCwgYWNjb3VudGluZyBhbmQgcmVzb3Vy Y2UgbWFuYWdlbWVudC4KClVuZm9ydHVuYXRlbHkgYWxsIGJveGVzIEnKvG0gZmluZGluZyBjb21l IHByZS1maWxsZWQgd2l0aCBTYW5kIHRoYXTigJlzIGltcG9zc2libGUgdG8gZ2V0IHJpZCBvZiwg b3IgbGFjayBzb21lIG9mIHRoZSBvdGhlciByZXF1aXJlbWVudHMuCgpBbiBhbmFsb2d1ZSBmb3Ig c3VjaCBhbiBBUEkgY2FuIGJlIGZvdW5kIGluIFNvbGFyaXMgQ29udHJhY3RzIGFuZCBMaW51eCBD Z3JvdXBzLgoKQ2dyb3VwcyBhcmUgZXh0ZW5zaXZlbHkgdXNlZCBpbiBtYW55IGluaXQgaW1wbGVt ZW50YXRpb25zIG9uIExpbnV4IHRvIGNvbnRhaW4sIGlkZW50aWZ5IGFuZCBjb250cm9sIHNlcnZp Y2VzLgpQZXJzb25hbGx5LCBpIHVzZWQgY29udHJhY3RzIG9uIFNvbGFyaXMgZm9yIHRoYXQgdmVy eSBwdXJwb3NlLCBpbiBhbiBpbi1ob3VzZSBkZXBsb3ltZW50IHNvZnR3YXJlLCBsb25nIGJlZm9y ZSBzeXN0ZW1kIHdhcyBoaXAuCgpJIHdvdWxkIGxpa2UgdG8gZG8gc29tZXRoaW5nIGxpa2UgdGhh dCBpbiBpbml0L3JjIG9uIEZyZWVCU0QuIEJ1dCBpIGNhbuKAmXQgc2VlbSB0byBmaW5kIGFuIEFQ SSB0aGF0IGZpdHMgdGhlIGJpbGwuCgpUaGUgbm9ybWFsIFBPU0lYIHByb2Nlc3MgbWFuYWdlbWVu dCBmYWNpbGl0aWVzLCBsaWtlIHByb2Nlc3MgZ3JvdXBzIGFuZCBwcm9jZXNzIHNlc3Npb25zIGFy ZSB0b28gZWFzeSB0byBlc2NhcGUuIEJ1dCBnaXZlbiB0aGVpciBwcmltYXJ5IHVzZSBpcyBqb2Ig Y29udHJvbCwgdGhhdOKAmXMgYSBmZWF0dXJlLCBub3QgYSBidWcuIEVpdGhlciB3YXksIHRoZXkg Y2FuIGJlIHJ1bGVkIG91dC4KClRoZSBjbG9zZXN0IHdlIGhhdmUgYXJlIGphaWwoMikgYW5kIENh cHNpY3VtKDQpLCBhcyB3ZWxsIGFzIHRoZSBmdW5jdGlvbnMgc3VwcGxlbWVudGluZyB0aGUgbGF0 dGVyLCBzdWNoIGFzIHBkZm9yaygyKSAmIGNvLgoKSmFpbHMgYXJlIHJlYWxseSBnb29kIGF0IGNv bnRhaW5pbmcgYW5kIGlkZW50aWZ5aW5nIHByb2Nlc3MgZ3JvdXBzLCBhbmQgZm9yIG1hbmFnaW5n IHRoZWlyIHJlc291cmNlcy4KSG93ZXZlciwgamFpbGluZyBldmVyeSBzaW5nbGUgZGFlbW9uIG9u IGEgc3lzdGVtLCB3aGlsZSBkZXNpcmFibGUgYnkgc29tZSwgd291bGQgbWFrZSBtYW55IGRhZW1v bnMgdXNlbGVzcy4gc3NoZCBhbmQgbW9uaXRvcmluZyBzb2Z0d2FyZSBhcmUgbXkgcHJpbWUgZXhh bXBsZXMsIGJ1dCB0aGVyZSBhcmUgcHJvYmFibHkgbWFueSBtb3JlLgoKQ2Fwc2ljdW0sIGFnYWlu LCBzZWVtcyBtb3JlIGNvbmNlcm5lZCB3aXRoIHRoZSBTYW5kIHRoYW4gdGhlIGJveDogaSBjYW5u b3Qgc2VlbSB0byBmaW5kIGFueXRoaW5nIHRoYXQgd291bGQgaGVscCBpZGVudGlmeSBhIHByb2Nl c3MgZ3JvdXAuIEV4Y2VwdCBmb3IgcGRmb3JrKDIpJ3MgcHJvY2Rlc2MoNCkuCgpGb3J0dWl0b3Vz bHksIHRoZXNlIGNhbiBiZSB1c2VkIGluZGVwZW5kZW50bHkgb2YgQ2Fwc2ljdW0sIHNvIHRoYXQg bWVhbnMgd2UgaGF2ZSBhbiBpZGVudGl0eSBhbmQgY29udHJvbCwgYnV0IGkgY2Fu4oCZdCBmaW5k IGFueSBpbmZvcm1hdGlvbiBvbiB3aGV0aGVyIGFsbCBkZXNjZW50YW50cyB3aWxsIHJlbWFpbiB3 aXRoaW4gdGhlIHNhbWUgcHJvY2Rlc2MuCgpUaGF04oCZcyBhcyBmYXIgYXMgbXkgcmVzZWFyY2gg aGFzIGdvdHRlbiBtZSwgc28gYmVmb3JlIGkgZGlnIG15c2VsZiBhbnkgZGVlcGVyLCBp4oCZZCBs aWtlIHRvIGFzayBmb3IgeW91ciBoZWxwLgoKQW5kLCBtb3N0IGltcG9ydGFudGx5LCBpZiBubyBz dWNoIEFQSSBleGlzdHMgeWV0LCB3aGVyZSB0byBzdGFydCBpbiBhZGRpbmcgaXQsIG9yIG1vZGlm eWluZyBhbiBleGlzdGluZyBBUEkgdG8gc3VpdCB0aGVzZSByZXF1aXJlbWVudHM/CgpCZXN0IHJl Z2FyZHMsCgpNaW5hCmh0dHBzOi8vYWxwaGEucGtnYmFzZS5saXZlLw== From owner-freebsd-hackers@freebsd.org Thu Jan 7 20:14:34 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D51B74DAB6F for ; Thu, 7 Jan 2021 20:14:34 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net (static-24-113-41-81.wavecable.com [24.113.41.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "ultimatedns.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DBcs62rN0z3n4m for ; Thu, 7 Jan 2021 20:14:33 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.16.1/8.16.1) with ESMTP id 107KEZmf077084; Thu, 7 Jan 2021 12:14:41 -0800 (PST) (envelope-from bsd-lists@bsdforge.com) MIME-Version: 1.0 Date: Thu, 07 Jan 2021 12:14:35 -0800 From: Chris To: freebsd-hackers Cc: =?UTF-8?Q?Mina_Gali=C4=87?= Subject: Re: A Box API without the Sand In-Reply-To: References: User-Agent: UDNSMS/17.0 Message-ID: X-Sender: bsd-lists@bsdforge.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DBcs62rN0z3n4m X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; ASN(0.00)[asn:11404, ipnet:24.113.0.0/16, country:US]; local_wl_ip(0.00)[24.113.41.81] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jan 2021 20:14:34 -0000 On 2021-01-07 11:42, Mina Galić wrote: > Hello FreeBSD developers > > I’m trying to find an (existing) API on FreeBSD which allows me to box in a > process and all of its descendants. That box should be easy to identify, and > allow > process management, accounting and resource management. > > Unfortunately all boxes Iʼm finding come pre-filled with Sand that’s > impossible to > get rid of, or lack some of the other requirements. > > An analogue for such an API can be found in Solaris Contracts and Linux > Cgroups. > > Cgroups are extensively used in many init implementations on Linux to > contain, > identify and control services. > Personally, i used contracts on Solaris for that very purpose, in an > in-house > deployment software, long before systemd was hip. > > I would like to do something like that in init/rc on FreeBSD. But i can’t > seem to > find an API that fits the bill. > > The normal POSIX process management facilities, like process groups and > process > sessions are too easy to escape. But given their primary use is job control, > that’s a feature, not a bug. Either way, they can be ruled out. > > The closest we have are jail(2) and Capsicum(4), as well as the functions > supplementing the latter, such as pdfork(2) & co. > > Jails are really good at containing and identifying process groups, and for > managing their resources. > However, jailing every single daemon on a system, while desirable by some, > would > make many daemons useless. sshd and monitoring software are my prime > examples, but > there are probably many more. > > Capsicum, again, seems more concerned with the Sand than the box: i cannot > seem to > find anything that would help identify a process group. Except for > pdfork(2)'s > procdesc(4). > > Fortuitously, these can be used independently of Capsicum, so that means we > have > an identity and control, but i can’t find any information on whether all > descentants will remain within the same procdesc. > > That’s as far as my research has gotten me, so before i dig myself any > deeper, i’d > like to ask for your help. > > And, most importantly, if no such API exists yet, where to start in adding > it, or > modifying an existing API to suit these requirements? If I understand you correctly. I'm unable to understand where jail(8) doesn't satisfy your needs. Nothing prevents you from running multiple daemons within a singe jail. You can easily control access restrictions in and out of the jail(s) via pf(4). Apologies if I didn't correctly understand your needs/intentions. --Chris > > Best regards, > > Mina > https://alpha.pkgbase.live/ > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Fri Jan 8 14:51:34 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 16DE44CBFE2 for ; Fri, 8 Jan 2021 14:51:34 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [IPv6:2001:470:1:474::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DC5dx5K3lz4Tqy for ; Fri, 8 Jan 2021 14:51:33 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 989C2252FA for ; Fri, 8 Jan 2021 14:51:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 989C2252FA Subject: Re: A Box API without the Sand To: freebsd-hackers@freebsd.org References: From: Allan Jude Message-ID: <31d89aed-4871-a42a-fbc8-2be617381a95@freebsd.org> Date: Fri, 8 Jan 2021 09:51:26 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DC5dx5K3lz4Tqy X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; local_wl_from(0.00)[freebsd.org] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2021 14:51:34 -0000 On 2021-01-07 14:42, Mina Galić wrote: > Hello FreeBSD developers > > I’m trying to find an (existing) API on FreeBSD which allows me to box in a process and all of its descendants. That box should be easy to identify, and allow process management, accounting and resource management. > > Unfortunately all boxes Iʼm finding come pre-filled with Sand that’s impossible to get rid of, or lack some of the other requirements. > > An analogue for such an API can be found in Solaris Contracts and Linux Cgroups. > > Cgroups are extensively used in many init implementations on Linux to contain, identify and control services. > Personally, i used contracts on Solaris for that very purpose, in an in-house deployment software, long before systemd was hip. > > I would like to do something like that in init/rc on FreeBSD. But i can’t seem to find an API that fits the bill. > > The normal POSIX process management facilities, like process groups and process sessions are too easy to escape. But given their primary use is job control, that’s a feature, not a bug. Either way, they can be ruled out. > > The closest we have are jail(2) and Capsicum(4), as well as the functions supplementing the latter, such as pdfork(2) & co. > > Jails are really good at containing and identifying process groups, and for managing their resources. > However, jailing every single daemon on a system, while desirable by some, would make many daemons useless. sshd and monitoring software are my prime examples, but there are probably many more. > > Capsicum, again, seems more concerned with the Sand than the box: i cannot seem to find anything that would help identify a process group. Except for pdfork(2)'s procdesc(4). > > Fortuitously, these can be used independently of Capsicum, so that means we have an identity and control, but i can’t find any information on whether all descentants will remain within the same procdesc. > > That’s as far as my research has gotten me, so before i dig myself any deeper, i’d like to ask for your help. > > And, most importantly, if no such API exists yet, where to start in adding it, or modifying an existing API to suit these requirements? > > Best regards, > > Mina > https://alpha.pkgbase.live/ > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > A few different ideas in this area have been discussed before. I remember a conversation with John Baldwin at one of the FreeBSD Storage Summits, about the idea of a 'null' jail. Basically a jail with no restrictions, one that uses / instead of a chroot, inherits the network and shared memory namespaces, etc. The main advantage to this abstraction is that it builds on a lot of existing functionality. It gives you a top level identifier (jail id), and can already have a lot of policies applied to it (cpuset, RCTL resource limits, etc) You can mostly already control what devices from /dev are visible. I think what we would need to do is define what else needs to be different from a jail. Like do we need the ability to allow processes inside the container to see the processes from the host? There may be a few places that will be more problematic. ZFS automatically enforces its restrictions if your jail ID is not 0. -- Allan Jude