From nobody Wed Sep 7 14:55:24 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MN50J204Kz4cbgd for ; Wed, 7 Sep 2022 14:55:28 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MN50H4CfVz3T2x; Wed, 7 Sep 2022 14:55:27 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from shw-obgw-4002a.ext.cloudfilter.net ([10.228.9.250]) by cmsmtp with ESMTP id VruIoaNOBS8WrVwScowSPc; Wed, 07 Sep 2022 14:55:26 +0000 Received: from spqr.komquats.com ([70.66.148.124]) by cmsmtp with ESMTPA id VwSaojPanC3uhVwSboN3Q5; Wed, 07 Sep 2022 14:55:26 +0000 X-Authority-Analysis: v=2.4 cv=a6MjSGeF c=1 sm=1 tr=0 ts=6318b0de a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=kj9zAlcOel0A:10 a=xOM3xZuef0cA:10 a=pGLkceISAAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=EkcXrb_YAAAA:8 a=9rOpkhTHWMKq_9iKNLkA:9 a=CjuIK1q_8ugA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id A715E768; Wed, 7 Sep 2022 07:55:24 -0700 (PDT) Received: by slippy.cwsent.com (Postfix, from userid 1000) id 7C9881FB; Wed, 7 Sep 2022 07:55:24 -0700 (PDT) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Alan Somers cc: Konstantin Belousov , FreeBSD CURRENT Subject: Re: Header symbols that shouldn't be visible to ports? In-reply-to: References: Comments: In-reply-to Alan Somers message dated "Mon, 05 Sep 2022 08:41:58 -0600." List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 07 Sep 2022 07:55:24 -0700 Message-Id: <20220907145524.7C9881FB@slippy.cwsent.com> X-CMAE-Envelope: MS4xfPfgOUSt8YIbnSN/CQ9oauMdEgdszkYNHkGMdlBVH/6WXhE97DP0bT6ehrK9ZUzB7nZiI0/HfTMBEMt1/NKwCjUnlwMYFxgqF4mhaJuJySvb0eC552fo wpTymBzt/VEO92vVaWXjYL9et4ccNRPJqDLZo9kzURs+cR02hT0brGlJF1clfsfoMWm+TeQRH+nH16nUuYfQ3DDW/MsNRJCH0l4ydGe0/fTT2M2Iwq/yKcKR v8ESdT9JSbFLNZlW9POxdLP6wvKoZOc7bQplsid3jA0= X-Rspamd-Queue-Id: 4MN50H4CfVz3T2x X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 3.97.99.32) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-0.80 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_LONG(-1.00)[-0.998]; MV_CASE(0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[3.97.99.32:from]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_SPF_NA(0.00)[no SPF record]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[cschubert.com: no valid DMARC record]; TO_MATCH_ENVRCPT_SOME(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N In message , Alan Somers writes: > On Sat, Sep 3, 2022 at 11:10 PM Konstantin Belousov wro > te: > > > > On Sat, Sep 03, 2022 at 10:19:12AM -0600, Alan Somers wrote: > > > Our /usr/include headers define a lot of symbols that are used by > > > critical utilities in the base system like ps and ifconfig, but aren't > > > stable across major releases. Since they aren't stable, utilities > > > built for older releases won't run correctly on newer ones. Would it > > > make sense to guard these symbols so they can't be used by programs in > > > the ports tree? There is some precedent for that, for example > > > _WANT_SOCKET and _WANT_MNTOPTNAMES. > > _WANT_SOCKET is clearly about exposing parts of the kernel definitions > > for userspace code that wants to dig into kernel structures. Similarly > > for _WANT_MNTOPTNAMES, but in fact this thing is quite stable. The > > definitions are guarded by additional defines not due to their instability, > > but because using them in userspace requires (much) more preparation from > > userspace environment, which is either not trivial (_WANT_SOCKET) or > > contradicts to standartized use of the header (_WANT_MNTOPTNAMES + > > sys/mount.h). > > > > > > > > I'm particular, I'm thinking about symbols like the following: > > > MINCORE_SUPER > > Why this symbol should be hidden? It is implementation-defined and > > intended to be exposed to userspace. All MINCORE_* not only MINCORE_SUPER > > are under BSD_VISIBLE braces, because POSIX does not define the symbols. > > Because it isn't stable. It changed for example in rev 847ab36bf22 > for 13.0. Programs using the older value (including virtually every > Rust program) won't work on 13.0 and later. > > > > > > TDF_* > > These symbols coming from non-standard header sys/proc.h. If userspace > > includes the header, it is already outside any formal standard, and I > > do not see a reason to make the implementation more convoluted there. > > > > > PRI_MAX* > > > PRI_MIN* > > > PI_*, PRIBIO, PVFS, etc > > > IFCAP_* > > These are all implementation-specific and come from non-standard headers, > > unless I am mistaken, then please correct me. > > > > > RLIM_NLIMITS > > > IFF_* > > Same. > > > > > *_MAXID > > This is too broad. > > I'm talking about symbols like IPV6CTL_MAXID, which record the size of > sysctl lists. Obviously, these symbols can't be stable, and probably > aren't useful outside of the base system. > > > > > > > > > Clearly delineating private symbols like this would ease the > > > maintenance burden on languages that rely on FFI, like Ruby and Rust. > > > FFI basically assumes that symbols once defined will never change. > > > > Why e.g. sys/proc.h is ever consumed by FFI wrappers? > > I should add a little detail. Rust uses FFI to access C functions, > and #define'd constants are redefined in the Rust bindings. For most > Rust programs, the build process doesn't check the contents of > /usr/include in any way. Instead, all of that stuff is hard-coded in > the Rust bindings. That makes cross-compiling a breeze! But it does > cause problems when the C library changes. Adding a new symbol, like > copy_file_range, isn't so bad. If your Rust program doesn't use it, > then the Rust binding will become an unused symbol and get eliminated > by the linker. If your Rust program does use it OTOH, then it will be > resolved by the dynamic linker at runtime - if you're running on > FreeBSD 13 or newer. Otherwise, your program will fail to run. A > bigger problem is with symbols that change. For example, the 64-bit > inode stuff. Rust programs still use a FreeBSD 11 ABI (we're working > on that). But other symbols change more frequently. Things like > PRI_MAX_REALTIME can change between any two releases. That creates a > big maintenance burden to keep track of them in the FFI bindings. And > they also aren't very useful in cross-compiled programs targeting a > FreeBSD 11 ABI. Instead, they really need to have bindings > automatically generated at build time. That's possible, but it's not > the default. This is exactly what happened with DMD D. When 64-bit statfs was introduced all DMD D compiled programs failed to run and recompiling didn't help. The DMD upstream failed to understand the problem. Eventually the port had to be removed. > > So what the Rust community really needs is a way to know which symbols > will be stable across releases, and which might vary. Are you > suggesting that anything from a non-POSIX header file should be > considered variable? > Rust and every other community. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0