From nobody Sat Feb 28 11:55:45 2026 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fNNx673f9z6T5g0 for ; Sat, 28 Feb 2026 11:57:46 +0000 (UTC) (envelope-from ivy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fNNx66Xrcz3D6Q for ; Sat, 28 Feb 2026 11:57:46 +0000 (UTC) (envelope-from ivy@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772279866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yGppljTkJiK9S4GI1XTAfePvC7pkJPcAXBexmCyolHE=; b=xTdU4SxvxCdwPHxAI6tT6FjaJPud0+oLWAqkiwhTFyI3eKdYXWN+8fUbSVwKQfZs1scplU f6vNMugv3VSRrCxwzPSI25o6XsjbfO05clwv5lbGiW6HSucgT3pBqqVJktvdE4Spo6O63Z 4C4FxTG9MRdhziECzPtqBnXcyB5Z7P4RTatH387+eoVtuO3RmLmSoCaJV0Uv+z3x6IV3du mkV0Zi7DqLPK9FMs8ePmdKLong9jmJsKiP8wzXcFqodcEV6KqGs6tC5v40LALGfU15aesE blmyuNF1fDYrqDbKzTspfRkuakGFOb7fdZhcfyq2dGQ841TcZAKekaBQ13cuIg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1772279866; a=rsa-sha256; cv=none; b=bdI2B3jhma5SjVYFWkP+Nl+iUd9iNrSiNMgwZ2yfnpXI6t5EyR0LFceShKyTOndsZyBdB9 zUNdKj+V857jQpQlHdjs3iWozxYEbqnqM3pGuB2zQdDyKLs/y4gsge/QKEcJ+fG7PfmEN/ vy2mra3tV5ijEiKjJCt5vRI1QkdFv71RU/enuj+zTHTmw2sfvFHYCq7spM4j+744/cLMaW PyBY1+U9+1DcLTJGa3/6AXObFb7twqsDQje/BLvxrs78DlTxDm8kobUUwAf2UftJ0Jbbm0 bSOFt1epNwF9yFUsgCf1GFemroM+YAMUzx/tOiOY59KCS0tkFF8uTo7Q+WJ5XA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1772279866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yGppljTkJiK9S4GI1XTAfePvC7pkJPcAXBexmCyolHE=; b=BWIhtpF82Jej7S97xMMWdJWlSu4Maxu9vRo84/17JGu4jCVJB8qog60ANfaqG+cQAd++8c gcJwVBtFnQBH63WocV8nIm4A/kbWI2Ks5gyXhaMGVzMO5yIZ8itSk3wehE58LVPvR/9X8E 65QJnxT3TL07SGCszHXA0KIm0acmHepWeKXFCtCDaNY3WI67MtAkNQ8DNK7aiQVnIM6I2N cBY9/ClX+gTKNq4nALv0A3DuEsp6El6OYC2zVTqmnUwxe7+75uv4++wAMfjbVeNF9vSXjF tGmqj8v/GT0mQ0tFzsT05NiuCtV0lkk5zZIbFLFSiBQdfB7WzPAzl5Y7n4Akgw== Received: from amaryllis.le-fay.org (amaryllis.le-fay.org [IPv6:2a00:1098:6b:400::9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: ivy/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4fNNx64H0yzvyT for ; Sat, 28 Feb 2026 11:57:46 +0000 (UTC) (envelope-from ivy@freebsd.org) Date: Sat, 28 Feb 2026 11:55:45 +0000 From: Lexi Winter To: freebsd-current@freebsd.org Subject: Re: pkgbase and customised builds via ${SRC}/release/release.sh Message-ID: Mail-Followup-To: freebsd-current@freebsd.org References: <20fb908c0954e62977dffdcd4a883678@riseup.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="c5f+Q6D2munpdaYa" Content-Disposition: inline In-Reply-To: --c5f+Q6D2munpdaYa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Alastair Hogge wrote in : > The biggest problem I had come across, was the "tags=3Dpackage=3D{foo}" > declarations in ${SRC}/etc/mtree/*.dist. I had to remove a couple of > these from a 2 or 3 mtree dists. What is going on in this part of the > build infrastructure? There is a history of similar commits[1][2][3], to > the changes I now have to keep locally, yet, des@ recently added[4] more > tags. =20 short answer: as of last week, des's commit is correct and the previous commits removing the tags should be reverted, i just didn't get around to doing it yet. the problem is that previously, tags=3Dpackage=3Dfoo would cause the 'foo' package to be unconditionally created, even if it was otherwise empty. this first came to light when i introduced several of these in commit 436618a427b4[0], which created several of these empty packages in specific circumstances i hadn't tested (mostly involving non-amd64 builds and certain src.conf options). [0] https://cgit.freebsd.org/src/commit/?id=3D436618a427b4baaf42d8221ef07d1= 4e3ba787d3a "etc/mtree: Add package tags for /usr/include" in at least a couple of cases this broke the build or install because we'd end up with a 'foo-dev' package having a depend on a 'foo' package that doesn't exist. the post-436618a427b4 reverts are to fix specific instances of this without having to revert the entire initial commit which was broadly correct. this was on my to-do list for a long time until it came up again in D55408[1], the review for des's commit that you mentioned, at which point i fixed the problem properly in 7965c93e4d41[2] by not creating packages which only contain directories. this also fixes all the previous examples of this problem, which is why the workarounds can now be reverted. [1] https://reviews.freebsd.org/D55408 "build: Move all of lp under LPR option" [2] https://cgit.freebsd.org/src/commit/?id=3D7965c93e4d4103ba6ed7ac1e5f159= 9c93cbcdbf7 "packages: Don't create empty packages" if you're on stable/15 (or 15.0), i didn't yet MFC 7965c93e4d41, but it's fairly self-contained and should be fine to cherry-pick. if you're rather wait, it will land in 15.1. > From what I understand when investigating this, the build infra uses > both the mtree dists, and ${SRC}/release/packages/generate-set-ucl.lua > to generate the pkgbase sets? Is that correct? Does that mean there is > duplicated work here, and one of these paths is redundant? pkgbase *sets* are generated purely based on annotations (a type of metadata) in the packages we previously built, i.e., we do the full pkgbase build, then scan that for 'set' annotations and use that to generate the set packages. this is driven by release/packages/create-sets.sh, which invokes generate-set-ucl.lua as part of its work. but note that this is only about set packages (i.e., "FreeBSD-set-*"), which never contain any files or directories and should never be referenced in mtree. mtree is involved in the staging process we do prior to the initial (non-set) pkgbase build. that process is driven by Makefile.inc1,=20 and looks roughly like this: * a 'make stageworld' is done to populate /worldstage. stageworld is essentially the same thing as installworld, with some minor tweaks to prevent the host system state from affecting the installed world. part of this process involves running mtree to create the directory structure. this stageworld is done with NO_ROOT and METALOG set, which means a complete log of everything installed (including by mtree) is written to a special file called METALOG in the worldstage/ directory. * then we run release/scripts/mtree-to-plist.awk on the METALOG file to generate the individual *.plist files for each package. this looks at the 'package' tag for each file or directory in the METALOG to determine which package the file belongs to. despite the name, this script is never run on mtree files from src. it's called mtree-to-plist because the METALOG is in mtree format. * based on the plist files, we run release/packages/generate-ucl.sh to generate a UCL file containing pkg-create(8) metadata for each package. * we build the packages and copy them to ${REPODIR}. i find this process both overcomplicated and somewhat fragile, so i'd like to improve it at some point, but there are more urgent things to fix like release media builds and some kernel-related issues (e.g. D54542[4] and D54282[5], if you're interested). [4] https://reviews.freebsd.org/D54542 "release: Build the release media from packages" [5] https://reviews.freebsd.org/D54282 "packages: Always install kernel as /boot/kernel.NAME" > I also noted that a few WITHOUT_foo options had no effect on the World > building stage 1.2, bootstrap tools. Here, when WITHOUT_KERBEROS is > defined, it is still built. I even still have a Kerberos KDC pkgbase > component. i am not sure what's going on here since i'm not very familiar with how the src bootstrap works. if no one else has an idea, you could ask on hackers@ (or in a new thread, since people might ignore pkgbase threads.) --c5f+Q6D2munpdaYa Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQSyjTg96lp3RifySyn1nT63mIK/YAUCaaLXvgAKCRD1nT63mIK/ YNdPAP4xoZTgBS2JOrh0TKAIjTef2NeqDZ35vlhRvi30jwR4eQEA7uwSydYmNeIi gcB6yoWDsRc+aWxJT7mZ/wznGa58kg4= =UFho -----END PGP SIGNATURE----- --c5f+Q6D2munpdaYa--