Date: Sat, 28 Feb 2026 11:55:45 +0000
From: Lexi Winter <ivy@freebsd.org>
To: freebsd-current@freebsd.org
Subject: Re: pkgbase and customised builds via ${SRC}/release/release.sh
Message-ID: <aaLXwSlnYJ4VZYpm@amaryllis.le-fay.org>
In-Reply-To: <bff85ee4a5836465e4199e41ec1b543e@riseup.net>
References: <20fb908c0954e62977dffdcd4a883678@riseup.net> <aUpxOug8XTWQviqA@amaryllis.le-fay.org> <bff85ee4a5836465e4199e41ec1b543e@riseup.net>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
Alastair Hogge wrote in <bff85ee4a5836465e4199e41ec1b543e@riseup.net>:
> The biggest problem I had come across, was the "tags=package={foo}"
> declarations in ${SRC}/etc/mtree/*.dist. I had to remove a couple of
> these from a 2 or 3 mtree dists. What is going on in this part of the
> build infrastructure? There is a history of similar commits[1][2][3], to
> the changes I now have to keep locally, yet, des@ recently added[4] more
> tags.
short answer: as of last week, des's commit is correct and the previous
commits removing the tags should be reverted, i just didn't get around
to doing it yet.
the problem is that previously, tags=package=foo would cause the 'foo'
package to be unconditionally created, even if it was otherwise empty.
this first came to light when i introduced several of these in commit
436618a427b4[0], which created several of these empty packages in
specific circumstances i hadn't tested (mostly involving non-amd64
builds and certain src.conf options).
[0] https://cgit.freebsd.org/src/commit/?id=436618a427b4baaf42d8221ef07d14e3ba787d3a
"etc/mtree: Add package tags for /usr/include"
in at least a couple of cases this broke the build or install because
we'd end up with a 'foo-dev' package having a depend on a 'foo' package
that doesn't exist. the post-436618a427b4 reverts are to fix specific
instances of this without having to revert the entire initial commit
which was broadly correct.
this was on my to-do list for a long time until it came up again in
D55408[1], the review for des's commit that you mentioned, at which
point i fixed the problem properly in 7965c93e4d41[2] by not creating
packages which only contain directories. this also fixes all the
previous examples of this problem, which is why the workarounds can
now be reverted.
[1] https://reviews.freebsd.org/D55408
"build: Move all of lp under LPR option"
[2] https://cgit.freebsd.org/src/commit/?id=7965c93e4d4103ba6ed7ac1e5f1599c93cbcdbf7
"packages: Don't create empty packages"
if you're on stable/15 (or 15.0), i didn't yet MFC 7965c93e4d41, but
it's fairly self-contained and should be fine to cherry-pick. if
you're rather wait, it will land in 15.1.
> From what I understand when investigating this, the build infra uses
> both the mtree dists, and ${SRC}/release/packages/generate-set-ucl.lua
> to generate the pkgbase sets? Is that correct? Does that mean there is
> duplicated work here, and one of these paths is redundant?
pkgbase *sets* are generated purely based on annotations (a type of
metadata) in the packages we previously built, i.e., we do the full
pkgbase build, then scan that for 'set' annotations and use that to
generate the set packages. this is driven by
release/packages/create-sets.sh, which invokes generate-set-ucl.lua
as part of its work. but note that this is only about set packages
(i.e., "FreeBSD-set-*"), which never contain any files or directories
and should never be referenced in mtree.
mtree is involved in the staging process we do prior to the initial
(non-set) pkgbase build. that process is driven by Makefile.inc1,
and looks roughly like this:
* a 'make stageworld' is done to populate <objdir>/worldstage.
stageworld is essentially the same thing as installworld, with
some minor tweaks to prevent the host system state from affecting
the installed world.
part of this process involves running mtree to create the directory
structure.
this stageworld is done with NO_ROOT and METALOG set, which means
a complete log of everything installed (including by mtree) is
written to a special file called METALOG in the worldstage/
directory.
* then we run release/scripts/mtree-to-plist.awk on the METALOG
file to generate the individual *.plist files for each package.
this looks at the 'package' tag for each file or directory in the
METALOG to determine which package the file belongs to.
despite the name, this script is never run on mtree files from src.
it's called mtree-to-plist because the METALOG is in mtree format.
* based on the plist files, we run release/packages/generate-ucl.sh
to generate a UCL file containing pkg-create(8) metadata for
each package.
* we build the packages and copy them to ${REPODIR}.
i find this process both overcomplicated and somewhat fragile,
so i'd like to improve it at some point, but there are more urgent
things to fix like release media builds and some kernel-related
issues (e.g. D54542[4] and D54282[5], if you're interested).
[4] https://reviews.freebsd.org/D54542
"release: Build the release media from packages"
[5] https://reviews.freebsd.org/D54282
"packages: Always install kernel as /boot/kernel.NAME"
> I also noted that a few WITHOUT_foo options had no effect on the World
> building stage 1.2, bootstrap tools. Here, when WITHOUT_KERBEROS is
> defined, it is still built. I even still have a Kerberos KDC pkgbase
> component.
i am not sure what's going on here since i'm not very familiar with how
the src bootstrap works. if no one else has an idea, you could ask on
hackers@ (or in a new thread, since people might ignore pkgbase threads.)
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQSyjTg96lp3RifySyn1nT63mIK/YAUCaaLXvgAKCRD1nT63mIK/
YNdPAP4xoZTgBS2JOrh0TKAIjTef2NeqDZ35vlhRvi30jwR4eQEA7uwSydYmNeIi
gcB6yoWDsRc+aWxJT7mZ/wznGa58kg4=
=UFho
-----END PGP SIGNATURE-----
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?aaLXwSlnYJ4VZYpm>