From nobody Fri Aug 15 04:44:02 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4c38db0lWQz64XKW; Fri, 15 Aug 2025 04:44:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4c38dZ55Fcz3qHK; Fri, 15 Aug 2025 04:44:02 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1755233042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zPX2PLr17fP4ailneazAGqyEJkY3vRKKbfHbfBm1GIk=; b=DjgQAKbxnXruZeLT7E7OgwellHgLFMdlliigX5YcKSgfIirJA6f179EZMwVaUSy79HAbTK NgkOAI6p7ydCnWE1ALGFVibKDGFMd1r/hragEaVaBTjw6S2f4WaV0NWnsoZhULGNVVsc6k uf8E5kNZxjGg7YsmLAQK5YTgNKkOzASK3J1Fc2dvNnxy20UxuTr1SzTpOy65GFbDFpuDJD WxQipXt1X9WKwAY4KGkJFzKEnf2Kfd8oVY54pbj4HYYD5LJq2xDJ+0m7C5NfIPAGq7UOGB RJ6Ro2DKDba4KBeITH1vPlwgcR3Y0C2ckjzL2g75GRtuCztwNON3pzTQQ+Qr/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1755233042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zPX2PLr17fP4ailneazAGqyEJkY3vRKKbfHbfBm1GIk=; b=rw68Z6wOu7bJmrwoPB1sF7kkTv0X1Or1chLjNLtBQHPsfI6oWuEH4D6vvXwwhp8VD/jStP ya9TIqMlk6hmwpuL6NUyHxPRS/rsQDIZimFSyEcxlM3Qg/i5BTfGJm/+pXuloePh7aFDfc np2BidpE3BaQ4PvQS4KgT3SFG/RcQJW3wKQD1fwQYTC3nebbDSZP7Q7t97FtI86/Udejlv d97Z2xO2gl1XNLYpxO+Z9HRRfYgMPi1LDL3XOSW2YV8+lDU6Ojn6nI/E6q16CvfE2vpY5r yehRZzQH19mOFrRYp38KXU3uz9fOv4ZCUNrDFy4rNsAmHPo575BeAPqG9skoHw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1755233042; a=rsa-sha256; cv=none; b=P3Ut9gYR1VW0gJxzttDrxWND7seWig4k4LJ94TFHINHmv0PdyrrESOC5ld8AfSNH3LbTBA qVQL+wdW+ngy9qKVszzbWW5QRAMy7h8sCbrxmi83IClFTQALX0txWc3Md8ySSTkvUqUADq p6k8RTN6yNx8piaDwtAsz+gfkFHcwpGa2xbDse9zZLrzI7/gUcGXA3k3KcBjFYQfxPU3FG Bcwi+VOHmXFGd1OPWp9e9UeyC6JY3/H2R7r/8JR0irI0Jz125pCZh/jKJg/Fkn0OprzLZM vOMggqiWSmfwoQbfSiITnkLTRspl5kWWYgdNw359qZDKqA161CmjaR9qFPlWWg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4c38dZ4Xdhz7xF; Fri, 15 Aug 2025 04:44:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 57F4i2BY006881; Fri, 15 Aug 2025 04:44:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 57F4i2fl006878; Fri, 15 Aug 2025 04:44:02 GMT (envelope-from git) Date: Fri, 15 Aug 2025 04:44:02 GMT Message-Id: <202508150444.57F4i2fl006878@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kyle Evans Subject: git: 0090096e5273 - main - build: introduce the notion of a build epoch List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0090096e52738383d6bf48ed5d27dffc850c8a01 Auto-Submitted: auto-generated The branch main has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=0090096e52738383d6bf48ed5d27dffc850c8a01 commit 0090096e52738383d6bf48ed5d27dffc850c8a01 Author: Kyle Evans AuthorDate: 2025-08-15 04:43:42 +0000 Commit: Kyle Evans CommitDate: 2025-08-15 04:43:42 +0000 build: introduce the notion of a build epoch Idea and file format shamelessly stolen from CheriBSD, but reimplemented in terms of the standard FreeBSD build system. We'll use this in some events that call for a deeper cleansing, typically reserved for situations where the dependencies are too complicated to unwind. This notably does not preclude us from doing separate cleansing of world for specific src.conf(5) knob changes that would require a rebuild. In the FreeBSD version, we either stamp the OBJTOP we're cleaning (bootstrap or the full OBJDIR) with the current epoch and bail out for unstamped objdirs, or we compare and either `rm -rf` or `cleandir` as necessary. Reviewed by: brooks, des Differential Revision: https://reviews.freebsd.org/D51848 --- .clean_build_epoch | 7 ++++ Makefile.inc1 | 5 ++- tools/build/depend-cleanup.sh | 85 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 93 insertions(+), 4 deletions(-) diff --git a/.clean_build_epoch b/.clean_build_epoch new file mode 100644 index 000000000000..74cb650f0015 --- /dev/null +++ b/.clean_build_epoch @@ -0,0 +1,7 @@ +# Add a new line containing a larger number than the last (e.g. YYYYMMDD) for +# every change that breaks ABI or requires a full rebuild for some other reason. +# Only the last entry is required, but having the full list may be useful for +# looking at frequency of ABI breakage, etc. + +# 4757b351ea9: openssl: Import version 3.5.1 +20250807 # All OpenSSL-using bits need rebuilt diff --git a/Makefile.inc1 b/Makefile.inc1 index e67bc7f5d1b1..cf32248b6b9d 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1082,7 +1082,7 @@ _cleanobj_fast_depend_hack: .PHONY @echo ">>> Deleting stale dependencies..."; MACHINE=${MACHINE} MACHINE_ARCH=${MACHINE_ARCH} \ ALL_libcompats=${_ALL_libcompats:Q} \ - sh ${.CURDIR}/tools/build/depend-cleanup.sh ${OBJTOP} + sh ${.CURDIR}/tools/build/depend-cleanup.sh ${OBJTOP} ${SRCTOP} _cleanworldtmp: .PHONY .if ${MK_CLEAN} == "yes" @@ -1175,7 +1175,8 @@ _cleanobj: ${_+_}cd ${.CURDIR}; ${LIB${LIBCOMPAT}WMAKE} _NO_INCLUDE_COMPILERMK=t -f Makefile.inc1 ${CLEANDIR} .endfor .else - ${_+_}cd ${.CURDIR}; ${WMAKE} _NO_INCLUDE_COMPILERMK=t _cleanobj_fast_depend_hack + ${_+_}cd ${.CURDIR}; env CLEANMK="_NO_INCLUDE_COMPILERMK=t ${CLEANDIR}" \ + MAKE=${MAKE} ${WMAKE} _cleanobj_fast_depend_hack .endif # ${MK_CLEAN} == "yes" _obj: @echo diff --git a/tools/build/depend-cleanup.sh b/tools/build/depend-cleanup.sh index c2cb7877dd04..cd51c59ff0e1 100755 --- a/tools/build/depend-cleanup.sh +++ b/tools/build/depend-cleanup.sh @@ -63,6 +63,10 @@ # if [ "$MACHINE_ARCH" = "amd64" ]; then # clean_dep lib/libc bcmp c # fi +# +# We also have a big hammer at the top of the tree, .clean_build_epoch, to be +# used in severe cases where we can't surgically remove just the parts that +# need rebuilt. This should be used sparingly. set -e set -u @@ -80,7 +84,7 @@ err() usage() { - echo "usage: $(basename $0) [-v] [-n] objtop" >&2 + echo "usage: $(basename $0) [-v] [-n] objtop srctop" >&2 } VERBOSE= @@ -101,17 +105,31 @@ while getopts vn o; do done shift $((OPTIND-1)) -if [ $# -ne 1 ]; then +if [ $# -ne 2 ]; then usage exit 1 fi OBJTOP=$1 shift +SRCTOP=$1 +shift + if [ ! -d "$OBJTOP" ]; then err "$OBJTOP: Not a directory" fi +if [ ! -d "$SRCTOP" -o ! -f "$SRCTOP/Makefile.inc1" ]; then + err "$SRCTOP: Not the root of a src tree" +fi + +: ${CLEANMK=""} +if [ -n "$CLEANMK" ]; then + if [ -z "${MAKE+set}" ]; then + err "MAKE not set" + fi +fi + if [ -z "${MACHINE+set}" ]; then err "MACHINE not set" fi @@ -151,6 +169,69 @@ clean_dep() done } +extract_epoch() +{ + [ -s "$1" ] || return 0 + + awk 'int($1) > 0 { epoch = $1 } END { print epoch }' "$1" +} + +clean_world() +{ + local buildepoch="$1" + + # The caller may set CLEANMK in the environment to make target(s) that + # should be invoked instead of just destroying everything. This is + # generally used after legacy/bootstrap tools to avoid over-cleansing + # since we're generally in the temporary tree's ancestor. + if [ -n "$CLEANMK" ]; then + echo "Cleaning up the object tree" + run $MAKE -C "$SRCTOP" -f "$SRCTOP"/Makefile.inc1 $CLEANMK + else + echo "Cleaning up the temporary build tree" + run rm -rf "$OBJTOP" + fi + + # We don't assume that all callers will have grabbed the build epoch, so + # we'll do it here as needed. This will be useful if we add other + # non-epoch reasons to force clean. + if [ -z "$buildepoch" ]; then + buildepoch=$(extract_epoch "$SRCTOP"/.clean_build_epoch) + fi + + mkdir -p "$OBJTOP" + echo "$buildepoch" > "$OBJTOP"/.clean_build_epoch + + exit 0 +} + +check_epoch() +{ + local srcepoch objepoch + + srcepoch=$(extract_epoch "$SRCTOP"/.clean_build_epoch) + if [ -z "$srcepoch" ]; then + err "Malformed .clean_build_epoch; please validate the last line" + fi + + # We don't discriminate between the varying degrees of difference + # between epochs. If it went backwards we could be bisecting across + # epochs, in which case the original need to clean likely still stands. + objepoch=$(extract_epoch "$OBJTOP"/.clean_build_epoch) + if [ -z "$objepoch" ] || [ "$srcepoch" -ne "$objepoch" ]; then + if [ "$VERBOSE" ]; then + echo "Cleaning - src epoch: $srcepoch, objdir epoch: ${objepoch:-unknown}" + fi + + clean_world "$srcepoch" + # NORETURN + fi +} + +check_epoch + +#### Typical dependency cleanup begins here. + # Date Rev Description # 20220326 fbc002cb72d2 move from bcmp.c to bcmp.S