Date: Mon, 29 Jun 2026 15:52:21 +0000 From: Dag-Erling=?utf-8?Q? Sm=C3=B8rg?=rav <des@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: e6467516f584 - main - pkgbasify: Add new utility Message-ID: <6a4294b5.3c62b.74319574@gitrepo.freebsd.org>
index | next in thread | raw e-mail
The branch main has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=e6467516f584505ac29fd370966abd530a7b5454 commit e6467516f584505ac29fd370966abd530a7b5454 Author: Dag-Erling Smørgrav <des@FreeBSD.org> AuthorDate: 2026-06-29 00:41:11 +0000 Commit: Dag-Erling Smørgrav <des@FreeBSD.org> CommitDate: 2026-06-29 13:49:22 +0000 pkgbasify: Add new utility The pkgbasify utility converts a system installed from distribution sets to packaged base by registering, without actually installing, a set of packages that corresponds to what is already installed. MFC after: 1 week --- usr.sbin/Makefile | 1 + usr.sbin/pkgbasify/Makefile | 6 + usr.sbin/pkgbasify/pkgbasify.8 | 215 +++++++++++++++ usr.sbin/pkgbasify/pkgbasify.sh | 596 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 818 insertions(+) diff --git a/usr.sbin/Makefile b/usr.sbin/Makefile index 9e885d5d45c5..2b478b26d3d8 100644 --- a/usr.sbin/Makefile +++ b/usr.sbin/Makefile @@ -57,6 +57,7 @@ SUBDIR= adduser \ nologin \ pciconf \ periodic \ + pkgbasify \ pnfsdscopymr \ pnfsdsfile \ pnfsdskill \ diff --git a/usr.sbin/pkgbasify/Makefile b/usr.sbin/pkgbasify/Makefile new file mode 100644 index 000000000000..cbd8637b78c3 --- /dev/null +++ b/usr.sbin/pkgbasify/Makefile @@ -0,0 +1,6 @@ +SCRIPTS=pkgbasify.sh +LINKS= ${BINDIR}/pkgbasify ${BINDIR}/depkgbasify +MAN= pkgbasify.8 +MLINKS= pkgbasify.8 depkgbasify.8 + +.include <bsd.prog.mk> diff --git a/usr.sbin/pkgbasify/pkgbasify.8 b/usr.sbin/pkgbasify/pkgbasify.8 new file mode 100644 index 000000000000..9c44740e4132 --- /dev/null +++ b/usr.sbin/pkgbasify/pkgbasify.8 @@ -0,0 +1,215 @@ +.\"- +.\" Copyright (c) 2026 Dag-Erling Smørgrav +.\" +.\" SPDX-License-Identifier: BSD-2-Clause +.\" +.Dd June 28, 2026 +.Dt PKGBASIFY 8 +.Os +.Sh NAME +.Nm pkgbasify , +.Nm depkgbasify +.Nd convert a system to and from pkgbase +.Sh SYNOPSIS +.Nm pkgbasify +.Op Fl hnqtv +.Op Fl a | Fl y +.Op Fl b Ar basedir +.Op Fl j Ar jail +.Op Fl r Ar repo +.Nm depkgbasify +.Op Fl hnqv +.Op Fl a | Fl y +.Op Fl b Ar basedir +.Op Fl j Ar jail +.Sh DESCRIPTION +The +.Nm pkgbasify +utility converts a system installed from distribution sets to packaged +base by registering, without actually installing, a set of packages +that corresponds to what is already installed. +See +.Sx Converting to Packaged Base +below. +.Pp +The +.Nm depkgbasify +utility converts a packaged base system back to distribution sets by +unregistering all base packages. +See +.Sx Converting from Packaged Base +below. +.Pp +In both cases, the user is urged to take a backup or snapshot of the +system and perform a dry run before the actual conversion. +.Pp +The following options are available: +.Bl -tag -width indent +.It Fl a +Automatic mode. +Do not prompt the user, but skip certain optional steps. +Mutually exclusive with +.Fl y . +.It Fl b Ar basedir +Operate on a system mounted at +.Ar basedir . +.It Fl h +Print a usage message and exit. +.It Fl j Ar jail +Operate on the specified jail. +.It Fl n +Dry run. +Examine the system and download the necessary files but do not +register or unregister anything. +.It Fl q +Quiet mode. +.It Fl r Ar repo +.Po +.Nm pkgbasify +only +.Pc +Use the specified packaged base repository instead of the default. +.It Fl t +.Po +.Nm pkgbasify +only +.Pc +Trim. +Not yet implemented. +.It Fl v +Verbose mode. +.It Fl y +Assume the answer to every prompt is +.Dq yes . +Mutually exclusive with +.Fl a . +.El +.Ss Converting to Packaged Base +The +.Nm pkgbasify +utility begins by inspecting the system to identify which distribution +sets are installed. +It then downloads and registers the sets of base packages that +correspond to those distribution sets. +If not running in automatic mode, it then performs a forced upgrade +and enables the +.Dv BACKUP_LIBRARIES +options in the +.Xr pkg 8 +configuration. +.Pp +It is important to note that after the conversion, files that should +have been present on the system but weren't will still be recorded in +the package database. +This is why a non-automatic +.Nm pkgbasify +offers to perform a forced upgrade: to reset the system to a state +where the files on disk match the recorded inventory. +However, since +.Nm pkgbasify +does not make a backup of the system before converting it, this forced +upgrade is irreversible, which is also why it is skipped in automatic +mode. +.Ss Converting from Packaged Base +The +.Nm depkgbasify +utility begins by identifying the repository from which the base +system was installed. +It then attempts to identify which package sets are partially or +completely installed. +If not running in automatic mode, it will then offer to complete any +partially installed sets, remove the source tree (if installed from +packages), and perform a last upgrade before the conversion. +Finally, it unregisters all remaining base packages without removing +any of the files. +.Ss Round-trip Conversion +When run in automatic mode, neither +.Nm pkgbasify +nor +.Nm depkgbasify +will make any changes to the system that the other cannot undo. +Thus, starting with a system installed from distribution sets and +running +.Ql pkgbasify -a +followed by +.Ql depkgbasify -a +should have a net effect of no changes to the system except for the +package cache. +Conversely, starting with a freshly updated packaged base system and +running +.Ql depkgbasify -a +followed by +.Ql pkgbasify -a +should also have a net zero effect. +.Sh ENVIRONMENT +.Bl -tag -width indent +.It Ev BASEDIR +base directory as if passed to the +.Fl b +option. +.It Ev BASEREPO +repository as if passed to the +.Fl r +option. +.It Ev JAIL +jail id or name as if passed to the +.Fl j +option. +.El +.Sh EXIT STATUS +.Ex -std pkgbasify depkgbasify +.Sh SEE ALSO +.Xr build 7 , +.Xr pkg 7 , +.Xr pkgbase 7 , +.Xr freebsd-update 8 , +.Xr pkg 8 +.Sh HISTORY +The +.Nm pkgbasify +and +.Nm depkgbasify +utilities first appeared in +.Fx 15.2 . +.Sh AUTHORS +The +.Nm pkgbasify +and +.Nm depkgbasify +utilities and this manual page were written by +.An Dag-Erling Sm\(/orgrav Aq Mt des@FreeBSD.org . +.Sh CAVEATS +The +.Nm pkgbasify +utility is intended for use on systems installed from distribution +sets. +It will also work on systems installed from source, but if the system +has been customized through the use of build options, the conversion +will round it back up, so to speak, to the smallest set of +distribution sets which include everything present on the system. +Conversely, +.Nm pkgbasify +will not notice or register elements of the system which are disabled +by default but have been locally enabled, built and installed, unless +installing from a repository that has been built with the same build +options. +.Pp +When converting a system installed from source to packaged base, the +recommended procedure is to build a set of packages from the same +source tree and using the same build options, then configure a +repository pointing to those packages, and perform the conversion +using that repository before switching to another and doing a forced +upgrade. +.Pp +The +.Nm depkgbasify +utility merely unregisters the base packages. +It does not revert the binaries to a state that could have been +produced by an official installation image or +.Xr freebsd-update 8 . +However, if the system was using base packages from the official +.Fx +packaged base repository prior to the conversion, the contents should +match what +.Xr freebsd-update 8 +expects. diff --git a/usr.sbin/pkgbasify/pkgbasify.sh b/usr.sbin/pkgbasify/pkgbasify.sh new file mode 100755 index 000000000000..86f926bfcd83 --- /dev/null +++ b/usr.sbin/pkgbasify/pkgbasify.sh @@ -0,0 +1,596 @@ +#!/bin/sh +#- +# Copyright (c) 2026 Dag-Erling Smørgrav +# +# SPDX-License-Identifier: BSD-2-Clause +# + +set -eu +exec 3>&2 + +progname="${0##*/}" +localbase="$(/sbin/sysctl -qn user.localbase || echo /usr/local)" + +# +# All the top-level sets except src +# +topsets="base base-dbg kernels kernels-dbg lib32 lib32-dbg tests" + +# +# Default repository +# +OS=FreeBSD +BASEREPO=${BASEREPO:-${OS}-base} + +# +# Command-line options +# +auto= +base=${BASEDIR:-} +dryrun= +jail=${JAIL:-} +quiet= +repo= +trim= +verbose= +yes= + +# +# Print a message +# +msg() { + local width=$((${COLUMNS:-80} - ${#progname} - 4)) + echo "$@" | /usr/bin/fmt -sw${width} | + /usr/bin/sed "s|^|${progname}: |" +} + +# +# Print an error message and exit +# +err() { + msg "$@" >&3 + exit 1 +} + +# +# Print a warning message +# +warn() { + if ! [ "${quiet}" ]; then + msg "$@" >&3 + fi +} + +# +# Print an informational message +# +info() { + if ! [ "${quiet}" ]; then + msg "$@" + fi +} + +# +# Ask the user for confirmation +# +confirm() { + local ans= + if [ "${yes}" ]; then + return 0 + fi + while :; do + read -p "$@ " -r ans + case ${ans} in + [Yy]|[Yy][Ee][Ss]) + return 0 + ;; + [Nn]|[Nn][Oo]) + return 1 + ;; + *) + echo "Please answer with yes or no." >&3 + ;; + esac + done +} + +# +# Execute a command, but print it first if verbose +# +v() { + if [ "${verbose}" ]; then + echo "$@" >&3 + fi + "$@" +} + +# +# Compute the path of a file in the target environment +# +path() { + local path="${base}$1" + if [ -n "${jail}" ]; then + path="$(/usr/sbin/jls -j "${jail}" path)${path}" + fi + echo "${path}" +} + +# +# Check if a file exists in the target environment +# +exists() { + v test -e "$(path "$@")" +} + +# +# Run pkg(7) +# +# pkg(7) currently lacks support for -j and -r, otherwise we could run +# every pkg command through it, which would prepare us for when pkg +# eventually moves into base. For now, we can use jexec for -j, but +# there is no way to emulate -r. +# +pkg7() { + if [ -n "${base}" ]; then + err "pkg(7) currently lacks -r option" + elif [ -n "${jail}" ]; then + v /usr/sbin/jexec "${jail}" /usr/sbin/pkg "$@" + else + v /usr/sbin/pkg "$@" + fi +} + +# +# Run pkg(8) +# +pkg() { + v "${localbase}"/sbin/pkg ${jail:+-j"${jail}"} ${base:+-r"${base}"} "$@" +} + +# +# Run pkg bootstrap +# +pkg_bootstrap() { + if ! pkg7 -N 2>/dev/null; then + warn "Package manager not installed" + if confirm "Bootstrap the package manager?"; then + pkg7 bootstrap -r"${repo}" -y + else + err "Unable to proceed without package manager" + fi + fi +} + +# +# Check if a package exists in the repository +# +pkg_exists() { + pkg rquery -r"${repo}" -U %v "$@" >/dev/null +} + +# +# Run pkg fetch +# +pkg_fetch() { + pkg fetch -r"${repo}" -U ${quiet:+-q} "$@" +} + +# +# Run pkg install +# +pkg_install() { + pkg install -r"${repo}" -U ${dryrun:+-n} ${quiet:+-q} "$@" +} + +# +# Check if a package is installed +# +pkg_installed() { + pkg info -q "$@" +} + +# +# Run pkg query +# +pkg_query() { + pkg query "$@" +} + +# +# Run pkg install --register-only +# +pkg_register() { + pkg install -r"${repo}" --register-only ${dryrun:+-n} ${quiet:+-q} "$@" +} + +# +# Run pkg remove +# +pkg_remove() { + pkg remove ${dryrun:+-n} ${quiet:+-q} "$@" +} + +# +# Run pkg rquery +# +pkg_rquery() { + pkg rquery -r"${repo}" -U "$@" +} + +# +# Run pkg unregister +# +pkg_unregister() { + pkg unregister ${dryrun:+-n} ${quiet:+-q} "$@" +} + +# +# Run pkg update +# +pkg_update() { + pkg update -r"${repo}" ${quiet:+-q} "$@" +} + +# +# Run pkg upgrade +# +pkg_upgrade() { + pkg upgrade -r"${repo}" ${dryrun:+-n} ${quiet:+-q} "$@" +} + +# +# Generate a complete transitive list of dependencies +# +pkg_alldeps() { + local deps=$(pkg rquery %dn "$@") + if [ "${deps}" ]; then + echo "${deps}" + pkg_alldeps ${deps} + fi | sort -u +} + +# +# Check if pkg is new enough +# +check_pkg() { + local ver=$(pkg -v) + case ${ver} in + 1.*|2.[0-6].*|2.7.[0-4]) + err "pkg ${ver} is too old, I need pkg 2.7.5 or newer" + ;; + esac +} + +# +# Try to ascertain if the system is running pkgbase +# +ispkgbase() { + pkg which -q /sbin/init >/dev/null +} + +# +# Convert to pkgbase +# +pkgbasify() { + local sets=base set= + + # Check if we're running pkgbase + if ispkgbase; then + err "The system is already running packaged base" + fi + + # Update the repository + pkg_update + + # Some paths + local debug=/usr/lib/debug + local ld=/libexec/ld-elf.so.1 + local ld32=/libexec/ld-elf32.so.1 + local test=/usr/tests/Kyuafile + local src=/usr/src + + # Sanity check + if ! exists "${ld}"; then + err "No base system found" + fi + + # Decide whether to register base-dbg + if ! exists "${debug}${ld}.debug"; then + warn "No debugging symbols found" + else + sets="${sets} base-dbg" + fi + + # Decide whether to register lib32 and lib32-dbg + if ! pkg_exists ${OS}-set-lib32; then + # Not available + elif ! exists "${ld32}"; then + warn "No 32-bit loader found" + else + sets="${sets} lib32" + if exists "${debug}${ld32}.debug"; then + sets="${sets} lib32-dbg" + fi + fi + + # Decide whether to register tests + if ! exists "${test}"; then + warn "No test suite found" + else + sets="${sets} tests" + fi + + # Decide whether to register kernels and kernels-dbg + local kernel=/boot/kernel/kernel + if [ -z "${jail}${base}" ]; then + if ! kernel=$(/sbin/sysctl -qn kern.bootfile); then + err "Unable to identify the running kernel" + fi + case ${kernel%/kernel} in + *.old) + kernel=${kernel%.old/kernel}/kernel + ;; + esac + case ${kernel} in + /boot/kernel/kernel) + ;; + *) + kernel= + ;; + esac + fi + if [ -z "${kernel}" ] || ! exists "${kernel}"; then + warn "Non-standard kernel, or no kernel installed" + kernel= + else + sets="${sets} kernels" + if exists "${debug}${kernel}.debug"; then + sets="${sets} kernels-dbg" + fi + fi + + # Decide whether to register src + if ! exists "${src}/Makefile.inc1"; then + warn "No source tree found" + elif exists "${src}/.git"; then + warn "Source tree managed by Git " + else + sets="${sets} src" + fi + + # We now have our list of sets + sets=$(for set in ${sets}; do echo ${OS}-set-$set; done) + + if ! [ "${quiet}" ] || ! [ "${yes}" ]; then + echo "The following package sets will be installed:" + echo + pkg_rquery "%n %c" ${sets} | \ + /usr/bin/sed 's/ (metapackage)//' | + /usr/bin/column -tl2 | + /usr/bin/sed 's/^/ /' + echo + fi + + # Final checkpoint + if ! [ "${auto}" ] && ! confirm "Proceed with conversion?"; then + err "Conversion abandoned" + fi + + # Download the packages first + warn "Downloading packages, this may take a while..." + pkg_fetch -d -y ${sets} + + # Now register them + pkg_register -y ${sets} + + # Force-upgrade + if ! [ "${auto}" ] && confirm "Reinstall base packages?"; then + pkg_upgrade -f -y + fi + + # Check if BACKUP_LIBRARIES is turned on + case $(pkg config BACKUP_LIBRARIES) in + no) + warn "The recommended BACKUP_LIBRARIES option is not enabled" + if ! [ "${auto}" ] && + confirm "Enable BACKUP_LIBRARIES option?"; then + if ! [ "${dryrun}" ]; then + v /usr/bin/sed -i.bak \ + -e $'1i\\\nBACKUP_LIBRARIES = true;' \ + $(path ${localbase}/etc/pkg.conf) + fi + info "BACKUP_LIBRARIES option enabled" + fi + ;; + esac + + # Done! + if ! [ "${dryrun}" ]; then + info "Conversion complete" + fi +} + +# +# Convert from pkgbase +# +depkgbasify() { + local havesrc= pkg= set= + + # Check if we're running pkgbase + if ! ispkgbase; then + err "The system is not running packaged base" + fi + + # Identify and update the repository + repo=$(pkg_query %R ${OS}-runtime) + info "Using repository ${repo}" + pkg_update + + # Check which sets are partially or completely installed + info "Examining the system..." + for set in ${topsets}; do + if pkg_installed ${OS}-set-${set}; then + info "The ${set} set is installed" + continue + elif ! pkg_exists %v ${OS}-set-${set}; then + # set does not exist + continue + fi + local want=$(pkg_alldeps ${OS}-set-${set} | grep -v ${OS}-set) + local have=$(pkg_query %n ${want} || true) + if [ "${have}" = "${want}" ]; then + info "The ${set} set is complete" + elif [ "${have}" ]; then + info "The ${set} set is partially installed" + if ! [ "${auto}" ] && + confirm "Complete the ${set} set?"; then + pkg_fetch -d -y ${OS}-set-${set} + pkg_install -y ${OS}-set-${set} + fi + else + info "The ${set} set is absent" + fi + done + + # Check if the source tree is installed + for pkg in set-src src-sys src; do + if pkg_installed ${OS}-${pkg}; then + havesrc="${havesrc} ${OS}-${pkg}" + fi + done + if [ "${havesrc}" ]; then + info "The source tree appears to be installed. You may want" \ + "to uninstall it before converting to make room for a" \ + "Git clone." + if ! [ "${auto}" ] && confirm "Uninstall the source tree?"; then + pkg_remove -fy ${havesrc} + fi + fi + + # One final upgrade? + if ! [ "${auto}" ] && confirm "Upgrade installed base packages?"; then + pkg_upgrade -y + fi + + # Final checkpoint + if ! [ "${auto}" ] && ! confirm "Proceed with conversion?"; then + err "Conversion abandoned" + fi + + # Unregister all base packages + pkg_unregister -f -y $(pkg_query -e '%o ~ base/*' '%n') + + # Done! + if ! [ "${dryrun}" ]; then + info "Conversion complete" + fi +} + +# +# Print usage message and exit +# +usage() { + exec >&3 + echo -n "usage: ${progname} " + case ${progname%.sh} in + pkgbasify) + echo "[-hnqtv] [-a | -y] [-b basedir] [-j jail] [-r repo]" + ;; + depkgbasify) + echo "[-hnqv] [-a | -y] [-b basedir] [-j jail]" + ;; + *) + echo "-h" + ;; + esac + exit 1 +} + +# +# Entry point +# +main() { + local opt + + # Check program name and set defaults + case ${progname%.sh} in + pkgbasify) + repo=${BASEREPO} + ;; + depkgbasify) + ;; + *) + usage + ;; + esac + + # Parse options + while getopts "ab:hj:nr:qtvy" opt; do + case ${opt} in + a) + auto=true + ;; + b) + base="${OPTARG}" + ;; + h) + usage + ;; + j) + jail="${OPTARG}" + ;; + n) + dryrun=true + ;; + r) + repo="${OPTARG}" + ;; + q) + quiet=true + ;; + t) + trim=true + ;; + v) + verbose=true + ;; + y) + yes=true + ;; + *) + usage + ;; + esac + done + shift $((OPTIND - 1)) + + # Check for conflicting options + if [ "${auto}" ] && [ "${yes}" ]; then + usage + fi + case ${progname%.sh} in + depkgbasify) + if [ "${repo}" ] || [ "${trim}" ]; then + usage + fi + ;; + esac + + # Bootstrap pkg if necessary + pkg_bootstrap + + # Check the pkg version + check_pkg + + # Do what we were asked to do + case ${progname%.sh} in + pkgbasify) + pkgbasify "$@" + ;; + depkgbasify) + depkgbasify "$@" + ;; + esac +} + +main "$@"home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6a4294b5.3c62b.74319574>