From nobody Thu Sep 25 13:12:09 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cXYyx5rWDz68VZ0; Thu, 25 Sep 2025 13:12:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cXYyx3g7sz3CK7; Thu, 25 Sep 2025 13:12:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1758805929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LvcIkGRIlXIX7FZ8oS90YBFEP4usGdR4u1aEIQ0uv7Q=; b=naDZhBQYqs8do0THbEVjW2c2YhnjBhtXD7Zq66e/nuyjuxqnHrISkUxHltOe06jgXyAGed f9pb3EcwzPV3e2tlF/XoCtl/xQLJf3l7xIv248mTXVQsaYSnIx4n8HmVqkBxTS8n/lydIk jbkD9mSiwGC2dUzNoQfRx73J9L7wyv8Z40Mvr5ZCG1tJBgs4ZisRbsvl1WparJ1S5tONJV b7lFLxCbxkoCRa9tFaOkTxAivNw7wvhyokAgaQ0Cw0rtdVDRiQPW4DxGaszloDKcshwSzN 12LeMJrdxNoxVk2udzt3AcGzYA1KJUB0u12oIF0obPX0N0Oz0mu39Av1LFuEhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1758805929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LvcIkGRIlXIX7FZ8oS90YBFEP4usGdR4u1aEIQ0uv7Q=; b=Wsoz8AxpUxvY6eHIJ3YK9ul30vw8568jFgr0N5dUCvsmH+/3H7SC9r+jRRssBNf6ThzIuL W7sS1oZifj6ggTily3xTc5myeEO9YwM4EQI57eu9ZX5tUwZERIYsl+iRoCAkG4y31mwZLy M5keu7VZUpPUEeKWRvi/V4IHIfKx/NJt6q7lgpU7DewC7FSVqPFnzN40to5FOGXgvhNXXl MhldFKAKrCDmz7s41rLU2B7mZm2maVOXAujqiyNTlcSd9tQcRQNSLoTQIE9ecGdYPxN2E3 8UXlumtWlOycHJ5xj0ngAY+v8qg8Pev0h8XEgVIb3ce/1pzQXUZ2ByfiPEwgWg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1758805929; a=rsa-sha256; cv=none; b=xLD9D05nDFMn0NjRPFvH0BPSt7l1m6Q+9l9QHeygmWSCs1RB4IMczZdy09A3dieFJrJdsC +bfGSxc+PCyFYBe2muy7YFH8tv6OsXD6FUpVaFleE9zvpyVa4bpo4Pdq2TCUDAjrsTKKzu ixnzFrS9IlmzLBXMSzbjmUJKRm2ixlNRm4PzK3K52zwqPvN23t+suDMbvuyBTIp++Fogdo uri0ngkyVZHqAHxvz357W2drnu7qn59vkcu01QC52KA5yGeTjtFXxnwzhZfXhD0B0O2CV6 3vFK1R3L5LxIeSuvrWBKY/8Solgy61iaO0UOFq1hLjORwXc56DNC3IiekoscwA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cXYyx3DwZz1Hj1; Thu, 25 Sep 2025 13:12:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 58PDC9D2061278; Thu, 25 Sep 2025 13:12:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 58PDC9SU061275; Thu, 25 Sep 2025 13:12:09 GMT (envelope-from git) Date: Thu, 25 Sep 2025 13:12:09 GMT Message-Id: <202509251312.58PDC9SU061275@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Ed Maste Subject: git: 4227d51b6e09 - main - release: create pkgbase VM and cloud images List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: emaste X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4227d51b6e09ceaaae0f5832c85ec700d80ef18f Auto-Submitted: auto-generated The branch main has been updated by emaste: URL: https://cgit.FreeBSD.org/src/commit/?id=4227d51b6e09ceaaae0f5832c85ec700d80ef18f commit 4227d51b6e09ceaaae0f5832c85ec700d80ef18f Author: Isaac Freund AuthorDate: 2025-07-24 08:43:43 +0000 Commit: Ed Maste CommitDate: 2025-09-25 12:55:56 +0000 release: create pkgbase VM and cloud images VM and cloud images are now built as packaged base systems by default, matching the default for installation media. Setting -DNOPKGBASE allows building as non-pkgbase systems. Reviewed by: emaste Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D51483 --- release/Makefile | 3 ++- release/Makefile.vm | 8 ++++++-- release/tools/ec2-builder.conf | 9 +++++++++ release/tools/ec2-small.conf | 9 +++++++++ release/tools/vmimage.subr | 36 ++++++++++++++++++++++++++++++++---- 5 files changed, 58 insertions(+), 7 deletions(-) diff --git a/release/Makefile b/release/Makefile index fb2a60130700..d131a36157d7 100644 --- a/release/Makefile +++ b/release/Makefile @@ -23,7 +23,8 @@ # (by default, the directory above this one) # NODISTSETS: if set, do not include dist sets or MANIFEST # NOPKGBASE: if set, include dist tarballs rather than pkgbase packages in -# disc1 and dvd1 installation media +# disc1 and dvd1 installation media and build VM/cloud images using +# make installkernel installworld. # PORTSDIR: location of ports tree to distribute (default: /usr/ports) # XTRADIR: xtra-bits-dir argument for /mkisoimages.sh # NOPKG: if set, do not distribute third-party packages diff --git a/release/Makefile.vm b/release/Makefile.vm index 479547f76da1..eafd1d6abf71 100644 --- a/release/Makefile.vm +++ b/release/Makefile.vm @@ -126,7 +126,7 @@ ${_CW:tu}${_FS:tu}${_FMT:tu}IMAGE= ${_CW:tl}.${_FS}.${_FMT} cw-${_CW:tl}-${_FS}-${_FMT}: cw-ec2-base-${_FS}-${_FMT} .endif -cw-${_CW:tl}-${_FS}-${_FMT}: ${QEMUTGT} +cw-${_CW:tl}-${_FS}-${_FMT}: ${QEMUTGT} ${PKGBASE_REPO_DIR} mkdir -p ${.OBJDIR}/${.TARGET} env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} SWAPSIZE=${SWAPSIZE} \ QEMUSTATIC=${QEMUSTATIC} \ @@ -137,6 +137,8 @@ cw-${_CW:tl}-${_FS}-${_FMT}: ${QEMUTGT} PKG_REPOS_DIR=${PKG_REPOS_DIR:U${.OBJDIR}/${.TARGET}/etc/pkg} \ PKG_REPO_NAME=${PKG_REPO_NAME:UFreeBSD-ports} \ PKG_INSTALL_EPOCH=${PKG_INSTALL_EPOCH:U${GITEPOCH}} \ + NOPKGBASE=${NOPKGBASE} \ + PKGBASE_REPO_DIR=${.OBJDIR}/pkgbase-repo-dir \ ${.CURDIR}/scripts/mk-vmimage.sh \ -C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \ -F ${"${_CW:MEC2-BUILDER}" != "":?ufs:${_FS}} \ @@ -180,7 +182,7 @@ CLEANFILES+= ${VMBASE}.${FS}.${FORMAT} vm-base: vm-image -vm-image: ${QEMUTGT} +vm-image: ${QEMUTGT} ${PKGBASE_REPO_DIR} .if defined(WITH_VMIMAGES) && !empty(WITH_VMIMAGES) . for FORMAT in ${VMFORMATS} . for FS in ${VMFSLIST} @@ -193,6 +195,8 @@ vm-image: ${QEMUTGT} PKG_REPOS_DIR=${PKG_REPOS_DIR:U${.OBJDIR}/${.TARGET}-${FORMAT}-${FS}/etc/pkg} \ PKG_REPO_NAME=${PKG_REPO_NAME:UFreeBSD-ports} \ PKG_INSTALL_EPOCH=${PKG_INSTALL_EPOCH:U${GITEPOCH}} \ + NOPKGBASE=${NOPKGBASE} \ + PKGBASE_REPO_DIR=${.OBJDIR}/pkgbase-repo-dir \ ${.CURDIR}/scripts/mk-vmimage.sh \ -C ${.CURDIR}/tools/vmimage.subr \ -d ${.OBJDIR}/${.TARGET}-${FORMAT}-${FS} -F ${FS} \ diff --git a/release/tools/ec2-builder.conf b/release/tools/ec2-builder.conf index 993dffc565eb..cf4276fc80ec 100644 --- a/release/tools/ec2-builder.conf +++ b/release/tools/ec2-builder.conf @@ -11,6 +11,15 @@ export VMSIZE=8000m export INSTALLOPTS="WITHOUT_DEBUG_FILES=YES WITHOUT_KERNEL_SYMBOLS=YES \ WITHOUT_LIB32=YES WITHOUT_TESTS=YES WITHOUT_LLDB=YES" +# Equivalent to INSTALLOPTS for pkgbase +vm_extra_filter_base_packages() { + grep -v \ + -e '.*-dbg$' \ + -e '.*-lib32$' \ + -e '^FreeBSD-tests.*' \ + -e '^FreeBSD-lldb.*' +} + # Packages to install into the image we're creating. In addition to packages # present on all EC2 AMIs, we install: # * ec2-scripts, which provides a range of EC2ification startup scripts, diff --git a/release/tools/ec2-small.conf b/release/tools/ec2-small.conf index e2a348d6ba8f..32d02cbb79e4 100644 --- a/release/tools/ec2-small.conf +++ b/release/tools/ec2-small.conf @@ -14,6 +14,15 @@ export VMSIZE=5000m export INSTALLOPTS="WITHOUT_DEBUG_FILES=YES WITHOUT_KERNEL_SYMBOLS=YES \ WITHOUT_LIB32=YES WITHOUT_TESTS=YES WITHOUT_LLDB=YES" +# Equivalent to INSTALLOPTS for pkgbase +vm_extra_filter_base_packages() { + grep -v \ + -e '.*-dbg$' \ + -e '.*-lib32$' \ + -e '^FreeBSD-tests.*' \ + -e '^FreeBSD-lldb.*' +} + # Packages to install into the image we're creating. In addition to packages # present on all EC2 AMIs, we install: # * ec2-scripts, which provides a range of EC2ification startup scripts, diff --git a/release/tools/vmimage.subr b/release/tools/vmimage.subr index b3187efd6526..577abaac73cf 100644 --- a/release/tools/vmimage.subr +++ b/release/tools/vmimage.subr @@ -70,13 +70,41 @@ vm_copy_base() { return 0 } +vm_filter_base_packages() { + # Reads a list of all base system packages from stdin. + # Writes a list of base system packages to install to stdout. + grep -v -e '^FreeBSD-src.*' -e '^FreeBSD-kernel.*' + # There are several kernel variants available in separate packages. + # For VMs it is sufficient to install only the generic kernel. + echo "FreeBSD-kernel-man" + echo "FreeBSD-kernel-generic" + echo "FreeBSD-kernel-generic-dbg" +} + +vm_extra_filter_base_packages() { + # Prototype. When overridden, allows further filtering of base system + # packages, reading package names from stdin and writing to stdout. + cat +} + vm_install_base() { # Installs the FreeBSD userland/kernel to the virtual machine disk. - cd ${WORLDDIR} && \ - make DESTDIR=${DESTDIR} ${INSTALLOPTS} \ - installworld installkernel distribution || \ - err "\n\nCannot install the base system to ${DESTDIR}." + if [ -z "${NOPKGBASE}" ]; then + local pkg_cmd + pkg_cmd="pkg --rootdir ${DESTDIR} --repo-conf-dir ${PKGBASE_REPO_DIR} + -o ASSUME_ALWAYS_YES=yes -o IGNORE_OSVERSION=yes + -o INSTALL_AS_USER=yes " + $pkg_cmd update + selected=$($pkg_cmd rquery -U -r FreeBSD-base %n | \ + vm_filter_base_packages | vm_extra_filter_base_packages) + $pkg_cmd install -U -r FreeBSD-base $selected + else + cd ${WORLDDIR} && \ + make DESTDIR=${DESTDIR} ${INSTALLOPTS} \ + installworld installkernel distribution || \ + err "\n\nCannot install the base system to ${DESTDIR}." + fi # Bootstrap etcupdate(8) database. mkdir -p ${DESTDIR}/var/db/etcupdate