From owner-freebsd-pkgbase@freebsd.org Mon Jan 18 18:25:26 2021 Return-Path: Delivered-To: freebsd-pkgbase@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 40E0B4F2C30 for ; Mon, 18 Jan 2021 18:25:26 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DKKw61Mq8z4nsQ; Mon, 18 Jan 2021 18:25:26 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 1CE5622C03; Mon, 18 Jan 2021 18:25:26 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qt1-f170.google.com with SMTP id z6so4857096qtn.0; Mon, 18 Jan 2021 10:25:26 -0800 (PST) X-Gm-Message-State: AOAM532hf/+FTJkTvtGIz3XWT2o4wsIyeHzwqmBSdm/pTc7fSHjf/A3F MXyE/Rv7DyKrJV4FJXeKVTnNnl5w0TLJ/i+7AMs= X-Google-Smtp-Source: ABdhPJxICIXn/HdRfCOQKnPGpqodv6cg5JAvp3UH2b6g9yPtqmbJk2tXKyhjUbrWkBzGhy7N1HTa6i03m68/SV/7qrY= X-Received: by 2002:ac8:7141:: with SMTP id h1mr835654qtp.211.1610994325554; Mon, 18 Jan 2021 10:25:25 -0800 (PST) MIME-Version: 1.0 References: <202011021823.0A2INo93086367@repo.freebsd.org> In-Reply-To: <202011021823.0A2INo93086367@repo.freebsd.org> From: Kyle Evans Date: Mon, 18 Jan 2021 12:25:12 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r367277 - head To: Emmanuel Vadot Cc: freebsd-pkgbase@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jan 2021 18:25:26 -0000 On Mon, Nov 2, 2020 at 12:24 PM Emmanuel Vadot wrote: > > Author: manu > Date: Mon Nov 2 18:23:50 2020 > New Revision: 367277 > URL: https://svnweb.freebsd.org/changeset/base/367277 > > Log: > pkgbase: Add incremental packages > > This adds a new target update-packages which will create the new packages > compared to the last run. > > This is how to use it: > At this point we cut a release > $ make buildworld ... > $ make buildkernel > $ make packages > > There is now a PKG_VERSION directory with latest link pointing to it > Distribute the packages to server > > $ something something that update the source tree > $ make buildworld ... > $ make buildkernel > $ make update-packages > You know have a PKG_VERSION directory in the REPODIR and latest link pointing to it. > In PKG_VERSION dir only the packages which differs from the latest run are > named PKG_VERSION, otherwise the old packages are there. > > The process is : > Build the new packages in the PKG_VERSION directory > Compare the internal data with the PKG_VERSION_FROM version. The comparison is done > by checking the internal hash of the packages. > By default PKG_VERSION_FROM is set to what the latest link points to. > If the old and new version matches, we rm the new package and cp the old one. > > Differential Revision: https://reviews.freebsd.org/D25984 > > Modified: > head/Makefile > head/Makefile.inc1 > > [.. snip ..] > Modified: head/Makefile.inc1 > ============================================================================== > --- head/Makefile.inc1 Mon Nov 2 17:39:59 2020 (r367276) > +++ head/Makefile.inc1 Mon Nov 2 18:23:50 2020 (r367277) > [... snip ...] > @@ -1867,6 +1880,28 @@ package-pkg: .PHONY > > real-packages: stage-packages create-packages sign-packages .PHONY > > +real-update-packages: stage-packages .PHONY > + ${_+_}${MAKE} -C ${.CURDIR} PKG_VERSION=${PKG_VERSION} create-packages > + @echo "==> Checking for new packages (comparing ${PKG_VERSION} to ${PKG_VERSION_FROM})" > + @for pkg in ${REPODIR}/${PKG_ABI}/${PKG_VERSION_FROM}/${PKG_NAME_PREFIX}-*; do \ > + pkgname=$$(pkg query -F $${pkg} '%n' | sed 's/${PKG_NAME_PREFIX}-\(.*\)/\1/') ; \ > + newpkgname=${PKG_NAME_PREFIX}-$${pkgname}-${PKG_VERSION}.${PKG_FORMAT} ; \ > + oldsum=$$(pkg query -F $${pkg} '%X') ; \ > + if [ ! -f ${REPODIR}/${PKG_ABI}/${PKG_VERSION}/$${newpkgname} ]; then \ > + continue; \ > + fi ; \ > + newsum=$$(pkg query -F ${REPODIR}/${PKG_ABI}/${PKG_VERSION}/$${newpkgname} '%X') ; \ > + if [ "$${oldsum}" == "$${newsum}" ]; then \ > + echo "==> Keeping old ${PKG_NAME_PREFIX}-$${pkgname}-${PKG_VERSION_FROM}.${PKG_FORMAT}" ; \ > + rm ${REPODIR}/${PKG_ABI}/${PKG_VERSION}/$${newpkgname} ; \ > + cp $${pkg} ${REPODIR}/${PKG_ABI}/${PKG_VERSION} ; \ > + else \ > + echo "==> New package $${newpkgname}" ; \ > + fi ; \ > + done > + ${_+_}@cd ${.CURDIR}; \ > + ${MAKE} -f Makefile.inc1 PKG_VERSION=${PKG_VERSION} sign-packages > + > stage-packages-world: .PHONY > @mkdir -p ${WSTAGEDIR} > ${_+_}@cd ${.CURDIR}; \ Any objection to making update-packages harmless to run the first time, too? It'd be nicer for some external scripting to not worry about the distinction between packages and update-packages if the real-update-packages target skipped the loop entirely if ${REPODIR}/${PKG_ABI} didn't exist at all. Thanks, Kyle Evans