From owner-svn-src-all@freebsd.org Wed Aug 8 19:36:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D2F41068C48; Wed, 8 Aug 2018 19:36:30 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE071904DB; Wed, 8 Aug 2018 19:36:29 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 826BA702B; Wed, 8 Aug 2018 19:36:29 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w78JaTpp067515; Wed, 8 Aug 2018 19:36:29 GMT (envelope-from bdrewery@FreeBSD.org) Received: (from bdrewery@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w78JaSpK067513; Wed, 8 Aug 2018 19:36:29 GMT (envelope-from bdrewery@FreeBSD.org) Message-Id: <201808081936.w78JaSpK067513@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bdrewery set sender to bdrewery@FreeBSD.org using -f From: Bryan Drewery Date: Wed, 8 Aug 2018 19:36:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337499 - stable/11 X-SVN-Group: stable-11 X-SVN-Commit-Author: bdrewery X-SVN-Commit-Paths: stable/11 X-SVN-Commit-Revision: 337499 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2018 19:36:30 -0000 Author: bdrewery Date: Wed Aug 8 19:36:28 2018 New Revision: 337499 URL: https://svnweb.freebsd.org/changeset/base/337499 Log: MFC r320280,r320281,r320282,r320283,r320284,r320285,r320692,r322362,r322401,r322402,r336181: r320280: packages: Allow stageworld/stagekernel to run with make jobs. r320281: packages: Allow staging world/kernel in parallel. r320282: packages: Allow creating kernel/world packages in parallel. r320283: packages: Allow actually building individual world packages in parallel. r320284: packages: Parallelize individual kernel packaging. r320285: Expose only the create-packages-* targets since they set needed DEST/DIRDIR. r320692: Fix create-kernel-packages with multiple BUILDKERNELS after r320284 r322362: Indent nested conditionals for readability. r322401: Avoid creating kernel-dbg.txz distribution sets and kernel-debug packages when MK_DEBUG_FILES is 'no'. r322402: Fix indentation from r322401. r336181: Fix parsing of create-kernel-packages Modified: stable/11/Makefile stable/11/Makefile.inc1 Directory Properties: stable/11/ (props changed) Modified: stable/11/Makefile ============================================================================== --- stable/11/Makefile Wed Aug 8 19:32:28 2018 (r337498) +++ stable/11/Makefile Wed Aug 8 19:36:28 2018 (r337499) @@ -131,7 +131,7 @@ TGTS= all all-man buildenv buildenvvars buildkernel bu build32 distribute32 install32 buildsoft distributesoft installsoft \ builddtb xdev xdev-build xdev-install \ xdev-links native-xtools stageworld stagekernel stage-packages \ - create-world-packages create-kernel-packages create-packages \ + create-packages-world create-packages-kernel create-packages \ packages installconfig real-packages sign-packages package-pkg \ print-dir test-system-compiler Modified: stable/11/Makefile.inc1 ============================================================================== --- stable/11/Makefile.inc1 Wed Aug 8 19:32:28 2018 (r337498) +++ stable/11/Makefile.inc1 Wed Aug 8 19:36:28 2018 (r337499) @@ -1371,20 +1371,24 @@ packagekernel: .PHONY @${DESTDIR}/${DISTDIR}/kernel.meta | \ ${XZ_CMD} > ${PACKAGEDIR}/kernel.txz .endif +.if ${MK_DEBUG_FILES} != "no" cd ${DESTDIR}/${DISTDIR}/kernel; \ tar cvf - --include '*/*/*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.meta | \ ${XZ_CMD} > ${DESTDIR}/${DISTDIR}/kernel-dbg.txz +.endif .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes" .for _kernel in ${BUILDKERNELS:[2..-1]} cd ${DESTDIR}/${DISTDIR}/kernel.${_kernel}; \ tar cvf - --exclude '*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta | \ ${XZ_CMD} > ${PACKAGEDIR}/kernel.${_kernel}.txz +.if ${MK_DEBUG_FILES} != "no" cd ${DESTDIR}/${DISTDIR}/kernel.${_kernel}; \ tar cvf - --include '*/*/*.debug' \ @${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta | \ ${XZ_CMD} > ${DESTDIR}/${DISTDIR}/kernel.${_kernel}-dbg.txz +.endif .endfor .endif .else @@ -1393,17 +1397,21 @@ packagekernel: .PHONY tar cvf - --exclude '*.debug' . | \ ${XZ_CMD} > ${PACKAGEDIR}/kernel.txz .endif +.if ${MK_DEBUG_FILES} != "no" cd ${DESTDIR}/${DISTDIR}/kernel; \ tar cvf - --include '*/*/*.debug' $$(eval find .) | \ ${XZ_CMD} > ${DESTDIR}/${DISTDIR}/kernel-dbg.txz +.endif .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes" .for _kernel in ${BUILDKERNELS:[2..-1]} cd ${DESTDIR}/${DISTDIR}/kernel.${_kernel}; \ tar cvf - --exclude '*.debug' . | \ ${XZ_CMD} > ${PACKAGEDIR}/kernel.${_kernel}.txz +.if ${MK_DEBUG_FILES} != "no" cd ${DESTDIR}/${DISTDIR}/kernel.${_kernel}; \ tar cvf - --include '*/*/*.debug' $$(eval find .) | \ ${XZ_CMD} > ${DESTDIR}/${DISTDIR}/kernel.${_kernel}-dbg.txz +.endif .endfor .endif .endif @@ -1439,49 +1447,80 @@ package-pkg: .PHONY real-packages: stage-packages create-packages sign-packages .PHONY -stage-packages: .PHONY - @mkdir -p ${REPODIR} ${WSTAGEDIR} ${KSTAGEDIR} +stage-packages-world: .PHONY + @mkdir -p ${WSTAGEDIR} ${_+_}@cd ${.CURDIR}; \ - ${MAKE} DESTDIR=${WSTAGEDIR} -DNO_ROOT -B stageworld ; \ - ${MAKE} DESTDIR=${KSTAGEDIR} -DNO_ROOT -B stagekernel + ${MAKE} DESTDIR=${WSTAGEDIR} -DNO_ROOT stageworld -create-packages: _pkgbootstrap .PHONY +stage-packages-kernel: .PHONY + @mkdir -p ${KSTAGEDIR} + ${_+_}@cd ${.CURDIR}; \ + ${MAKE} DESTDIR=${KSTAGEDIR} -DNO_ROOT stagekernel + +stage-packages: .PHONY stage-packages-world stage-packages-kernel + +_repodir: .PHONY @mkdir -p ${REPODIR} + +create-packages-world: _pkgbootstrap _repodir .PHONY ${_+_}@cd ${.CURDIR}; \ - ${MAKE} DESTDIR=${WSTAGEDIR} \ - PKG_VERSION=${PKG_VERSION} create-world-packages ; \ - ${MAKE} DESTDIR=${KSTAGEDIR} \ + ${MAKE} -f Makefile.inc1 \ + DESTDIR=${WSTAGEDIR} \ + PKG_VERSION=${PKG_VERSION} create-world-packages + +create-packages-kernel: _pkgbootstrap _repodir .PHONY + ${_+_}@cd ${.CURDIR}; \ + ${MAKE} -f Makefile.inc1 \ + DESTDIR=${KSTAGEDIR} \ PKG_VERSION=${PKG_VERSION} DISTDIR=kernel \ create-kernel-packages +create-packages: .PHONY create-packages-world create-packages-kernel + create-world-packages: _pkgbootstrap .PHONY @rm -f ${WSTAGEDIR}/*.plist 2>/dev/null || : @cd ${WSTAGEDIR} ; \ awk -f ${SRCDIR}/release/scripts/mtree-to-plist.awk \ ${WSTAGEDIR}/METALOG @for plist in ${WSTAGEDIR}/*.plist; do \ - plist=$${plist##*/} ; \ - pkgname=$${plist%.plist} ; \ - sh ${SRCDIR}/release/packages/generate-ucl.sh -o $${pkgname} \ - -s ${SRCDIR} -u ${WSTAGEDIR}/$${pkgname}.ucl ; \ - done - @for plist in ${WSTAGEDIR}/*.plist; do \ - plist=$${plist##*/} ; \ - pkgname=$${plist%.plist} ; \ - awk -F\" ' \ - /^name/ { printf("===> Creating %s-", $$2); next } \ - /^version/ { print $$2; next } \ - ' ${WSTAGEDIR}/$${pkgname}.ucl ; \ - ${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh -o ALLOW_BASE_SHLIBS=yes \ - create -M ${WSTAGEDIR}/$${pkgname}.ucl \ - -p ${WSTAGEDIR}/$${pkgname}.plist \ - -r ${WSTAGEDIR} \ - -o ${REPODIR}/$$(${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh config ABI)/${PKG_VERSION} ; \ - done + plist=$${plist##*/} ; \ + pkgname=$${plist%.plist} ; \ + echo "_PKGS+= $${pkgname}" ; \ + done > ${WSTAGEDIR}/packages.mk + ${_+_}@cd ${.CURDIR}; \ + ${MAKE} -f Makefile.inc1 create-world-packages-jobs \ + .MAKE.JOB.PREFIX= -create-kernel-packages: _pkgbootstrap .PHONY +.if make(create-world-packages-jobs) +.include "${WSTAGEDIR}/packages.mk" +.endif + +create-world-packages-jobs: .PHONY +.for pkgname in ${_PKGS} +create-world-packages-jobs: create-world-package-${pkgname} +create-world-package-${pkgname}: .PHONY + @sh ${SRCDIR}/release/packages/generate-ucl.sh -o ${pkgname} \ + -s ${SRCDIR} -u ${WSTAGEDIR}/${pkgname}.ucl + @awk -F\" ' \ + /^name/ { printf("===> Creating %s-", $$2); next } \ + /^version/ { print $$2; next } \ + ' ${WSTAGEDIR}/${pkgname}.ucl ; \ + ${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh -o ALLOW_BASE_SHLIBS=yes \ + create -M ${WSTAGEDIR}/${pkgname}.ucl \ + -p ${WSTAGEDIR}/${pkgname}.plist \ + -r ${WSTAGEDIR} \ + -o ${REPODIR}/$$(${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh config ABI)/${PKG_VERSION} +.endfor + +_default_flavor= -default .if exists(${KSTAGEDIR}/kernel.meta) -.for flavor in "" -debug +. if ${MK_DEBUG_FILES} != "no" +_debug=-debug +. endif +create-kernel-packages: .PHONY +. for flavor in "" ${_debug} +create-kernel-packages: create-kernel-packages-flavor${flavor:C,^""$,${_default_flavor},} +create-kernel-packages-flavor${flavor:C,^""$,${_default_flavor},}: _pkgbootstrap .PHONY @cd ${KSTAGEDIR}/${DISTDIR} ; \ awk -f ${SRCDIR}/release/scripts/mtree-to-plist.awk \ -v kernel=yes -v _kernconf=${INSTALLKERNEL} \ @@ -1506,12 +1545,17 @@ create-kernel-packages: _pkgbootstrap .PHONY -p ${KSTAGEDIR}/${DISTDIR}/kernel.${INSTALLKERNEL}${flavor}.plist \ -r ${KSTAGEDIR}/${DISTDIR} \ -o ${REPODIR}/$$(${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh config ABI)/${PKG_VERSION} -.endfor +. endfor .endif .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes" -.for _kernel in ${BUILDKERNELS:[2..-1]} -.if exists(${KSTAGEDIR}/kernel.${_kernel}.meta) -.for flavor in "" -debug +. for _kernel in ${BUILDKERNELS:[2..-1]} +. if exists(${KSTAGEDIR}/kernel.${_kernel}.meta) +. if ${MK_DEBUG_FILES} != "no" +_debug=-debug +. endif +. for flavor in "" ${_debug} +create-kernel-packages: create-kernel-packages-extra-flavor${flavor:C,^""$,${_default_flavor},}-${_kernel} +create-kernel-packages-extra-flavor${flavor:C,^""$,${_default_flavor},}-${_kernel}: _pkgbootstrap .PHONY @cd ${KSTAGEDIR}/kernel.${_kernel} ; \ awk -f ${SRCDIR}/release/scripts/mtree-to-plist.awk \ -v kernel=yes -v _kernconf=${_kernel} \ @@ -1536,9 +1580,9 @@ create-kernel-packages: _pkgbootstrap .PHONY -p ${KSTAGEDIR}/kernel.${_kernel}/kernel.${_kernel}${flavor}.plist \ -r ${KSTAGEDIR}/kernel.${_kernel} \ -o ${REPODIR}/$$(${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh config ABI)/${PKG_VERSION} -.endfor -.endif -.endfor +. endfor +. endif +. endfor .endif sign-packages: _pkgbootstrap .PHONY