From owner-svn-src-head@freebsd.org Tue Sep 3 14:06:16 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 14EB0DC428; Tue, 3 Sep 2019 14:06:09 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46N7z36HwGz4PDc; Tue, 3 Sep 2019 14:06:07 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1452) id 351081A1AF; Tue, 3 Sep 2019 14:05:59 +0000 (UTC) X-Original-To: yuripv@localmail.freebsd.org Delivered-To: yuripv@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [96.47.72.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id CB81E62CE; Wed, 3 Apr 2019 21:54:51 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44C9F6F6B0; Wed, 3 Apr 2019 21:54:51 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 538) id 1C12A62A5; Wed, 3 Apr 2019 21:54:51 +0000 (UTC) Delivered-To: src-committers@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 94D4162A3 for ; Wed, 3 Apr 2019 21:54:48 +0000 (UTC) (envelope-from cperciva@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4AD7D6F6AC; Wed, 3 Apr 2019 21:54:48 +0000 (UTC) (envelope-from cperciva@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 245011C0EE; Wed, 3 Apr 2019 21:54:48 +0000 (UTC) (envelope-from cperciva@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x33LsmNr013078; Wed, 3 Apr 2019 21:54:48 GMT (envelope-from cperciva@FreeBSD.org) Received: (from cperciva@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x33LslFs013075; Wed, 3 Apr 2019 21:54:47 GMT (envelope-from cperciva@FreeBSD.org) Message-Id: <201904032154.x33LslFs013075@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cperciva set sender to cperciva@FreeBSD.org using -f From: Colin Percival To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r345858 - in head/release: . tools X-SVN-Group: head X-SVN-Commit-Author: cperciva X-SVN-Commit-Paths: in head/release: . tools X-SVN-Commit-Revision: 345858 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk X-Loop: FreeBSD.org Sender: owner-src-committers@freebsd.org X-Rspamd-Queue-Id: 44C9F6F6B0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Status: O X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Tue, 03 Sep 2019 14:06:17 -0000 X-Original-Date: Wed, 3 Apr 2019 21:54:47 +0000 (UTC) X-List-Received-Date: Tue, 03 Sep 2019 14:06:17 -0000 Author: cperciva Date: Wed Apr 3 21:54:47 2019 New Revision: 345858 URL: https://svnweb.freebsd.org/changeset/base/345858 Log: Add support for cross-building cloudware images. If MACHINE_ARCH doesn't match TARGET_ARCH, and we're not in the special case of building i386 images on an amd64 host, we need to pull in the qemu-user-static package; this allows us to run some commands inside the VM disk image chroot, most notably to install packages. Reviewed by: gjb MFC after: 2 weeks Sponsored by: FreeBSD/EC2 patreon (https://www.patreon.com/cperciva) Modified: head/release/Makefile.vm head/release/tools/ec2.conf head/release/tools/vmimage.subr Modified: head/release/Makefile.vm ============================================================================== --- head/release/Makefile.vm Wed Apr 3 21:48:09 2019 (r345857) +++ head/release/Makefile.vm Wed Apr 3 21:54:47 2019 (r345858) @@ -39,6 +39,24 @@ VAGRANT-VMWARE_FORMAT= vmdk VAGRANT-VMWARE_DESC= Vagrant Image for VMWare VAGRANT-VMWARE_DISK= ${OSRELEASE}.vmware.${VAGRANT_FORMAT} +emulator-portinstall: +.if ${TARGET_ARCH} != ${MACHINE_ARCH} +.if ( ${TARGET_ARCH} != "i386" ) || ( ${MACHINE_ARCH} != "amd64" ) +.if !exists(/usr/local/bin/qemu-${TARGET_ARCH}-static) +.if exists(${PORTSDIR}/emulators/qemu-user-static/Makefile) + env - PATH=$$PATH make -C ${PORTSDIR}/emulators/qemu-user-static BATCH=1 all install clean +.else +.if !exists(/usr/local/sbin/pkg-static) + env ASSUME_ALWAYS_YES=yes pkg bootstrap -y +.endif + env ASSUME_ALWAYS_YES=yes pkg install -y emulators/qemu-user-static +.endif +.endif + +QEMUSTATIC=/usr/local/bin/qemu-${TARGET_ARCH}-static +.endif +.endif + .if defined(WITH_CLOUDWARE) && !empty(WITH_CLOUDWARE) && !empty(CLOUDWARE) . for _CW in ${CLOUDWARE} CLOUDTARGETS+= cw-${_CW:tl} @@ -53,9 +71,10 @@ ${_CW:tu}IMAGE= ${_CW:tl}.${${_CW:tu}_FORMAT} ${_CW:tu}CONF?= ${.CURDIR}/tools/${_CW:tl}.conf . endif -cw-${_CW:tl}: +cw-${_CW:tl}: emulator-portinstall mkdir -p ${.OBJDIR}/${.TARGET} env TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} SWAPSIZE=${SWAPSIZE} \ + QEMUSTATIC=${QEMUSTATIC} \ ${.CURDIR}/scripts/mk-vmimage.sh \ -C ${.CURDIR}/tools/vmimage.subr -d ${.OBJDIR}/${.TARGET} \ -i ${.OBJDIR}/${_CW:tl}.img -s ${VMSIZE} -f ${${_CW:tu}_FORMAT} \ Modified: head/release/tools/ec2.conf ============================================================================== --- head/release/tools/ec2.conf Wed Apr 3 21:48:09 2019 (r345857) +++ head/release/tools/ec2.conf Wed Apr 3 21:54:47 2019 (r345858) @@ -40,7 +40,7 @@ vm_extra_pre_umount() { # catalogue and install or update pkg when the instance first # launches, so these files would just be replaced anyway; removing # them from the image allows it to boot faster. - chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ + chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \ /usr/sbin/pkg delete -f -y pkg rm ${DESTDIR}/var/db/pkg/repo-*.sqlite Modified: head/release/tools/vmimage.subr ============================================================================== --- head/release/tools/vmimage.subr Wed Apr 3 21:48:09 2019 (r345857) +++ head/release/tools/vmimage.subr Wed Apr 3 21:54:47 2019 (r345858) @@ -148,10 +148,15 @@ vm_install_base() { hostname="$(echo $(uname -o) | tr '[:upper:]' '[:lower:]')" echo "hostname=\"${hostname}\"" >> ${DESTDIR}/etc/rc.conf + if ! [ -z "${QEMUSTATIC}" ]; then + export EMULATOR=/qemu + cp ${QEMUSTATIC} ${DESTDIR}/${EMULATOR} + fi + mkdir -p ${DESTDIR}/dev mount -t devfs devfs ${DESTDIR}/dev - chroot ${DESTDIR} /usr/bin/newaliases - chroot ${DESTDIR} /etc/rc.d/ldconfig forcestart + chroot ${DESTDIR} ${EMULATOR} /usr/bin/newaliases + chroot ${DESTDIR} ${EMULATOR} /bin/sh /etc/rc.d/ldconfig forcestart umount_loop ${DESTDIR}/dev cp /etc/resolv.conf ${DESTDIR}/etc/resolv.conf @@ -188,9 +193,9 @@ vm_extra_install_packages() { fi mkdir -p ${DESTDIR}/dev mount -t devfs devfs ${DESTDIR}/dev - chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ + chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \ /usr/sbin/pkg bootstrap -y - chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ + chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \ /usr/sbin/pkg install -y ${VM_EXTRA_PACKAGES} umount_loop ${DESTDIR}/dev @@ -210,13 +215,16 @@ vm_extra_pre_umount() { # Note: When overriding this function, removing resolv.conf in the # disk image must be included. + if ! [ -z "${QEMUSTATIC}" ]; then + rm -f ${DESTDIR}/${EMULATOR} + fi rm -f ${DESTDIR}/etc/resolv.conf return 0 } vm_extra_pkg_rmcache() { if [ -e ${DESTDIR}/usr/local/sbin/pkg ]; then - chroot ${DESTDIR} env ASSUME_ALWAYS_YES=yes \ + chroot ${DESTDIR} ${EMULATOR} env ASSUME_ALWAYS_YES=yes \ /usr/local/sbin/pkg clean -y -a fi