Date: Wed, 5 Mar 2014 23:17:53 +0000 (UTC) From: Glen Barber <gjb@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r262810 - in head: release release/arm release/tools share/man/man7 Message-ID: <201403052317.s25NHrVB057044@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: gjb Date: Wed Mar 5 23:17:53 2014 New Revision: 262810 URL: http://svnweb.freebsd.org/changeset/base/262810 Log: Merge ^/projects/release-embedded into ^/head. After several months of testing and fixing (and breaking) various parts of release/release.sh changes, it is now possible to build FreeBSD/arm images as part of the release process. When EMBEDDEDBUILD is set in the release.conf file, release.sh will create the build environment, then run a separate script in release/${XDEV}/release.sh [1]. Currently, only arm is supported. The release/${XDEV}/release.sh configures the build environment specific for the target image, such as installing gcc(1), installing additional third-party software from the ports tree, and fetching external sources. Once the build environment is set up, release/${XDEV}/release.sh runs Crochet, written by Tim Kientzle, which builds the userland and kernel, and creates an image that can be written to an SD card with dd(1). Many thanks to Tim for his work on Crochet. Sample configurations for FreeBSD/arm boards are in the release/arm/ directory, and Crochet configuration files for each board are located in release/tools/arm/. Supported boards at this time are: BEAGLEBONE, PANDABOARD, RPI-B, and WANDBOARD-QUAD. Adding support for additional boards will continue in the projects/release-embedded/ branch, and incrementally merged back to head/. Many thanks to the FreeBSD Foundation for the support and sponsorship of this project. [1] XDEV is used in order to keep the various configurations organized by architecture, but since TARGET and TARGET_ARCH are used to build the chroot, the values of those variables cannot be used. MFC after: 1 month Sponsored by: The FreeBSD Foundation Added: head/release/arm/ - copied from r262314, projects/release-embedded/release/arm/ head/release/tools/ - copied from r262314, projects/release-embedded/release/tools/ Modified: head/release/arm/BEAGLEBONE.conf head/release/arm/PANDABOARD.conf head/release/arm/RPI-B.conf head/release/arm/WANDBOARD-QUAD.conf head/release/arm/release.sh head/release/release.conf.sample head/release/release.sh head/share/man/man7/release.7 Directory Properties: head/ (props changed) Modified: head/release/arm/BEAGLEBONE.conf ============================================================================== --- projects/release-embedded/release/arm/BEAGLEBONE.conf Fri Feb 21 23:18:35 2014 (r262314) +++ head/release/arm/BEAGLEBONE.conf Wed Mar 5 23:17:53 2014 (r262810) @@ -13,7 +13,7 @@ fi # Build chroot configuration TARGET="amd64" TARGET_ARCH="amd64" -SVNROOT="svn://svn.FreeBSD.org" +SVNROOT="svn://svn.FreeBSD.org/" SRCBRANCH="base/head@rHEAD" DOCBRANCH="doc/head@rHEAD" PORTBRANCH="ports/head@rHEAD" @@ -24,7 +24,7 @@ NODOC=yes # script, these must be exported. set -a WORLD_FLAGS="-j $(sysctl -n hw.ncpu)" -KERNEL_FLAGS="-j $(expr \( $(sysctl -n hw.ncpu) + 1 \) / 2)" +KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))" CHROOTDIR="/scratch" EMBEDDEDBUILD=1 EMBEDDEDPORTS="lang/python textproc/gsed" Modified: head/release/arm/PANDABOARD.conf ============================================================================== --- projects/release-embedded/release/arm/PANDABOARD.conf Fri Feb 21 23:18:35 2014 (r262314) +++ head/release/arm/PANDABOARD.conf Wed Mar 5 23:17:53 2014 (r262810) @@ -13,7 +13,7 @@ fi # Build chroot configuration TARGET="amd64" TARGET_ARCH="amd64" -SVNROOT="svn://svn.FreeBSD.org" +SVNROOT="svn://svn.FreeBSD.org/" SRCBRANCH="base/head@rHEAD" DOCBRANCH="doc/head@rHEAD" PORTBRANCH="ports/head@rHEAD" @@ -24,7 +24,7 @@ NODOC=yes # script, these must be exported. set -a WORLD_FLAGS="-j $(sysctl -n hw.ncpu)" -KERNEL_FLAGS="-j $(expr \( $(sysctl -n hw.ncpu) + 1 \) / 2)" +KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))" CHROOTDIR="/scratch" EMBEDDEDBUILD=1 EMBEDDEDPORTS="lang/python textproc/gsed" Modified: head/release/arm/RPI-B.conf ============================================================================== --- projects/release-embedded/release/arm/RPI-B.conf Fri Feb 21 23:18:35 2014 (r262314) +++ head/release/arm/RPI-B.conf Wed Mar 5 23:17:53 2014 (r262810) @@ -13,7 +13,7 @@ fi # Build chroot configuration TARGET="amd64" TARGET_ARCH="amd64" -SVNROOT="svn://svn.FreeBSD.org" +SVNROOT="svn://svn.FreeBSD.org/" SRCBRANCH="base/head@rHEAD" DOCBRANCH="doc/head@rHEAD" PORTBRANCH="ports/head@rHEAD" @@ -24,7 +24,7 @@ NODOC=yes # script, these must be exported. set -a WORLD_FLAGS="-j $(sysctl -n hw.ncpu)" -KERNEL_FLAGS="-j $(expr \( $(sysctl -n hw.ncpu) + 1 \) / 2)" +KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))" CHROOTDIR="/scratch" EMBEDDEDBUILD=1 EMBEDDEDPORTS="lang/python textproc/gsed" Modified: head/release/arm/WANDBOARD-QUAD.conf ============================================================================== --- projects/release-embedded/release/arm/WANDBOARD-QUAD.conf Fri Feb 21 23:18:35 2014 (r262314) +++ head/release/arm/WANDBOARD-QUAD.conf Wed Mar 5 23:17:53 2014 (r262810) @@ -13,7 +13,7 @@ fi # Build chroot configuration TARGET="amd64" TARGET_ARCH="amd64" -SVNROOT="svn://svn.FreeBSD.org" +SVNROOT="svn://svn.FreeBSD.org/" SRCBRANCH="base/head@rHEAD" DOCBRANCH="doc/head@rHEAD" PORTBRANCH="ports/head@rHEAD" @@ -24,7 +24,7 @@ NODOC=yes # script, these must be exported. set -a WORLD_FLAGS="-j $(sysctl -n hw.ncpu)" -KERNEL_FLAGS="-j $(expr \( $(sysctl -n hw.ncpu) + 1 \) / 2)" +KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))" CHROOTDIR="/scratch" EMBEDDEDBUILD=1 EMBEDDEDPORTS="lang/python textproc/gsed" Modified: head/release/arm/release.sh ============================================================================== --- projects/release-embedded/release/arm/release.sh Fri Feb 21 23:18:35 2014 (r262314) +++ head/release/arm/release.sh Wed Mar 5 23:17:53 2014 (r262810) @@ -1,4 +1,31 @@ #!/bin/sh +#- +# Copyright (c) 2013, 2014 The FreeBSD Foundation +# All rights reserved. +# +# This software was developed by Glen Barber +# under sponsorship from the FreeBSD Foundation. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. # # $FreeBSD$ # @@ -55,7 +82,7 @@ install_crochet() { install_uboot() { # Only fetch u-boot sources if UBOOTSRC is set; otherwise it is # not needed. - if [ "X${UBOOTSRC}" != "X" ]; then + if [ -n "${UBOOTSRC}" ]; then continue else return 0 Modified: head/release/release.conf.sample ============================================================================== --- head/release/release.conf.sample Wed Mar 5 22:43:30 2014 (r262809) +++ head/release/release.conf.sample Wed Mar 5 23:17:53 2014 (r262810) @@ -45,5 +45,17 @@ PORTBRANCH="ports/head@rHEAD" ## Set miscellaneous 'make release' settings. #NODOC= #NOPORTS= -#RELSTRING= #WITH_DVD= + +## Set when building embedded images. +#EMBEDDEDBUILD= + +## Set to skip the chroot environment buildworld/installworld/distribution +## step if it is expected the build environment will exist via alternate +## means. +#CHROOTBUILD_SKIP= + +## Set to pass additional flags to make(1) for the build chroot setup, such +## as TARGET/TARGET_ARCH. +#CHROOT_MAKEENV= + Modified: head/release/release.sh ============================================================================== --- head/release/release.sh Wed Mar 5 22:43:30 2014 (r262809) +++ head/release/release.sh Wed Mar 5 23:17:53 2014 (r262810) @@ -41,6 +41,7 @@ export PATH # The directory within which the release will be built. CHROOTDIR="/scratch" +RELENGDIR="$(realpath $(dirname $(basename ${0})))" # The default version control system command to obtain the sources. VCSCMD="svn checkout" @@ -52,6 +53,9 @@ SRCBRANCH="base/head@rHEAD" DOCBRANCH="doc/head@rHEAD" PORTBRANCH="ports/head@rHEAD" +# Set for embedded device builds. +EMBEDDEDBUILD= + # Sometimes one needs to checkout src with --force svn option. # If custom kernel configs copied to src tree before checkout, e.g. SRC_FORCE_CHECKOUT= @@ -108,9 +112,18 @@ SRCBRANCH="${SVNROOT}${SRCBRANCH}" DOCBRANCH="${SVNROOT}${DOCBRANCH}" PORTBRANCH="${SVNROOT}${PORTBRANCH}" +if [ -n "${EMBEDDEDBUILD}" ]; then + if [ -z "${XDEV}" ] || [ -z "${XDEV_ARCH}" ]; then + echo "ERROR: XDEV and XDEV_ARCH must be set in ${RELEASECONF}." + exit 1 + fi + WITH_DVD= + NODOC=yes +fi + # If PORTS is set and NODOC is unset, force NODOC=yes because the ports tree # is required to build the documentation set. -if [ "x${NOPORTS}" != "x" ] && [ "x${NODOC}" = "x" ]; then +if [ -n "${NOPORTS}" ] && [ -z "${NODOC}" ]; then echo "*** NOTICE: Setting NODOC=1 since ports tree is required" echo " and NOPORTS is set." NODOC=yes @@ -120,10 +133,10 @@ fi # The release makefile verifies definedness of NOPORTS/NODOC variables # instead of their values. DOCPORTS= -if [ "x${NOPORTS}" != "x" ]; then +if [ -n "${NOPORTS}" ]; then DOCPORTS="NOPORTS=yes " fi -if [ "x${NODOC}" != "x" ]; then +if [ -n "${NODOC}" ]; then DOCPORTS="${DOCPORTS}NODOC=yes" fi @@ -131,12 +144,12 @@ fi # this file, unless overridden by release.conf. In most cases, these # will not need to be changed. CONF_FILES="__MAKE_CONF=${MAKE_CONF} SRCCONF=${SRC_CONF}" -if [ "x${TARGET}" != "x" ] && [ "x${TARGET_ARCH}" != "x" ]; then +if [ -n "${TARGET}" ] && [ -n "${TARGET_ARCH}" ]; then ARCH_FLAGS="TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH}" else ARCH_FLAGS= fi -CHROOT_MAKEENV="MAKEOBJDIRPREFIX=${CHROOTDIR}/tmp/obj" +CHROOT_MAKEENV="${CHROOT_MAKEENV} MAKEOBJDIRPREFIX=${CHROOTDIR}/tmp/obj" CHROOT_WMAKEFLAGS="${MAKE_FLAGS} ${WORLD_FLAGS} ${CONF_FILES}" CHROOT_IMAKEFLAGS="${CONF_FILES}" CHROOT_DMAKEFLAGS="${CONF_FILES}" @@ -147,11 +160,11 @@ RELEASE_RMAKEFLAGS="${ARCH_FLAGS} KERNCO # Force src checkout if configured FORCE_SRC_KEY= -if [ "x${SRC_FORCE_CHECKOUT}" != "x" ]; then +if [ -n "${SRC_FORCE_CHECKOUT}" ]; then FORCE_SRC_KEY="--force" fi -if [ ! ${CHROOTDIR} ]; then +if [ -z "${CHROOTDIR}" ]; then echo "Please set CHROOTDIR." exit 1 fi @@ -166,19 +179,21 @@ set -e # Everything must succeed mkdir -p ${CHROOTDIR}/usr ${VCSCMD} ${FORCE_SRC_KEY} ${SRCBRANCH} ${CHROOTDIR}/usr/src -if [ "x${NODOC}" = "x" ]; then +if [ -z "${NODOC}" ]; then ${VCSCMD} ${DOCBRANCH} ${CHROOTDIR}/usr/doc fi -if [ "x${NOPORTS}" = "x" ]; then +if [ -z "${NOPORTS}" ]; then ${VCSCMD} ${PORTBRANCH} ${CHROOTDIR}/usr/ports fi -cd ${CHROOTDIR}/usr/src -env ${CHROOT_MAKEENV} make ${CHROOT_WMAKEFLAGS} buildworld -env ${CHROOT_MAKEENV} make ${CHROOT_IMAKEFLAGS} installworld \ - DESTDIR=${CHROOTDIR} -env ${CHROOT_MAKEENV} make ${CHROOT_DMAKEFLAGS} distribution \ - DESTDIR=${CHROOTDIR} +if [ -z "${CHROOTBUILD_SKIP}" ]; then + cd ${CHROOTDIR}/usr/src + env ${CHROOT_MAKEENV} make ${CHROOT_WMAKEFLAGS} buildworld + env ${CHROOT_MAKEENV} make ${CHROOT_IMAKEFLAGS} installworld \ + DESTDIR=${CHROOTDIR} + env ${CHROOT_MAKEENV} make ${CHROOT_DMAKEFLAGS} distribution \ + DESTDIR=${CHROOTDIR} +fi mount -t devfs devfs ${CHROOTDIR}/dev cp /etc/resolv.conf ${CHROOTDIR}/etc/resolv.conf trap "umount ${CHROOTDIR}/dev" EXIT # Clean up devfs mount on exit @@ -194,6 +209,29 @@ if [ -e ${SRC_CONF} ] && [ ! -c ${SRC_CO cp ${SRC_CONF} ${CHROOTDIR}/${SRC_CONF} fi +# Embedded builds do not use the 'make release' target. +if [ -n "${EMBEDDEDBUILD}" ]; then + # If a crochet configuration file exists in *this* checkout of + # release/, copy it to the /tmp/external directory within the chroot. + # This allows building embedded releases without relying on updated + # scripts and/or configurations to exist in the branch being built. + if [ -e ${RELENGDIR}/tools/${XDEV}/crochet-${KERNEL}.conf ] && \ + [ -e ${RELENGDIR}/${XDEV}/release.sh ]; then + mkdir -p ${CHROOTDIR}/tmp/external/${XDEV}/ + cp ${RELENGDIR}/tools/${XDEV}/crochet-${KERNEL}.conf \ + ${CHROOTDIR}/tmp/external/${XDEV}/crochet-${KERNEL}.conf + /bin/sh ${RELENGDIR}/${XDEV}/release.sh + fi + # If the script does not exist for this architecture, exit. + # This probably should be checked earlier, but allowing the rest + # of the build process to get this far will at least set up the + # chroot environment for testing. + exit 0 +else + # Not embedded. + continue +fi + if [ -d ${CHROOTDIR}/usr/ports ]; then # Run ldconfig(8) in the chroot directory so /var/run/ld-elf*.so.hints # is created. This is needed by ports-mgmt/pkg. @@ -201,7 +239,7 @@ if [ -d ${CHROOTDIR}/usr/ports ]; then ## Trick the ports 'run-autotools-fixup' target to do the right thing. _OSVERSION=$(sysctl -n kern.osreldate) - if [ -d ${CHROOTDIR}/usr/doc ] && [ "x${NODOC}" = "x" ]; then + if [ -d ${CHROOTDIR}/usr/doc ] && [ -z "${NODOC}" ]; then PBUILD_FLAGS="OSVERSION=${_OSVERSION} BATCH=yes" PBUILD_FLAGS="${PBUILD_FLAGS}" chroot ${CHROOTDIR} make -C /usr/ports/textproc/docproj \ @@ -209,13 +247,9 @@ if [ -d ${CHROOTDIR}/usr/ports ]; then fi fi -if [ "x${RELSTRING}" = "x" ]; then - RELSTRING="$(chroot ${CHROOTDIR} uname -s)-${OSRELEASE}-${TARGET_ARCH}" -fi - eval chroot ${CHROOTDIR} make -C /usr/src ${RELEASE_WMAKEFLAGS} buildworld eval chroot ${CHROOTDIR} make -C /usr/src ${RELEASE_KMAKEFLAGS} buildkernel eval chroot ${CHROOTDIR} make -C /usr/src/release ${RELEASE_RMAKEFLAGS} \ - release RELSTRING=${RELSTRING} + release eval chroot ${CHROOTDIR} make -C /usr/src/release ${RELEASE_RMAKEFLAGS} \ - install DESTDIR=/R RELSTRING=${RELSTRING} + install DESTDIR=/R Modified: head/share/man/man7/release.7 ============================================================================== --- head/share/man/man7/release.7 Wed Mar 5 22:43:30 2014 (r262809) +++ head/share/man/man7/release.7 Wed Mar 5 23:17:53 2014 (r262810) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd February 25, 2014 +.Dd March 5, 2014 .Dt RELEASE 7 .Os .Sh NAME @@ -138,6 +138,11 @@ configuration file supports the followin .Bl -tag -width Ev .It Va CHROOTDIR The directory within which the release will be built. +.It Va CHROOT_MAKEENV +Additional +.Xr make 1 +arguments to pass through, which directly affect the +tuning of the build chroot. .It Va SVNROOT The .Xr svn 1 @@ -249,6 +254,72 @@ target. The command run to obtain the source trees. Defaults to .Qq Cm svn checkout . +.It Va CHROOTBUILD_SKIP +If defined, the +.Li buildworld , +.Li installworld , +and +.Li distribution +stages of the +.Xr chroot 8 +build environment setup are skipped. +This is intended solely for cases where the +.Xr chroot 8 +userland are provided by alternate means. +.El +.Sh EMBEDDED BUILDS +The following +.Fa release.conf +variables are relevant only to release builds for embedded systems: +.Bl -tag -width Ev +.It Va EMBEDDEDBUILD +Set to a non-null value to enable functionality for embedded device +release builds. +.Pq This option is considered highly experimental. +.Pp +When set, +.Va WITH_DVD +is unset, and +.Va NODOC +is defined. +Additionally, +.Va XDEV +and +.Va XDEV_ARCH +must also be defined. +When the build environment is created, +.Fa release.sh +runs a separate build script located in an architecture-specific +directory in +.Pa src/release/${XDEV}/ . +.It Va EMBEDDEDPORTS +Set to the list of any ports that are required for the target device +in the format of +.Fa category/port . +The +.Fa devel/subversion +port is built by default. +.It Va CROCHETSRC +Set to the source URL for the Crochet build tool. +.It Va CROCHETBRANCH +Set to the subversion branch from +.Va ${CROCHETSRC} +to use. +Defaults to +.Pa trunk . +.It Va UBOOTSRC +Set to the source URL of u-boot, if required. +.It Va UBOOTBRANCH +Set to the subversion branch from +.Va ${UBOOTSRC} +to use. +Defaults to +.Pa trunk . +.It Va UBOOTDIR +Set to the target directory within +.Va ${CHROOTDIR} +to check out +.Va ${UBOOTSRC}/${UBOOTBRANCH} . .El .Sh MAKEFILE TARGETS The release makefile
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201403052317.s25NHrVB057044>