From owner-svn-src-all@freebsd.org Tue Nov 6 07:10:39 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6CF611071F2; Tue, 6 Nov 2018 07:10:38 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 123DA8FE25; Tue, 6 Nov 2018 07:10:38 +0000 (UTC) (envelope-from melounmichal@gmail.com) Received: by mail-wm1-x32f.google.com with SMTP id b203-v6so10357271wme.5; Mon, 05 Nov 2018 23:10:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:reply-to:subject:to:references:openpgp:autocrypt:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=DbYy6GRWpnK99uuUsMV1pXUdHFeZboNvwwK4jSTTnec=; b=AkWpb2xrr2NDq2ovLADCNDnftMD6UOc+67hTEF7YdkwCNLk+ZWXJ1ytn0/svhKgqd8 G1LZ5Qrtj74nl70ejScLRKjwgJQ8xuFnl5SpcXC3x+CwRaXHwZybkqkyFVUo4/PKOH41 ULnA7vEuGRQ1N3M0dWAGJ30HriVBTBGr0gvI32DHPLaCRgzA2O+wWTq6xJVmPpZryE2E cE2u5eahk6/uSOL4EbR2CTzIC6nxcq6xDKPxVHZ+8eapy7QP1Tg53t84gc5YaJfKKtUU dyqxsIiACn6xMvYR0cokwYx3fubuP9+d/lFYMoh9D4GVWSLiEgCpiOsd2NFxAhd9LXUU OnLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:reply-to:subject:to:references:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=DbYy6GRWpnK99uuUsMV1pXUdHFeZboNvwwK4jSTTnec=; b=FewXH8KUP4kt1OzrnUGwJqCQ9w5zwBflkauZ7geCjZl/WUgRYTo5Da4IS04sRZuk28 y5UpbVLkoDKFleaSyNJCV7hvHPu4k/bA41sLohdZpzjpXEbcquCUAq8IjHayS6EeWB4S 0H4dwy3BnEtm7Pn7HvVj8xGiZ/wyCRrN8d4e51Q2iJx0soNWk9NySo7Z+bH/vMlACPmQ UEHronAZix9OkcvXNDay4gyrFjTBndZwYqkEUNo+LiiXapWc608gb3DHutOhbvBwPzng lZLRJFjLq0a+VZxH+fSfHg2tBmyRN8sx6VGWAtNKtItc+qRCgQ/46kU7Ppb+sFgJvLhg 6ohQ== X-Gm-Message-State: AGRZ1gKYyNBZjUOa00F7MUtOqVLfS9Us5FDxMyyqZ8NOFLDiDM71L1El Z2r59gt0Sd+BP6TLz1PyczivoCNF X-Google-Smtp-Source: AJdET5f1p67Ovt2whlnbDYBTAIuCFRFewnYSzA1s4Rxy9UUzsA/hDZxcsMVR1ylCVAy0eVeC1ZbX/Q== X-Received: by 2002:a1c:27c5:: with SMTP id n188-v6mr897668wmn.88.1541488236562; Mon, 05 Nov 2018 23:10:36 -0800 (PST) Received: from [88.208.79.100] (halouny.humusoft.cz. [88.208.79.100]) by smtp.gmail.com with ESMTPSA id h9-v6sm4483878wrw.90.2018.11.05.23.10.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Nov 2018 23:10:35 -0800 (PST) From: Michal Meloun X-Google-Original-From: Michal Meloun Reply-To: mmel@freebsd.org Subject: Re: svn commit: r340157 - in head: . tools/build tools/tools/nanobsd/embedded To: Alex Richardson , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201811051951.wA5JpB5q088141@repo.freebsd.org> Openpgp: preference=signencrypt Autocrypt: addr=mmel@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFYuVRkBCADZiwLCCne3wG9b9k+R2Neo5zVo2bLaZRfNNY/v9kg283i0sb1Da4EdEiNT 15El5UyozhphUIbIR/zrVpxF1TvvFdoCyzx6a68bNY2d9dBrDcNDZC+XnyDdHQoobN87DWT1 mRVkmbg9LHZ/SVUOkGYuWyE+8UYeDAcUizuXwSK5zFWmeTyIoWNa68ifrWLfQe0p4x5jC/AI VURCi17p360vU4fhgwoMvEEhrRBWCr4DYHToFjIt2WdBy3GR1qoO0+Xkd6G+OoBULo+XDfgu L2WdPvh0K69F9/LgHkMmG5Il7SCe62QGpG2vaCgRV7BQhLX+kxlvM+WrdRatWRml4Y/3ABEB AAHNIE1pY2hhbCBNZWxvdW4gPG1tZWxAZnJlZWJzZC5vcmc+wsCABBMBCgAqAhsDBQsJCAcD BRUKCQgLBRYDAgEAAh4BAheAAhkBBQJZjBHDBQkHICOqAAoJEGkesmtexaqqIKMIAJ9xTp1w ge86ns2ZYOac5++mAgpFatohSlxYUR3gwud3Y3Ej0eumavpv/C26N6dsLnspwRenKdLbIPKe 0N8lI7CcDBIJGiFyY3c4H79QjIkYpRgbWFyCM85zEyVJpB+U7BhsgXE2uwVjE9RNhEP0KBoj sp357uqq1B1+VUO4GJ+RjdmYSOcNrjR8tTfy02456qovGjJ4JcJBlhyK6GzBKvnZSoA0s+QP OMn3gd8gdomMLEJdS3kTsfhLh2rQPZa9EmzafIyjXrirWq4+4fVFgd8SiMZyyTM+Kz30ZSUe 6SmfaQTQ/WLRIl5jku2uYQWlrRIKT9xaQzRWtZO9UgtXFRHOwE0EVi5VGQEIALqgRkfS21D/ OqWE9mXfh2bIjrp9uC8T0MCuimbsrAdLKNNorGu2nE+rebgX8n5nYM377HOnalPGyOuXvCbQ 8MFVRdWOHxenJjXJialNdBsOf2wLva3vSSVsdoPzibWDIcJqhBOQ3EuhsILyWSPvYYKEiy95 mfhrDtuTTOAYVR9aNQBOENztB2TDJyMx/qZmtGroGV3N0Hqde/znHPtQO8RG5/FQGMfHMI5G FMuycr1ceHnLo/ovrqAl4TYV+UHSHJ+FDE9dt9wXHclWbWbC0yNugchZq6rho5Jjfv4a2v7P pyn3HoDinh1lWP7hYA0ZNExGHekLnXWVqO/lzGS6bMEAEQEAAcLAZQQYAQoADwIbDAUCWYwR wwUJByAjqgAKCRBpHrJrXsWqqrsrB/4g4ESK5TLxUxi8pLWcLPyvwtN4Fmf7VsCVefkhakaG rDPmfvfnG+OFwN60Xqoni7GBeakl01xwT4RINfvVfShDy6cHpLS7QL/M8pzfulVX38MkVkOD yGZhwjE+jyT/kZNA1Olaw3N3IefHq3brskQ7G4d9oPep2DDbw7C4Q76uOBjxy34JVB0WOsB6 NyMQB9h6LGljQtdEddyUqwnRZzzHiGvp0hPtdYQHQZlqbj4FV9lTRK7a8Ega+y7MgmeMiztG zeXyjNP02r3PRHCPagwa57bPxH2aAh4Q7UzBBZ0GTMm7DLKNtCP58WDxblrrhZ+7kHqGK8Fs bdeUpDdEYLVd Message-ID: Date: Tue, 6 Nov 2018 08:10:34 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <201811051951.wA5JpB5q088141@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 123DA8FE25 X-Spamd-Result: default: False [-3.67 / 200.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[mmel@freebsd.org]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.96)[-0.957,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; IP_SCORE(-0.70)[ipnet: 2a00:1450::/32(-1.96), asn: 15169(-1.48), country: US(-0.08)]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; MID_RHS_MATCH_TO(1.00)[]; RCVD_IN_DNSWL_NONE(0.00)[f.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 07:10:39 -0000 On 05.11.2018 20:51, Alex Richardson wrote: > Author: arichardson > Date: Mon Nov 5 19:51:10 2018 > New Revision: 340157 > URL: https://svnweb.freebsd.org/changeset/base/340157 > > Log: > Allow building world without inheriting $PATH > > Inheriting $PATH during the build phase can cause the build to fail when > compiling on a different system due to missing build tools or incompatible > versions somewhere in $PATH. This has cause build failures for us before > due to the jenkins slaves still running FreeBSD 10. > Listing the tools we depend on explicitly instead of just using whatever > happens to be in $PATH allows us to check that we don't accidentally add a > new build dependency. > > All tools that do no need to be bootstrapped will now be symlinked to > ${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain > ${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can > be set to force compiling almost all bootstrap tools instead of symlinking > them. This will not bootstrap tools such as cp,mv, etc. since they may be > used during the build and for those we should really only be using POSIX > compatible options. > > Furthermore, this change is required in order to be able to build on > non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they > often accept different flags or produce incompatible output. > > Approved By: brooks (mentor) > Differential Revision: https://reviews.freebsd.org/D16815 > > Modified: > head/Makefile > head/Makefile.inc1 > head/tools/build/Makefile > head/tools/tools/nanobsd/embedded/common > > Modified: head/Makefile > ============================================================================== > --- head/Makefile Mon Nov 5 19:25:57 2018 (r340156) > +++ head/Makefile Mon Nov 5 19:51:10 2018 (r340157) > @@ -610,10 +610,13 @@ _need_lld_${target}_${target_arch} != \ > # XXX: Passing HOST_OBJTOP into the PATH would allow skipping legacy, > # bootstrap-tools, and cross-tools. Need to ensure each tool actually > # supports all TARGETS though. > +# For now we only pass UNIVERSE_TOOLCHAIN_PATH which will be added at the end > +# of STRICTTMPPATH to ensure that the target-specific binaries come first. > MAKE_PARAMS_${target}+= \ > XCC="${HOST_OBJTOP}/tmp/usr/bin/cc" \ > XCXX="${HOST_OBJTOP}/tmp/usr/bin/c++" \ > - XCPP="${HOST_OBJTOP}/tmp/usr/bin/cpp" > + XCPP="${HOST_OBJTOP}/tmp/usr/bin/cpp" \ > + UNIVERSE_TOOLCHAIN_PATH=${HOST_OBJTOP}/tmp/usr/bin > .endif > .if defined(_need_lld_${target}_${target_arch}) && \ > ${_need_lld_${target}_${target_arch}} == "yes" > > Modified: head/Makefile.inc1 > ============================================================================== > --- head/Makefile.inc1 Mon Nov 5 19:25:57 2018 (r340156) > +++ head/Makefile.inc1 Mon Nov 5 19:51:10 2018 (r340157) > @@ -580,8 +580,21 @@ BUILD_ARCH!= uname -p > WORLDTMP?= ${OBJTOP}/tmp > BPATH= ${CCACHE_WRAPPER_PATH_PFX}${WORLDTMP}/legacy/usr/sbin:${WORLDTMP}/legacy/usr/bin:${WORLDTMP}/legacy/bin > XPATH= ${WORLDTMP}/usr/sbin:${WORLDTMP}/usr/bin > -STRICTTMPPATH= ${BPATH}:${XPATH} > + > +# When building we want to find the cross tools before the host tools in ${BPATH}. > +# We also need to add UNIVERSE_TOOLCHAIN_PATH so that we can find the shared > +# toolchain files (clang, lld, etc.) during make universe/tinderbox > +STRICTTMPPATH= ${XPATH}:${BPATH}:${UNIVERSE_TOOLCHAIN_PATH} > +# We should not be using tools from /usr/bin accidentally since this could cause > +# the build to break on other systems that don't have that tool. For now we > +# still allow using the old behaviour (inheriting $PATH) if > +# BUILD_WITH_STRICT_TMPPATH is set to 0 but this will eventually be removed. > +BUILD_WITH_STRICT_TMPPATH?=1 > +.if ${BUILD_WITH_STRICT_TMPPATH} != 0 > +TMPPATH= ${STRICTTMPPATH} > +.else > TMPPATH= ${STRICTTMPPATH}:${PATH} > +.endif > > # > # Avoid running mktemp(1) unless actually needed. > @@ -589,8 +602,16 @@ TMPPATH= ${STRICTTMPPATH}:${PATH} > # when in the middle of installing over this system. > # > .if make(distributeworld) || make(installworld) || make(stageworld) > -INSTALLTMP!= mktemp -d -u -t install > +.if ${BUILD_WITH_STRICT_TMPPATH} != 0 > +MKTEMP=${WORLDTMP}/legacy/usr/bin/mktemp > +.if !exists(${MKTEMP}) > +.error "mktemp binary doesn't exist in expected location: ${MKTEMP}" > .endif > +.else > +MKTEMP=mktemp > +.endif > +INSTALLTMP!= ${MKTEMP} -d -u -t install > +.endif > > .if make(stagekernel) || make(distributekernel) > TAGS+= kernel > @@ -647,6 +668,8 @@ CROSSENV+= BUILD_TOOLS_META=.NOMETA > CROSSENV+= ${TARGET_CFLAGS} > .endif > > +BOOTSTRAPPING_OSRELDATE?=${OSRELDATE} > + > # bootstrap-tools stage > BMAKEENV= INSTALL="sh ${.CURDIR}/tools/install.sh" \ > TOOLS_PREFIX=${TOOLS_PREFIX_UNDEF:U${WORLDTMP}} \ > @@ -658,7 +681,7 @@ BSARGS= DESTDIR= \ > OBJTOP='${WORLDTMP}/obj-tools' \ > OBJROOT='$${OBJTOP}/' \ > MAKEOBJDIRPREFIX= \ > - BOOTSTRAPPING=${OSRELDATE} \ > + BOOTSTRAPPING=${BOOTSTRAPPING_OSRELDATE} \ > BWPHASE=${.TARGET:C,^_,,} \ > SSP_CFLAGS= \ > MK_HTML=no NO_LINT=yes MK_MAN=no \ > @@ -680,7 +703,7 @@ TMAKE= \ > ${BMAKEENV} ${MAKE} ${WORLD_FLAGS} -f Makefile.inc1 \ > TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} \ > DESTDIR= \ > - BOOTSTRAPPING=${OSRELDATE} \ > + BOOTSTRAPPING=${BOOTSTRAPPING_OSRELDATE} \ > BWPHASE=${.TARGET:C,^_,,} \ > SSP_CFLAGS= \ > -DNO_LINT \ > @@ -705,7 +728,7 @@ KTMAKE= \ > OBJTOP='${WORLDTMP}/obj-kernel-tools' \ > OBJROOT='$${OBJTOP}/' \ > MAKEOBJDIRPREFIX= \ > - BOOTSTRAPPING=${OSRELDATE} \ > + BOOTSTRAPPING=${BOOTSTRAPPING_OSRELDATE} \ > SSP_CFLAGS= \ > MK_HTML=no -DNO_LINT MK_MAN=no \ > -DNO_PIC MK_PROFILE=no -DNO_SHARED \ > @@ -874,6 +897,9 @@ WORLDTMP_MTREEFLAGS= -deUW > DESTDIR_MTREEFLAGS+= -W > .endif > MTREE?= mtree > +.if ${BUILD_WITH_STRICT_TMPPATH} != 0 > +MTREE= ${WORLDTMP}/legacy/usr/sbin/mtree > +.endif > WORLDTMP_MTREE= ${MTREE} ${WORLDTMP_MTREEFLAGS} > DESTDIR_MTREE= ${MTREE} ${DESTDIR_MTREEFLAGS} > > @@ -948,10 +974,15 @@ _worldtmp: .PHONY > .if !defined(NO_CLEAN) > rm -rf ${WORLDTMP} > .else > +# Note: for delete-old we need to set $PATH to also include the host $PATH > +# since otherwise a partial build with missing symlinks in ${WORLDTMP}/legacy/ > +# will fail to run due to missing binaries. $WMAKE sets PATH to only ${TMPPATH} > +# so we remove that assingnment from $WMAKE and prepend the new $PATH > ${_+_}@if [ -e "${WORLDTMP}" ]; then \ > echo ">>> Deleting stale files in build tree..."; \ > - cd ${.CURDIR}; ${WMAKE} -DBATCH_DELETE_OLD_FILES _NO_INCLUDE_COMPILERMK=t \ > - delete-old delete-old-libs >/dev/null; \ > + cd ${.CURDIR}; env PATH=${TMPPATH}:${PATH} ${WMAKE:NPATH=*} \ > + _NO_INCLUDE_COMPILERMK=t -DBATCH_DELETE_OLD_FILES delete-old \ > + delete-old-libs >/dev/null; \ > fi > rm -rf ${WORLDTMP}/legacy/usr/include > .if ${USING_SYSTEM_COMPILER} == "yes" > @@ -972,6 +1003,11 @@ _worldtmp: .PHONY > # available on the target system (this happens e.g. when building on non-FreeBSD) > cd ${.CURDIR}/tools/build; \ > ${MAKE} DIRPRFX=tools/build/ DESTDIR=${WORLDTMP}/legacy installdirs > +# In order to build without inheriting $PATH we need to add symlinks to the host > +# tools in $WORLDTMP for the tools that we don't build during bootstrap-tools > + cd ${.CURDIR}/tools/build; \ > + ${MAKE} DIRPRFX=tools/build/ DESTDIR=${WORLDTMP}/legacy host-symlinks > + > _legacy: > @echo > @echo "--------------------------------------------------------------" > @@ -1969,6 +2005,29 @@ legacy: .PHONY > # > _bt= _bootstrap-tools > > +# We want to run the build with only ${WORLDTMP} in $PATH to ensure we don't > +# accidentally run tools that are incompatible but happen to be in $PATH. > +# This is especially important when building on Linux/MacOS where many of the > +# programs used during the build accept different flags or generate different > +# output. On those platforms we only symlink the tools known to be compatible > +# (e.g. basic utilities such as mkdir) into ${WORLDTMP} and build all others > +# from the FreeBSD sources during the bootstrap-tools stage. > +# We want to build without the user's $PATH starting in the bootstrap-tools > +# phase so the tools used in that phase (ln, cp, etc) must have already been > +# linked to $WORLDTMP. The tools are listed in the _host_tools_to_symlink > +# variable in tools/build/Makefile and are linked during the legacy phase. > +# Since they could be Linux or MacOS binaries, too we must only use flags that > +# are portable across operating systems. > + > +# If BOOTSTRAP_ALL_TOOLS is set we will build all the required tools from the > +# current source tree. Otherwise we create a symlink to the version found in > +# $PATH during the bootstrap-tools stage. > +.if defined(BOOTSTRAP_ALL_TOOLS) > +# BOOTSTRAPPING will be set on the command line so we can't override it here. > +# Instead set BOOTSTRAPPING_OSRELDATE so that the value 0 is set ${BSARGS} > +BOOTSTRAPPING_OSRELDATE:= 0 > +.endif > + > .if ${MK_GAMES} != "no" > _strfile= usr.bin/fortune/strfile > .endif > @@ -1981,12 +2040,21 @@ _gperf= gnu/usr.bin/gperf > _vtfontcvt= usr.bin/vtfontcvt > .endif > > +# If we are not building the bootstrap because BOOTSTRAPPING is sufficient > +# we symlink the host version to $WORLDTMP instead. By doing this we can also > +# detect when a bootstrap tool is being used without the required MK_FOO. > +# If you add a new bootstrap tool where we could also use the host version, > +# please ensure that you also add a .else case where you add the tool to the > +# _bootstrap_tools_links variable. > .if ${BOOTSTRAPPING} < 1000033 > _m4= usr.bin/m4 > _lex= usr.bin/lex > - > -${_bt}-usr.bin/m4: ${_bt}-lib/libopenbsd > -${_bt}-usr.bin/lex: ${_bt}-usr.bin/m4 > +# Note: lex needs m4 to build but m4 also depends on lex. However, lex can be > +# bootstrapped so we build lex first. > +${_bt}-usr.bin/m4: ${_bt}-lib/libopenbsd ${_bt}-usr.bin/yacc ${_bt}-${_lex} > +_bt_lex_depend=${_bt}-usr.bin/lex ${_bt}-usr.bin/m4 > +.else > +_bootstrap_tools_links+=m4 lex > .endif > > # r245440 mtree -N support added > @@ -1998,17 +2066,23 @@ _nmtree= lib/libmd \ > > ${_bt}-lib/libnetbsd: ${_bt}-lib/libmd > ${_bt}-usr.sbin/nmtree: ${_bt}-lib/libnetbsd > +.else > +_bootstrap_tools_links+=mtree > .endif > > # r246097: log addition login.conf.db, passwd, pwd.db, and spwd.db with cat -l > .if ${BOOTSTRAPPING} < 1000027 > _cat= bin/cat > +.else > +_bootstrap_tools_links+=cat > .endif > > # r277259 crunchide: Correct 64-bit section header offset > # r281674 crunchide: always include both 32- and 64-bit ELF support > .if ${BOOTSTRAPPING} < 1100078 > _crunchide= usr.sbin/crunch/crunchide > +.else > +_bootstrap_tools_links+=crunchide > .endif > > # r285986 crunchen: use STRIPBIN rather than STRIP > @@ -2018,12 +2092,16 @@ _crunchide= usr.sbin/crunch/crunchide > (${MK_AUTO_OBJ} == "yes" && ${BOOTSTRAPPING} < 1100114) || \ > (${MK_META_MODE} == "yes" && ${BOOTSTRAPPING} < 1200006) > _crunchgen= usr.sbin/crunch/crunchgen > +.else > +_bootstrap_tools_links+=crunchgen > .endif > > # r296926 -P keymap search path, MFC to stable/10 in r298297 > .if ${BOOTSTRAPPING} < 1003501 || \ > (${BOOTSTRAPPING} >= 1100000 && ${BOOTSTRAPPING} < 1100103) > _kbdcontrol= usr.sbin/kbdcontrol > +.else > +_bootstrap_tools_links+=kbdcontrol > .endif > > _yacc= lib/liby \ > @@ -2059,6 +2137,10 @@ _dtc= usr.bin/dtc > _dtc= gnu/usr.bin/dtc > .endif > > +.if ${MK_LOCALES} != "no" > +_localedef= usr.bin/localedef > +.endif > + > .if ${MK_KERBEROS} != "no" > _kerberos5_bootstrap_tools= \ > kerberos5/tools/make-roken \ > @@ -2069,12 +2151,108 @@ _kerberos5_bootstrap_tools= \ > usr.bin/compile_et > > .ORDER: ${_kerberos5_bootstrap_tools:C/^/${_bt}-/g} > +.for _tool in ${_kerberos5_bootstrap_tools} > +${_bt}-${_tool}: ${_bt}-usr.bin/yacc ${_bt_lex_depend} > +.endfor > .endif > > ${_bt}-usr.bin/mandoc: ${_bt}-lib/libopenbsd > > -bootstrap-tools: .PHONY > +# The tools listed in _basic_bootstrap_tools will generally not be > +# bootstrapped unless BOOTSTRAP_ALL_TOOL is set. However, when building on a > +# Linux or MacOS host the host versions are incompatible so we need to build > +# them from the source tree. Usually the link name will be the same as the subdir, > +# but some directories such as grep or test install multiple binaries. In that > +# case we use the _basic_bootstrap_tools_multilink variable which is a list of > +# subdirectory and comma-separated list of files. > +_basic_bootstrap_tools_multilink=usr.bin/grep grep,egrep,fgrep > +_basic_bootstrap_tools_multilink+=bin/test test,[ > +# bootstrap tools needed by buildworld: > +_basic_bootstrap_tools=usr.bin/awk usr.bin/cut bin/expr usr.bin/gencat \ > + usr.bin/join usr.bin/mktemp bin/rmdir usr.bin/sed usr.bin/sort \ > + usr.bin/truncate usr.bin/tsort > +# elf2aout is required for sparc64 build > +_basic_bootstrap_tools+=usr.bin/elf2aout > +# file2c is required for building usr.sbin/config: > +_basic_bootstrap_tools+=usr.bin/file2c > +# uuencode/uudecode required for share/tabset > +_basic_bootstrap_tools+=usr.bin/uuencode usr.bin/uudecode > +# xargs is required by mkioctls > +_basic_bootstrap_tools+=usr.bin/xargs > +# cap_mkdb is required for share/termcap: > +_basic_bootstrap_tools+=usr.bin/cap_mkdb > +# ldd is required for installcheck (TODO: just always use /usr/bin/ldd instead?) > +_basic_bootstrap_tools+=usr.bin/ldd > +# services_mkdb/pwd_mkdb are required for installworld: > +_basic_bootstrap_tools+=usr.sbin/services_mkdb usr.sbin/pwd_mkdb > +# sysctl/chflags are required for installkernel: > +_basic_bootstrap_tools+=sbin/sysctl bin/chflags > > +.if ${MK_AMD} != "no" > +# unifdef is only used by usr.sbin/amd/libamu/Makefile > +_basic_bootstrap_tools+=usr.bin/unifdef > +.endif > + > +.if ${MK_BOOT} != "no" > +_basic_bootstrap_tools+=usr.sbin/btxld btxld exist only on amd64/i386, so this commit breaks native buildworld for all other architectures. See: -------------------------------------------------------------- >>> stage 1.2: bootstrap tools -------------------------------------------------------------- cd /usr/src; INSTALL="sh /usr/src/tools/install.sh" TOOLS_PREFIX=/usr/obj/usr/src/arm.armv7/tmp PATH=/usr/obj/usr/src/arm.armv7/tmp/legacy/usr/sbin:/usr/obj/usr/src/arm.armv7/tmp/legacy/usr/bin:/usr/obj/usr/src/arm.armv7/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin WORLDTMP=/usr/obj/usr/src/arm.armv7/tmp MAKEFLAGS="-m /usr/src/tools/build/mk -m /usr/src/share/mk" make -f Makefile.inc1 DESTDIR= OBJTOP='/usr/obj/usr/src/arm.armv7/tmp/obj-tools' OBJROOT='${OBJTOP}/' MAKEOBJDIRPREFIX= BOOTSTRAPPING=1300002 BWPHASE=bootstrap-tools SSP_CFLAGS= MK_HTML=no NO_LINT=yes MK_MAN=no -DNO_PIC MK_PROFILE=no -DNO_SHARED -DNO_CPU_CFLAGS MK_WARNS=no MK_CTF=no MK_CLANG_EXTRAS=no MK_CLANG_FULL=no MK_LLDB=no MK_RETPOLINE=no MK_TESTS=no MK_INCLUDES=yes MK_LLVM_TARGET_ALL=no bootstrap-tools /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/m4 -> /usr/bin/m4 /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/lex -> /usr/bin/lex /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/mtree -> /usr/sbin/mtree /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/cat -> /bin/cat /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/crunchide -> /usr/bin/crunchide /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/crunchgen -> /usr/bin/crunchgen /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/kbdcontrol -> /usr/sbin/kbdcontrol /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/awk -> /usr/bin/awk /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/cut -> /usr/bin/cut /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/expr -> /bin/expr /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/gencat -> /usr/bin/gencat /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/join -> /usr/bin/join /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/mktemp -> /usr/bin/mktemp /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/rmdir -> /bin/rmdir /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/sed -> /usr/bin/sed /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/sort -> /usr/bin/sort /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/truncate -> /usr/bin/truncate /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/tsort -> /usr/bin/tsort /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/elf2aout -> /usr/bin/elf2aout /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/file2c -> /usr/bin/file2c /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/uuencode -> /usr/bin/uuencode /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/uudecode -> /usr/bin/uudecode /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/xargs -> /usr/bin/xargs /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/cap_mkdb -> /usr/bin/cap_mkdb /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/ldd -> /usr/bin/ldd /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/services_mkdb -> /usr/sbin/services_mkdb /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/pwd_mkdb -> /usr/sbin/pwd_mkdb /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/sysctl -> /sbin/sysctl /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/chflags -> /bin/chflags /usr/obj/usr/src/arm.armv7/tmp/legacy/bin/unifdef -> /usr/bin/unifdef *** Error code 1 Stop. make[2]: stopped in /usr/src *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. > +_basic_bootstrap_tools+=bin/dd > +# xz/unxz is used by EFI > +_basic_bootstrap_tools_multilink+=usr.bin/xz xz,unxz > +# md5 is used by boot/beri (and possibly others) > +_basic_bootstrap_tools+=sbin/md5 > +.if defined(BOOTSTRAP_ALL_TOOLS) > +${_bt}-sbin/md5: ${_bt}-lib/libmd > +.endif > +.endif > + > +.if ${MK_ZONEINFO} != "no" > +_basic_bootstrap_tools+=usr.sbin/zic usr.sbin/tzsetup > +.endif > + > +.if defined(BOOTSTRAP_ALL_TOOLS) > +_other_bootstrap_tools+=${_basic_bootstrap_tools} > +.for _subdir _links in ${_basic_bootstrap_tools_multilink} > +_other_bootstrap_tools+=${_subdir} > +.endfor > +${_bt}-usr.bin/awk: ${_bt_lex_depend} ${_bt}-usr.bin/yacc > +${_bt}-bin/expr: ${_bt_lex_depend} ${_bt}-usr.bin/yacc > +# If we are bootstrapping file2c, we have to build it before config: > +${_bt}-usr.sbin/config: ${_bt}-usr.bin/file2c ${_bt_lex_depend} > +# Note: no symlink to make/bmake in the !BOOTSTRAP_ALL_TOOLS case here since > +# the links to make/bmake make links will have already have been created in the > +# `make legacy` step. Not adding a link to make is important on non-FreeBSD > +# since "make" will usually point to GNU make there. > +_other_bootstrap_tools+=usr.bin/bmake > +.else > +# All tools in _basic_bootstrap_tools have the same name as the subdirectory > +# so we can use :T to get the name of the symlinks that we need to create. > +_bootstrap_tools_links+=${_basic_bootstrap_tools:T} > +.for _subdir _links in ${_basic_bootstrap_tools_multilink} > +_bootstrap_tools_links+=${_links:S/,/ /g} > +.endfor > +.endif # defined(BOOTSTRAP_ALL_TOOLS) > + > +# Link the tools that we need for building but don't need to bootstrap because > +# the host version is known to be compatible into ${WORLDTMP}/legacy > +# We do this before building any of the bootstrap tools in case they depend on > +# the presence of any of the links (e.g. as m4/lex/awk) > +${_bt}-links: .PHONY > + > +.for _tool in ${_bootstrap_tools_links} > +${_bt}-link-${_tool}: .PHONY .MAKE > + @if [ ! -e "${WORLDTMP}/legacy/bin/${_tool}" ]; then \ > + source_path=`which ${_tool}`; \ > + if [ ! -e "$${source_path}" ] ; then \ > + echo "Cannot find host tool '${_tool}'"; false; \ > + fi; \ > + ln -sfnv "$${source_path}" "${WORLDTMP}/legacy/bin/${_tool}"; \ > + fi > +${_bt}-links: ${_bt}-link-${_tool} > +.endfor > + > + > +bootstrap-tools: ${_bt}-links .PHONY > + > # Please document (add comment) why something is in 'bootstrap-tools'. > # Try to bound the building of the bootstrap-tool to just the > # FreeBSD versions that need the tool built at this stage of the build. > @@ -2093,6 +2271,7 @@ bootstrap-tools: .PHONY > ${_yacc} \ > ${_m4} \ > ${_lex} \ > + ${_other_bootstrap_tools} \ > usr.bin/xinstall \ > ${_gensnmptree} \ > usr.sbin/config \ > @@ -2101,10 +2280,13 @@ bootstrap-tools: .PHONY > ${_nmtree} \ > ${_vtfontcvt} \ > ${_localedef} > -${_bt}-${_tool}: .PHONY .MAKE > +${_bt}-${_tool}: ${_bt}-links .PHONY .MAKE > ${_+_}@${ECHODIR} "===> ${_tool} (obj,all,install)"; \ > cd ${.CURDIR}/${_tool}; \ > if [ -z "${NO_OBJWALK}" ]; then ${MAKE} DIRPRFX=${_tool}/ obj; fi; \ > + if [ "${_tool}" = "usr.bin/lex" ]; then \ > + ${MAKE} DIRPRFX=${_tool}/ bootstrap; \ > + fi; \ > ${MAKE} DIRPRFX=${_tool}/ all; \ > ${MAKE} DIRPRFX=${_tool}/ DESTDIR=${WORLDTMP}/legacy install > > @@ -2235,9 +2417,14 @@ _gcc= gnu/usr.bin/cc > _usb_tools= stand/usb/tools > .endif > > +.if ${BUILD_WITH_STRICT_TMPPATH} != 0 || defined(BOOTSTRAP_ALL_TOOLS) > +_ar=usr.bin/ar > +.endif > + > cross-tools: .MAKE .PHONY > .for _tool in \ > ${LOCAL_XTOOL_DIRS} \ > + ${_ar} \ > ${_clang_libs} \ > ${_clang} \ > ${_lld} \ > > Modified: head/tools/build/Makefile > ============================================================================== > --- head/tools/build/Makefile Mon Nov 5 19:25:57 2018 (r340156) > +++ head/tools/build/Makefile Mon Nov 5 19:51:10 2018 (r340157) > @@ -59,15 +59,64 @@ SUBDIR= cross-build > # Needed to build config (since it uses libnv) > SYSINCS+= ${SRCTOP}/sys/sys/nv.h ${SRCTOP}/sys/sys/cnv.h > > +# We want to run the build with only ${WORLDTMP} in $PATH to ensure we don't > +# accidentally run tools that are incompatible but happen to be in $PATH. > +# This is especially important when building on Linux/MacOS where many of the > +# programs used during the build accept different flags or generate different > +# output. On those platforms we only symlink the tools known to be compatible > +# (e.g. basic utilities such as mkdir) into ${WORLDTMP} and build all others > +# from the FreeBSD sources during the bootstrap-tools stage. > > +# basic commands: It is fine to use the host version for all of these even on > +# Linux/MacOS since we only use flags that are supported by all of them. > +_host_tools_to_symlink= basename bzip2 bunzip2 chmod chown cmp comm cp date \ > + dirname echo env false find fmt gzip gunzip head hostname id ln ls \ > + mkdir mv nice patch rm realpath sh sleep stat tee touch tr true uname \ > + uniq wc which > + > +# We also need a symlink to the absolute path to the make binary used for > +# the toplevel makefile. This is not necessarily the same as `which make` > +# since e.g. on Linux and MacOS that will be GNU make. > +_make_abs!= which "${MAKE}" > +_host_abs_tools_to_symlink= ${_make_abs}:make ${_make_abs}:bmake > + > +host-symlinks: > + @echo "Linking host tools into ${DESTDIR}/bin" > +.for _tool in ${_host_tools_to_symlink} > + @if [ ! -e "${DESTDIR}/bin/${_tool}" ]; then \ > + source_path=`which ${_tool}`; \ > + if [ ! -e "$${source_path}" ] ; then \ > + echo "Cannot find host tool '${_tool}'"; false; \ > + fi; \ > + ln -sfnv "$${source_path}" "${DESTDIR}/bin/${_tool}"; \ > + fi > +.endfor > +.for _tool in ${_host_abs_tools_to_symlink} > + @source_path="${_tool:S/:/ /:[1]}"; \ > + target_path="${DESTDIR}/bin/${_tool:S/:/ /:[2]}"; \ > + if [ ! -e "$${target_path}" ] ; then \ > + if [ ! -e "$${source_path}" ] ; then \ > + echo "Host tool '${src_path}' is missing"; false; \ > + fi; \ > + ln -sfnv "$${source_path}" "$${target_path}"; \ > + fi > +.endfor > + > # Create all the directories that are needed during the legacy, bootstrap-tools > # and cross-tools stages. We do this here using mkdir since mtree may not exist > # yet (this happens if we are crossbuilding from Linux/Mac). > installdirs: > -.for _dir in bin sbin usr/bin usr/sbin usr/lib usr/include lib/geom lib/casper > +.for _dir in bin usr/lib usr/include lib/geom lib/casper > mkdir -p "${DESTDIR}/${_dir}" > .endfor > - > +# Link usr/bin, sbin, and usr/sbin to bin so that it doesn't matter whether a > +# bootstrap tool was added to WORLTMP with a symlink or by building it in the > +# bootstrap-tools phase. We could also overrride BINDIR when building bootstrap > +# tools but adding the symlinks is easier and means all tools are also > +# in the directory that they are installed to normally. > + ln -sf bin ${DESTDIR}/sbin > + ln -sf ../bin ${DESTDIR}/usr/bin > + ln -sf ../bin ${DESTDIR}/usr/sbin > .for _group in ${INCSGROUPS:NINCS} > mkdir -p "${DESTDIR}/${${_group}DIR}" > .endfor > > Modified: head/tools/tools/nanobsd/embedded/common > ============================================================================== > --- head/tools/tools/nanobsd/embedded/common Mon Nov 5 19:25:57 2018 (r340156) > +++ head/tools/tools/nanobsd/embedded/common Mon Nov 5 19:51:10 2018 (r340157) > @@ -671,4 +671,4 @@ create_diskimage ( ) ( > # Set the path to the same path we use for buldworld to use latest mkimg > NANO_TARGET=$(cd ${NANO_SRC}; ${NANO_MAKE} TARGET_ARCH=${NANO_ARCH} -V _TARGET) > NANO_TMPPATH=$(cd ${NANO_SRC}; ${NANO_MAKE} MK_AUTO_OBJ=no TARGET=${NANO_TARGET} TARGET_ARCH=${NANO_ARCH} -f Makefile.inc1 buildenv -V TMPPATH) > -export PATH="${NANO_TMPPATH}" > +export PATH="${NANO_TMPPATH}:${PATH}" >