From owner-svn-src-head@freebsd.org Fri Aug 7 16:26:57 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F041F3BEDA0; Fri, 7 Aug 2020 16:26:57 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BNW3567xnz4dPw; Fri, 7 Aug 2020 16:26:57 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B73671A99C; Fri, 7 Aug 2020 16:26:57 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 077GQvWv038729; Fri, 7 Aug 2020 16:26:57 GMT (envelope-from imp@FreeBSD.org) Received: (from imp@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 077GQvd8038727; Fri, 7 Aug 2020 16:26:57 GMT (envelope-from imp@FreeBSD.org) Message-Id: <202008071626.077GQvd8038727@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: imp set sender to imp@FreeBSD.org using -f From: Warner Losh Date: Fri, 7 Aug 2020 16:26:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r364030 - in head: . tools/build X-SVN-Group: head X-SVN-Commit-Author: imp X-SVN-Commit-Paths: in head: . tools/build X-SVN-Commit-Revision: 364030 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Aug 2020 16:26:58 -0000 Author: imp Date: Fri Aug 7 16:26:56 2020 New Revision: 364030 URL: https://svnweb.freebsd.org/changeset/base/364030 Log: The practice of creating symbolic links is somewhat fragile. Always make copies instead. There's too many times that we can't run the new binaries with old libraries. Making the links when things are known to be 'safe' is a nice optimization, but a copy of all the binaries is only 30MB, so saving the copies at the cost of increased support when new symbols are added and used as part of the bootstrap seems to be unwise. There may be additional optimizations possible here, especially for !FreeBSD hosts. However, that's beyond the scope of the problem I'm trying to fix with make failing mid-way through an installworld across change r363679. This optimization there caused us to run a new binary with an old library once a new make was installed due to the symbolic link. One could just copy make, but then other binaries fail as well, so rather than play whack-a-mole, I opted to take us back to the old way. Before r340157 or so we did copies (thogh of a lot fewer artifacts), and we didn't have issues like this. Reviewed by: arichards@ Differential Revision: https://reviews.freebsd.org/D25967 Modified: head/Makefile.inc1 head/UPDATING head/tools/build/Makefile Modified: head/Makefile.inc1 ============================================================================== --- head/Makefile.inc1 Fri Aug 7 16:20:07 2020 (r364029) +++ head/Makefile.inc1 Fri Aug 7 16:26:56 2020 (r364030) @@ -2293,13 +2293,12 @@ ${_bt}-links: .PHONY .for _tool in ${_bootstrap_tools_links} ${_bt}-link-${_tool}: .PHONY .MAKE - @if [ ! -e "${WORLDTMP}/legacy/bin/${_tool}" ]; then \ - source_path=`which ${_tool}`; \ - if [ ! -e "$${source_path}" ] ; then \ - echo "Cannot find host tool '${_tool}'"; false; \ - fi; \ - ln -sfnv "$${source_path}" "${WORLDTMP}/legacy/bin/${_tool}"; \ - fi + @rm -f "${WORLDTMP}/legacy/bin/${_tool}"; \ + source_path=`which ${_tool}`; \ + if [ ! -e "$${source_path}" ] ; then \ + echo "Cannot find host tool '${_tool}'"; false; \ + fi; \ + cp -f "$${source_path}" "${WORLDTMP}/legacy/bin/${_tool}" ${_bt}-links: ${_bt}-link-${_tool} .endfor Modified: head/UPDATING ============================================================================== --- head/UPDATING Fri Aug 7 16:20:07 2020 (r364029) +++ head/UPDATING Fri Aug 7 16:26:56 2020 (r364030) @@ -26,6 +26,12 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 13.x IS SLOW: disable the most expensive debugging functionality run "ln -s 'abort:false,junk:false' /etc/malloc.conf".) +20200807: + Makefile.inc has been updated to work around the issue documented in + 20200729. It was a case where the optimization of using symbolic links + to point to binaries created a situation where we'd run new binaries + with old libraries starting midway through the installworld process. + 20200729: r363679 has redefined some undefined behavior in regcomp(3); notably, extraneous escapes of most ordinary characters will no longer be Modified: head/tools/build/Makefile ============================================================================== --- head/tools/build/Makefile Fri Aug 7 16:20:07 2020 (r364029) +++ head/tools/build/Makefile Fri Aug 7 16:26:56 2020 (r364030) @@ -119,26 +119,25 @@ _host_abs_tools_to_symlink= ${_make_abs}:make ${_make_ host-symlinks: @echo "Linking host tools into ${DESTDIR}/bin" .for _tool in ${_host_tools_to_symlink} - @if [ ! -e "${DESTDIR}/bin/${_tool}" ]; then \ - source_path=`which ${_tool}`; \ - if [ ! -e "$${source_path}" ] ; then \ - echo "Cannot find host tool '${_tool}'"; false; \ - fi; \ - ln -sfnv "$${source_path}" "${DESTDIR}/bin/${_tool}"; \ - fi + @source_path=`which ${_tool}`; \ + if [ ! -e "$${source_path}" ] ; then \ + echo "Cannot find host tool '${_tool}'"; false; \ + fi; \ + rm -f "${DESTDIR}/bin/${_tool}"; \ + cp -f "$${source_path}" "${DESTDIR}/bin/${_tool}" .endfor .for _tool in ${_host_abs_tools_to_symlink} @source_path="${_tool:S/:/ /:[1]}"; \ target_path="${DESTDIR}/bin/${_tool:S/:/ /:[2]}"; \ - if [ ! -e "$${target_path}" ] ; then \ - if [ ! -e "$${source_path}" ] ; then \ - echo "Host tool '${src_path}' is missing"; false; \ - fi; \ - ln -sfnv "$${source_path}" "$${target_path}"; \ - fi + if [ ! -e "$${source_path}" ] ; then \ + echo "Host tool '${src_path}' is missing"; false; \ + fi; \ + rm -f "$${target_path}"; \ + cp -f "$${source_path}" "$${target_path}" .endfor .if exists(/usr/libexec/flua) - ln -sf /usr/libexec/flua ${DESTDIR}/usr/libexec/flua + rm -f ${DESTDIR}/usr/libexec/flua + cp -f /usr/libexec/flua ${DESTDIR}/usr/libexec/flua .endif # Create all the directories that are needed during the legacy, bootstrap-tools