From owner-dev-commits-src-main@freebsd.org Tue Apr 20 20:23:35 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 01DA25E7994; Tue, 20 Apr 2021 20:23:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FPw9y6fSJz3hhv; Tue, 20 Apr 2021 20:23:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D7086191B2; Tue, 20 Apr 2021 20:23:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 13KKNYdt004233; Tue, 20 Apr 2021 20:23:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 13KKNY7n004232; Tue, 20 Apr 2021 20:23:34 GMT (envelope-from git) Date: Tue, 20 Apr 2021 20:23:34 GMT Message-Id: <202104202023.13KKNY7n004232@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 0611aec3cf3a - main - etcupdate: Always extract to a temporary tree. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0611aec3cf3a373e6a06f103699dbc91c3d6d472 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Apr 2021 20:23:35 -0000 The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=0611aec3cf3a373e6a06f103699dbc91c3d6d472 commit 0611aec3cf3a373e6a06f103699dbc91c3d6d472 Author: John Baldwin AuthorDate: 2021-04-20 20:21:42 +0000 Commit: John Baldwin CommitDate: 2021-04-20 20:21:42 +0000 etcupdate: Always extract to a temporary tree. etcupdate has had a somewhat nasty race condition since its creation in that its state machine can get very confused if it is interrupted while building the tree to compare against. This is exacerbated by the fact that etcupdate doesn't emit any output while building the tree which can take several seconds (especially in recent years with the addition of the tree-wide buildconfig/installconfig passes). To mitigate this, always install a new tree into a temporary directory created via mktemp as was previously done only for dry-runs via -n. The existing trees are only rotated and the new tree installed as /var/db/etcupdate/current after the update command has completed. Reported by: dim, np (and many others) Reviewed by: imp MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D29843 --- usr.sbin/etcupdate/etcupdate.sh | 148 +++++++++++++++++++++++++--------------- 1 file changed, 92 insertions(+), 56 deletions(-) diff --git a/usr.sbin/etcupdate/etcupdate.sh b/usr.sbin/etcupdate/etcupdate.sh index 87d269f2ca69..72fa3e9d524d 100755 --- a/usr.sbin/etcupdate/etcupdate.sh +++ b/usr.sbin/etcupdate/etcupdate.sh @@ -226,9 +226,11 @@ build_tree() return 0 } -# Generate a new NEWTREE tree. If tarball is set, then the tree is +# Generate a new tree. If tarball is set, then the tree is # extracted from the tarball. Otherwise the tree is built from a # source tree. +# +# $1 - directory to store new tree in extract_tree() { local files @@ -239,16 +241,16 @@ extract_tree() if [ -n "$preworld" ]; then files="$PREWORLD_FILES" fi - if ! (mkdir -p $NEWTREE && tar xf $tarball -C $NEWTREE $files) \ + if ! (mkdir -p $1 && tar xf $tarball -C $1 $files) \ >&3 2>&1; then echo "Failed to extract new tree." - remove_tree $NEWTREE + remove_tree $1 exit 1 fi else - if ! build_tree $NEWTREE; then + if ! build_tree $1; then echo "Failed to build new tree." - remove_tree $NEWTREE + remove_tree $1 exit 1 fi fi @@ -1353,14 +1355,28 @@ extract_cmd() log "extract command: tarball=$tarball" + # Create a temporary directory to hold the tree + dir=`mktemp -d $WORKDIR/etcupdate-XXXXXXX` + if [ $? -ne 0 ]; then + echo "Unable to create temporary directory." + exit 1 + fi + + extract_tree $dir + if [ -d $NEWTREE ]; then if ! remove_tree $NEWTREE; then echo "Unable to remove current tree." + remove_tree $dir exit 1 fi fi - extract_tree + if ! mv $dir $NEWTREE >&3 2>&1; then + echo "Unable to rename temp tree to current tree." + remove_tree $dir + exit 1 + fi } # Resolve conflicts left from an earlier merge. @@ -1420,7 +1436,7 @@ status_cmd() # source tree. update_cmd() { - local dir + local dir new old if [ $# -ne 0 ]; then usage @@ -1449,60 +1465,44 @@ update_cmd() exit 1 fi + # Save tree names to use for rotation later. + old=$OLDTREE + new=$NEWTREE if [ -z "$rerun" ]; then - # For a dryrun that is not a rerun, do not rotate the existing - # stock tree. Instead, extract a tree to a temporary directory - # and use that for the comparison. - if [ -n "$dryrun" ]; then - dir=`mktemp -d $WORKDIR/etcupdate-XXXXXXX` - if [ $? -ne 0 ]; then - echo "Unable to create temporary directory." - exit 1 - fi - - # A pre-world dryrun has already set OLDTREE to - # point to the current stock tree. - if [ -z "$preworld" ]; then - OLDTREE=$NEWTREE - fi - NEWTREE=$dir - - # For a pre-world update, blow away any pre-existing - # NEWTREE. - elif [ -n "$preworld" ]; then - if ! remove_tree $NEWTREE; then - echo "Unable to remove pre-world tree." - exit 1 - fi + # Extract the new tree to a temporary directory. The + # trees are only rotated after a successful update to + # avoid races if an update command is interrupted + # before it completes. + dir=`mktemp -d $WORKDIR/etcupdate-XXXXXXX` + if [ $? -ne 0 ]; then + echo "Unable to create temporary directory." + exit 1 + fi - # Rotate the existing stock tree to the old tree. - elif [ -d $NEWTREE ]; then - # First, delete the previous old tree if it exists. - if ! remove_tree $OLDTREE; then - echo "Unable to remove old tree." - exit 1 - fi + # Populate the new tree. + extract_tree $dir - # Move the current stock tree. - if ! mv $NEWTREE $OLDTREE >&3 2>&1; then - echo "Unable to rename current stock tree." - exit 1 - fi + # Compare the new tree against the previous tree. For + # the preworld case OLDTREE already points to the + # current stock tree. + if [ -z "$preworld" ]; then + OLDTREE=$NEWTREE fi + NEWTREE=$dir + fi - if ! [ -d $OLDTREE ]; then - cat <&3 2>&1; then + echo "Unable to rename old tree." + exit 1 + fi + fi + + # Rotate the new tree. Remove a previous pre-world + # tree if it exists. + if [ -d $new ]; then + if [ -z "$preworld" ]; then + panic "New tree should be rotated to old" + fi + if ! remove_tree $old; then + echo "Unable to remove previous pre-world tree." + exit 1 + fi + fi + + if ! mv $NEWTREE $new >&3 2>&1; then + echo "Unable to rename current tree." + exit 1 + fi fi }