From owner-dev-commits-src-all@freebsd.org Tue Mar 23 19:21:49 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3E96E5B8FE2; Tue, 23 Mar 2021 19:21:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F4h7d1L6zz3qGr; Tue, 23 Mar 2021 19:21:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 210F226928; Tue, 23 Mar 2021 19:21:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 12NJLn4w093092; Tue, 23 Mar 2021 19:21:49 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 12NJLnFA093091; Tue, 23 Mar 2021 19:21:49 GMT (envelope-from git) Date: Tue, 23 Mar 2021 19:21:49 GMT Message-Id: <202103231921.12NJLnFA093091@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Nathan Whitehorn Subject: git: 4601382e1362 - stable/13 - Fix scripted installs on EFI systems after default mounting of the ESP. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: nwhitehorn X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4601382e1362352f17a33e4ed38db5dcfe3f6be5 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2021 19:21:49 -0000 The branch stable/13 has been updated by nwhitehorn: URL: https://cgit.FreeBSD.org/src/commit/?id=4601382e1362352f17a33e4ed38db5dcfe3f6be5 commit 4601382e1362352f17a33e4ed38db5dcfe3f6be5 Author: Nathan Whitehorn AuthorDate: 2021-03-23 13:19:42 +0000 Commit: Nathan Whitehorn CommitDate: 2021-03-23 19:21:33 +0000 Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Approved by: re (gjb) Reviewed by: gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 (cherry picked from commit c2f16c595eb51c6e0cb6ece3f6f078d738019059) --- usr.sbin/bsdinstall/partedit/partedit.c | 39 +++++++++++++++++++++++++++++++++ usr.sbin/bsdinstall/scripts/script | 30 ++++++++++++++++++------- 2 files changed, 61 insertions(+), 8 deletions(-) diff --git a/usr.sbin/bsdinstall/partedit/partedit.c b/usr.sbin/bsdinstall/partedit/partedit.c index c5fa28e73582..6d045428dd32 100644 --- a/usr.sbin/bsdinstall/partedit/partedit.c +++ b/usr.sbin/bsdinstall/partedit/partedit.c @@ -323,6 +323,22 @@ validate_setup(void) return (TRUE); } +static int +mountpoint_sorter(const void *xa, const void *xb) +{ + struct partition_metadata *a = *(struct partition_metadata **)xa; + struct partition_metadata *b = *(struct partition_metadata **)xb; + + if (a->fstab == NULL && b->fstab == NULL) + return 0; + if (a->fstab == NULL) + return 1; + if (b->fstab == NULL) + return -1; + + return strcmp(a->fstab->fs_file, b->fstab->fs_file); +} + static int apply_changes(struct gmesh *mesh) { @@ -386,6 +402,29 @@ apply_changes(struct gmesh *mesh) free(__DECONST(char *, items[i*2])); free(items); + /* Sort filesystems for fstab so that mountpoints are ordered */ + { + struct partition_metadata **tobesorted; + struct partition_metadata *tmp; + int nparts = 0; + TAILQ_FOREACH(md, &part_metadata, metadata) + nparts++; + tobesorted = malloc(sizeof(struct partition_metadata *)*nparts); + nparts = 0; + TAILQ_FOREACH_SAFE(md, &part_metadata, metadata, tmp) { + tobesorted[nparts++] = md; + TAILQ_REMOVE(&part_metadata, md, metadata); + } + qsort(tobesorted, nparts, sizeof(tobesorted[0]), + mountpoint_sorter); + + /* Now re-add everything */ + while (nparts-- > 0) + TAILQ_INSERT_HEAD(&part_metadata, + tobesorted[nparts], metadata); + free(tobesorted); + } + if (getenv("PATH_FSTAB") != NULL) fstab_path = getenv("PATH_FSTAB"); else diff --git a/usr.sbin/bsdinstall/scripts/script b/usr.sbin/bsdinstall/scripts/script index 4d0a91833644..1d8e52a9d6d3 100755 --- a/usr.sbin/bsdinstall/scripts/script +++ b/usr.sbin/bsdinstall/scripts/script @@ -116,14 +116,28 @@ fi # Unpack distributions bsdinstall checksum -for set in $DISTRIBUTIONS; do - f_dprintf "Extracting $BSDINSTALL_DISTDIR/$set" - # XXX: this will fail if any mountpoints are FAT, due to inability to - # set ctime/mtime on the root of FAT partitions. tar has no option to - # ignore this. We probably need to switch back to distextract here - # to properly support EFI. - tar -xf "$BSDINSTALL_DISTDIR/$set" -C $BSDINSTALL_CHROOT -done +if [ -t 0 ]; then + # If install is a tty, use distextract as normal + bsdinstall distextract +else + # Otherwise, we need to use tar (see https://reviews.freebsd.org/D10736) + for set in $DISTRIBUTIONS; do + f_dprintf "Extracting $BSDINSTALL_DISTDIR/$set" + # XXX: The below fails if any mountpoints are FAT, due to + # inability to set ctime/mtime on the root of FAT partitions, + # which is needed to support e.g. EFI system partitions. tar has + # no option to ignore this (distextract ignores them internally + # through a hack), and returns 1 on any warning or error, + # effectively turning all warnings into fatal errors. + # + # Work around this in an extremely lame way for the specific + # case of EFI system partitions only. This *ONLY WORKS* if + # /boot/efi is empty and does not handle analagous problems on + # other systems (ARM, PPC64). + tar -xf "$BSDINSTALL_DISTDIR/$set" -C $BSDINSTALL_CHROOT --exclude boot/efi + mkdir -p $BSDINSTALL_CHROOT/boot/efi + done +fi # Configure bootloader if needed bsdinstall bootconfig