From owner-dev-commits-src-all@freebsd.org Tue Mar 23 13:30:13 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A53895AEC0F; Tue, 23 Mar 2021 13:30:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F4XKx4K2Zz4rP2; Tue, 23 Mar 2021 13:30:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8743C218D2; Tue, 23 Mar 2021 13:30:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 12NDUDl9095861; Tue, 23 Mar 2021 13:30:13 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 12NDUD7t095851; Tue, 23 Mar 2021 13:30:13 GMT (envelope-from git) Date: Tue, 23 Mar 2021 13:30:13 GMT Message-Id: <202103231330.12NDUD7t095851@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Nathan Whitehorn Subject: git: c2f16c595eb5 - main - Fix scripted installs on EFI systems after default mounting of the ESP. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: nwhitehorn X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c2f16c595eb51c6e0cb6ece3f6f078d738019059 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2021 13:30:13 -0000 The branch main has been updated by nwhitehorn: URL: https://cgit.FreeBSD.org/src/commit/?id=c2f16c595eb51c6e0cb6ece3f6f078d738019059 commit c2f16c595eb51c6e0cb6ece3f6f078d738019059 Author: Nathan Whitehorn AuthorDate: 2021-03-23 13:19:42 +0000 Commit: Nathan Whitehorn CommitDate: 2021-03-23 13:29:54 +0000 Fix scripted installs on EFI systems after default mounting of the ESP. Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to extract a directory at that point post-mount when the system is installed. Normally, this is fine, since tar can happily set whatever properties it wants. For FAT32 file systems, however, like the ESP, tar will attempt to set mtime on the root directory, which FAT does not support, and tar will interpret this as a fatal error, breaking the install (see https://github.com/libarchive/libarchive/issues/1516). This issue would also break scripted installs on bare-metal POWER8, POWER9, and PS3 systems, as well as some ARM systems. This patch solves the problem in two ways: - If stdout is a TTY, use the distextract stage instead of tar, as in interactive installs. distextract solves this problem internally and provides a nicer UI to boot, but requires a TTY. - If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude boot/efi from tarball extraction and then add it by hand. This is a hack, and better solutions (as in the libarchive ticket above) will obsolete it, but it solves the most common case, leaving only unattended TTY-less installs on a few tier-2 platforms broken. In addition, fix a bug with fstab generation uncovered once the tar issue is fixed that umount(8) can depend on the ordering of lines in fstab in a way that mount(8) does not. The partition editor now writes out fstab in mount order, making sure umount (run at the end of scripted, but not interactive, installs) succeeds. PR: 254395 Reviewed by: gjb, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D29380 --- usr.sbin/bsdinstall/partedit/partedit.c | 39 +++++++++++++++++++++++++++++++++ usr.sbin/bsdinstall/scripts/script | 30 ++++++++++++++++++------- 2 files changed, 61 insertions(+), 8 deletions(-) diff --git a/usr.sbin/bsdinstall/partedit/partedit.c b/usr.sbin/bsdinstall/partedit/partedit.c index c5fa28e73582..6d045428dd32 100644 --- a/usr.sbin/bsdinstall/partedit/partedit.c +++ b/usr.sbin/bsdinstall/partedit/partedit.c @@ -323,6 +323,22 @@ validate_setup(void) return (TRUE); } +static int +mountpoint_sorter(const void *xa, const void *xb) +{ + struct partition_metadata *a = *(struct partition_metadata **)xa; + struct partition_metadata *b = *(struct partition_metadata **)xb; + + if (a->fstab == NULL && b->fstab == NULL) + return 0; + if (a->fstab == NULL) + return 1; + if (b->fstab == NULL) + return -1; + + return strcmp(a->fstab->fs_file, b->fstab->fs_file); +} + static int apply_changes(struct gmesh *mesh) { @@ -386,6 +402,29 @@ apply_changes(struct gmesh *mesh) free(__DECONST(char *, items[i*2])); free(items); + /* Sort filesystems for fstab so that mountpoints are ordered */ + { + struct partition_metadata **tobesorted; + struct partition_metadata *tmp; + int nparts = 0; + TAILQ_FOREACH(md, &part_metadata, metadata) + nparts++; + tobesorted = malloc(sizeof(struct partition_metadata *)*nparts); + nparts = 0; + TAILQ_FOREACH_SAFE(md, &part_metadata, metadata, tmp) { + tobesorted[nparts++] = md; + TAILQ_REMOVE(&part_metadata, md, metadata); + } + qsort(tobesorted, nparts, sizeof(tobesorted[0]), + mountpoint_sorter); + + /* Now re-add everything */ + while (nparts-- > 0) + TAILQ_INSERT_HEAD(&part_metadata, + tobesorted[nparts], metadata); + free(tobesorted); + } + if (getenv("PATH_FSTAB") != NULL) fstab_path = getenv("PATH_FSTAB"); else diff --git a/usr.sbin/bsdinstall/scripts/script b/usr.sbin/bsdinstall/scripts/script index 4d0a91833644..1d8e52a9d6d3 100755 --- a/usr.sbin/bsdinstall/scripts/script +++ b/usr.sbin/bsdinstall/scripts/script @@ -116,14 +116,28 @@ fi # Unpack distributions bsdinstall checksum -for set in $DISTRIBUTIONS; do - f_dprintf "Extracting $BSDINSTALL_DISTDIR/$set" - # XXX: this will fail if any mountpoints are FAT, due to inability to - # set ctime/mtime on the root of FAT partitions. tar has no option to - # ignore this. We probably need to switch back to distextract here - # to properly support EFI. - tar -xf "$BSDINSTALL_DISTDIR/$set" -C $BSDINSTALL_CHROOT -done +if [ -t 0 ]; then + # If install is a tty, use distextract as normal + bsdinstall distextract +else + # Otherwise, we need to use tar (see https://reviews.freebsd.org/D10736) + for set in $DISTRIBUTIONS; do + f_dprintf "Extracting $BSDINSTALL_DISTDIR/$set" + # XXX: The below fails if any mountpoints are FAT, due to + # inability to set ctime/mtime on the root of FAT partitions, + # which is needed to support e.g. EFI system partitions. tar has + # no option to ignore this (distextract ignores them internally + # through a hack), and returns 1 on any warning or error, + # effectively turning all warnings into fatal errors. + # + # Work around this in an extremely lame way for the specific + # case of EFI system partitions only. This *ONLY WORKS* if + # /boot/efi is empty and does not handle analagous problems on + # other systems (ARM, PPC64). + tar -xf "$BSDINSTALL_DISTDIR/$set" -C $BSDINSTALL_CHROOT --exclude boot/efi + mkdir -p $BSDINSTALL_CHROOT/boot/efi + done +fi # Configure bootloader if needed bsdinstall bootconfig