From owner-freebsd-questions@freebsd.org Thu Apr 15 14:34:09 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F4255F274E for ; Thu, 15 Apr 2021 14:34:09 +0000 (UTC) (envelope-from scott.gasch@gmail.com) Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FLhg45mcmz4SLG for ; Thu, 15 Apr 2021 14:34:08 +0000 (UTC) (envelope-from scott.gasch@gmail.com) Received: by mail-lj1-f170.google.com with SMTP id c1so19085073ljd.7 for ; Thu, 15 Apr 2021 07:34:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fUZoowSM6tQ6JBsflS+mrrTXe2BrvBxQfvlap2W9HNs=; b=ByMmtQgIDJE9Yd1GhXnCdtaoDlcM6GS48bOOIKLc3h2q22Vv5gYN4z+/bNs4auf/Od LYoUjKnWppgEBebSyLPd/R4oB57t2bxcFolIQjUDevzxQ8iSuR9EHNhS4aES1JvWxx+C eNODApquZ3/3kR1KUJ7D85TK5HoTuRx9SopC+jpkyQ9TRm9wzAGTgnDMefHod8l7z4s5 uZTON/7GQkh6ExCWTzGhM22V64P6Ur/UfekI3Fq/rP06rGDFJdg7FYif0BYbIYqSL50v /Ud6OyUMB5PsCgbO7dkyc9pXtmkdW4xh6K9zcYEBP4+4/hnfAjWJqoIO9LgogjnWdyAU T2RQ== X-Gm-Message-State: AOAM530/PHcg1GMIi6iRs3FKAnM4k4DDOBRpJvQ1if01KXDWJzrJQNhP 5Wx/i0weRatSwb+YRMfvjpQtq7r4n2mD9yB1puxUKQep X-Google-Smtp-Source: ABdhPJzug7ECozGdybqb7yEqJpS8PwRxjAKF6VEhD4XI1ZXLNisle4HY8keq8Xgm+gcfY1d/GZXS44QFyCN7AM/ZGi4= X-Received: by 2002:a2e:700b:: with SMTP id l11mr1975665ljc.413.1618497244691; Thu, 15 Apr 2021 07:34:04 -0700 (PDT) MIME-Version: 1.0 References: <20210415133528.lanr3zvxcrd4m5vl@ozzmosis.com> In-Reply-To: <20210415133528.lanr3zvxcrd4m5vl@ozzmosis.com> From: Scott Gasch Date: Thu, 15 Apr 2021 07:33:37 -0700 Message-ID: Subject: Re: FreeBSD 13-RELEASE install forgest SSD-based ZFS volume To: andrew clarke Cc: freebsd-questions@freebsd.org X-Rspamd-Queue-Id: 4FLhg45mcmz4SLG X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of scottgasch@gmail.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=scottgasch@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[scott@gasch.org,scottgasch@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.208.170:from]; R_DKIM_NA(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_NEQ_ENVFROM(0.00)[scott@gasch.org,scottgasch@gmail.com]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[gasch.org]; SPAMHAUS_ZRD(0.00)[209.85.208.170:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.208.170:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.208.170:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions] X-Mailman-Approved-At: Thu, 15 Apr 2021 16:55:15 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2021 14:34:09 -0000 Thanks. It's as you say: when I rebooted with the 13.0 kernel and the 12.2 userland the /var mountpoint (and /home, and /root, and /usr/src, etc...) sit on the zssd pool which has disappeared. The system comes up but my home directory is gone and so is root's. I can login but the missing pool means I can't finish upgrading userland to 13.0. My first instinct was to restore /home which is when I found that creating the missing pool anew does not survive a reboot in this state. Apparently creating an ssd based mirror (or stripe, or single provider pool) with 12.2 zpool and creating some volumes on it with 12.2 zfs then rebooting the machine causes the newly created pool to vanish again. I'm considering mounting /var on the HDD zfs pool temporarily and trying to freebsd-update -r 13.0-RELEASE upgrade again so that I can get userland to 13.0 on the hope that, in that configuration, I'll be able to finish the update and then maybe restore the missing data from backups. Thanks for any ideas and suggestions, much appreciated. Scott On Thu, Apr 15, 2021 at 6:35 AM andrew clarke wrote: > On 2021-04-14 22:55:43, Scott Gasch (scott.gasch@gmail.com) wrote: > > > I'm trying to upgrade a system from 12.2 to 13.0. The system has two ZFS > > volumes: z and zssd. The former is a mirror set based on spinning > platter > > disks and the latter is a mirror set based on two ssd devices. > > > > I've gotten as far as installing the new kernel and rebooting at which > > point the system is screwed up because, on boot, zssd no longer exists. > > Note: the kernel is 13.0 while userland is still 12.2 at this point. > > My experience upgrading to 13.0 was similar, but resolved itself after > upgrading userland and rebooting: > > I have two ZFS pools (both mirrors): zroot and tank. FreeBSD boots from > zroot. > > I upgraded from 12.2 to 13.0 with "freebsd-update upgrade -r > 13.0-RELEASE", ran > "freebsd-update install" then rebooted to load the 13.0 kernel. At this > point > userland was still at 12.2, and tank was missing from "zpool status". Only > zroot was visible. > > I didn't try to import tank manually. > > After running "freebsd-update install" a second time to install 13.0 > userland > and rebooting again, FreeBSD 13.0 booted from zroot and both it and tank > was > visible once more with "zpool status", and from there it was business as > usual. > > I imagine I'd have been a bit stuck if /var/db/freebsd-update/ happened to > be a > symlink to a location on the "tank" pool, because freebsd-update wouldn't > be > able to complete the upgrade. But that wasn't the case, and spreading > the base system over multiple pools seems unwise anyway. > > I don't know if any of this is helpful. If not you'll need to elaborate on > "the system is screwed up". >