From nobody Thu Nov 28 14:52:45 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XzfSB6ZLhz5dpb7 for ; Thu, 28 Nov 2024 14:52:58 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XzfSB46Jwz49sc; Thu, 28 Nov 2024 14:52:58 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-5cefa22e9d5so977128a12.3; Thu, 28 Nov 2024 06:52:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732805577; x=1733410377; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=G/ZCwB9hDNaGed150n6byhQNCP+Xl44ozauxN44kdZ8=; b=VHyarrnzVhR3qFs8X2tkGXqbJg521TOgjTnmQvAW6s9upgLm/FHBadmNT0J+yFmzYY iBCytiGkE4Hfmkc1MNMjGX4A+dksu4ssoM0lQobWuNb5O6JIXZaNqGvoAGCmvOwMYRr3 evMnqi8tScMe3g6zbYRIFike2Tn0wla/k4iPPOs577r2HpMb8QEfp6I9KcBgaEN7i9O9 4y7IQ6msdRwVa1JcULifm1t+FVllMPkLWU3QWRvyAXfZWXs+Ox4VTq0pvY76KbSeti7Q e2xmMCEy1dYFSbi1qWXCRRyoEZfe9zYPAMv473xlo36AF2zxYbZ5K2G5ObzpZ4W5TnXL G+uA== X-Forwarded-Encrypted: i=1; AJvYcCWY6zWZBqef5edIIxU1890t4dguXmpAyUJ2DZ4MZinpsHQbgmMfsH1dveb47nJkDZ3Q9VtbbSLeVwcFwq37Avg=@freebsd.org X-Gm-Message-State: AOJu0YwUacyW15kHhN9ee1ehqWQJrG/489f+0T/kb9Oe8HjoK1vFt/fh kGob7RnNL2eStXABjZWgmZXZ2VUmgyP6Q68FwSiRUeOmwuiC+Qk9Lf1H3rxQWC0Zo31ZrhW2rk1 K47G79TNUNSbsUmXvhmTR5/GjvEM7Jw== X-Gm-Gg: ASbGncu+cVTtdmJsWlBtkMgSbJ5Rbce99A338cOeiykekvawKWJjnf8rlLRTjNLmGQ3 80hLMIPFAsYgkVKd5MEcRY6T1CLrouAB67a4dBkxgZ6n25LxgL3S8Y+1Q9MjDwQwQ X-Google-Smtp-Source: AGHT+IEg1fnYOxVVPmSzvH7OhgkuAIYjHBZkVTKTqrFsxGeFhAPtIhtrgNKDnUUMm9Ew7sRJFNiGV3ruLKcJhEXMatA= X-Received: by 2002:a05:6402:5243:b0:5d0:225b:f4e4 with SMTP id 4fb4d7f45d1cf-5d080c5cdaamr5601003a12.29.1732805576964; Thu, 28 Nov 2024 06:52:56 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> In-Reply-To: <22187e59-b6e9-4f2e-ba9b-f43944d1a37b@blastwave.org> From: Alan Somers Date: Thu, 28 Nov 2024 08:52:45 -0600 Message-ID: Subject: Re: zpools no longer exist after boot To: Dennis Clarke Cc: Alan Somers , Current FreeBSD Content-Type: multipart/alternative; boundary="000000000000e1437d0627fa3c5e" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4XzfSB46Jwz49sc X-Spamd-Bar: ---- --000000000000e1437d0627fa3c5e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 28, 2024, 8:45=E2=80=AFAM Dennis Clarke = wrote: > On 11/28/24 08:52, Alan Somers wrote: > > On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke > wrote: > > > >> > >> This is a baffling problem wherein two zpools no longer exist after > >> boot. This is : > . > . > . > > Do you have zfs_enable=3D"YES" set in /etc/rc.conf? If not then nothing > will > > get imported. > > > > Regarding the cachefile property, it's expected that "zpool import" wil= l > > change it, unless you do "zpool import -O cachefile=3Dwhatever". > > > > The rc script seems to do something slightly different with zpool import > -c $FOOBAR thus : > > > titan# cat /etc/rc.d/zpool > #!/bin/sh > # > # > > # PROVIDE: zpool > # REQUIRE: hostid disks > # BEFORE: mountcritlocal > # KEYWORD: nojail > > . /etc/rc.subr > > name=3D"zpool" > desc=3D"Import ZPOOLs" > rcvar=3D"zfs_enable" > start_cmd=3D"zpool_start" > required_modules=3D"zfs" > > zpool_start() > { > local cachefile > > for cachefile in /etc/zfs/zpool.cache /boot/zfs/zpool.cache; do > if [ -r $cachefile ]; then > zpool import -c $cachefile -a -N > if [ $? -ne 0 ]; then > echo "Import of zpool cache > ${cachefile} failed," \ > "will retry after root mount hold > release" > root_hold_wait > zpool import -c $cachefile -a -N > fi > break > fi > done > } > > load_rc_config $name > run_rc_command "$1" > titan# > > > > I may as well nuke the pre-existing cache file and start over : > > > titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 1424 Jan 16 2024 /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache > titan# > titan# > titan# rm /boot/zfs/zpool.cache > titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" t0 > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache > titan# > titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" leaf > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache > titan# > titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" proteus > titan# > titan# ls -l /boot/zfs/zpool.cache > -rw-r--r-- 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache > titan# > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile /boot/zfs/zpool.cache local > titan# > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile /boot/zfs/zpool.cache local > titan# > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile /boot/zfs/zpool.cache local > titan# > > titan# > titan# reboot > Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to > stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 0 0 0 0 0 0 done > All buffers synced. > Uptime: 2h38m57s > GEOM_MIRROR: Device swap: provider destroyed. > GEOM_MIRROR: Device swap destroyed. > uhub5: detached > uhub1: detached > uhub4: detached > uhub2: detached > uhub3: detached > uhub6: detached > uhub0: detached > ix0: link state changed to DOWN > . > . > . > > Starting iscsid. > Starting iscsictl. > Clearing /tmp. > Updating /var/run/os-release done. > Updating motd:. > Creating and/or trimming log files. > Starting syslogd. > No core dumps found. > Starting local daemons:failed to open cache file: No such file or directo= ry > . > Starting ntpd. > Starting powerd. > Mounting late filesystems:. > Starting cron. > Performing sanity check on sshd configuration. > Starting sshd. > Starting background file system > FreeBSD/amd64 (titan) (ttyu0) > > login: root > Password: > Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0 > Last login: Thu Nov 28 14:33:45 on ttyu0 > FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 > main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > > Welcome to FreeBSD! > > Release Notes, Errata: https://www.FreeBSD.org/releases/ > Security Advisories: https://www.FreeBSD.org/security/ > FreeBSD Handbook: https://www.FreeBSD.org/handbook/ > FreeBSD FAQ: https://www.FreeBSD.org/faq/ > Questions List: https://www.FreeBSD.org/lists/questions/ > FreeBSD Forums: https://forums.FreeBSD.org/ > > Documents installed with the system are in the > /usr/local/share/doc/freebsd/ > directory, or can be installed later with: pkg install en-freebsd-doc > For other languages, replace "en" with a language code like de or fr. > > Show the version of FreeBSD installed: freebsd-version ; uname -a > Please include that output and any error messages when posting questions. > Introduction to manual pages: man man > FreeBSD directory layout: man hier > > To change this login announcement, see motd(5). > You have new mail. > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > leaf 18.2T 984K 18.2T - - 0% 0% 1.00x > ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x > ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x > ONLINE - > titan# > > This is progress ... however the cachefile property is wiped out again : > > titan# zpool get cachefile t0 > NAME PROPERTY VALUE SOURCE > t0 cachefile - default > titan# zpool get cachefile leaf > NAME PROPERTY VALUE SOURCE > leaf cachefile - default > titan# zpool get cachefile proteus > NAME PROPERTY VALUE SOURCE > proteus cachefile - default > titan# > > Also, strangely, none of the filesystem in proteus are mounted : > > titan# > titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r > proteus > NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT > proteus on sha512 on no none > proteus/bhyve off sha512 on no /bhyve > proteus/bhyve/disk off sha512 on no /bhyve/disk > proteus/bhyve/isos off sha512 on no /bhyve/isos > proteus/obj on sha512 on no /usr/obj > proteus/src on sha512 on no /usr/src > titan# > > If I reboot again without doing anything will the zpools re-appear ? > > > titan# > titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0 > > titan# reboot > Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to > stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 0 0 0 0 0 done > All buffers synced. > Uptime: 4m50s > GEOM_MIRROR: Device swap: provider destroyed. > GEOM_MIRROR: Device swap destroyed. > uhub4: detached > uhub1: detached > uhub5: detached > uhub0: detached > uhub3: detached > uhub6: detached > uhub2: detached > ix0: link state changed to DOWN > . > . > . > Starting iscsid. > Starting iscsictl. > Clearing /tmp. > Updating /var/run/os-release done. > Updating motd:. > Creating and/or trimming log files. > Starting syslogd. > No core dumps found. > Starting local daemons:failed to open cache file: No such file or directo= ry > . > Starting ntpd. > Starting powerd. > Mounting late filesystems:. > Starting cron. > Performing sanity check on sshd configuration. > Starting sshd. > Starting background file system > FreeBSD/amd64 (titan) (ttyu0) > > login: root > Password: > Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0 > Last login: Thu Nov 28 14:36:29 on ttyu0 > FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1 > main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 > > Welcome to FreeBSD! > > Release Notes, Errata: https://www.FreeBSD.org/releases/ > Security Advisories: https://www.FreeBSD.org/security/ > FreeBSD Handbook: https://www.FreeBSD.org/handbook/ > FreeBSD FAQ: https://www.FreeBSD.org/faq/ > Questions List: https://www.FreeBSD.org/lists/questions/ > FreeBSD Forums: https://forums.FreeBSD.org/ > > Documents installed with the system are in the > /usr/local/share/doc/freebsd/ > directory, or can be installed later with: pkg install en-freebsd-doc > For other languages, replace "en" with a language code like de or fr. > > Show the version of FreeBSD installed: freebsd-version ; uname -a > Please include that output and any error messages when posting questions. > Introduction to manual pages: man man > FreeBSD directory layout: man hier > > To change this login announcement, see motd(5). > You have new mail. > titan# > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > leaf 18.2T 1.01M 18.2T - - 0% 0% 1.00x > ONLINE - > proteus 1.98T 361G 1.63T - - 1% 17% 1.00x > ONLINE - > t0 444G 91.2G 353G - - 27% 20% 1.00x > ONLINE - > titan# > titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r > proteus > NAME EXEC CHECKSUM CANMOUNT MOUNTED MOUNTPOINT > proteus on sha512 on no none > proteus/bhyve off sha512 on no /bhyve > proteus/bhyve/disk off sha512 on no /bhyve/disk > proteus/bhyve/isos off sha512 on no /bhyve/isos > proteus/obj on sha512 on no /usr/obj > proteus/src on sha512 on no /usr/src > titan# > > OKay so the zpools appear to be back in spite of the strange situation > with the cachefile property is empty everywhere. My guess is the zpool > rc script is bring in information during early boot. > > Why the zfs filesystems on proteus do not mount? Well that is a strange > problem but at least the zpool can be used. > > -- > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > For "zpool import", the "-c" argument instructs zfs which cachefile to search for importable pools. "-O", on the other hand, specifies how the cachefile property should be set after the pool is imported. > --000000000000e1437d0627fa3c5e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Nov 28, 2024, 8:45=E2=80=AFAM Dennis Clarke &l= t;dclarke@blastwave.org> wr= ote:
On 11/28/24= 08:52, Alan Somers wrote:
> On Thu, Nov 28, 2024, 7:06=E2=80=AFAM Dennis Clarke <dclarke@bla= stwave.org> wrote:
>
>>
>> This is a baffling problem wherein two zpools no longer exist afte= r
>> boot. This is :
.
.
.
> Do you have zfs_enable=3D"YES" set in /etc/rc.conf? If not t= hen nothing will
> get imported.
>
> Regarding the cachefile property, it's expected that "zpool i= mport" will
> change it, unless you do "zpool import -O cachefile=3Dwhatever&qu= ot;.
>

The rc script seems to do something slightly different with zpool import -c $FOOBAR thus :


titan# cat=C2=A0 /etc/rc.d/zpool
#!/bin/sh
#
#

# PROVIDE: zpool
# REQUIRE: hostid disks
# BEFORE: mountcritlocal
# KEYWORD: nojail

. /etc/rc.subr

name=3D"zpool"
desc=3D"Import ZPOOLs"
rcvar=3D"zfs_enable"
start_cmd=3D"zpool_start"
required_modules=3D"zfs"

zpool_start()
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0local cachefile

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for cachefile in /etc/zfs/zpool.cache /bo= ot/zfs/zpool.cache; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if [ -r $cach= efile ]; then
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0zpool import -c $cachefile -a -N
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0if [ $? -ne 0 ]; then
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0echo "Import of zpool cac= he
${cachefile} failed," \
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"will retry= after root mount hold
release"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0root_hold_wait
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0zpool import -c $cachefile -a = -N
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0fi
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0break
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0fi
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0done
}

load_rc_config $name
run_rc_command "$1"
titan#



I may as well nuke the pre-existing cache file and start over :


titan# ls -l /etc/zfs/zpool.cache /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 1424 Jan 16=C2=A0 2024 /boot/zfs/zpool.cache<= br> -rw-r--r--=C2=A0 1 root wheel 4960 Nov 28 14:15 /etc/zfs/zpool.cache
titan#
titan#
titan# rm /boot/zfs/zpool.cache
titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" t0
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 1456 Nov 28 14:27 /boot/zfs/zpool.cache
titan#
titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" leaf
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 3536 Nov 28 14:28 /boot/zfs/zpool.cache
titan#
titan# zpool set cachefile=3D"/boot/zfs/zpool.cache" proteus
titan#
titan# ls -l /boot/zfs/zpool.cache
-rw-r--r--=C2=A0 1 root wheel 4960 Nov 28 14:28 /boot/zfs/zpool.cache
titan#
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 /boot/zfs/zpool.cache=C2=A0 local
titan#
titan# zpool get cachefile leaf
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 SOURCE
leaf=C2=A0 cachefile=C2=A0 /boot/zfs/zpool.cache=C2=A0 local
titan#
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 SOURCE
proteus=C2=A0 cachefile=C2=A0 /boot/zfs/zpool.cache=C2=A0 local
titan#

titan#
titan# reboot
Nov 28 14:34:05 Waiting (max 60 seconds) for system process `vnlru' to =
stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 0 done
All buffers synced.
Uptime: 2h38m57s
GEOM_MIRROR: Device swap: provider destroyed.
GEOM_MIRROR: Device swap destroyed.
uhub5: detached
uhub1: detached
uhub4: detached
uhub2: detached
uhub3: detached
uhub6: detached
uhub0: detached
ix0: link state changed to DOWN
.
.
.

Starting iscsid.
Starting iscsictl.
Clearing /tmp.
Updating /var/run/os-release done.
Updating motd:.
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting local daemons:failed to open cache file: No such file or directory=
.
Starting ntpd.
Starting powerd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system
FreeBSD/amd64 (titan) (ttyu0)

login: root
Password:
Nov 28 14:36:29 titan login[4162]: ROOT LOGIN (root) ON ttyu0
Last login: Thu Nov 28 14:33:45 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1
main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/=
Security Advisories:=C2=A0 =C2=A0https://www.FreeBSD.org= /security/
FreeBSD Handbook:=C2=A0 =C2=A0 =C2=A0 https://www.FreeBS= D.org/handbook/
FreeBSD FAQ:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://= www.FreeBSD.org/faq/
Questions List:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https:= //www.FreeBSD.org/lists/questions/
FreeBSD Forums:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https://forums.Free= BSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd= /
directory, or can be installed later with:=C2=A0 pkg install en-freebsd-doc=
For other languages, replace "en" with a language code like de or= fr.

Show the version of FreeBSD installed:=C2=A0 freebsd-version ; uname -a
Please include that output and any error messages when posting questions. Introduction to manual pages:=C2=A0 man man
FreeBSD directory layout:=C2=A0 =C2=A0 =C2=A0 man hier

To change this login announcement, see motd(5).
You have new mail.
titan#
titan# zpool list
NAME=C2=A0 =C2=A0 =C2=A0 SIZE=C2=A0 ALLOC=C2=A0 =C2=A0FREE=C2=A0 CKPOINT=C2= =A0 EXPANDSZ=C2=A0 =C2=A0FRAG=C2=A0 =C2=A0 CAP=C2=A0 DEDUP
HEALTH=C2=A0 ALTROOT
leaf=C2=A0 =C2=A0 =C2=A018.2T=C2=A0 =C2=A0984K=C2=A0 18.2T=C2=A0 =C2=A0 =C2= =A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A00%=C2=A0= =C2=A0 =C2=A00%=C2=A0 1.00x
ONLINE=C2=A0 -
proteus=C2=A0 1.98T=C2=A0 =C2=A0361G=C2=A0 1.63T=C2=A0 =C2=A0 =C2=A0 =C2=A0= -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A01%=C2=A0 =C2=A0 17= %=C2=A0 1.00x
ONLINE=C2=A0 -
t0=C2=A0 =C2=A0 =C2=A0 =C2=A0 444G=C2=A0 91.2G=C2=A0 =C2=A0353G=C2=A0 =C2= =A0 =C2=A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 27%=C2= =A0 =C2=A0 20%=C2=A0 1.00x
ONLINE=C2=A0 -
titan#

This is progress ... however the cachefile property is wiped out again :
titan# zpool get cachefile t0
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOURCE
t0=C2=A0 =C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default=
titan# zpool get cachefile leaf
NAME=C2=A0 PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOURCE
leaf=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default
titan# zpool get cachefile proteus
NAME=C2=A0 =C2=A0 =C2=A0PROPERTY=C2=A0 =C2=A0VALUE=C2=A0 =C2=A0 =C2=A0 SOUR= CE
proteus=C2=A0 cachefile=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 default titan#

Also, strangely, none of the filesystem in proteus are mounted :

titan#
titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteu= s
NAME=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 EXEC=C2=A0 CHEC= KSUM=C2=A0 =C2=A0CANMOUNT=C2=A0 MOUNTED=C2=A0 MOUNTPOINT
proteus=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha5= 12=C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 = =C2=A0none
proteus/bhyve=C2=A0 =C2=A0 =C2=A0 =C2=A0off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0= =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve proteus/bhyve/disk=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/disk
proteus/bhyve/isos=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/isos
proteus/obj=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/obj
proteus/src=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/src
titan#

If I reboot again without doing anything will the zpools re-appear ?


titan#
titan# Nov 28 14:37:08 titan su[4199]: admsys to root on /dev/pts/0

titan# reboot
Nov 28 14:40:29 Waiting (max 60 seconds) for system process `vnlru' to =
stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 done
All buffers synced.
Uptime: 4m50s
GEOM_MIRROR: Device swap: provider destroyed.
GEOM_MIRROR: Device swap destroyed.
uhub4: detached
uhub1: detached
uhub5: detached
uhub0: detached
uhub3: detached
uhub6: detached
uhub2: detached
ix0: link state changed to DOWN
.
.
.
Starting iscsid.
Starting iscsictl.
Clearing /tmp.
Updating /var/run/os-release done.
Updating motd:.
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting local daemons:failed to open cache file: No such file or directory=
.
Starting ntpd.
Starting powerd.
Mounting late filesystems:.
Starting cron.
Performing sanity check on sshd configuration.
Starting sshd.
Starting background file system
FreeBSD/amd64 (titan) (ttyu0)

login: root
Password:
Nov 28 14:43:01 titan login[4146]: ROOT LOGIN (root) ON ttyu0
Last login: Thu Nov 28 14:36:29 on ttyu0
FreeBSD 15.0-CURRENT (GENERIC-NODEBUG) #1
main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/=
Security Advisories:=C2=A0 =C2=A0https://www.FreeBSD.org= /security/
FreeBSD Handbook:=C2=A0 =C2=A0 =C2=A0 https://www.FreeBS= D.org/handbook/
FreeBSD FAQ:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0https://= www.FreeBSD.org/faq/
Questions List:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https:= //www.FreeBSD.org/lists/questions/
FreeBSD Forums:=C2=A0 =C2=A0 =C2=A0 =C2=A0 https://forums.Free= BSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd= /
directory, or can be installed later with:=C2=A0 pkg install en-freebsd-doc=
For other languages, replace "en" with a language code like de or= fr.

Show the version of FreeBSD installed:=C2=A0 freebsd-version ; uname -a
Please include that output and any error messages when posting questions. Introduction to manual pages:=C2=A0 man man
FreeBSD directory layout:=C2=A0 =C2=A0 =C2=A0 man hier

To change this login announcement, see motd(5).
You have new mail.
titan#
titan# zpool list
NAME=C2=A0 =C2=A0 =C2=A0 SIZE=C2=A0 ALLOC=C2=A0 =C2=A0FREE=C2=A0 CKPOINT=C2= =A0 EXPANDSZ=C2=A0 =C2=A0FRAG=C2=A0 =C2=A0 CAP=C2=A0 DEDUP
HEALTH=C2=A0 ALTROOT
leaf=C2=A0 =C2=A0 =C2=A018.2T=C2=A0 1.01M=C2=A0 18.2T=C2=A0 =C2=A0 =C2=A0 = =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A00%=C2=A0 =C2= =A0 =C2=A00%=C2=A0 1.00x
ONLINE=C2=A0 -
proteus=C2=A0 1.98T=C2=A0 =C2=A0361G=C2=A0 1.63T=C2=A0 =C2=A0 =C2=A0 =C2=A0= -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A01%=C2=A0 =C2=A0 17= %=C2=A0 1.00x
ONLINE=C2=A0 -
t0=C2=A0 =C2=A0 =C2=A0 =C2=A0 444G=C2=A0 91.2G=C2=A0 =C2=A0353G=C2=A0 =C2= =A0 =C2=A0 =C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-=C2=A0 =C2=A0 27%=C2= =A0 =C2=A0 20%=C2=A0 1.00x
ONLINE=C2=A0 -
titan#
titan# zfs list -o name,exec,checksum,canmount,mounted,mountpoint -r proteu= s
NAME=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 EXEC=C2=A0 CHEC= KSUM=C2=A0 =C2=A0CANMOUNT=C2=A0 MOUNTED=C2=A0 MOUNTPOINT
proteus=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha5= 12=C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 = =C2=A0none
proteus/bhyve=C2=A0 =C2=A0 =C2=A0 =C2=A0off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0= =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve proteus/bhyve/disk=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/disk
proteus/bhyve/isos=C2=A0 off=C2=A0 =C2=A0sha512=C2=A0 =C2=A0 =C2=A0on=C2=A0= =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/bhyve/isos
proteus/obj=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/obj
proteus/src=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0on=C2=A0 =C2=A0 sha512=C2=A0 = =C2=A0 =C2=A0on=C2=A0 =C2=A0 =C2=A0 =C2=A0 no=C2=A0 =C2=A0 =C2=A0 =C2=A0/us= r/src
titan#

OKay so the zpools appear to be back in spite of the strange situation
with the cachefile property is empty everywhere.=C2=A0 My guess is the zpoo= l
rc script is bring in information during early boot.

Why the zfs filesystems on proteus do not mount? Well that is a strange problem but at least the zpool can be used.

--
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

For "zpool import", the "-c" argum= ent instructs zfs which cachefile to search for importable pools. "-O&= quot;, on the other hand, specifies how the cachefile property should be se= t after the pool is imported.=C2=A0
--000000000000e1437d0627fa3c5e--