From nobody Thu Nov 28 15:02:26 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xzfg83r7Jz5dqPq for ; Thu, 28 Nov 2024 15:02:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xzfg83Mc7z4D8c; Thu, 28 Nov 2024 15:02:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732806148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsNR7zc21fUAeDNY6arFEhppR1vjE/1RxufU4C51Dis=; b=q4Jnbz93t0wv2uQFd9qNh5sMUrPF+qimJ0mK+DG7RiBlhpQxjRuP5c3rd6it4Xt8uT5/b3 dEDPOxho2WboTZHCQmf0DCdquQ0od3NOWNCGc2SrI4HHo34ADOcbT44KA2C6yKGCXKrm6D 6YtLPVoQWMl0tUj7BkvRyA1iCAttN/jNcwzr8VaQKTHaYruFs+6ZPbJwF7foUS3kmf5LQw 5lNXHNAL9QcZscOGZIKfIEI3kMmXr9RiEgqi1QT388DzRhJGhW87g6CGAmGmOoD41OUZLX e0tzSRElvaM2K54PbEJa90TTmr3aDhGt4KJUiwJ5NyzKQ4ANKoBwjQyKDCMBRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732806148; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsNR7zc21fUAeDNY6arFEhppR1vjE/1RxufU4C51Dis=; b=gzGJwFLapWwDDwq1isVC5Ib2HcVxmW9Z1RWloXDX1z8eBXwjY7Ioq8c6L1h0yng1KOogdj fN9Y3H2VX+ikGGQvCH0UFH6emedhyILUOOoX+kNQcl7y6hCb46bEMYWnjGW73Gmf2bMljz VI6Dt/RAKbKqiccziAbHYAmzx3ZAQPzee1R3namvdGmG7iI0S8Ms0KNjl+CnYDHfXB/z3J M5ufkXaEjdEi7dEh9JlQGojPPggKXHBcJOBJCFKOTNoxP4wu8NHbb3yZNITE6d21xyMdGg zFqwK2ccajIQw7npvyNUjo0PddUyITdr0MFohXauIsb30X+aCS1gH93Tw7PxzA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732806148; a=rsa-sha256; cv=none; b=ODCYRIbJG6ObZjS1I68FT8Ph1wREZp4s3J+lauoqDGWvqgM70S0a/bFUGdjPu322+kFNZb LERe+7lL8fMRb+/1wUk+tCUDP5t49PV00tjrFX1ZMC9w22W2hJ294vudkB1T+iVpUdz5Np fwMKGqf+MCQ3rZYRpA0/1X8cNnt95Q+CCiwSZm5iABSNSjoqRd1RwY5a4MIb201AsRwmi+ Q8imqVRZ2Jk5mpks1YWDKnro00EnTTRYngwOJjgajPYr5lnYcPdsMGLk22x1weskxedu2+ mr1vcCdqRMUHPAHx/GwfODrEmC8fSEjbksbkJaqhORu0f8QDjtkL/JXZfoztAg== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R10" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Xzfg82R59z12kV; Thu, 28 Nov 2024 15:02:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id E5F1951CB3; Thu, 28 Nov 2024 16:02:26 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.9\)) Subject: Re: zpools no longer exist after boot From: Dimitry Andric In-Reply-To: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> Date: Thu, 28 Nov 2024 16:02:26 +0100 Cc: Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <5165C4DB-8BA0-4579-A808-F98FBF3B797F@FreeBSD.org> References: <5798b0db-bc73-476a-908a-dd1f071bfe43@blastwave.org> To: Dennis Clarke X-Mailer: Apple Mail (2.3731.700.6.1.9) On 28 Nov 2024, at 14:05, Dennis Clarke wrote: >=20 > This is a baffling problem wherein two zpools no longer exist after > boot. This is : >=20 > titan# uname -apKU > FreeBSD titan 15.0-CURRENT FreeBSD 15.0-CURRENT #1 = main-n273749-4b65481ac68a-dirty: Wed Nov 20 15:08:52 GMT 2024 = root@titan:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 amd64 = 1500027 1500027 > titan# >=20 > titan# zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT > t0 444G 91.2G 353G - - 27% 20% 1.00x = ONLINE - > titan# >=20 > The *only* zpool that seems to exist in any reliable way is the little > NVME based unit for booting. The other two zpools vanished and yet the > devices exist just fine : >=20 > titan# > titan# camcontrol devlist > at scbus0 target 0 lun 0 = (pass0,ada0) > at scbus1 target 0 lun 0 = (pass1,ada1) > at scbus2 target 0 lun 0 = (ses0,pass2) > at scbus6 target 0 lun 0 = (ses1,pass3) > at scbus7 target 0 lun 1 = (pass4,nda0) > at scbus8 target 0 lun 0 = (da0,pass5) It has been my experience that with disks connected to external = enclosures, these sometimes take a long time to come up. If they come up = only after the kernel initially detects ZFS pools, they will not be = shown or imported at all. If you look at dmesg or boot logs, are the external disks detected = around the same time as the NVMe device? -Dimitry