From nobody Fri Apr 1 18:11:43 2022 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8F7B41A5743E for ; Fri, 1 Apr 2022 18:11:46 +0000 (UTC) (envelope-from rrsum@summerhill.org) Received: from chance.summerhill.org (104-179-148-192.lightspeed.livnmi.sbcglobal.net [104.179.148.192]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "chance", Issuer "chance" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KVSt94VBtz3CLk for ; Fri, 1 Apr 2022 18:11:45 +0000 (UTC) (envelope-from rrsum@summerhill.org) Received-SPF: pass (chance.summerhill.org: 192.168.1.14 is whitelisted) receiver=chance.summerhill.org; client-ip=192.168.1.14; helo=[192.168.1.14]; envelope-from=rrsum@summerhill.org; x-software=spfmilter 2.001 http://www.acme.com/software/spfmilter/ with libspf2-1.2.10; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=summerhill.org; s=home; t=1648836704; bh=WXKyzbPLkcqL0+5HltGve8H1tMhmf6notgIP8Gv4UC8=; h=Date:Subject:To:References:From:In-Reply-To; b=GxYOUqw8PevMoA6DGfi4z7r6QPNzDK15E7SbcvBKo0r29swiuWqz6RligFWEH9Zx6 YVLTuK1UvYf+3M4ZM8ExJRg33BSJ6LtVrqZyPVichX1F93JElC/phnSglsmxs0BbGp ZKIDrIiv+pAX4uzWUZmuywKWW21oLbCH6BtB/j+I7lpIe8X7BXvdg93XdU7ZH0JFq4 a64Ki+K7iGlyRGehs69RL0avXlnNSrY3kPHfrlQART8++gfrkj/IPMuxq0rmWAzBaV 85OR6xOSqBrKLIrIvXBGXoLQliZ8KKvGIqLApowKJHo5G2EyCdlQs8fBnUKXNMWyg7 47NK73jrUEW2g== Received: from [192.168.1.14] (jake.summerhill.org [192.168.1.14]) (authenticated bits=0) by chance.summerhill.org (8.17.1/8.17.1) with ESMTPSA id 231IBhq1043048 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Fri, 1 Apr 2022 14:11:44 -0400 (EDT) (envelope-from rrsum@summerhill.org) X-Authentication-Warning: chance.summerhill.org: Host jake.summerhill.org [192.168.1.14] claimed to be [192.168.1.14] Message-ID: <32e33149-8013-9c5e-868b-0beef4c63889@summerhill.org> Date: Fri, 1 Apr 2022 14:11:43 -0400 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: zpools disappearing on reboot with 13.1Beta3 (and now RC1) Content-Language: en-US To: questions@freebsd.org References: <383af7d8-294f-bd43-9a4e-660b56261d3e@summerhill.org> <20220329210600.b4e405ae3a5006eceaaf44cb@sohara.org> From: Rick Summerhill In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4KVSt94VBtz3CLk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=summerhill.org header.s=home header.b=GxYOUqw8; dmarc=pass (policy=reject) header.from=summerhill.org; spf=pass (mx1.freebsd.org: domain of rrsum@summerhill.org designates 104.179.148.192 as permitted sender) smtp.mailfrom=rrsum@summerhill.org X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[summerhill.org:s=home]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; HAS_XAW(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[summerhill.org:+]; DMARC_POLICY_ALLOW(-0.50)[summerhill.org,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[questions]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7018, ipnet:104.176.0.0/12, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 4/1/22 2:01 PM, Rick Summerhill wrote: > On 3/29/22 6:20 PM, R Richard Summerhill wrote: >> On 3/29/22 16:06, Steve O'Hara-Smith wrote: >>> On Tue, 29 Mar 2022 15:37:19 -0400 >>> Rick Summerhill wrote: >>> >>>> I'm running a server under 13.1Beta3 and it has 2 zpools, one a mirror >>>> and the other a raidz1.  When I reboot the server, both pools disappear >>>> but come back when imported.  Do I have something set wrong on the >>>> server? >>>     Does your /etc/rc.conf contain >>> >>> zfs_enable="YES" >> >> >> Yes, it does.  Let me give you a bit more info.  I rebuilt 13.1Beta3 >> with just the mirror pool.  Rebooted several times to test everything >> and it came back correctly each time.  Then I added one of these 5x1 >> port multiplier arrays to an esata port (this is one of my test boxes) >> and rebooted.  All the disks were there (they are all labeled >> appropriately), and the mirror zpool (which uses the labels) was not >> there.  Importing it, however, brought it back.  Then I created the >> raidz1 pool on the multiplier array.  Both pools showed up with a good >> status.  On rebooting, the disks are there and the pools are not, but >> on importing, both come back. >> >> Not that the multiplier array has been working fine under 13.0Release >> and ZFS. >> >> > > I upgraded to 13.1RC1 and the condition persists.  Also, dmesg shows > > ZFS filesystem version: 5 > ZFS storage pool version: features support (5000) > pid 49 (zpool), jid 0, uid 0: exited on signal 6 > > and the disks for the pools appear after that in dmesg. > > I also tried adding the following lines to loader.conf > > kern.cam.boot_delay="5000"  # Delay (in ms) of root mount for CAM bus > kern.cam.scsi_delay="2000"  # Delay (in ms) before probing SCSI > > That did not solve the problem, however.  Any thoughts? One other thing I should have pointed out: The system disks are on nvd0 and the line right before the zfs lines in dmesg say: Trying to mount root from ufs:/dev/nvd0p2 [rw]... Rick -- Rick Summerhill Retired, Chief Technology Officer, Internet2 10233 Timberhill Rd Manchester, MI 48158 USA Home: 734-428-1422 Web: http://www.rick.summerhill.org