From nobody Thu Feb 24 15:57:14 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6B2AB19D77B0 for ; Thu, 24 Feb 2022 15:57:17 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K4Hbc4zj2z4twX; Thu, 24 Feb 2022 15:57:16 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=+i4bG89VZm9005+mZsXIkj4LyXlrzf841nlwXIHPNL8=; b=JIv4kh/PyidLPb+8Kux2ZRJL0g 4Qp4leaFUHAyA/eKQBFwPfQ8pss5Xfq/Xt0iWu8POWj8HlxogbxBFLQ4C9tVMsrTYYTHcE94csVM4 Kk28geUAQPSsBWOOryr7Ea/4XXobd8LMzyXU/FYJ9SnRpil6GInCLihxMhC8MAbFB/KwFq1vmY1OS OwyT4jv9AmzWUDPgYsfCTn7FswdEhI+afPqbpl3et+Fr1o55IawvqdAyUcgUW0NpPDIs2hBcyqjOT OclN1/ayNXqarfHxC8S73amDR5DwTEC30MQMI+3RHr506l8uLVKolioY1/NUd+eikoUdLbs/I8Vdx tXk7JndQ==; Received-SPF: pass (thebighonker.lerctr.org: domain of lerctr.org designates 2001:470:1f0f:3ad:bb:dcff:fe50:d900 as permitted sender) client-ip=2001:470:1f0f:3ad:bb:dcff:fe50:d900; envelope-from=ler@lerctr.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:63382 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1nNGUV-0008gX-0F; Thu, 24 Feb 2022 09:57:15 -0600 Received: from 2600:1700:210:b18f:cc53:f3ee:b613:c3a6 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Feb 2022 09:57:14 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Thu, 24 Feb 2022 09:57:14 -0600 From: Larry Rosenman To: Alexander Motin Cc: Freebsd current Subject: Re: ZFS PANIC: HELP. In-Reply-To: <2bc12c9ee3e6dd71b65079116ff2b845@lerctr.org> References: <07c0c9c34b4a4133acab597c96867d27@lerctr.org> <95c7c326-e2f9-6e66-7b97-b9fb2671f4ad@FreeBSD.org> <1b6b2017ba69e6fda1ca237c3016ac61@lerctr.org> <6182c57bf1859482e72af78037c399e4@lerctr.org> <7abcbe88-446f-44b9-c3a7-0997d3430b57@FreeBSD.org> <2bc12c9ee3e6dd71b65079116ff2b845@lerctr.org> Message-ID: <5930f3d2b71c0932f903bb5b88a3c87d@lerctr.org> X-Sender: ler@lerctr.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4K4Hbc4zj2z4twX X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=lerctr.org header.s=ler2019 header.b="JIv4kh/P"; dmarc=pass (policy=none) header.from=lerctr.org; spf=pass (mx1.freebsd.org: domain of ler@lerctr.org designates 192.147.25.65 as permitted sender) smtp.mailfrom=ler@lerctr.org X-Spamd-Result: default: False [-0.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[lerctr.org:s=ler2019]; FREEFALL_USER(0.00)[ler]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[lerctr.org:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[lerctr.org,none]; SUBJ_ALL_CAPS(1.20)[16]; MLMMJ_DEST(0.00)[freebsd-current]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8166, ipnet:192.147.25.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 02/23/2022 9:27 pm, Larry Rosenman wrote: > On 02/23/2022 9:15 pm, Alexander Motin wrote: >> On 23.02.2022 22:01, Larry Rosenman wrote: >>> On 02/23/2022 8:58 pm, Alexander Motin wrote: >>>> On 23.02.2022 21:52, Larry Rosenman wrote: >>>>> On 02/23/2022 8:41 pm, Alexander Motin wrote: >>>>>> Hi Larry, >>>>>> >>>>>> The panic you are getting is an assertion, enabled by kernel built >>>>>> with INVARIANTS option.  On 13 you may just not have that >>>>>> debugging >>>>>> enabled to hit the issue.  But that may be only a consequence. >>>>>> Original problem I guess in possibly corrupted ZFS intent log >>>>>> records >>>>>> (or false positive), that could happen so due to use of -F >>>>>> recovery >>>>>> option on `zpool import`, that supposed to try import pool at >>>>>> earlier >>>>>> transaction group if there is some metadata corruption found.  It >>>>>> is >>>>>> not supposed to work 100% and only a last resort.  Though may be >>>>>> that >>>>>> assertion is just excessively strict for that specific recovery >>>>>> case. >>>>>> If as you say pool can be imported and scrubbed on 13, then I'd >>>>>> expect >>>>>> following clean export should allow later import on 14 without -F. >>>>>> >>>>>> On 23.02.2022 21:21, Larry Rosenman wrote: >>>>>>> >>>>>>> 've got my main dev box that crashes on 14 with the screen shot >>>>>>> at https://www.lerctr.org/~ler/14-zfs-crash.png. >>>>>>> Booting from a 13-REL USB installer it imports and scrubs. >>>>>>> >>>>>>> Ideas? >>>>>>> >>>>>>> I can either video conference with shared screen or give access >>>>>>> to the console via my Dominion KVM. >>>>>>> >>>>>>> Any help/ideas/etc welcome I really need to get this box back. >>>>>>> >>>>>>> >>>>> How can I import the pool withOUT it mounting the FileSystems so I >>>>> can export it cleanly on the 13 system? >>>> >>>> Why do you need to import without mounting file systems?  I think >>>> you >>>> may actually wish them to be mounted to replay their ZILs.  Just use >>>> -R option to mount file systems in some different place. >>> >>> I get the errors shown at: >>> https://www.lerctr.org/~ler/14-mount-R-output.png >>> >>> Should I worry?  Or do something(tm) here? >> >> This looks weird, but may possibly depend on mount points topology, >> whether /mnt is writable, etc. What happen if you export it now and >> try to import it in normal way on 14 without -F? > > It crashes just after root mount (this is the boot pool and only pool > on the system), > seeL > https://www.lerctr.org/~ler/14-BOOT-Crash.png Where do I go from here? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106