From nobody Thu Feb 24 03:27:45 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F190E19EDA71 for ; Thu, 24 Feb 2022 03:27:47 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K3yyq1rm2z4XYZ; Thu, 24 Feb 2022 03:27:47 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Content-Transfer-Encoding:Content-Type:Message-ID:References: In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ij7UO/0xQeWQYWwcBpATilBdEBPdE/SZqt5/Chly1CI=; b=x4eGpLXE/AWMan17VXHAN8z5ZK AOnbjzQM3+R357vK//2GvEdu18iBJ8SgW7ISanvshf6y7zjJXkYD1fqGkZXpQIKGAXe3Vqc4nkpg/ MRyePYYygXGP1e9cqLeeX8xlju64FJWAcY8P0OpxzL8hHci2NyPyWuC+oD6Xzs0IBvcOMcL6AXWlR SA1zCAoUWQNEZJ7+JqTmnRataSrJlYQaViLxduhctYrTA1yJRFHVt57NWbtfelt3F7bL6iArqsNho +kFta9Uxj1qLXg+ieR3sSX4gqrKqepTSjlUtBjAVE7KhHa5XimsfpQdUBIdXSyO/hwi4ctmqatLLg gYDc8WEw==; Received-SPF: pass (thebighonker.lerctr.org: domain of lerctr.org designates 2001:470:1f0f:3ad:bb:dcff:fe50:d900 as permitted sender) client-ip=2001:470:1f0f:3ad:bb:dcff:fe50:d900; envelope-from=ler@lerctr.org; helo=webmail.lerctr.org; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:58369 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1nN4nB-000NC1-Eu; Wed, 23 Feb 2022 21:27:45 -0600 Received: from 2600:1700:210:b18f:cd39:c42e:bc5b:5db5 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Wed, 23 Feb 2022 21:27:45 -0600 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Wed, 23 Feb 2022 21:27:45 -0600 From: Larry Rosenman To: Alexander Motin Cc: Freebsd current Subject: Re: ZFS PANIC: HELP. In-Reply-To: <7abcbe88-446f-44b9-c3a7-0997d3430b57@FreeBSD.org> References: <07c0c9c34b4a4133acab597c96867d27@lerctr.org> <95c7c326-e2f9-6e66-7b97-b9fb2671f4ad@FreeBSD.org> <1b6b2017ba69e6fda1ca237c3016ac61@lerctr.org> <6182c57bf1859482e72af78037c399e4@lerctr.org> <7abcbe88-446f-44b9-c3a7-0997d3430b57@FreeBSD.org> Message-ID: <2bc12c9ee3e6dd71b65079116ff2b845@lerctr.org> X-Sender: ler@lerctr.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4K3yyq1rm2z4XYZ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=lerctr.org header.s=ler2019 header.b=x4eGpLXE; dmarc=pass (policy=none) header.from=lerctr.org; spf=pass (mx1.freebsd.org: domain of ler@lerctr.org designates 192.147.25.65 as permitted sender) smtp.mailfrom=ler@lerctr.org X-Spamd-Result: default: False [-2.75 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[lerctr.org:s=ler2019]; FREEFALL_USER(0.00)[ler]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[lerctr.org:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[lerctr.org,none]; SUBJ_ALL_CAPS(1.20)[16]; MLMMJ_DEST(0.00)[freebsd-current]; NEURAL_HAM_SHORT(-0.95)[-0.955]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8166, ipnet:192.147.25.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 02/23/2022 9:15 pm, Alexander Motin wrote: > On 23.02.2022 22:01, Larry Rosenman wrote: >> On 02/23/2022 8:58 pm, Alexander Motin wrote: >>> On 23.02.2022 21:52, Larry Rosenman wrote: >>>> On 02/23/2022 8:41 pm, Alexander Motin wrote: >>>>> Hi Larry, >>>>> >>>>> The panic you are getting is an assertion, enabled by kernel built >>>>> with INVARIANTS option.  On 13 you may just not have that debugging >>>>> enabled to hit the issue.  But that may be only a consequence. >>>>> Original problem I guess in possibly corrupted ZFS intent log >>>>> records >>>>> (or false positive), that could happen so due to use of -F recovery >>>>> option on `zpool import`, that supposed to try import pool at >>>>> earlier >>>>> transaction group if there is some metadata corruption found.  It >>>>> is >>>>> not supposed to work 100% and only a last resort.  Though may be >>>>> that >>>>> assertion is just excessively strict for that specific recovery >>>>> case. >>>>> If as you say pool can be imported and scrubbed on 13, then I'd >>>>> expect >>>>> following clean export should allow later import on 14 without -F. >>>>> >>>>> On 23.02.2022 21:21, Larry Rosenman wrote: >>>>>> >>>>>> 've got my main dev box that crashes on 14 with the screen shot at >>>>>> https://www.lerctr.org/~ler/14-zfs-crash.png. >>>>>> Booting from a 13-REL USB installer it imports and scrubs. >>>>>> >>>>>> Ideas? >>>>>> >>>>>> I can either video conference with shared screen or give access to >>>>>> the console via my Dominion KVM. >>>>>> >>>>>> Any help/ideas/etc welcome I really need to get this box back. >>>>>> >>>>>> >>>> How can I import the pool withOUT it mounting the FileSystems so I >>>> can export it cleanly on the 13 system? >>> >>> Why do you need to import without mounting file systems?  I think you >>> may actually wish them to be mounted to replay their ZILs.  Just use >>> -R option to mount file systems in some different place. >> >> I get the errors shown at: >> https://www.lerctr.org/~ler/14-mount-R-output.png >> >> Should I worry?  Or do something(tm) here? > > This looks weird, but may possibly depend on mount points topology, > whether /mnt is writable, etc. What happen if you export it now and > try to import it in normal way on 14 without -F? It crashes just after root mount (this is the boot pool and only pool on the system), seeL https://www.lerctr.org/~ler/14-BOOT-Crash.png -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106