From nobody Wed Mar 19 00:04:12 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZHTTj6D5kz5qBSJ for ; Wed, 19 Mar 2025 00:04:25 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZHTTj1RyTz3y82; Wed, 19 Mar 2025 00:04:25 +0000 (UTC) (envelope-from delphij@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2ff80290debso5901798a91.3; Tue, 18 Mar 2025 17:04:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1742342664; x=1742947464; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=FeOhrqsHbnUGKOxlFlIfp3Dl7BveuS9oTnalJKbRZPw=; b=QcBtFNqVE9pTDl5SBWAu9f2oztJD10DhfBK/+/MsHeMzvUTD8tPwaR5Ds9Y9APSbXu ZS6jl0mxNOFx+dCM1j4BS2rd7NHItv1IjMJiKWChax4A73dnY+DNTjFOeWggT4FgTR2Y q8if37KVr4DO5T2jbYlIAkhcqOb8TM7IwQhedcPrkrpcy9wFxBLyJ952w0sl8qT2yzh6 sV3Z5CNihTMEfw0mZOx61u+Q8ISI0vlB0Fyj3Umupw+QhJmWjj9NfOuXSBgFXmcFt2Vj Ds5JCdEzxL0Axp4z7z0tQCm2ZegZ4ntz2cCik0iO6vF4lCLZK6HfwxldJFjsw3hMdWeU 7Z0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742342664; x=1742947464; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FeOhrqsHbnUGKOxlFlIfp3Dl7BveuS9oTnalJKbRZPw=; b=K3Nxst/loVojkTKd+a4xi3FyJmU453bpBqOqjifmzFaMyaVlhyOoWMD/fORO0GhOzE O8N3cDrh1hrWhWfFOClonTLxFhQILSJoWQvqxlyuP5ybyjjh4Wo45lmM1gc69xNNTPVU /cnptfSdvmnGbjJ7GT/xd2m0k3z1l1ZQj1hx5z383CrWgHVpxckkrMAu+r2sudU5D6WO OSznp7IkU/NH6aXWcsbUA8D5cdvck/haxjU8Wd2olJQKLADBiL9pAowmykqbCBCBREOv gfp5B6IDqYPrVB6cc5AgBt83ZjF8/bTNqSGV3XJ/HpzowSmb1UAZXQkw3CChn2JWQhWh iLJA== X-Forwarded-Encrypted: i=1; AJvYcCVZ3KOHwYLG+5MYQLiEQ76ncFmpmXG7vdNmpRSEU4CLaWr/G2AjHxM0V0lOXCMzJbqhRfXSvo2wxs8wjvBfqic=@freebsd.org, AJvYcCVwbd6LqlKgvDo5bNzuDJP61YuigMM+eLYZPIBOJgUSBo0ShVwB0pECSZfQmp2nti91fYo1@freebsd.org X-Gm-Message-State: AOJu0YwW8AuJSgLOEX+0+E7SBELoyi5Z+0FqhpBp8hRRxKCvxjrINzw9 TiSBUZ354ZPPvEf6RdUe8fImRz/hpNf6fs+iU97c5D16o4mjFC8bnK8HZKj/TWftlx7TKN168uH 6SIPk5NDSjdyx5xPsRpG+Gof4fuum3bSw X-Gm-Gg: ASbGncv7FGrH5uoct0Ojm9av19/G7akeQ0X6O9iPvLlkDvSqMdrizyM45cAMBJEgsLW QccrQAhXSurW7NXXr6MSGx6V5U/X/ult0mvwE/TovZnPkc18m0IluiWI5Hv1JxkpZdshGs5MGK4 /1ccYaDyLAp+p2K0PTd1zbp367yYcjLGN7XAQMRmGkhhBfIk5e0HiCsA2X X-Google-Smtp-Source: AGHT+IEIfE1Gb3QhpzfiQb21CLZhFVdGIXJTjI2eu7CewgLu+cMkJ38hyXG/OPXTEx/MJHZ7ZGXGXQQ4p7GJ09wj+jY= X-Received: by 2002:a17:90b:1b0e:b0:2ff:53a4:74f0 with SMTP id 98e67ed59e1d1-301be2163d6mr825297a91.29.1742342663392; Tue, 18 Mar 2025 17:04:23 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <887C8BE0-9670-4892-9B60-9CA09675D40C@spmzt.net> <0f55b357-2a9f-4b04-bd12-a0afe92da5d7@spmzt.net> In-Reply-To: <0f55b357-2a9f-4b04-bd12-a0afe92da5d7@spmzt.net> From: Xin LI Date: Tue, 18 Mar 2025 17:04:12 -0700 X-Gm-Features: AQ5f1Jq8bnQ4Tr_cUimoyzdn9zzaiN22y9GLcnUcfL9zz-34Lms3pleFwbYCXvI Message-ID: Subject: Re: zap_increment panic on specific zpool To: Seyed Pouria Mousavizadeh Tehrani Cc: dch@skunkwerks.at, freebsd-current@freebsd.org, mav@freebsd.org Content-Type: multipart/alternative; boundary="0000000000008756660630a6c3db" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4ZHTTj1RyTz3y82 X-Spamd-Bar: ---- --0000000000008756660630a6c3db Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Mar 18, 2025 at 2:40=E2=80=AFPM Seyed Pouria Mousavizadeh Tehrani < info@spmzt.net> wrote: > I created another pool and backed up all my data using rsync, which avoid= s > block level cloning. It's working! > This probably has nothing to do with block cloning. The assertion was that zap_increment() succeeded (0), while EIO (5) was returned instead. This is likely the result of data corruption with correct checksums (in that case it's likely that you would see ECKSUM=3D122). When this happens, it's usually a permanent damage, the poo= l should still be read-only importable but you can't revive it to read-write state. Common causes of this would be a permanent damage which could be the result of a memory corruption (could be a bug in the kernel, or more usually, faulty memory that was not detected or panicked your system earlier). > Should I create a PR for this issue on FreeBSD or OpenZFS github > repository? > If you have a way to reliably trigger the issue please do file a FreeBSD PR or open an issue at OpenZFS as it would be very useful. Otherwise, it's likely to be faulty memory and there is not a lot that can be done from software. > > On 3/18/25 16:57, Dave Cottlehuber wrote: > > On Tue, 18 Mar 2025, at 12:35, Seyed Pouria Mousavizadeh Tehrani wrote: > > Hi, > > I was working on my desktop when it suddenly panicked, and Now, I'm > unable to boot into multi-user mode anymore. Additionally, in single > user-mode with my bpool, whenever I execute the "zpool import it > rpool", the panic occurs shortly after. > > My kdb backtrace log is attached to this url. > > I'm using FreeBSD 15.0-CURRENT main-n275975-5963423232e8. > > This looks like an assert; you may be able to boot again using > either a RELEASE if your zpool hasn't been upgraded past that > point, or a GENERIC-NODEBUG from CURRENT. This would at least get > you working, but it won't fix the underlying cause. > > hope it helps, > A+ > Dave > > > -- > Seyed Pouria Mousavizadeh Tehrani > Hoopad Cloud (AS214145) > > --0000000000008756660630a6c3db Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, Mar 1= 8, 2025 at 2:40=E2=80=AFPM Seyed Pouria Mousavizadeh Tehrani <info@spmzt.net> wrote:
=20 =20 =20 =20

I created another pool and backed up all my data using rsync, which avoids block level cloning. It's working!


This probably has nothing to do with block = cloning.

The assertion was that=C2=A0zap_i= ncrement() succeeded (0), while EIO (5) was returned instead.=C2=A0 This is= likely the result of data corruption with correct checksums (in that case = it's likely=C2=A0that you would see ECKSUM=3D122).=C2=A0 When this happ= ens, it's=C2=A0usually=C2=A0a permanent=C2=A0damage, the pool should st= ill be read-only importable but you can't revive it to read-write state= .=C2=A0 Common causes of this would be a permanent damage which could be th= e result of a memory corruption (could be a bug in the kernel, or more usua= lly, faulty memory that was not detected or panicked your system earlier).<= /span>=C2=A0

Should I create a PR for this issue = on FreeBSD or OpenZFS github repository?

If you have a way to reliab= ly trigger the issue please do file a FreeBSD PR or open an issue at OpenZF= S as it would be very useful.=C2=A0 Otherwise, it's likely to be faulty= memory and there is not a lot that can be done from software.

=C2=A0


On 3/18/25 16:57, Dave Cottlehuber wrote:
On Tue, 18 Mar 2025, at 12:35, Seyed Pouria Mousavizadeh Tehrani=
 wrote:

Hi,=20

I was working on my desktop when it suddenly panicked, and Now, I'm=20
unable to boot into multi-user mode anymore. Additionally, in single=20
user-mode with my bpool, whenever I execute the "zpool import it=20
rpool", the panic occurs shortly after.

My kdb backtrace log is attached to this url.
<https://ibb.co/v6=
d4Kr7K>

I'm using FreeBSD 15.0-CURRENT main-n275975-5963423232e8.

This looks like an assert; you may be able to boot again using
either a RELEASE if your zpool hasn't been upgraded past that
point, or a GENERIC-NODEBUG from CURRENT. This would at least get
you working, but it won't fix the underlying cause.

hope it helps,
A+
Dave


--=20
Seyed Pouria Mousavizadeh Tehrani
Hoopad Cloud (AS214145)
--0000000000008756660630a6c3db--