From nobody Mon Aug 19 21:19:50 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WnlqQ5VL6z5TXHs for ; Mon, 19 Aug 2024 21:20:02 +0000 (UTC) (envelope-from boyvalue@gmail.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WnlqQ1D6Vz4rMV for ; Mon, 19 Aug 2024 21:20:02 +0000 (UTC) (envelope-from boyvalue@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=d+sttAKN; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of boyvalue@gmail.com designates 2607:f8b0:4864:20::836 as permitted sender) smtp.mailfrom=boyvalue@gmail.com Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-44fe9cf83c7so26817891cf.0 for ; Mon, 19 Aug 2024 14:20:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724102401; x=1724707201; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=ImDhKFKhcqOEtqtubBHyoB8fhmSVWUzooSBBqqGhG5o=; b=d+sttAKNQqyfMthtQLRSGkpH+KjgTjLT3Z5pXA1Isg3FzWsu5vRmLTy0qkucAN8Lk0 YpBJBW9c52dbdp9r9A+e5Qm5gl1TjFa7BG34wSaOgqGPyJ6k7j6JxlgeflbWR7W4GyeY l4GvCvQW9/z6C2cAdtGFK3hSbzTLO+HiTAyPcLW0tEJAPLCvpdkVS1rZOZHk7WO/44St Uhj30DCd+6QIxW6S76juZ6e9asxi6MW9Gr0i5FMfXPRN4KhQLMzaovgvDJN6PcUlla1V YlJk2SAwNYXKkEn3tk5MpjA34wxYosnDR5tMdH2ZOiL2mEMolKILc4o5L7AQMTopt6Cu wvlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724102401; x=1724707201; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ImDhKFKhcqOEtqtubBHyoB8fhmSVWUzooSBBqqGhG5o=; b=nUYoMHW727JVheLhR2Et/tgN6njiKfPODFFpMpnPIkX/V9E3MxwChVk28fGT+2YI4/ qWVhIjdUYInCUnHU4SVcwJRRSiNSZEmpA6uyAVJxUhtjOZkgLhC8A2iJNKOdWpI6Amzf cPgccLGdb5SOgxgWo+65PGYFqwu2Bc9xaPlg2jpIB2211S7sima9TnlKPoLCRxKGysza sC5YMg85lOLWZWNbNQRDc9jKqf+GFq9fs4uDjNgNzTNdcmOKLHrA0lYUNIbHffFGQR13 /oOSrrS+X3L6fG5gBHYY/D8Tgk5l5abkwjmN7eFuH3eYN3YvajJVDDG7PUqe8+uB8rLA i1sg== X-Gm-Message-State: AOJu0YxuemnbMOI5MrDAApu0zsBeWV3CWDx9sBopWUESZvJBmjNGCzu5 dglrYhnD9O3swYTpkG+jY0+WFO1Og4k5oyGV/ha8GcJJLu4CYFW+4I90X/fq6NHrAAuenCdvjgA e3UIV0car2PWQ1/XXNWVmewI36qw9n1w= X-Google-Smtp-Source: AGHT+IHQH9mZKeXOXHrOScPcf63k1ed+O5mLYZdi8TmrlCVGF/IdgZPCq7ra0pN9D819Xif63y+y6bis7x042W/1wKE= X-Received: by 2002:a05:622a:4819:b0:44f:e2ba:2d66 with SMTP id d75a77b69052e-453742189admr204472501cf.18.1724102400899; Mon, 19 Aug 2024 14:20:00 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 From: Pamela Ballantyne Date: Mon, 19 Aug 2024 15:19:50 -0600 Message-ID: Subject: ZFS: Suspended Pool due to allegedly uncorrectable I/O error To: freebsd-fs@freebsd.org Content-Type: multipart/alternative; boundary="00000000000029846c06200fdf50" X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::836:from] X-Rspamd-Queue-Id: 4WnlqQ1D6Vz4rMV --00000000000029846c06200fdf50 Content-Type: text/plain; charset="UTF-8" Hi, So, this is long, so here's TL;DR: ZFS suspended a pool for presumably good reasons, but on reboot, there didn't seem to be any good reason for it. As a background, I'm an early ZFS adopter of ZFS. I have a remote server running ZFS continuously since late 2010, 24x7. I also use ZFS on my home machines. While I do not claim to be a ZFS expert, I've managed to handle the various issues that have come up over the years and haven't had to ask for help from the experts. But now I am completely baffled and would appreciate any help, advice, pointers, links, whatever. On Sunday Morning, 08/11, I upgraded the server from 12.4-RELEASE-p9 to 13.3-RELEASE-p5. The upgrade went smoothly; there was no problem, and the server worked flawlessly post-upgrade. On Thursday evening, 8/15, the server became unreachable. It would still respond to pings via the IP address, but that was it. I used to be able to access the server via IPMI, but that ability disappeared several company mergers ago. The current NOC staff sent me a screenshot of the server output, which showed repeated messages saying: "Solaris: WARNING: Pool 'zroot' has encountered an uncorrectable I/O failure and has been suspended." There had been no warnings in the log files, nothing. There was no sign from the S.M.A.R.T. monitoring system, nothing. It's a simple mirrored setup with just two drives. So I expected a catastrophic hardware failure. Maybe the HBA had failed (this is on a SuperMicro Blade server), or both drives had manage to die at the same time. Without any way to log in remotely, I requested a reboot. The server rebooted without errors. I could ssh into my account and poke around. Everything was normal. There were no log entries related to the crash. I realize post-crash there would be no filesystem to write to, but there was still nothing leading up to it - no hardware or disk-related messages of any kind. The only sign of any problem I could find was 2 checksum errors listed on only one of the drives in the mirror when I did zpool status. I ran a scrub, which completed without any problem or error. About 30 minutes after the scrub, the two checksum errors disappeared without manually clearing them. I've run some drive tests and they both pass with flying colors. And it's now been a few days and the system has been performing flawlessly. So, I am completely flummoxed. I am trying to understand why the pool was suspended when it looks like something ZFS should have easily handled. I've had complete drive failures, and ZFS just kept on going. Is there any bug or incompatibility in 13.3-p5? Is this something that will recur on each full moon? So thanks in advance for any advice, shared experiences, or whatever you can offer. Best, Pammy --00000000000029846c06200fdf50 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

So, this is long, so here's TL;= DR:=C2=A0 ZFS suspended a pool for presumably
good reasons, but o= n reboot, there didn't seem to be any good reason for it.
As a background, I'm an early ZFS adopter of ZFS. I have a = remote server running ZFS
continuously since late 2010, 24x7.= I also use ZFS on my home machines. While I do not
claim to be a= ZFS expert, I've managed to handle the various issues that have come u= p over=C2=A0
the years and haven't had to ask for help from t= he experts.

But now I am completely baffled and wo= uld appreciate any help, advice, pointers, links, whatever.

<= /div>
On Sunday Morning, 08/11, I upgraded the server from 12.4-RELEASE= -p9 to 13.3-RELEASE-p5.
The upgrade went smoothly; there was no p= roblem, and the server worked flawlessly post-upgrade.

=
On Thursday evening, 8/15, the server became unreachable. It would sti= ll respond to pings via=C2=A0
the IP address, but that was it.=C2= =A0 I used to be able to access the server via IPMI, but that ability disap= peared
several company mergers ago. The current NOC staff sent me= a screenshot of the server output,
which showed repeated message= s saying:

"Solaris: WARNING: Pool 'zroot&= #39; has encountered an uncorrectable I/O failure and has been suspended.&q= uot;

There had been no warnings in the log fil= es, nothing. There was no sign from the S.M.A.R.T. monitoring system, nothi= ng.

It's a simple mirrored setup with just two= drives. So I expected a catastrophic hardware failure. Maybe the HBA had= =C2=A0
failed (this is on a SuperMicro Blade server), or both dri= ves had manage to die at the same time.=C2=A0

With= out any way to log in remotely, I requested a reboot.=C2=A0 The server rebo= oted without errors. I could
ssh into my account and poke around.= =C2=A0 Everything was normal. There were no log entries related to the cras= h. I realize post-crash
there would be no filesystem to write to,= but there was still nothing leading up to it - no hardware or disk-related=
messages of any kind.=C2=A0 The only sign of any problem I could= find was 2 checksum errors listed on only one of the
drives in t= he mirror when I did zpool status.

I ran a scrub, = which completed without any problem or error. About 30 minutes after the sc= rub, the=C2=A0
two checksum errors disappeared without manually c= learing them. I've run some drive tests and
they both pass wi= th flying colors. And it's now been a few days and=C2=A0the system has = been performing flawlessly.

So, I am completely fl= ummoxed. I am trying to understand why the pool was suspended when it looks= like
something ZFS should have easily handled. I've had comp= lete drive failures, and ZFS just kept on going.
Is there any bug= or incompatibility in 13.3-p5?=C2=A0 Is this something that will recur on = each full moon?

So thanks in advance for any advic= e, shared experiences, or whatever you can offer.

= Best,
Pammy



--00000000000029846c06200fdf50-- From nobody Tue Aug 20 03:41:13 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WnwHb28zmz5V7y1 for ; Tue, 20 Aug 2024 03:41:31 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [IPv6:2001:470:0:19b::b869:801b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "holgerdanske.com", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WnwHZ06Rtz4G2q for ; Tue, 20 Aug 2024 03:41:29 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=holgerdanske.com header.s=nov-20210719-112354 header.b=R6omItU4; dmarc=pass (policy=none) header.from=holgerdanske.com; spf=pass (mx1.freebsd.org: domain of dpchrist@holgerdanske.com designates 2001:470:0:19b::b869:801b as permitted sender) smtp.mailfrom=dpchrist@holgerdanske.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=holgerdanske.com; s=nov-20210719-112354; t=1724125278; bh=c7b1wwpeeB5T37ybRs7FwR35b03B7YLq/J9ix7khw/c=; h=Received:Message-ID:Date:MIME-Version:User-Agent:Subject:To: References:Content-Language:From:In-Reply-To:Content-Type: Content-Transfer-Encoding; b=R6omItU4LIV5NJS7kLdRsC5u9YR18K42mc9GPluStoM35iq0cothHKpDTAyjAYJ25 4ZkyJ3FBR0qwuy/og6W7Ik16yRYJy0F8+NHZvB2js09GE3PNB6bZK4BD+2vKmnfUVi YsBAg3RVtMfyblf2vOM37P+V/0fWO4UaqKIyILSTMpoQ0HX9kX1B9sBYGlUlQh5qDD YiMhOst8AqmooQ2Pzf0W6evEC7SsCp5zZ4DReRfanr9EY7JAHl1YXyhTxIbTi7wKJn Rti9jkl1VPYSv34c0zUfHeEDidUNfEKpemMLgPTopHCcyvIrznmiAnR7QbbNOPYFH7 hw7W6DiONdMPMkQoS06ec95jFGJWKEWzNnP/LKV5byDpwkWS0599V3v2Dfi0cbossD fEpu/RzM2A4UdUIMwzP+INpvxHy4lz5bUk2WFaoaj5OwpjIgcVIgoV/Okt5Ddm0GxY s3Naa/daOAYgm7AYxq1QcgMNTQS2K1cQlcMM3Im3qoOdkIPGbnN9niFQhks4Yb/9tl leP8z6qd+Z2Chv3ltYy4II61+49v/iLemb/qqK63yLtb8vaCvSoTryW0D8nZIBwCdf oyfe78+7P6Qrsg+JaAhMWheJFNpc0Bu+uCT4Kt9oCxFpSWnZVEHltA9+bqkENqMTQz ULaM+axvnf3CTgRo6e3yVXoA= Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Mon, 19 Aug 2024 20:41:18 -0700 Message-ID: <86142b7d-0f19-41c8-95ed-19a5d589ac86@holgerdanske.com> Date: Mon, 19 Aug 2024 20:41:13 -0700 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: ZFS: Suspended Pool due to allegedly uncorrectable I/O error To: freebsd-fs@freebsd.org References: Content-Language: en-US From: David Christensen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.87 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.976]; DMARC_POLICY_ALLOW(-0.50)[holgerdanske.com,none]; R_SPF_ALLOW(-0.20)[+a]; R_DKIM_ALLOW(-0.20)[holgerdanske.com:s=nov-20210719-112354]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[holgerdanske.com:+] X-Rspamd-Queue-Id: 4WnwHZ06Rtz4G2q On 8/19/24 14:19, Pamela Ballantyne wrote: > > On Sunday Morning, 08/11, I upgraded the server from 12.4-RELEASE-p9 to > 13.3-RELEASE-p5. > The upgrade went smoothly; there was no problem, and the server worked > flawlessly post-upgrade. > > On Thursday evening, 8/15, the server became unreachable. It would still > respond to pings via > the IP address, but that was it. I used to be able to access the server > via IPMI, but that ability disappeared > several company mergers ago. The current NOC staff sent me a screenshot of > the server output, > which showed repeated messages saying: > > "Solaris: WARNING: Pool 'zroot' has encountered an uncorrectable I/O > failure and has been suspended." > I have a SOHO network and have been running a FreeBSD/ ZFS on a few machines for a few years, including a 24x7 file server. AIUI FreeBSD 12-R and prior used IllumOS ZFS code and FreeBSD 13-R and newer use OpenZFS code. AIUI, to do an in-place upgrade of FreeBSD 12-R with Illumos ZFS pools to FreeBSD 13-R with OpenZFS code, you needed to follow a specific procedure to pre-upgrade (?) the Illumos pools before upgrading FreeBSD. (Especially for root-on-ZFS.) OpenZFS had a data destruction bug last year (November 2023?), which resurfaced this year (February 2024?). Those events caused me to delay upgrading FreeBSD/ ZFS. A few weeks ago, I did a fresh install of 13-R with UFS onto a repurposed machine, added HDD's/ SSD's, built a fresh OpenZFS pool, and replicated the data from the old file server to the new file server. The new 13-R/OpenZFS file server has been up 24x7 since then. I have since repurposed/ rebuilt a backup server, and then the removable single-drive backup disks/ pools. Everything is now FreeBSD 13-R and OpenZFS. I have noted some differences in how OpenZFS does incremental replication versus how IllumOS ZFS did, but am still learning. I expect I will be reworking my ZFS-related scripts as I figure things out. I understand that in-place upgrading a FOSS computer over many years is a source of pride for many people. I tried that, and it did not work out for me. Since then, I have invested myself in fresh installs, minimal sysadmin changes, thorough documentation, scripting, version control, backup, restore, and multiple layers of redundancy. The efforts are far more predictable and the results are far more reliable. David From nobody Tue Aug 20 08:10:13 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wp2Fn4y5bz5SbX2 for ; Tue, 20 Aug 2024 08:10:21 +0000 (UTC) (envelope-from SRS0=aXLp=PT=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wp2Fn0GV3z4m8K for ; Tue, 20 Aug 2024 08:10:20 +0000 (UTC) (envelope-from SRS0=aXLp=PT=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Tue, 20 Aug 2024 10:10:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1724141413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=WQ4dtxkGeSAOGGwq1TD59+Zxr6FB6ll3n3V9skRWayY=; b=ASEJjdMGkf5W+oOa8FdBp6in2j1ypKWh0fp6y6AzezuI3UzCxACfykXM5ypgChux2AhyRJ Duq+fedxCLaz+Q3Fh3j9VLp5FEl4gC4oykZLgOEl2U7F7IjrBatkQuIzA4zJyxm2zZ5Bgj 0bd+3apl3lhbk9ePCGhA3IzCWZ8Z8vDBuN0v/jQ4RlFmzyT17nHgaHax31EiFrTK44TJxY d+AT3zip96aREYzX8We2v9qWk6D7zuoRuAn3lePZephW/torOD5RftRrXapPOB9IlBMJVe 7boxRs48g+Ga1M0QX+dHgd+NB4mvQIi3VYezuPded+VN1MrpX/SZe5kOs0MnyQ== From: Ronald Klop To: Pamela Ballantyne Cc: freebsd-fs@freebsd.org Message-ID: <146942683.2571.1724141413478@localhost> In-Reply-To: Subject: Re: ZFS: Suspended Pool due to allegedly uncorrectable I/O error List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_2570_13676940.1724141413473" X-Mailer: Realworks (716.31) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4Wp2Fn0GV3z4m8K ------=_Part_2570_13676940.1724141413473 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hi, This happens on my Raspberry pi when usb senses a disconnect of the disk.It= does have a message about this usb event on the serial console but because= of the repetition of other messages it goes off screen quickly. And as the= disk is unavailable it didn=E2=80=99t write anything locally. Remote loggi= ng could help.=20 I just did =E2=80=98zpool set failmode=3Dpanic zroot=E2=80=99 to at least h= ave a workable system if it happens.login via serial/ipmi doesn=E2=80=99t w= ork either because you don=E2=80=99t have any executable available to run Regards,Ronald. Van: Pamela Ballantyne Datum: 19 augustus 2024 23:20 Aan: freebsd-fs@freebsd.org Onderwerp: ZFS: Suspended Pool due to allegedly uncorrectable I/O error >=20 >=20 >=20 >=20 > Hi, >=20 > So, this is long, so here's TL;DR: ZFS suspended a pool for presumably > good reasons, but on reboot, there didn't seem to be any good reason for = it. >=20 > As a background, I'm an early ZFS adopter of ZFS. I have a remote server = running ZFS >=20 > continuously since late 2010, 24x7. I also use ZFS on my home machines. W= hile I do not > claim to be a ZFS expert, I've managed to handle the various issues that = have come up over=20 > the years and haven't had to ask for help from the experts. >=20 > But now I am completely baffled and would appreciate any help, advice, po= inters, links, whatever. >=20 > On Sunday Morning, 08/11, I upgraded the server from 12.4-RELEASE-p9 to 1= 3.3-RELEASE-p5. > The upgrade went smoothly; there was no problem, and the server worked fl= awlessly post-upgrade. >=20 > On Thursday evening, 8/15, the server became unreachable. It would still = respond to pings via=20 > the IP address, but that was it. I used to be able to access the server = via IPMI, but that ability disappeared > several company mergers ago. The current NOC staff sent me a screenshot o= f the server output, > which showed repeated messages saying: >=20 > "Solaris: WARNING: Pool 'zroot' has encountered an uncorrectable I/O fail= ure and has been suspended." >=20 >=20 > There had been no warnings in the log files, nothing. There was no sign f= rom the S.M.A.R.T. monitoring system, nothing. >=20 > It's a simple mirrored setup with just two drives. So I expected a catast= rophic hardware failure. Maybe the HBA had=20 > failed (this is on a SuperMicro Blade server), or both drives had manage = to die at the same time.=20 >=20 > Without any way to log in remotely, I requested a reboot. The server reb= ooted without errors. I could > ssh into my account and poke around. Everything was normal. There were n= o log entries related to the crash. I realize post-crash > there would be no filesystem to write to, but there was still nothing lea= ding up to it - no hardware or disk-related > messages of any kind. The only sign of any problem I could find was 2 ch= ecksum errors listed on only one of the > drives in the mirror when I did zpool status. >=20 > I ran a scrub, which completed without any problem or error. About 30 min= utes after the scrub, the=20 > two checksum errors disappeared without manually clearing them. I've run = some drive tests and > they both pass with flying colors. And it's now been a few days and the s= ystem has been performing flawlessly. >=20 > So, I am completely flummoxed. I am trying to understand why the pool was= suspended when it looks like > something ZFS should have easily handled. I've had complete drive failure= s, and ZFS just kept on going. > Is there any bug or incompatibility in 13.3-p5? Is this something that w= ill recur on each full moon? >=20 > So thanks in advance for any advice, shared experiences, or whatever you = can offer. >=20 > Best, > Pammy >=20 >=20 >=20 >=20 >=20 ------=_Part_2570_13676940.1724141413473 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi,

This happens on my R= aspberry pi when usb senses a disconnect of the disk.
It does have a message about this usb event on the serial console but beca= use of the repetition of other messages it goes off screen quickly. And as the disk is unavailable it didn=E2=80=99t write= anything locally. Remote logging could help. 
<= br>I just did =E2=80=98zpool set failmode=3Dpanic zroot=E2=80=99 to at leas= t have a workable system if it happens.
login via ser= ial/ipmi doesn=E2=80=99t work either because you don=E2=80=99t have any exe= cutable available to run

Regards,
Ronald.

Van: Pa= mela Ballantyne <boyvalue@gmail.com>
Datum: 19 au= gustus 2024 23:20
Aan: freebsd-fs@freebsd.org
Onderwerp: ZFS: Suspended Pool due to allegedly uncorrectable I/= O error

Hi,

So, this is lo= ng, so here's TL;DR:  ZFS suspended a pool for presumably
good reasons, but on reboot, there didn't seem to be any good reas= on for it.

As a background, = I'm an early ZFS adopter of ZFS. I have a remote server running ZFS
continuously since late 2010, 24x7. I also use ZFS on my = home machines. While I do not
claim to be a ZFS expert= , I've managed to handle the various issues that have come up over 
the years and haven't had to ask for help from the exper= ts.

But now I am completely = baffled and would appreciate any help, advice, pointers, links, whatever.

On Sunday Morning, 08/11, I u= pgraded the server from 12.4-RELEASE-p9 to 13.3-RELEASE-p5.
The upgrade went smoothly; there was no problem, and the server worke= d flawlessly post-upgrade.

O= n Thursday evening, 8/15, the server became unreachable. It would still res= pond to pings via 
the IP address, but that was i= t.  I used to be able to access the server via IPMI, but that ability = disappeared
several company mergers ago. The current N= OC staff sent me a screenshot of the server output,
wh= ich showed repeated messages saying:

"Solaris: WARNING: Pool 'zroot' has encountered an uncorrectable I= /O failure and has been suspended."

There had been no warnings in the log files, nothing. There was= no sign from the S.M.A.R.T. monitoring system, nothing.

It's a simple mirrored setup with just two dri= ves. So I expected a catastrophic hardware failure. Maybe the HBA had =
failed (this is on a SuperMicro Blade server), or bot= h drives had manage to die at the same time. 
Without any way to log in remotely, I requested a re= boot.  The server rebooted without errors. I could
ssh into my account and poke around.  Everything was normal. There w= ere no log entries related to the crash. I realize post-crash
there would be no filesystem to write to, but there was still nothi= ng leading up to it - no hardware or disk-related
mess= ages of any kind.  The only sign of any problem I could find was 2 che= cksum errors listed on only one of the
drives in the m= irror when I did zpool status.

I ran a scrub, which completed without any problem or error. About 30 mi= nutes after the scrub, the 
two checksum errors d= isappeared without manually clearing them. I've run some drive tests and
they both pass with flying colors. And it's now been a f= ew days and the system has been performing flawlessly.

So, I am completely flummoxed. I am trying = to understand why the pool was suspended when it looks like
something ZFS should have easily handled. I've had complete drive fai= lures, and ZFS just kept on going.
Is there any bug or= incompatibility in 13.3-p5?  Is this something that will recur on eac= h full moon?

So thanks in ad= vance for any advice, shared experiences, or whatever you can offer.
<= div class=3D"">
Best,
Pammy





------=_Part_2570_13676940.1724141413473-- From nobody Tue Aug 20 15:28:45 2024 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WpCzx753zz5TKRy for ; Tue, 20 Aug 2024 15:29:01 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WpCzx0KzSz42fr for ; Tue, 20 Aug 2024 15:29:01 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=ng2ewLDg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jdavidlists@gmail.com designates 2607:f8b0:4864:20::42e as permitted sender) smtp.mailfrom=jdavidlists@gmail.com Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-714186ce2f2so401850b3a.0 for ; Tue, 20 Aug 2024 08:29:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724167739; x=1724772539; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ab2Kxl3mjh2lnSs/DSQoPAjCBl7Mz7JmDUBP/fPHob8=; b=ng2ewLDgguHwE9cuXsmwNUHS3afG9g4x7uOX/HUbKN4bkXOoDiucAsktATmMvnK60X TwX+GVlYknLYcFhkm6fOmv0cETrS0HrcYeZn+Al53WeUTyIyF5Nmmn0kSIUQypvAhqP0 jA7TxalGNJI02Z8foO4y4Q2RJNMxQ2wb6TOCHE21ZiupCfbxs5qhBcUiempjo1YQlbI8 B7a0P6zJX5ypQuNXKu/V95mxDYfXUWzEkp2War+YkVcZEcWapwQvdAJT8kimbziPU42L fjgYHzdFbtFqgUlW/LmlUL8nb0UYo9uw7BBZzNwncZPdxBgTnfcz7HOaGPXlX4amWWw7 KDgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724167739; x=1724772539; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ab2Kxl3mjh2lnSs/DSQoPAjCBl7Mz7JmDUBP/fPHob8=; b=lm69xz+9NkxzQX/uqlhb4XgcZsk07gdxtAEuEv/Fp536orqvIelOQnDhtGFlcsipZx Mth6jKGoqSz0Wwzl5OlhZqmQuBIIGSGxSwb122ui5b+FkirEigJXxiDeRDy5w87wSb87 tLsaUfsSjJ22KeEgcx/9YWBUfP7dtpPXmu9ipksuo6Hr8dDmKCNK9nb0AWM6naAEsHB/ 7IwtUdshV+4R4qOWRBxpXlx8TiS/QKdivLWyi63KN4Tqt4jBTfUd+VoC/X0dSbjCCtI5 iO/KlsqC1ONcYAtlVYhWBKA+mLfUXEE+EKej2AnVirFalvQwBsEOYOQD1FC2ftor7KXE plxQ== X-Gm-Message-State: AOJu0YyG9VPxJ4AaB2d4L3gWtefW9UsbisTpYIFZSZdEkOcJwHvZSIpm HREXvwCf5OTHx6CUfeO7rNRv96uWEiuDywluAC2g0JEBt/rpGUsKm3rLf5pUxq746pLUbZOCL9n pmiUJUK2N9SEHanVN9g6FPeObGkI= X-Google-Smtp-Source: AGHT+IG4C6gZ6P50eJue8Oo9X96uzP6v8pz6klFUjGgDvSbttgHj+20PuVh3IA32UfSQL54IJmZ1h/o0YTwYZa2gK/8= X-Received: by 2002:a05:6a00:148f:b0:70d:34aa:6d51 with SMTP id d2e1a72fcca58-713c4df24e9mr14839168b3a.6.1724167739384; Tue, 20 Aug 2024 08:28:59 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: J David Date: Tue, 20 Aug 2024 11:28:45 -0400 Message-ID: Subject: Re: NFS, intermittent 'RPC struct is bad' errors To: Rick Macklem Cc: fs@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TAGGED_FROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[fs@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; TAGGED_RCPT(0.00)[]; MLMMJ_DEST(0.00)[fs@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::42e:from] X-Rspamd-Queue-Id: 4WpCzx0KzSz42fr On Wed, Jun 19, 2024 at 10:05=E2=80=AFAM Rick Macklem wrote: > On Tue, Jun 18, 2024 at 11:32=E2=80=AFPM Lexi Winter wr= ote: > > i have a few systems running NFSv4 on FreeBSD, using Kerberos (MIT > > Kerberos KDC), with the server exporting ZFS filesystems. > > > > recently i've noticed intermittent errors of 'RPC struct is bad' when > > writing to the NFS server, which usually resolves itself after retrying= . > > for example: > [...] > No one else has reported anything like this recently, We are also seeing intermittent "RPC struct is bad" from FreeBSD NFSv4.2 clients accessing ZFS filesystems. There are a few differences between our situation and that reported by Lexi Winter: - NFS servers are Debian 12. - We do not use kerberos, noncontigwr, or delegations. - It does not resolve itself after retrying. In our case, it seems to "infect" directories as viewed from a certain FreeBSD client machine after attempting to modify (usually delete) that directory. once a directory is affected, it can no longer be viewed or removed from that client. Other clients (FreeBSD or otherwise) are not affected. It doesn't seem to be cached or anything. I.e., it doesn't go away if you leave that directory alone for a while (whether a few minutes, few hours, or a full day). Once the directory is removed from another (identical) client, everything is fine. It seems to happen randomly, on the order of once every few billion NFS calls, so whatever it is, it's a very, very edge case. ZFS+NFS seems to have a few of those, and I'm sure cross-platform mounts aren't helping. Thanks! From nobody Tue Aug 20 18:25:11 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WpHxy29rMz5TZwZ for ; Tue, 20 Aug 2024 18:27:34 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WpHxw1FzDz4QYQ for ; Tue, 20 Aug 2024 18:27:31 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of li-fbsd@citylink.dinoex.sub.org designates 2a0b:f840::12 as permitted sender) smtp.mailfrom=li-fbsd@citylink.dinoex.sub.org; arc=pass ("uucp.dinoex.org:s=M20221114:i=1") Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840:0:0:0:0:0:12]) by uucp.dinoex.org (8.18.1/8.18.1) with ESMTPS id 47KIR6mP078426 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Tue, 20 Aug 2024 20:27:07 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) ARC-Seal: i=1; a=rsa-sha256; d=uucp.dinoex.org; s=M20221114; t=1724178429; cv=none; b=CcIdwud1wsklRWXrhiy/fy+zpGx6bO5sPnfdCKUYe0oeMWnNWpo4eLweiNslTc6XZsCB9zu+2P+6HbPfA3bfnP0SPbHNIdQLCWbpPvZ5xc60TONDpRid8OWEAk68bDT6wlmurcQ7Ah2FyiQP8N+SKi/+3CzdOnl5URELamRB7Nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=uucp.dinoex.org; s=M20221114; t=1724178429; c=relaxed/simple; bh=d8/6AMxx9fbyP+syDD69vKbShT1yfCroC8p1DErLPWc=; h=Received:Received:Received:X-Authentication-Warning:From: X-Newsgroups:Subject:Date:Message-ID:References:Injection-Date: Injection-Info:User-Agent:To:X-Milter:X-Greylist; b=Y0eSiFTAG3gy1RvpZEG2jSbdXiCXONc90fGs/xBMRmLfKNeJqVXWWMRA747zLVCcB3BRduZ3uFCp4JRZXceybL1WnK8QuvV68fu51GNWRnOoyONo3n5F/znl4YXRZqAvgG11gIlm0o440qDKvW9bvSnQ6DTSdApfDu4XgicioYg= ARC-Authentication-Results: i=1; uucp.dinoex.org X-MDaemon-Deliver-To: Received: (from uucp@localhost) by uucp.dinoex.org (8.18.1/8.18.1/Submit) with UUCP id 47KIR6a8078422 for freebsd-fs@freebsd.org; Tue, 20 Aug 2024 20:27:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from admn.intra.daemon.contact (localhost [127.0.0.1]) by admn.intra.daemon.contact (8.18.1/8.18.1) with ESMTPS id 47KIPVjj002777 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Tue, 20 Aug 2024 20:25:31 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from intra.daemon.contact (news@localhost) by admn.intra.daemon.contact (8.18.1/8.18.1/Submit) with NNTP id 47KIPBwW000813 for freebsd-fs@freebsd.org; Tue, 20 Aug 2024 20:25:11 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: admn.intra.daemon.contact: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: "Peter 'PMc' Much" X-Newsgroups: m2n.fbsd.fs Subject: Re: ZFS: Suspended Pool due to allegedly uncorrectable I/O error Date: Tue, 20 Aug 2024 18:25:11 -0000 (UTC) Message-ID: References: Injection-Date: Tue, 20 Aug 2024 18:25:11 -0000 (UTC) Injection-Info: admn.intra.daemon.contact; logging-data="807"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: slrn/1.0.3 (FreeBSD) To: freebsd-fs@freebsd.org X-Milter: Spamilter (Reciever: uucp.dinoex.org; Sender-ip: 0:0:2a0b:f840::; Sender-helo: uucp.dinoex.org;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [IPv6:2a0b:f840:0:0:0:0:0:12]); Tue, 20 Aug 2024 20:27:09 +0200 (CEST) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; ARC_ALLOW(-1.00)[uucp.dinoex.org:s=M20221114:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.986]; FORGED_SENDER(0.30)[pmc@citylink.dinoex.sub.org,li-fbsd@citylink.dinoex.sub.org]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; FROM_NEQ_ENVFROM(0.00)[pmc@citylink.dinoex.sub.org,li-fbsd@citylink.dinoex.sub.org]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[sub.org]; RCVD_TLS_LAST(0.00)[]; TO_DN_NONE(0.00)[] X-Rspamd-Queue-Id: 4WpHxw1FzDz4QYQ List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org On 2024-08-19, Pamela Ballantyne wrote: > --00000000000029846c06200fdf50 > Content-Type: text/plain; charset="UTF-8" > > Hi, > > So, this is long, so here's TL;DR: ZFS suspended a pool for presumably > good reasons, but on reboot, there didn't seem to be any good reason for it. > > As a background, I'm an early ZFS adopter of ZFS. I have a remote server > running ZFS > continuously since late 2010, 24x7. I also use ZFS on my home machines. > While I do not > claim to be a ZFS expert, I've managed to handle the various issues that > have come up over > the years and haven't had to ask for help from the experts. > > But now I am completely baffled and would appreciate any help, advice, > pointers, links, whatever. > > On Sunday Morning, 08/11, I upgraded the server from 12.4-RELEASE-p9 to > 13.3-RELEASE-p5. > The upgrade went smoothly; there was no problem, and the server worked > flawlessly post-upgrade. > > On Thursday evening, 8/15, the server became unreachable. It would still > respond to pings via > the IP address, but that was it. I used to be able to access the server > via IPMI, but that ability disappeared > several company mergers ago. The current NOC staff sent me a screenshot of > the server output, > which showed repeated messages saying: > > "Solaris: WARNING: Pool 'zroot' has encountered an uncorrectable I/O > failure and has been suspended." > > There had been no warnings in the log files, nothing. There was no sign > from the S.M.A.R.T. monitoring system, nothing. > > It's a simple mirrored setup with just two drives. So I expected a > catastrophic hardware failure. Maybe the HBA had > failed (this is on a SuperMicro Blade server), or both drives had manage to > die at the same time. > > Without any way to log in remotely, I requested a reboot. The server > rebooted without errors. I could > ssh into my account and poke around. Everything was normal. There were no > log entries related to the crash. I realize post-crash > there would be no filesystem to write to, but there was still nothing > leading up to it - no hardware or disk-related > messages of any kind. The only sign of any problem I could find was 2 > checksum errors listed on only one of the > drives in the mirror when I did zpool status. > > I ran a scrub, which completed without any problem or error. About 30 > minutes after the scrub, the > two checksum errors disappeared without manually clearing them. I've run > some drive tests and > they both pass with flying colors. And it's now been a few days and the > system has been performing flawlessly. > > So, I am completely flummoxed. I am trying to understand why the pool was > suspended when it looks like > something ZFS should have easily handled. I've had complete drive failures, > and ZFS just kept on going. > Is there any bug or incompatibility in 13.3-p5? Is this something that > will recur on each full moon? Well, in fact it is a bit late for Lughnasadh. But yes, these things can happen. Some disk gets a bad mood on the controller, and is detached. On rare occasion another disk gets a bad mood on the controller, and, well... And after reboot everything is fine again. (Because, if the disk doesn't like the environment for some reason, then it goes offline to the controller, and comes back only after a reset or power switch.) It would be interesting to read the kernel messages, as they would tell what the controller did believe. (That is one reason I do not use root-on-zfs. My /var/log, however, is on zfs). In my case, the main reason for these problems was thermal drift (+ageing/oxidation) on the connectors, combined with a PS working at it's limits (and, obviousely, me smoking). But then, mine is not a stock machine: I'm running Xeon-EP put together from scrap parts. Such failures *SHOULD* not happen on a state of the art machine. How old is it, and how much temperature walk does it do? > So thanks in advance for any advice, shared experiences, or whatever you > can offer. cheerio, PMc From nobody Fri Aug 23 13:39:28 2024 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wr1Q84t0Rz5TJ5C for ; Fri, 23 Aug 2024 13:39:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wr1Q83jsKz4s4k for ; Fri, 23 Aug 2024 13:39:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724420368; a=rsa-sha256; cv=none; b=Rew55g5kt6QoGIA6HA7H7WaGhUuiMJSDa3Omj8Mxhn2vAExKKDNyzqJQqxMDkAgZq05u/h XOc56mXi+yvT3gyYjoahh6yMZmvOgqacG9bR4MdT6nfFNTzscB89ht7cb+EXoDSKV1wB9B Jn2kIqoFtT7tYaQB02n9aDI4L+xMk1wiHLnoxc5NZ6FCpXPwrkDGC3B2geDJx8QHsbOiOw yyYFobaxYvhbPeH34B+EEeWJQP4E0gGcdXOG0PKcfS2osNMpoCjQrx2iPKlWetM2Tp891l XHFq0gjS9FRPE/emv8Y5Dx8T/l1ngE9CjQ8K6fN/ds1xoLGZGWMVGzDZUTP5mA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724420368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P5sZRSH6Jcvro1aAdf5sUyph8HmQtrLFlQnwtzLZEbg=; b=N07wzo/uIj7zP1Jui6ss7O+GmXETKEhPAywtF58wWseV9y7J8uLRCODtv0UyLhjkgiLhbi CmNYanOhSlaStPE/6Iwk4hO9Ow53M+OuU0M9NY/K2w2bFk7OKHTWmF8llTZMVWmq0np8L1 9kPHZTNPzj1/ZLHhnc0OeAmjBTGaphwGosh232jkCS08aZnvjd9RIT0m6qefOfKUYLLUj7 UMR+03cafroK5NXclQqdfJrhSyd1pkqNNGGW+rLUyDMU8ICfNpAtHPwdMTDIPv03kTfB/O ranOk8+pBAnhKpznA6DD1QGzShVJOkRoDSzne7okXk7Rsb0pKD4zukvTOoAryg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wr1Q838k9z1C4x for ; Fri, 23 Aug 2024 13:39:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 47NDdSgh050866 for ; Fri, 23 Aug 2024 13:39:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 47NDdSFc050865 for fs@FreeBSD.org; Fri, 23 Aug 2024 13:39:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 281003] NFS client can open the server on ZFS with sharenfs dataset Date: Fri, 23 Aug 2024 13:39:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281003 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|ports-bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Fri Aug 23 15:00:01 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wr39r4ZLSz5TPd2 for ; Fri, 23 Aug 2024 14:58:56 +0000 (UTC) (envelope-from anthony.pankov@yahoo.com) Received: from sonic310-11.consmr.mail.ir2.yahoo.com (sonic310-11.consmr.mail.ir2.yahoo.com [77.238.177.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wr39q2Bwmz413y for ; Fri, 23 Aug 2024 14:58:55 +0000 (UTC) (envelope-from anthony.pankov@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="EFl/MUnA"; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of anthony.pankov@yahoo.com designates 77.238.177.32 as permitted sender) smtp.mailfrom=anthony.pankov@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1724425133; bh=FbSBxpQXmga4hY5/t4xoAJIa+ChdIVRAlw9BANuDpJQ=; h=Date:From:To:Subject:References:From:Subject:Reply-To; b=EFl/MUnArBcLUgis3kKLCqaWjzeft32Pik6SpbFaPnsaoHrfGzHHr2dg2zb0hMuse48f4bCe6mRoZwriZlK2orrto6jH6g0cIU44s7fPdD33MQYhMaZzn4zJWriJPsq1De1r9QxI2WC88tbr9Shjsw+wUKUMEah2GDVToYsgilyev8zSNnNlri/0aJONjWQUtxlOBJhZAzT0QmTLpMO3CbJlMU9jsMYsHMNLbozUpr2sNfIT2TvP2BQ+PZIr2Sbuf/QOWOqS3f7x+aGkmeOFkrZRrXrX5IlEPnFCqcyoN0iaL7jBzXrt3SbTf6ZmnT3J7cAOgIWZ2BGzpVSaoSpsPg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1724425133; bh=8YBCWA6qf3MvotAN4+x9EAWGag4ws3PEHYRzjzB1Hvo=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=B8H/MMn/+BoNn0PmeVu+wGTeZFudZ3qd97T3Hcq+TibGrq38dTeJgTgly1SgvL0smuZinqyz11qR9jEoO/ob/ktLeK3SXoab92jB12Mk9drEk9iEOKzVhaquSfn7+u/Bn/I1GnTad4o3th9sZ1Cz3mXwW8XYvCey3nf6uCyI5E8PHCb3jxbPfyp+sgqHDJDorj93QIRaJt3Mk1btdUeMO12lv1Hxh11aOGFpjzYkyxPl4I3HBCT8ejMa1g9Lvr5G1pndPeCAtA4Rb1GzH/4hTglxQGyj6mAiDCqz85zafGJTZTH15RqL8XW/0lc7ALLht1shtyuBwPqVmsudJZfXqQ== X-YMail-OSG: a1d8eAsVM1k44Vdy8mAVr18jjPixi08AG4dwsA66X5aW6Mrw4hWxuqAOrq2x5Ud Z0ZoIi0LMIVL7PYg0PslUhRHTjzB_qtvDnZPVl1ZVJLWuZyqQyDvubJQiDcU.w778xMk2sHAoz9Z LQs_RrMoH6awL9YfRrGBJptIS_tX5B2jPPJJZ_C30KYXnOSdPSmsC3v6W0f_9j3aijbEZKP8ETsc _vrLkBNSn8vUXC0lkgUP.EwmByFUbj0cZJUxYH0.4mVoIQsaAaJh3GzZScCVWyNJKKPzrrBr8HDp 4iWKwo7fgWVachheJcoWVNhgXxIMMMvFObwkbw7bAN4irfdHFs_Sn1IXzRWhiEr6YzJyCd9xYovh 80rTEikC9WGXbX50s9wez4XyTTWtxXtOQgDtjror.LUj4XjBpnsYyokUZ8nPvHTO0ExFEVBY5yxi JOQ0Y1CZq7sttSld3US81SnHwg2uUBk6BATXf.6z9FyWnQc2s05298u70b6_5XG5w99qywk4kLZq ivepf2R94SO7KMEBArp5LB3GEDJ3XT3wMeE6n1rRWoGuU2NaHaMx9cRi904A3ElHhQITvc.g8tqB pSDvScw8L5hxR.kYJCKhqPALTlEMEmnSaqxYl7w.U8khB9pFIVQiCsMd2RAvcu8EotKrPasNGhkI rZv2iBzVcQ4d99Rrehqk2kljmQRHphkjkf679EeVwg2fnr9jPt5CfNW_AOHTJg_QoBYak6aC_n5p 19QvUHGSYEkEQjORPOh_As9vUZ7rOrxLdxpm5tMD_uS.zgNuXvFdy.BttXf0ozbuIjPyBpv9cCWU F0b4UuzH_vihqeculXip_ALngGAZxrRYwDCrUfw1XI4w9rf2qeG_MmDHWpnc2tLUGYXYMdq8Riwf 4znG07KY6oF.cluByBKLAdxCBeajNfhPVAgI_dFOr4vGKSyiaVpUGCH3q2t_w9.22T55Zjug1rZE 5XZyMfT_ghtTpZZWlHYBpkKEkTegINMWlb5MnWTWJSNUdQ154Nx9cAAukeNFXAeen6yB9FWo61Er YPCe_hi76rOoCYGCPJ1i9kVZwEnE9J9O.bbOdFtxyUOt0SAmmanYFQVzz5_W75g_T8AGF3wulatk E87Hvdt7NOxgT5whUFWiuyTdmYXEnlemRIW3rP7.is600WaeDnmFSFTGO4hQAlfy4F_JrqNIkO6o nQ0R.4W6O3V6LVeSdf31ZT5ca_ClrZwil1o8h63eBntK7VU4oVLi_lBWJAQcuYuaArwaJ3xlndmt qhCig0XhuMNOGTzRdIwp_pmCU8fcjW2V12n1Ta9RuuMOSasdQ4Jm1cLs9slVAJdYw_DAS61K1QDs W.r.dk9o45ioUXHApyvTsis7MvJAhNj2cUFNLIGCZVpmHqvQxt.SSSdf13PuOSBVzbTHYZf62daC txAYwMPurTkRu2c3C.UUNon4ku4IcsZTzvisyxDPp1i9VjKCno9QwnkspnQ1ncIabqciv0wya0aq 5G71Xb.7Qi.JvNItqKqV9nW09d10MDnIwwwXX4QevdoYBHx_9S2eO0lsWc8x54_Tl96yWJw1UWE2 TqGYe96JVSuMOgZn4oLIozZq.VbWbQyraNskNbro1DwpMATvri1glB1c4Rwdzur3SpYEpCG2huhM OeKHN.1LqWcybs96Sj8za0HVeS82wKYyteWMpWTVSJiY8APWSz2ND2TCnt9bCbtpAZry0FF1tE8B .B8DTzmyPHMw40IhaYylcMOQ3uX0xb4q.I9fUliIbeUOEBOiiJtlELJEr6ybtCiPcuXX3g0GPuG0 6RUnnyG.CvNyy1TZ5KyMDvZYZqbxZ.nt_nIqEHQbH.4rO9HjQ7dT7LIkeBPDzxoyREafbe5ozr77 oENV6vhqgP.GIvMmYURUMNdMZESQ_Isjb8emHL_VJyK0xIyz5x5xfiIKGArgQppe2S9w1jBbN3BO tyM9EGX5DcJcwxgGnfMk60ip3U5uRM8EQ2X7JrkAyQkp22w7LCoqBxn77fze1s_eh1lolCL6Kzzi qkBE5cEk9bcyajn4btPi8hAOZsftts60ZJ9.xnH1o_yKlyIApmW6tthG1hpGkMAWgu3mwg5676_q ejE22OkX6XUty6k8NKaUm5emRv7wxUgdywxF7YGoceq12iAkAdXp0CZby.ojfuMw_RWZ4T7azQtp 5S0X3T_GS9Z9eQwgAcJqptng.WcOOPjLktr4- X-Sonic-MF: X-Sonic-ID: cbe407ee-ef86-4bd2-9966-412668677aaa Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.ir2.yahoo.com with HTTP; Fri, 23 Aug 2024 14:58:53 +0000 Received: by hermes--production-ir2-6664f499fc-l95lq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f1e836545a151eaa81e7c0f7aa4977c5; Fri, 23 Aug 2024 14:58:51 +0000 (UTC) Date: Fri, 23 Aug 2024 18:00:01 +0300 From: Anthony Pankov Message-ID: <967684385.20240823180001@yahoo.com> To: freebsd-fs@FreeBSD.org Subject: scheduled run of background fsck List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit References: <967684385.20240823180001.ref@yahoo.com> X-Mailer: WebService/1.1.22544 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:34010, ipnet:77.238.176.0/22, country:GB]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[77.238.177.32:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_DN_NONE(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@FreeBSD.org]; DKIM_TRACE(0.00)[yahoo.com:+] X-Rspamd-Queue-Id: 4Wr39q2Bwmz413y Hello, I use UFS with regular (not journaled) softupdates (SU). Recently I've been told that it is a good idea to schedule a background fsck at least weekly. (I don't now much about UFS, the reason was something about "reclaiming blocks". Is it a right idea or am I missing something?) So I plan to add "fsck -B /data/meshare" to crontab. I have two issue with this: 1. When I manually run "fsck -B" nothing happen: # fsck -B /data/meshare # echo $? 0 and no fsck process in a background. Is it OK? 2. Some years ago I've used classic UFS+SU with background_fsck="YES" in rc.conf. But ended with production server rebooted after every 5-10 minutes. The problem appeared after rebooting after some software crash. The reason I've found in logs. Message was something like: FSCK: CAN'T WRITE PANIC As I understand background fsck was doing it job but when it reach part of filesystem with inconsistency it discover itself in read-only mode and panic to not leave filesystem with inconsistency. It was very frustrating. May I be sure for FreeBSD 12/STABLE+ that this will never happen again? Can I rely that 'fsck -B' will never cause system panic but gracefully exit and leave a message in log in case of a problem? -- Best regards, Anthony Pankov mailto:anthony.pankov@yahoo.com From nobody Fri Aug 23 22:54:57 2024 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WrFl60y6Gz5Tcpn for ; Fri, 23 Aug 2024 22:54:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WrFl5589Mz4hhr for ; Fri, 23 Aug 2024 22:54:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724453697; a=rsa-sha256; cv=none; b=fT7knr5eOuctC7Bg4XJ38o87rpIlOJsgznEJbhccuaiRduiQY4paS529rY3EfAz2b2Beq7 ADCuEWEy4z2B/M9hNZYEeP3BsSTa0MSs5hrmYSXrwdjIkP+6emN1cSIQzCQeTj7CLN3mRX AhcFxi70dOnxFQNXvkZdK2e/SzMdtJUT7W5LatqDi6gPpKJaE8KcYVBxHUE9127xZ+YLER SIY/aEboScmHFRnlVZeC20ggfcrPQh9UGC7YKQB6AKiNh/x5DWjr0mE5iyqtFMimD/e2Lw cW784fy5qPjLuizZAFLYDlmJ6cl/gyZ/aIoRNwl0QW3woVBEtNSs6YzuApjcMg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724453697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZfS2hQhRBZVbY37ghb2Ac01LGQViYCXcaz3eXXP5d/M=; b=rnLe5uCeRzLuO5jwMpkb+udSkOPCrlf8rdst9KcfnTH05o4cYUay1mzh/FK3cH0LyxcOqH cnIRTWpNZ2bfq8gcLfSYlIvufq9vycIlw7BvB+sopq2Aaee1aMu4sccq/zb+zc9872c70q mdazOCemvouRcUFQnjJctWKv5QB/gnn52TdOurMxoCM0y08zifokJXHTHhcrHZKGuAO12u 0YYfmA6Z3uAMr5ThFiSMmc78qlzrnnIbejvKhd3dVKT2oVpg9DnlAVnkLY+YvwXyz+FL1a tJX/x/fbYQ9KbR4wbumZUvtfwtbRov74VQ/0g48mbWc5+KpN74O1NqVKltoTeg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WrFl54f3DzTD6 for ; Fri, 23 Aug 2024 22:54:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 47NMsvNa045345 for ; Fri, 23 Aug 2024 22:54:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 47NMsvJb045340 for fs@FreeBSD.org; Fri, 23 Aug 2024 22:54:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 281003] NFS client can open the server on ZFS with sharenfs dataset Date: Fri, 23 Aug 2024 22:54:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: Unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: rmacklem@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: component cc assigned_to version product Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281003 Rick Macklem changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Individual Port(s) |bin CC| |rmacklem@FreeBSD.org Assignee|fs@FreeBSD.org |rmacklem@FreeBSD.org Version|Latest |Unspecified Product|Ports & Packages |Base System --- Comment #1 from Rick Macklem --- This appears to be a bug in mountd. -ro=3D192.168.1.56 is bogus, but mountd does not spot it as an error. Btw, both of your sharenfs settings are bogus, since -ro is not supposed to have "=3D192.168.1.56" after it. All the sharenfs property does is autogenerate an exports line. It can be found in /etc/zfs/exports and should obey the same syntax as "man 5 exports" describes. (Althougn the sharenfs parser in OpenZFS should be improved, the bogus case should not get past mountd.) I'll take a look at mountd.c and try to come up with a patch. I am not sure what you were trying to do by specifying one IP address followed by ro=3Danther-ip, but you cannot mix read/write and read-only exports on the same line. --> Until you have the very recent patch in PR#147881, you cannot generate multiple lines via sharenfs. --> To do multiple lines, you need to create them manually in /etc/exports and not use the sharenfs property. Thanks for reporting it, rick ps: And never put a "=3DXXX" after "ro" for an exports option. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Aug 24 16:10:04 2024 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WrhjT2857z5TN7w for ; Sat, 24 Aug 2024 16:10:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WrhjT0Rjlz4Dd1 for ; Sat, 24 Aug 2024 16:10:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724515805; a=rsa-sha256; cv=none; b=tzSzqibuiaPeDmc94yH9opP+owNTfeJYQwq5owhzALTbQ5cIs7JJ6FXe4NASGIXnSrNKsq o1qSp6qirCTwQGTHE24oNxnM+FSZ+K1bhXDUeaRow+YOIUU/fRqxhCbYwKSOZ4+P3T0luR cZSW7CjIYwfPoMoFrMwH9QaXsUx0OUw9n9MsNXq04hyF/EZSQ84HTzo60I6Rl1UIpIzRD0 fw6YWYdWnTu1mJMeKcq58b3azN1oj99CVGJSg8sjgiTGn/KpObWYIelW5TRj84Pi6RX8rs JNo/7eX7nDRnY44IpNWM3o1QP6+QPwz0iNbozKfv/r4j0qMT7gkH7Zlq8ww73w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724515805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4327+t9n9c9bF6ILK7z+xPt0C/ARO7G7YjG4Hg+sDqo=; b=G4dI+qIWYdj/eExLNFCMmPNg7qUTMjhHmT6XGggh1i6BUsDzu3cO6PnmrmRUmfEqE2OYh7 OSGaLgjJsYWPWO7q8q4p7I11O03T0yYBvsfhPrE3cBFrXJKNd2KmPdHSzSwqmrHFTRnQZP zCRNNVhKrzgxKc3Ii+Uh21EPv7KaGPysFFPUQ6HIYq3XxsgcEU8h1ZJXZWm2DX4RybYlRm /lJkEmLk+OUklDbUzm0y+Or3JSh8brd6XhvQSiSuRcqlsTW3GlqBnSZZvcPhIBLFoSQTUT VEpwQ5Rm382Eir+rhOfoTYnxuMwQXdXuKPHsz1J9QS2HSeur3zuk/T3tlfi/CQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WrhjS72PLz11vY for ; Sat, 24 Aug 2024 16:10:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 47OGA4Ri060418 for ; Sat, 24 Aug 2024 16:10:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 47OGA4FQ060413 for fs@FreeBSD.org; Sat, 24 Aug 2024 16:10:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 281033] [msdosfs] rm -f fails Date: Sat, 24 Aug 2024 16:10:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281033 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Aug 24 17:44:18 2024 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WrkpC2xnBz5TWHx for ; Sat, 24 Aug 2024 17:44:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WrkpB5SDTz4Nd8 for ; Sat, 24 Aug 2024 17:44:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724521458; a=rsa-sha256; cv=none; b=QOp9vIPzhKFvDeMTOzeKj50CMPm/J/Y1ZU6SV7/D8vEtUMraXDMUhlixjkor7sdlL2vo3e 6Go6hBh64Yq45GnSHkN16YKZxGnKV4rCpr3ds0pli0j0/uVqCBYsedfzgQCo7W6aFG2QYO mkcZLLUnRBvXh783uRQafkkwlIdOP564y7qScUXka4DBTxAYRSBkHyx/d20DiXUY2j30jY bikIHkjwNqEq8jW5NUiqkhNfEvzrtabrIcG2KxrJu3gzpwaYiVzeYYslnTqdwZgKInIGBI KGIt//FZdh0EwbM+6k8XfrnGRQywKEQO/6W+z3lnSBHQ3iCw5bscFksM8XSLKQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724521458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DamBCwKlrm7D1fiY1LxujVM1Q/LYz5UxnJ2RxdbPJds=; b=U1sTO1cGyJQg4g72mQ/OfTn3PtrOU2mP9IX9jwTKfWk8X85q0PntnNYYif74B/JUhrhGG6 RhhQUzszJ9Ag+yJfzchXYfvmZPT/ikETW6mxUcry1RuiLp9w5OZqA+ymo0BUciY1q/P258 3NMkTYeFcZiBZj+fD2Cbpa1IUZ7gK+PTOT9MPHNUSMedz28tsTc1kIO0WNNjt/85oFxo6P 9mFTHxcredSfhk3OklgQZRHjJex+yZ746p74GoXznjZ0pInB1PDZgQ8p30ZYzl6K9sFyVj hzEnBdlmteub4nzmUZefr7kmYs5dfcVAetCpP4dYjb6YiyPsACI7E8F0jpHGPQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WrkpB4yVSz14N7 for ; Sat, 24 Aug 2024 17:44:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 47OHiIHA069608 for ; Sat, 24 Aug 2024 17:44:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 47OHiIqu069602 for fs@FreeBSD.org; Sat, 24 Aug 2024 17:44:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 281033] [msdosfs] rm -f fails Date: Sat, 24 Aug 2024 17:44:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: se@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: se@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281033 Stefan E=C3=9Fer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |se@FreeBSD.org Assignee|fs@FreeBSD.org |se@FreeBSD.org Status|New |Open --- Comment #1 from Stefan E=C3=9Fer --- The patch that has been discussed as GitHub pull request 1329 (https://github.com/freebsd/freebsd-src/pull/1329) will be attached to this= PR. It has an undesirable side effect: the mv command returns "No such file or directory" instead of "Invalid argument" when the target file name is inval= id: # touch /boot/efi/A.DAT # mv /boot/efi/A.DAT /boot/efi/B*.DAT mv: rename /boot/efi/A.DAT to /boot/efi/B*.DAT: No such file or directory This is caused by namei() being called with the target filename and that invokes the patched msdosfs_lookup() and thus gets and returns the changed error code. But since the patch affects namei() and functions that call it, more vnode operations will return ENOENT instead of ```EINVAL`` with this patch (when given an invalid MSDOS file name on an MSDOSFS file system). The rename(2) man page mentions both EINVAL and ENOENT, but neither descrip= tion mentions the case of an invalid target file name: [ENOENT] A component of the from path does not exist, or a path prefix of to does not exist. [EINVAL] The from argument is a parent directory of to, or an attempt is ma= de to rename =E2=80=98.=E2=80=99 or =E2=80=98..=E2=80=99. Maybe the case of a target name not allowed on some file system should be added, e.g. one of: [EINVAL] The from argument is a parent directory of to, or an attempt is ma= de to rename =E2=80=98.=E2=80=99 or =E2=80=98..=E2=80=99, or the target filena= me is invalid. [ENOENT] A component of the from path does not exist, a path prefix of to d= oes not exist, or the target filename is invalid. If changing the return code of rename() (and possibly other system calls th= at return the error code of a failed namei() call unmodiified) is acceptable f= or this specific error case, I'll commit the patch to -CURRENT, but I'm not su= re whether POLA will allow merging to -STABLE. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Aug 25 03:03:40 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WrzCs18z0z5V7ht for ; Sun, 25 Aug 2024 03:03:53 +0000 (UTC) (envelope-from boyvalue@gmail.com) Received: from mail-qt1-x82d.google.com (mail-qt1-x82d.google.com [IPv6:2607:f8b0:4864:20::82d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WrzCr28GQz4Q22 for ; Sun, 25 Aug 2024 03:03:52 +0000 (UTC) (envelope-from boyvalue@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b="GA9a/mFa"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of boyvalue@gmail.com designates 2607:f8b0:4864:20::82d as permitted sender) smtp.mailfrom=boyvalue@gmail.com Received: by mail-qt1-x82d.google.com with SMTP id d75a77b69052e-44ff7bdb5a6so18718241cf.3 for ; Sat, 24 Aug 2024 20:03:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724555031; x=1725159831; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=Gim+wSI6Jwj04qreifMVEU+Qv0Qj9FUM398lXCh4LU0=; b=GA9a/mFaQhqQ3GxQk3V8S0odRFCwsSJatg3EfZKCXGDgKQJgLNCGGvvLkVv9gKiGO0 qjPUX8Vzc/FAfN6EFPDe5GZ0uxfeo/rmdx0GCR7QcDJowrnie72HFSi5NgISIRpi9sNH kQ68wh9jdrnygazHU2xGYOBYt6OA1V/jldw2KEs7RgYlXgbDtvK/HcGQLFLnW9tM9Qqh olLQy8oQEiP8oFvUbPREV+oiN41B0IBZDwNmvSSjuxNYOfrL1AhIqHb5h7+Ti0qtBsVD x3qQqLL0/IfSGgWj46m/eNrJWwayRMOIr7VyYhATMJcuIQZrbsxzH6J+2CnCBkO72Agg fS/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724555031; x=1725159831; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Gim+wSI6Jwj04qreifMVEU+Qv0Qj9FUM398lXCh4LU0=; b=mhFaYUublNKlG9XeALox2b5KA1KyjZiRiAAfgrNABzEfnwzhz2rtQcCxKyCI5SIzVk iBGKNeT4n9ZcgIxtoa3I3mbriCUg3tbueaVPWrRK1kb/vBmmKZsEtHU5k8pybNOy+lqE nICZxWeElLSZeKkLbrASD/0rE4RMAV2Zpq5adP9gAwCut47QzwbedskGx7yXQuodJ0u6 qBfhraLY8M7n2Y4Dj2drV+LFPqm2ZYInlbTb3wv1QYmSiWNnTjUiQ875t/+mf1uXZYuQ 0ayl20d/T0OlB1HpKvba0HgiYbtk2AMNIC3/QmTGz43ZnfqSAA0rm8CgpJEMCmhDivio u+Nw== X-Gm-Message-State: AOJu0YwHFstHAlNIFVmVuDTAZ714RFx6zuBNiNhwlHc/qpToL8wS+B5g ACML6s7YszQbC3WKi3GDWmfG20aS1otwae34CiSYy22sXc1CIlvMQyiDfScfdmAqfIhZU9yAuR8 N5Fi4hqbV2fl8myTZLm5pPapYozK3 X-Google-Smtp-Source: AGHT+IHJxG+hNVMTIjDfcs8ZwZp4J/KTKVSDuYqZqFk3oYxd3VgTy0f8iNv3Cp6HcSgmlrmaxE0OWUDJfHBQOmm8/0E= X-Received: by 2002:a05:622a:4a0b:b0:447:f79d:cf0b with SMTP id d75a77b69052e-455097c46fbmr78093311cf.41.1724555031091; Sat, 24 Aug 2024 20:03:51 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Pamela Ballantyne Date: Sat, 24 Aug 2024 21:03:40 -0600 Message-ID: Subject: Moving ZFS root popol to a virtual server was: ZFS: Suspended Pool due to allegedly uncorrectable I/O error To: freebsd-fs@freebsd.org Content-Type: multipart/alternative; boundary="0000000000000623e90620794290" X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.49)[-0.493]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MISSING_XM_UA(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::82d:from] X-Rspamd-Queue-Id: 4WrzCr28GQz4Q22 --0000000000000623e90620794290 Content-Type: text/plain; charset="UTF-8" Many thanks to David Christensen, Ronald Klop, and Peter Much for answering my plea for help. Further reviewing the available evidence, it seems that the most likely cause of the problem is someone at the data center removing the disk drives from our server. This was probably accidental, not malicious. I have reported this to the hosting company, but I'm not expecting much. It is ancient hardware, circa 2011. Although it still performs decently. It is probably time for an upgrade, particularly since the hosting company is dropping support for bare metal servers. I am not sure how to move the existing root pool to a virtual server. I've tried a few simulations with VirtualBox, creating a source server and a target. The pools get moved, but the target server never boots (even when I remember to set bootfs). Worse, the target disks are reported as being corrupted each time I do it. Once again, any experience/advice here would be very much appreciated. Thank you again, Pammy --0000000000000623e90620794290 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Many thanks to David Christensen, Ronald Klop, and Peter M= uch for answering my plea for help.

Further reviewing th= e available evidence,=C2=A0 it seems that the most likely cause of the prob= lem is someone at the data center
removing the disk drives from o= ur server.=C2=A0 This was probably accidental, not malicious. I have report= ed this to the hosting company, but
I'm not expecting much.

It is ancient hardware, circa 2011. Although=C2=A0i= t still performs decently. It is probably time for an upgrade, particularly=
since the hosting company is dropping support for bare metal ser= vers.

I am not sure how to move the existing root = pool to a virtual server. I've tried a few simulations with VirtualBox,= creating
a source server and a target. The pools get moved, but = the target server never boots (even when I remember to set bootfs).
Worse, the target disks are reported as being corrupted each time I do i= t.=C2=A0=C2=A0

Once again, any experience/advice h= ere would be very much appreciated.

Thank you agai= n,
Pammy


--0000000000000623e90620794290-- From nobody Sun Aug 25 05:25:02 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ws2MF1p7xz5VKgH for ; Sun, 25 Aug 2024 05:25:29 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [184.105.128.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "holgerdanske.com", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ws2MD0j5Dz4d0w for ; Sun, 25 Aug 2024 05:25:28 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=holgerdanske.com header.s=nov-20210719-112354 header.b=Q1wSj0Tt; dmarc=pass (policy=none) header.from=holgerdanske.com; spf=pass (mx1.freebsd.org: domain of dpchrist@holgerdanske.com designates 184.105.128.27 as permitted sender) smtp.mailfrom=dpchrist@holgerdanske.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=holgerdanske.com; s=nov-20210719-112354; t=1724563518; bh=8vaan5fqg8c5L56TInt0pe8RZ9ScywTVmrG1wbJh2g8=; h=Received:Message-ID:Date:MIME-Version:User-Agent:Subject:To: References:Content-Language:From:In-Reply-To:Content-Type: Content-Transfer-Encoding; b=Q1wSj0TtknsnesDGh5yzutuipkHhZmF3+ZFhbDMxwK1WcTnq9xaA9v/OPlY4/Ccf3 SwJsWzgQEXYTd9QXu79EN1E584Gh2rGhePmok62yBMcL9hmGmDxq+veskx8U70rsbT XsWZWSu6VIK6t88a/s/YJgRXVm4GiNezLzM3TZ/puGyRRWbYX8eOYrGYBzagS1NK82 CHEt861VumccwCHriD7sPhHYTJQ1GFYoLxKbcjUwetBzpi0spdoiav+4iVyQFmUCN/ UTeR06emQRttNJfCe4+kmGpiTqjC+YLgjMClzXhfg88ea788a390ZlB6wJ8T47rGHN F9J3iuV5hsY8c8oxTv8shRyBKBqL0RNd/mBJ3rOXPoHvpZs280Cqpjqa6jYMrAUNjZ ZKgKBBQWCqdAeARNhSsBVXBOGvPjnKvVZFiOluX4FivzVLZuq2bzWM3+A5jkSVwI8P hoCy8Sl5gEUlziMZPx26yEFu6g5Pg6IlwTMLWNDfi/nxkq0chBAtD1Sle5/CKrPIYA 22hZ0YHwvp//SpNBUD+8LHswBqFbN38UV5DDw/kc+oGOLTljAQM3Q2WB2QCtv96b1a Zk8gjJfogkOEUQUnkkfNhZ9j7vdfHn9Ipc4BixB6KQlXMJz5f5xbHVq3Tek2wfCpAa LGwQu8mxhLBYCJTqoRa23S1I= Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Sat, 24 Aug 2024 22:25:18 -0700 Message-ID: Date: Sat, 24 Aug 2024 22:25:02 -0700 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Moving ZFS root popol to a virtual server was: ZFS: Suspended Pool due to allegedly uncorrectable I/O error To: freebsd-fs@freebsd.org References: Content-Language: en-US From: David Christensen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.89 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[holgerdanske.com,none]; R_SPF_ALLOW(-0.20)[+a]; R_DKIM_ALLOW(-0.20)[holgerdanske.com:s=nov-20210719-112354]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:6939, ipnet:184.104.0.0/15, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[holgerdanske.com:+] X-Rspamd-Queue-Id: 4Ws2MD0j5Dz4d0w On 8/24/24 20:03, Pamela Ballantyne wrote: > Many thanks to David Christensen, Ronald Klop, and Peter Much for answering > my plea for help. > > Further reviewing the available evidence, it seems that the most likely > cause of the problem is someone at the data center > removing the disk drives from our server. This was probably accidental, > not malicious. I have reported this to the hosting company, but > I'm not expecting much. > > It is ancient hardware, circa 2011. Although it still performs decently. It > is probably time for an upgrade, particularly > since the hosting company is dropping support for bare metal servers. > > I am not sure how to move the existing root pool to a virtual server. I've > tried a few simulations with VirtualBox, creating > a source server and a target. The pools get moved, but the target server > never boots (even when I remember to set bootfs). > Worse, the target disks are reported as being corrupted each time I do it. > > Once again, any experience/advice here would be very much appreciated. > > Thank you again, > Pammy A networked version control system is incredibly useful for system administration. If you do not have such, I highly recommend it; both for this project and for everything else going forward. I have considered migrating an OS instance from a physical disk to a VM (or vice-versa) a few times, but always decided to do a fresh install on the intended target. I would build a fresh VM, install fresh software, and migrate configuration, data, and services from the old server to the new VM. Additional details include scheduling, network settings, validation, backup, and rollback. David From nobody Sun Aug 25 08:30:03 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ws6SF0kwrz5VbD4 for ; Sun, 25 Aug 2024 08:30:05 +0000 (UTC) (envelope-from SRS0=cvou=PY=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ws6SD6KSSz430t for ; Sun, 25 Aug 2024 08:30:04 +0000 (UTC) (envelope-from SRS0=cvou=PY=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Sun, 25 Aug 2024 10:30:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1724574603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eYQ6E0mcZkYgDoOBiN06knASp4mXku6XVV1Di6Koyg8=; b=GLEEuIh3yLG+jM3bqAWVnskkunG6CPHGDbVcpwIwzyItXb8ycnQua4mtLxpSE16LCJbog3 2o4fotATOEa0dG1m/8NWlnT52hg4yG7wmRUbRohDZkQ/1L01x+WmQe3brzgiJXyzAPdFoP Z0+cfKi5vC5Cj8z+7bkXjGR9S2WuBd30z9thLWA7q0eRUnIFLjMHVouIFQjRGTiS3maBtg DquIycxsoXkqaegMqz7UuefO6odeTzb9grwMhl9KG6HxwAL4RLOxex+V6pGqOx3ZEtBHBV z9fqcwqy5d8tu/631oBzrjC6ZkePf3QhFuXopvLPqqxXQHKiKnozIu3JCB0diw== From: Ronald Klop To: Pamela Ballantyne Cc: freebsd-fs@freebsd.org Message-ID: <1716220186.14048.1724574603123@localhost> In-Reply-To: References: Subject: Re: Moving ZFS root popol to a virtual server was: ZFS: Suspended Pool due to allegedly uncorrectable I/O error List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_14047_1171526651.1724574602983" X-Mailer: Realworks (717.35) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4Ws6SD6KSSz430t ------=_Part_14047_1171526651.1724574602983 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Pamela Ballantyne Datum: zondag, 25 augustus 2024 05:03 Aan: freebsd-fs@freebsd.org Onderwerp: Moving ZFS root popol to a virtual server was: ZFS: Suspended Pool due to allegedly uncorrectable I/O error > > Many thanks to David Christensen, Ronald Klop, and Peter Much for answering my plea for help. > > Further reviewing the available evidence, it seems that the most likely cause of the problem is someone at the data center > removing the disk drives from our server. This was probably accidental, not malicious. I have reported this to the hosting company, but > I'm not expecting much. > > It is ancient hardware, circa 2011. Although it still performs decently. It is probably time for an upgrade, particularly > since the hosting company is dropping support for bare metal servers. > > I am not sure how to move the existing root pool to a virtual server. I've tried a few simulations with VirtualBox, creating > a source server and a target. The pools get moved, but the target server never boots (even when I remember to set bootfs). > Worse, the target disks are reported as being corrupted each time I do it. > > Once again, any experience/advice here would be very much appreciated. > > Thank you again, > Pammy > > This is certainly possible but small details can result in a non-bootable pool on the target. Can you provide some information on how you moved the root pool to the virtual server? Did you dd the whole disk to a file? Or did you zfs send | zfs receive a snapshot? Or ... ? What boot-loader was used on the old server and what on the new one? Is the partitioning the same on both? I remember the old server had mirrored disks, how did you setup the virtual server? Regards, Ronald. ------=_Part_14047_1171526651.1724574602983 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Pamela Ballantyne <boyvalue@gmail.com>
Datum: zondag, 25 augustus 2024 05:03
Aan: freebsd-fs@freebsd.org
Onderwerp: Moving ZFS root popol to a virtual server was: ZFS: Suspended Pool due to allegedly uncorrectable I/O error

Many thanks to David Christensen, Ronald Klop, and Peter Much for answering my plea for help.
 
Further reviewing the available evidence,  it seems that the most likely cause of the problem is someone at the data center
removing the disk drives from our server.  This was probably accidental, not malicious. I have reported this to the hosting company, but
I'm not expecting much.
 
It is ancient hardware, circa 2011. Although it still performs decently. It is probably time for an upgrade, particularly
since the hosting company is dropping support for bare metal servers.
 
I am not sure how to move the existing root pool to a virtual server. I've tried a few simulations with VirtualBox, creating
a source server and a target. The pools get moved, but the target server never boots (even when I remember to set bootfs).
Worse, the target disks are reported as being corrupted each time I do it.  
 
Once again, any experience/advice here would be very much appreciated.
 
Thank you again,
Pammy
 
 


This is certainly possible but small details can result in a non-bootable pool on the target.

Can you provide some information on how you moved the root pool to the virtual server?
Did you dd the whole disk to a file? Or did you zfs send | zfs receive a snapshot? Or ... ?
What boot-loader was used on the old server and what on the new one? Is the partitioning the same on both?
I remember the old server had mirrored disks, how did you setup the virtual server?

Regards,
Ronald.
  ------=_Part_14047_1171526651.1724574602983-- From nobody Sun Aug 25 21:00:10 2024 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WsR5l52rNz5TWxY for ; Sun, 25 Aug 2024 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WsR5l0l6pz4Mft for ; Sun, 25 Aug 2024 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724619611; a=rsa-sha256; cv=none; b=CYcB3rLhiXeS5dVmO48i0X9v9GIxzTBV0PBTBiCxobItumcMr5rgVyAesehLGY7Nfc+Kw8 J7NcBwCJqNDkvKXqojoaOyCn4CjiLDmI3lfH8PznBo8FtaC2XVmcWvdaapbEcE8OP+00lP yV1BluBA7blGcKOiADEc7nnJ2q2CM3iXhTrXZ6ZpTCAURCyhFt6HDFcy4vJFHoA8gS7nTx fvhF4K/l+fTX2tz9b0drEH6ywVExAn5aIngjwwHSM2Jni9gw6tMalj9k6YeYB/mX7j8i0F bNF51suynXZArn6oqx4agQGv0JSggT8TYToLSMXFhcjEGHIFsnJTx4Al5YTwrQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724619611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MW/Dzjfp4a9tuWEFyaPgdFSW6W4krM8rHY/S4fl1u2Q=; b=CBx/WEQGixUn0T0VuBOmDndnJltyttZhZvhsZ1gYsiqTTqRDgj7L+q8e8cOpqqG7vRDfri UnwnLiXuROdLPo6S9h9GQHEJsRBsAY5vOpbaacrrw9yRKteLfOMsvSuRj0U6dTtMXNcTdj xb7bf8+kLDFtKkjTluTpcrKR3VYvjDpi19DHbs2bfdQMt8aSXxLTzYUldt2va6rFZ1OTJC XsKTTZJCOqsrln0QzTa1jov8nI6LC0efx2hH4U0FMudtXmN8SahvcSHDlpHQkI9HbaYovt nr8OFVHJPkswIiG4Wl8OxLni4jZrGThV8lMODE0CUVg9s7jrdlwktXCi2duVgg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WsR5l0MKMzfKy for ; Sun, 25 Aug 2024 21:00:11 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 47PL0AVX036467 for ; Sun, 25 Aug 2024 21:00:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 47PL0A9e036465 for fs@FreeBSD.org; Sun, 25 Aug 2024 21:00:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202408252100.47PL0A9e036465@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 25 Aug 2024 21:00:10 +0000 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17246196102.4E8559f.33182" Content-Transfer-Encoding: 7bit --17246196102.4E8559f.33182 Date: Sun, 25 Aug 2024 21:00:10 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 231794 | zfs: Panic due to ARC related KVA memory exhausti Open | 251035 | ZFS: Allow 64 bit ZFS to support 32 bit ioctls (W Open | 266409 | 13.1-RELEASE amd64 crashes in: sa_handle_destroy Open | 268162 | zfskeys_enable: each successful load of a key is Open | 269503 | docs.freebsd.org: default vfs.zfs.arc.meta_limit Open | 271384 | zfs_load is not suitably documented Open | 226130 | ZFS: solaris assert: zrl->zr_refcount == 0 (0x1 = 7 problems total for which you should take action. --17246196102.4E8559f.33182 Date: Sun, 25 Aug 2024 21:00:10 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    231794 | zfs: Panic due to ARC related KVA memory exhausti
Open        |    251035 | ZFS: Allow 64 bit ZFS to support 32 bit ioctls (W
Open        |    266409 | 13.1-RELEASE amd64 crashes in: sa_handle_destroy 
Open        |    268162 | zfskeys_enable: each successful load of a key is 
Open        |    269503 | docs.freebsd.org: default vfs.zfs.arc.meta_limit
Open        |    271384 | zfs_load is not suitably documented
Open        |    226130 | ZFS: solaris assert: zrl->zr_refcount == 0 (0x1 =

7 problems total for which you should take action.
--17246196102.4E8559f.33182--