From nobody Fri Nov 8 19:20:58 2024 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XlTLk5dXhz5cgDs for ; Fri, 08 Nov 2024 19:21:02 +0000 (UTC) (envelope-from void@f-m.fm) Received: from fhigh-a4-smtp.messagingengine.com (fhigh-a4-smtp.messagingengine.com [103.168.172.155]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XlTLj3PyPz4BpK for ; Fri, 8 Nov 2024 19:21:01 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=CtLcGwtb; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=DWuQyUk+; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 103.168.172.155 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from phl-compute-04.internal (phl-compute-04.phl.internal [10.202.2.44]) by mailfhigh.phl.internal (Postfix) with ESMTP id 8077D11400EC for ; Fri, 8 Nov 2024 14:21:00 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Fri, 08 Nov 2024 14:21:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1731093660; x=1731180060; bh=FQGlT2K7H0 UjM1BAB5a4NKjM2ATL0jkcWXNqGKs1c8A=; b=CtLcGwtbr9ASw8MkIFMjUn58X6 1K5I4YZKmnhTcVKK0PNj8A+z4SPpNfmNheS9I6Z5p+4ooSUzFdiT8Y7ExS4p+Jzp AGIfoYDPI6g8dnlEVU+8LZhS6TAAnFo+REnryqv2F1REwAr8hF0vKxPlLGNTfPpy pvq0Y6Ll3Hi1DUEleIzkPZzEppjNEwlV6JdwDVLB2mjkqx+M9Hrtt6tsh0FXduWz OM/hJ8NuJpH0tR3Lk17rKj+56bn40cJYuxTQb3NBRVrtm/K5/1aUv2yI2Gf3Lddr z6zQejw5p4WDkQn+++HUrOwFtOG5MJitEWACjsvX68+I9hkM3gCLZkroDr+g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1731093660; x=1731180060; bh=FQGlT2K7H0UjM1BAB5a4NKjM2ATL0jkcWXN qGKs1c8A=; b=DWuQyUk+1VJzirrIk1Me+bcCYmPLrI2baLaG3xu9c6hjIkrX0s5 sPR/vOLPF7JXwQErOSNLK3/FzwdB3IMhHCx0Fs330PN2KxsgeXsr7RwNjk3Jw2is 1gXuQYq0SH79zxCbhiboBhoBhOar/lQKEneVbOGXM3yt2fTkeqdOGsX2qGcPmbrU rfXBFe+yPQvNvlj6fk5FlieA7WMN3F3Q0w6Y9TP+hKSSSXVc4yD6vfdfSzX55wDu YQczB+xHdFzc5tc2w+g5l8yoY26LgkePGdaXJzLowDaA9TBz13FGu5Riy73B8d3g JUt4XyyhS8IWFqx9ZdQw9VrFMqiK2VbadtA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrtdeigdduvddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuf fkfhggtggujgesthdtredttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhm rdhfmheqnecuggftrfgrthhtvghrnhepkeeluddvlefhieelfefggffhffektdehleelgf dugfdvgeekjeejuddtheehgfeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm pehmrghilhhfrhhomhepvhhoihgusehfqdhmrdhfmhdpnhgspghrtghpthhtohepuddpmh houggvpehsmhhtphhouhhtpdhrtghpthhtohepfhhrvggvsghsugdqfhhssehfrhgvvggs shgurdhorhhg X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 8 Nov 2024 14:20:59 -0500 (EST) Date: Fri, 8 Nov 2024 19:20:58 +0000 From: void To: freebsd-fs@freebsd.org Subject: Re: zfs snapshot corruption when using encryption Message-ID: Mail-Followup-To: freebsd-fs@freebsd.org References: <03E4CCF5-0F9A-4B0E-A9DA-81C7C677860C@FreeBSD.org> <3E85AAAE-8B1E-47C7-B581-E3D98AB03907@FreeBSD.org> List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <3E85AAAE-8B1E-47C7-B581-E3D98AB03907@FreeBSD.org> X-Spamd-Result: default: False [-3.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:103.168.172.128/27]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[103.168.172.155:from]; FREEMAIL_FROM(0.00)[f-m.fm]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; MISSING_XM_UA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:209242, ipnet:103.168.172.0/24, country:US]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim] X-Rspamd-Queue-Id: 4XlTLj3PyPz4BpK X-Spamd-Bar: --- On Fri, Nov 08, 2024 at 03:29:22PM +0100, Palle Girgensohn wrote: >I cannot run `zfs send -I fs@previous_snap fs@problematic_snap`, I get >warning: cannot send 's@problematic_snap': Input/output error` >Removing the snapshot fixes the problem. What an odd problem. Like, it can write but not read. What zfs version? On 13.3 I'm using: zfs-2.1.14-FreeBSD_gd99134be8 zfs-kmod-2.1.14-FreeBSD_gd99134be8 on 14-stable: zfs-2.2.6-FreeBSD_g33174af15 zfs-kmod-2.2.6-FreeBSD_g33174af15 on 15-current: zfs-2.3.99-31-FreeBSD_gb2f6de7b5 zfs-kmod-2.3.99-31-FreeBSD_gb2f6de7b5 Is the encryption you're using is the GELI based whole-disk one (which has been around iirc for a few years) or the relatively recent zfs encryption that works per-filesystem. This is why I'm asking the zfs version. I thought the latter is relatively quite new, and I've never heard of it working on 14.0. The way I'd go about trying to address the issue would be to start from the lowest layer and work up. On a raidz2 array (the card in JBOD mode), I can identify all the drives as da0-7, so can use smartctl -x to query them all directly, looking for, in particular: Reallocated_Sector_Ct Reported_Uncorrect Current_Pending_Sector Offline_Uncorrectable If your disk arrangement is connected via something like HP Smart Array or similar, you'll need to look at the man page for smartctl for the exact syntax to query the disks behind the card. --