From nobody Thu Dec 14 22:36:21 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SrnJg0Mx1z53sq8 for ; Thu, 14 Dec 2023 22:36:35 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SrnJf4Z0Fz3GmG; Thu, 14 Dec 2023 22:36:34 +0000 (UTC) (envelope-from delphij@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-a22f59c6aeaso9890566b.2; Thu, 14 Dec 2023 14:36:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702593393; x=1703198193; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=6lrPqPkRSmQjTX7aNS3fq2uAGVoaFwdRXnty7j8U/0g=; b=kznmRq8+46+wLFCXWqNVZH/hpfyBZQwHkMlcM6oBM8JSIy9L7ylxDymr76YPzniZaf UbLR4gomyYA2u91lcZ9sbu7FkdBEGaazR+fge69hjdEHCj5r8h7UL+WSwx3j5txHzaBB qD8HHoH6VTr8R1ZP6Udl68uPLyeNlt4xuAdHkTafz/1d/6kMLZ7g0/EZZ/9otAmq0R/X cxjwhO5OlXe6rAty5OocwbrxyngZhwyOrEu89sIk3qc0B08x2oUthIUH3sK1iJzNJdJN vP7oQh2vLm3Cv2FVp1+2Ea1ptQ8UdnBP2+SBQeXXerbptsbzWdF5RpUQl3MNgp63d39C KT8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702593393; x=1703198193; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6lrPqPkRSmQjTX7aNS3fq2uAGVoaFwdRXnty7j8U/0g=; b=nnE6LwAI3/RMI1cCE5EzNLg1z6F8iK8sbEpFQtdNFXCVQ9C/UJthX9qhe5TK5FMLlB udZODgTnFTzap6OeQ0X40IUjEIMbMwbEVIxteP7E9M4LlKpLp8r9TPtutcpWZZuR+KQQ IZIGgsNBIpldQRF8fCIj165RfvKGP9tvkxI+XQONgrGt+izfPK8aGaXH2rDEymguNpEl Fy9hm5WrnMbEtn5ZmnEr2el3GZybrKFLNRsRC0IKlyPymmzdDcP5m736EUFM3aDaF8Hp v7W2sz5LG5auTsCQdh/uTw2k33kbf9Lm0a/hEgBqYIzU+cJ19I1hvQAQyQxJgPXZdSPq /ftw== X-Gm-Message-State: AOJu0Yy8tvSLKNnIkGqDEb4HtwuFqvlGLf9tXCG6coNyqPOMH0qZtHVc +P2uKIYr8XIV78k7i09QHh/hNmId/tUescrGTHJ0fhHbv07bZA== X-Google-Smtp-Source: AGHT+IFWbyfj6ZMqGrOnqGLQo5nJiXrMlEiZwN8yo6PxrQrAU1OdXMTx5amjBnxCSDUCARTDxWCERhWifP77iJQVEFY= X-Received: by 2002:a17:907:7d90:b0:a19:a19b:55c2 with SMTP id oz16-20020a1709077d9000b00a19a19b55c2mr6365710ejc.82.1702593392807; Thu, 14 Dec 2023 14:36:32 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <787CB64A-1687-49C3-9063-2CE3B6F957EF@le-fay.org> <1656A349-9E7D-4798-89D1-ECE39126FBC8@le-fay.org> <5fc39fab-3429-405b-9eac-1a7c94c3754f@FreeBSD.org> In-Reply-To: <5fc39fab-3429-405b-9eac-1a7c94c3754f@FreeBSD.org> From: Xin LI Date: Thu, 14 Dec 2023 14:36:21 -0800 Message-ID: Subject: Re: unusual ZFS issue To: Ronald Klop Cc: Lexi Winter , Pete Wright , "freebsd-fs@freebsd.org" Content-Type: multipart/alternative; boundary="000000000000601518060c7feacf" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4SrnJf4Z0Fz3GmG --000000000000601518060c7feacf Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Dec 14, 2023 at 2:32=E2=80=AFPM Ronald Klop wr= ote: > Did the IO error occur before or during the scrub? > This doesn't really matter. Regular reads can discover checksum errors and be reflected in zpool status too. > > Regards, > Ronald. > > > On 12/14/23 23:29, Lexi Winter wrote: > > On 14 Dec 2023, at 22:25, Xin LI wrote: > >> Try "zpool status -x" and see if it would show something useful? > > > > the output seems to be the same as =E2=80=98zpool status -v=E2=80=99: > > > > # zpool status -xv > > pool: data > > state: ONLINE > > status: One or more devices has experienced an error resulting in data > > corruption. Applications may be affected. > > action: Restore the file in question if possible. Otherwise restore th= e > > entire pool from backup. > > see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A > > scan: scrub in progress since Thu Dec 14 18:58:21 2023 > > 11.5T / 18.8T scanned at 962M/s, 8.71T / 18.8T issued at 726M/s > > 0B repaired, 46.41% done, 04:02:02 to go > > config: > > > > NAME STATE READ WRITE CKSUM > > data ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > da4p1 ONLINE 0 0 0 > > da6p1 ONLINE 0 0 0 > > da5p1 ONLINE 0 0 0 > > da7p1 ONLINE 0 0 0 > > da1p1 ONLINE 0 0 0 > > da0p1 ONLINE 0 0 0 > > da3p1 ONLINE 0 0 0 > > da2p1 ONLINE 0 0 0 > > logs > > mirror-2 ONLINE 0 0 0 > > ada0p4 ONLINE 0 0 0 > > ada1p4 ONLINE 0 0 0 > > cache > > ada1p5 ONLINE 0 0 0 > > ada0p5 ONLINE 0 0 0 > > > > errors: Permanent errors have been detected in the following files: > > > > i think this is expected since -x just filters the output to show pools > with errors? > > > > -x Only display status for pools that are exhibiting errors > or are > > otherwise unavailable. Warnings about pools not using th= e > latest > > on-disk format will not be included. > > > > thanks, lexi. > > --000000000000601518060c7feacf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: base64 PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIg c3R5bGU9ImZvbnQtZmFtaWx5Om1vbm9zcGFjZSxtb25vc3BhY2UiPjxicj48L2Rpdj48L2Rpdj48 YnI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiIGNsYXNzPSJnbWFpbF9h dHRyIj5PbiBUaHUsIERlYyAxNCwgMjAyMyBhdCAyOjMy4oCvUE0gUm9uYWxkIEtsb3AgJmx0Ozxh IGhyZWY9Im1haWx0bzpyb25hbGRAZnJlZWJzZC5vcmciPnJvbmFsZEBmcmVlYnNkLm9yZzwvYT4m Z3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxl PSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6MXB4IHNvbGlkIHJnYigyMDQs MjA0LDIwNCk7cGFkZGluZy1sZWZ0OjFleCI+RGlkIHRoZSBJTyBlcnJvciBvY2N1ciBiZWZvcmUg b3IgZHVyaW5nIHRoZSBzY3J1Yj88YnI+PC9ibG9ja3F1b3RlPjxkaXY+PGJyPjwvZGl2PjxkaXY+ PGRpdiBjbGFzcz0iZ21haWxfZGVmYXVsdCIgc3R5bGU9ImZvbnQtZmFtaWx5Om1vbm9zcGFjZSxt b25vc3BhY2UiPlRoaXMgZG9lc24mIzM5O3QgcmVhbGx5IG1hdHRlci7CoCBSZWd1bGFyIHJlYWRz IGNhbiBkaXNjb3ZlciBjaGVja3N1bSBlcnJvcnMgYW5kIGJlIHJlZmxlY3RlZCBpbiB6cG9vbCBz dGF0dXMgdG9vLjwvZGl2Pjxicj48L2Rpdj48ZGl2PsKgPC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9 ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0 OjFweCBzb2xpZCByZ2IoMjA0LDIwNCwyMDQpO3BhZGRpbmctbGVmdDoxZXgiPg0KPGJyPg0KUmVn YXJkcyw8YnI+DQpSb25hbGQuPGJyPg0KPGJyPg0KPGJyPg0KT24gMTIvMTQvMjMgMjM6MjksIExl eGkgV2ludGVyIHdyb3RlOjxicj4NCiZndDsgT24gMTQgRGVjIDIwMjMsIGF0IDIyOjI1LCBYaW4g TEkgJmx0OzxhIGhyZWY9Im1haWx0bzpkZWxwaGlqQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsi PmRlbHBoaWpAZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyPg0KJmd0OyZndDsgVHJ5ICZxdW90 O3pwb29sIHN0YXR1cyAteCZxdW90OyBhbmQgc2VlIGlmIGl0IHdvdWxkIHNob3cgc29tZXRoaW5n IHVzZWZ1bD88YnI+DQomZ3Q7IDxicj4NCiZndDsgdGhlIG91dHB1dCBzZWVtcyB0byBiZSB0aGUg c2FtZSBhcyDigJh6cG9vbCBzdGF0dXMgLXbigJk6PGJyPg0KJmd0OyA8YnI+DQomZ3Q7ICMgenBv b2wgc3RhdHVzIC14djxicj4NCiZndDvCoCDCoCBwb29sOiBkYXRhPGJyPg0KJmd0O8KgIMKgc3Rh dGU6IE9OTElORTxicj4NCiZndDsgc3RhdHVzOiBPbmUgb3IgbW9yZSBkZXZpY2VzIGhhcyBleHBl cmllbmNlZCBhbiBlcnJvciByZXN1bHRpbmcgaW4gZGF0YTxicj4NCiZndDvCoCDCoCDCoCDCoGNv cnJ1cHRpb24uwqAgQXBwbGljYXRpb25zIG1heSBiZSBhZmZlY3RlZC48YnI+DQomZ3Q7IGFjdGlv bjogUmVzdG9yZSB0aGUgZmlsZSBpbiBxdWVzdGlvbiBpZiBwb3NzaWJsZS7CoCBPdGhlcndpc2Ug cmVzdG9yZSB0aGU8YnI+DQomZ3Q7wqAgwqAgwqAgwqBlbnRpcmUgcG9vbCBmcm9tIGJhY2t1cC48 YnI+DQomZ3Q7wqAgwqAgwqBzZWU6IDxhIGhyZWY9Imh0dHBzOi8vb3Blbnpmcy5naXRodWIuaW8v b3Blbnpmcy1kb2NzL21zZy9aRlMtODAwMC04QSIgcmVsPSJub3JlZmVycmVyIiB0YXJnZXQ9Il9i bGFuayI+aHR0cHM6Ly9vcGVuemZzLmdpdGh1Yi5pby9vcGVuemZzLWRvY3MvbXNnL1pGUy04MDAw LThBPC9hPjxicj4NCiZndDvCoCDCoCBzY2FuOiBzY3J1YiBpbiBwcm9ncmVzcyBzaW5jZSBUaHUg RGVjIDE0IDE4OjU4OjIxIDIwMjM8YnI+DQomZ3Q7wqAgwqAgwqAgwqAxMS41VCAvIDE4LjhUIHNj YW5uZWQgYXQgOTYyTS9zLCA4LjcxVCAvIDE4LjhUIGlzc3VlZCBhdCA3MjZNL3M8YnI+DQomZ3Q7 wqAgwqAgwqAgwqAwQiByZXBhaXJlZCwgNDYuNDElIGRvbmUsIDA0OjAyOjAyIHRvIGdvPGJyPg0K Jmd0OyBjb25maWc6PGJyPg0KJmd0OyA8YnI+DQomZ3Q7wqAgwqAgwqAgwqBOQU1FwqAgwqAgwqAg wqAgU1RBVEXCoCDCoCDCoFJFQUQgV1JJVEUgQ0tTVU08YnI+DQomZ3Q7wqAgwqAgwqAgwqBkYXRh wqAgwqAgwqAgwqAgT05MSU5FwqAgwqAgwqAgwqAwwqAgwqAgwqAwwqAgwqAgwqAwPGJyPg0KJmd0 O8KgIMKgIMKgIMKgIMKgcmFpZHoyLTDCoCBPTkxJTkXCoCDCoCDCoCDCoDDCoCDCoCDCoDDCoCDC oCDCoDA8YnI+DQomZ3Q7wqAgwqAgwqAgwqAgwqAgwqBkYTRwMcKgIMKgT05MSU5FwqAgwqAgwqAg wqAwwqAgwqAgwqAwwqAgwqAgwqAwPGJyPg0KJmd0O8KgIMKgIMKgIMKgIMKgIMKgZGE2cDHCoCDC oE9OTElORcKgIMKgIMKgIMKgMMKgIMKgIMKgMMKgIMKgIMKgMDxicj4NCiZndDvCoCDCoCDCoCDC oCDCoCDCoGRhNXAxwqAgwqBPTkxJTkXCoCDCoCDCoCDCoDDCoCDCoCDCoDDCoCDCoCDCoDA8YnI+ DQomZ3Q7wqAgwqAgwqAgwqAgwqAgwqBkYTdwMcKgIMKgT05MSU5FwqAgwqAgwqAgwqAwwqAgwqAg wqAwwqAgwqAgwqAwPGJyPg0KJmd0O8KgIMKgIMKgIMKgIMKgIMKgZGExcDHCoCDCoE9OTElORcKg IMKgIMKgIMKgMMKgIMKgIMKgMMKgIMKgIMKgMDxicj4NCiZndDvCoCDCoCDCoCDCoCDCoCDCoGRh MHAxwqAgwqBPTkxJTkXCoCDCoCDCoCDCoDDCoCDCoCDCoDDCoCDCoCDCoDA8YnI+DQomZ3Q7wqAg wqAgwqAgwqAgwqAgwqBkYTNwMcKgIMKgT05MSU5FwqAgwqAgwqAgwqAwwqAgwqAgwqAwwqAgwqAg wqAwPGJyPg0KJmd0O8KgIMKgIMKgIMKgIMKgIMKgZGEycDHCoCDCoE9OTElORcKgIMKgIMKgIMKg MMKgIMKgIMKgMMKgIMKgIMKgMDxicj4NCiZndDvCoCDCoCDCoCDCoGxvZ3M8YnI+DQomZ3Q7wqAg wqAgwqAgwqAgwqBtaXJyb3ItMsKgIE9OTElORcKgIMKgIMKgIMKgMMKgIMKgIMKgMMKgIMKgIMKg MDxicj4NCiZndDvCoCDCoCDCoCDCoCDCoCDCoGFkYTBwNMKgIE9OTElORcKgIMKgIMKgIMKgMMKg IMKgIMKgMMKgIMKgIMKgMDxicj4NCiZndDvCoCDCoCDCoCDCoCDCoCDCoGFkYTFwNMKgIE9OTElO RcKgIMKgIMKgIMKgMMKgIMKgIMKgMMKgIMKgIMKgMDxicj4NCiZndDvCoCDCoCDCoCDCoGNhY2hl PGJyPg0KJmd0O8KgIMKgIMKgIMKgIMKgYWRhMXA1wqAgwqAgT05MSU5FwqAgwqAgwqAgwqAwwqAg wqAgwqAwwqAgwqAgwqAwPGJyPg0KJmd0O8KgIMKgIMKgIMKgIMKgYWRhMHA1wqAgwqAgT05MSU5F wqAgwqAgwqAgwqAwwqAgwqAgwqAwwqAgwqAgwqAwPGJyPg0KJmd0OyA8YnI+DQomZ3Q7IGVycm9y czogUGVybWFuZW50IGVycm9ycyBoYXZlIGJlZW4gZGV0ZWN0ZWQgaW4gdGhlIGZvbGxvd2luZyBm aWxlczo8YnI+DQomZ3Q7IDxicj4NCiZndDsgaSB0aGluayB0aGlzIGlzIGV4cGVjdGVkIHNpbmNl IC14IGp1c3QgZmlsdGVycyB0aGUgb3V0cHV0IHRvIHNob3cgcG9vbHMgd2l0aCBlcnJvcnM/PGJy Pg0KJmd0OyA8YnI+DQomZ3Q7wqAgwqAgwqAgwqAteMKgIMKgIMKgIE9ubHkgZGlzcGxheSBzdGF0 dXMgZm9yIHBvb2xzIHRoYXQgYXJlIGV4aGliaXRpbmcgZXJyb3JzIG9yIGFyZTxicj4NCiZndDvC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoG90aGVyd2lzZSB1bmF2YWlsYWJsZS7CoCBXYXJuaW5ncyBh Ym91dCBwb29scyBub3QgdXNpbmcgdGhlIGxhdGVzdDxicj4NCiZndDvCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoG9uLWRpc2sgZm9ybWF0IHdpbGwgbm90IGJlIGluY2x1ZGVkLjxicj4NCiZndDsgPGJy Pg0KJmd0O8KgIMKgIMKgIMKgdGhhbmtzLCBsZXhpLjxicj4NCjxicj4NCjwvYmxvY2txdW90ZT48 L2Rpdj48L2Rpdj4NCg== --000000000000601518060c7feacf--