Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 14 Dec 2023 21:17:06 +0000
From:      Lexi Winter <lexi@le-fay.org>
To:        "freebsd-fs@freebsd.org" <freebsd-fs@FreeBSD.org>
Subject:   unusual ZFS issue
Message-ID:  <787CB64A-1687-49C3-9063-2CE3B6F957EF@le-fay.org>

next in thread | raw e-mail | index | archive | help
hi list,

i=E2=80=99ve just hit this ZFS error:

# zfs list -rt snapshot data/vm/media/disk1
cannot iterate filesystems: I/O error
NAME                                                       USED  AVAIL  =
REFER  MOUNTPOINT
data/vm/media/disk1@autosnap_2023-12-13_12:00:00_hourly      0B      -  =
6.42G  -
data/vm/media/disk1@autosnap_2023-12-14_10:16:00_hourly      0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_11:17:00_hourly      0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_12:04:00_monthly     0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_12:15:00_hourly      0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_13:14:00_hourly      0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_14:38:00_hourly      0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_15:11:00_hourly      0B      -  =
6.46G  -
data/vm/media/disk1@autosnap_2023-12-14_17:12:00_hourly    316K      -  =
6.47G  -
data/vm/media/disk1@autosnap_2023-12-14_17:29:00_daily    2.70M      -  =
6.47G  -

the pool itself also reports an error:

# zpool status -v
  pool: data
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub in progress since Thu Dec 14 18:58:21 2023
	11.5T / 18.8T scanned at 1.46G/s, 6.25T / 18.8T issued at 809M/s
	0B repaired, 33.29% done, 04:30:20 to go
config:

	NAME        STATE     READ WRITE CKSUM
	data        ONLINE       0     0     0
	  raidz2-0  ONLINE       0     0     0
	    da4p1   ONLINE       0     0     0
	    da6p1   ONLINE       0     0     0
	    da5p1   ONLINE       0     0     0
	    da7p1   ONLINE       0     0     0
	    da1p1   ONLINE       0     0     0
	    da0p1   ONLINE       0     0     0
	    da3p1   ONLINE       0     0     0
	    da2p1   ONLINE       0     0     0
	logs
	  mirror-2  ONLINE       0     0     0
	    ada0p4  ONLINE       0     0     0
	    ada1p4  ONLINE       0     0     0
	cache
	  ada1p5    ONLINE       0     0     0
	  ada0p5    ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

(it doesn=E2=80=99t list any files, the output ends there.)

my assumption is that this indicates some sort of metadata corruption =
issue, but i can=E2=80=99t find anything that might have caused it.  =
none of the disks report any errors, and while all the disks are on the =
same SAS controller, i would have expected controller errors to be =
flagged as CKSUM errors.

my best guess is that this might be caused by a CPU or memory issue, but =
the system has ECC memory and hasn=E2=80=99t reported any issues.

- has anyone else encountered anything like this?

- i=E2=80=99m a bit worried that if i reboot, the system won=E2=80=99t =
be able to re-import the pool due to the =E2=80=9Ccannot iterate =
filesystems=E2=80=9D errors; is that a concern?

the system is running:

FreeBSD hemlock.eden.le-fay.org 14.0-RELEASE-p2 FreeBSD 14.0-RELEASE-p2 =
#4 releng/14.0-n265396-06497fbd52e2: Fri Dec  8 06:14:12 GMT 2023     =
root@hemlock.eden.le-fay.org:/data/src/obj/data/src/releng/14.0/amd64.amd6=
4/sys/HEMLOCK amd64

	thanks.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?787CB64A-1687-49C3-9063-2CE3B6F957EF>