From nobody Fri Aug 15 23:51:00 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4c3f5H0l1tz64TgW for ; Fri, 15 Aug 2025 23:51:15 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4c3f5G3KXSz41Mp for ; Fri, 15 Aug 2025 23:51:14 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-244582766a8so3829875ad.3 for ; Fri, 15 Aug 2025 16:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay.org; s=google; t=1755301872; x=1755906672; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=tAaAD9MUAXdEnC4BYDlUjZxj09EIQ9OLdq4Wla5QPQ4=; b=golquTNpBXuCUuCGdt9zZK0qxrgq19KttwMwjKEgZ9XoOFi75IbtUpWp4vzb3mzmYg My1BEg//QBTIbKVe2vPNXMy/NIEI9r+UV/KUpiZrYu8YFSoJrsMN0+osRvn74IkThPo1 LcpwY2++P9GgKcrY7zCZtrYRwTVusMJL+6M+g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755301872; x=1755906672; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tAaAD9MUAXdEnC4BYDlUjZxj09EIQ9OLdq4Wla5QPQ4=; b=qd0dsc917UXyaM+QiIB0Tr+QSfTUxswWstHBzURVfRQjNNdI/InxXuKUCOlBxNsrf7 hv8el9J156f2KNTViyuAql6UCYg5awT73WN0wLN0WlfsvcUi3hEO5Xn2AM/FJevhgz+f e5bF5ijHhpM+6ROQcqATRfAoKCWU1obCiAj0QcJ+mZjmgMpbU09d5Q9ZAa1x7LG6/x8c Bn3ZYtGlJoitv1E4thjVKcVVgHAN7XrYIgPjTrANbErAGMUQXJ5jgYO7WFvPbVxu+l1P VwwgdiIJLRL+M5rfPM9FL6Fx6sylltEugsy9+RApR4Z0naZ9VL8B5e5isq3eQyJPIJ7r w+aw== X-Gm-Message-State: AOJu0Yw8PiHx5NxsWM9KRkpbHtyHXXDcvg+sMU6axcpLy9Ty0fn6dGt7 VsOpa42uhxID74e1bzp+2JCtm8Dx2kB9UFdRNR1Fknb2xI+SrwMimnCyF6urqmGXLw== X-Gm-Gg: ASbGncsoDsXzGMfBgG1hc28UEXBTk+fh7J8uoumPjFyLn/zNJJXzysul+xxIvF9M0M5 YEgSY5sm5URJ0W8eU7mHVftZAftkuzdYx/i536hvQ+CseOfVfrqkXGyCoqC23RX8hekdCc8Rwjg hyh/JcyX83pdLE3aCw0vSl/9z8iLie3VNUQ2FXbV4v0GNJp+dzpD3TjV8W/uBhV06LS41wyYJrL zX3Wnot3FKePL9bgZE62jrn5LYOJrM4bGf8MBowZYd5NdSuZIF4kTzO+Wb/qg8u+atGAxP5T+qu IRE/sNf9TCK/64F8q6UkCYJZniUtDSOuMTXgEE0raXeuWX4suRU/Z0HvscQQ7b4/Yl5Zoc3Xtex EsR51SQbyz8aM90kKfjViz06JKHgVKxhh53i/h3E8c0oDhZLqPtoY66b9s35dcBJo/92UyAu2+o DsmC0ou4t+DRBJHxJwnPSE+RdixSRoaLVnKvVMF9Fn X-Google-Smtp-Source: AGHT+IHDYN1jRM53bqjqYrrwiEoYjJE6LrV3hiT49ZaBPivkd/zXRPKcBQoH0NJoGyBsSi2dN5og5g== X-Received: by 2002:a17:903:41c8:b0:240:58a7:892f with SMTP id d9443c01a7336-2446d82d032mr26400165ad.5.1755301871926; Fri, 15 Aug 2025 16:51:11 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2446d58ab11sm22849575ad.159.2025.08.15.16.51.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Aug 2025 16:51:11 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: zfs related panic From: Bakul Shah In-Reply-To: Date: Fri, 15 Aug 2025 16:51:00 -0700 Cc: FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: References: <14C5523D-1F66-434F-A4D9-E14DA4BBF1E9@iitbombay.org> To: Konstantin Belousov X-Mailer: Apple Mail (2.3826.700.81) X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4c3f5G3KXSz41Mp On Aug 15, 2025, at 3:51=E2=80=AFPM, Konstantin Belousov = wrote: >=20 > On Fri, Aug 15, 2025 at 11:19:55AM -0700, Bakul Shah wrote: >> Is this a known bug or may be something specific on my machine? >> If the latter, any way to "fsck" it? FYI, the zpool is a mirror >> (two files on the host via nvme). built from c992ac621327 commit hash >> (which has other issues but they seem to be separate from this). >> I saw the same panic when I booted from a day old snapshot. >>=20 >> Note that "ls /.zfs" panics but "ls /.zfs/snapshot" doesn't! >>=20 >> This is on a -current VM: >>=20 >> root@:/ # ls .zfs >> VNASSERT failed: oresid =3D=3D 0 || nresid !=3D oresid || = *(a)->a_eofflag =3D=3D 1 not true at vnode_if.c:1824 (VOP_READDIR_APV) >=20 > Try this, untested. Thanks for the quick patch! But I am afraid it didn't help. Let me know = if you want me to check things via gdb. [I have filed https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D288889 so we can continue debugging there] On the console (single user, RO root): # ls /.zfs VNASSERT failed: oresid =3D=3D 0 || nresid !=3D oresid || = *(a)->a_eofflag =3D=3D 1 not true at vnode_if.c:1824 (VOP_READDIR_APV) 0xfffff800059546e0: type VDIR state VSTATE_CONSTRUCTED op = 0xffffffff8272cfd0 usecount 1, writecount 0, refcount 1 seqc users 0 mountedhere 0 hold count flags () flags () lock type zfs: SHARED (count 1) name =3D .zfs parent_id =3D 0 id =3D 1 panic: VOP_READDIR: eofflag not set cpuid =3D 0 time =3D 1755276357 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe0053f83af0 vpanic() at vpanic+0x136/frame 0xfffffe0053f83c20 panic() at panic+0x43/frame 0xfffffe0053f83c80 VOP_READDIR_APV() at VOP_READDIR_APV+0x205/frame 0xfffffe0053f83cd0 kern_getdirentries() at kern_getdirentries+0x228/frame = 0xfffffe0053f83dd0 sys_getdirentries() at sys_getdirentries+0x29/frame 0xfffffe0053f83e00 amd64_syscall() at amd64_syscall+0x169/frame 0xfffffe0053f83f30 fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe0053f83f30 --- syscall (554, FreeBSD ELF64, getdirentries), rip =3D 0x331339f976aa, = rsp =3D 0x33133631ade8, rbp =3D 0x33133631ae20 --- KDB: enter: panic [ thread pid 23 tid 100211 ] Stopped at kdb_enter+0x33: movq $0,0x12313e2(%rip) db> Running gdb on the host (attached to tcp port): #16 0xffffffff80b7992b in vpanic ( fmt=3D0xffffffff812ddf30 "VOP_READDIR: eofflag not set", ap=3Dap@entry=3D0xfffffe0053f83c60) at /home/FreeBSD/current/sys/kern/kern_shutdown.c:962 #17 0xffffffff80b79793 in panic ( fmt=3D0xffffffff81d9eab0 = "\304\372\032\201\377\377\377\377") at /home/FreeBSD/current/sys/kern/kern_shutdown.c:887 #18 0xffffffff81195fd5 in VOP_READDIR_APV (vop=3D, a=3Da@entry=3D0xfffffe0053f83d30) at vnode_if.c:1824 #19 0xffffffff80c95e58 in VOP_READDIR (vp=3D0xfffff800059546e0, uio=3D0xfffffe0053f83d00, cred=3D, = eofflag=3D0xfffffe0053f83d6c, ncookies=3D0x0, cookies=3D0x0) at ./vnode_if.h:972 #20 kern_getdirentries (td=3D0xfffff8007e7c3780, fd=3D, buf=3D0x4ea30d020000 "\001", count=3D4096, basep=3Dbasep@entry=3D0xfffffe0053f83df0, residp=3Dresidp@entry=3D0x0,= bufseg=3DUIO_USERSPACE) at = /home/FreeBSD/current/sys/kern/vfs_syscalls.c:4353 #21 0xffffffff80c96289 in sys_getdirentries ( td=3D0xffffffff81d9eab0 , uap=3D0xfffff8007e7c3ba8) at /home/FreeBSD/current/sys/kern/vfs_syscalls.c:4287 #22 0xffffffff810ca8b9 in syscallenter (td=3D0xfffff8007e7c3780) at = /home/FreeBSD/current/sys/amd64/amd64/../../kern/subr_syscall.c:193 #23 amd64_syscall (td=3D0xfffff8007e7c3780, traced=3D0) at /home/FreeBSD/current/sys/amd64/amd64/trap.c:1208 >=20 > commit a97fc29bf2c03bbfc57b9c188ab3b24450d453bc > Author: Konstantin Belousov > Date: Sat Aug 16 01:50:42 2025 +0300 >=20 > zfs control dir: properly set eof >=20 > diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_ctldir.c = b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_ctldir.c > index 61d0bb26d1e5..725c02d47edf 100644 > --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_ctldir.c > +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_ctldir.c > @@ -1056,17 +1056,22 @@ zfsctl_snapdir_readdir(struct vop_readdir_args = *ap) > zfs_uio_t uio; > int *eofp =3D ap->a_eofflag; > off_t dots_offset; > + offset_t orig_resid; > int error; >=20 > zfs_uio_init(&uio, ap->a_uio); > + orig_resid =3D uio.uio->uio_resid; >=20 > ASSERT3S(vp->v_type, =3D=3D, VDIR); >=20 > error =3D sfs_readdir_common(ZFSCTL_INO_ROOT, ZFSCTL_INO_SNAPDIR, ap, > &uio, &dots_offset); > if (error !=3D 0) { > - if (error =3D=3D ENAMETOOLONG) /* ran out of destination space */ > + if (error =3D=3D ENAMETOOLONG) { /* ran out of destination space */ > error =3D 0; > + if (orig_resid =3D=3D uio.uio->uio_resid && eofp !=3D NULL) > + *eofp =3D 1; > + } > return (error); > } >=20 > @@ -1084,7 +1089,8 @@ zfsctl_snapdir_readdir(struct vop_readdir_args = *ap) > dsl_pool_config_exit(dmu_objset_pool(zfsvfs->z_os), FTAG); > if (error !=3D 0) { > if (error =3D=3D ENOENT) { > - if (eofp !=3D NULL) > + if (orig_resid =3D=3D uio.uio->uio_resid && > + eofp !=3D NULL) > *eofp =3D 1; > error =3D 0; > } > @@ -1099,8 +1105,12 @@ zfsctl_snapdir_readdir(struct vop_readdir_args = *ap) > entry.d_reclen =3D sizeof (entry); > error =3D vfs_read_dirent(ap, &entry, zfs_uio_offset(&uio)); > if (error !=3D 0) { > - if (error =3D=3D ENAMETOOLONG) > + if (error =3D=3D ENAMETOOLONG) { > error =3D 0; > + if (orig_resid =3D=3D uio.uio->uio_resid && > + eofp !=3D NULL) > + *eofp =3D 1; > + } > zfs_exit(zfsvfs, FTAG); > return (SET_ERROR(error)); > }