From owner-freebsd-fs@freebsd.org Sat Apr 17 12:39:32 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9427B5D4A58 for ; Sat, 17 Apr 2021 12:39:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FMt1w3dBDz4fcY for ; Sat, 17 Apr 2021 12:39:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 7C7D15D4B6D; Sat, 17 Apr 2021 12:39:32 +0000 (UTC) Delivered-To: fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7C4605D48E9 for ; Sat, 17 Apr 2021 12:39:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FMt1w339Rz4ffk for ; Sat, 17 Apr 2021 12:39:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5B6861815D for ; Sat, 17 Apr 2021 12:39:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 13HCdWZt095532 for ; Sat, 17 Apr 2021 12:39:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 13HCdWQW095531 for fs@FreeBSD.org; Sat, 17 Apr 2021 12:39:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 249985] kernel panic at shutdown in zfs_acl_free() and list_remove() Date: Sat, 17 Apr 2021 12:39:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: madpilot@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Apr 2021 12:39:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D249985 --- Comment #6 from Guido Falsi --- By chance I discovered something interesting. Now the machine is regularly down this: # zfs list -t snapshot internal error: cannot iterate filesystems: Invalid argument Abort (core dumped) (backtrace at the end of this comment, but I don't think this one is interesting) I tracked this down to a single snapshot that looks corrupted, if I try to analyze it with zfs zfs crashes,=20 If I try to destroy that snashot with: zfs destroy zroot/var/mail@2021-03-14_18.00.00--1w I cause a kernel panic, backtrace also at end of message. What I gather from this panic is that the openzfs code is returning EINVAL = at=20 I cause a kernel panic, backtrace also at end of message. I don't know enou= gh about ZFS to understand more than this, unluckily. Some more information: > uname -a FreeBSD ubik.madpilot.net 14.0-CURRENT FreeBSD 14.0-CURRENT main-n246069-112f007e128 MPNET amd64 The machine is an acer laptop, the disk is an nvd(4) device, and I'm runnin= g it eli encrypted, the disk layout was created by the installer when 13 was sti= ll current. I'm actually curious if there is a way to recover from this condition. I'll= try experimenting with zdb to see if I can gather some details about why this snapshot causes a crash. ----- zfs.core backtrace: #0 0x00000008015dd4ba in thr_kill () from /lib/libc.so.7 #1 0x0000000801552de4 in raise () from /lib/libc.so.7 #2 0x0000000801606dc9 in abort () from /lib/libc.so.7 #3 0x000000080112e75e in zfs_standard_error_fmt () from /lib/libzfs.so.4 #4 0x000000080112e2b5 in zfs_standard_error () from /lib/libzfs.so.4 #5 0x00000008011175a3 in zfs_iter_snapshots () from /lib/libzfs.so.4 #6 0x0000000001031182 in ?? () #7 0x00000008011172c2 in zfs_iter_filesystems () from /lib/libzfs.so.4 #8 0x000000000103114d in ?? () #9 0x00000008011172c2 in zfs_iter_filesystems () from /lib/libzfs.so.4 #10 0x000000000103114d in ?? () #11 0x00000008011092f9 in zfs_iter_root () from /lib/libzfs.so.4 #12 0x0000000001030968 in ?? () #13 0x000000000103454c in ?? () #14 0x000000000103145e in ?? () #15 0x00000000010303df in ?? () #16 0x0000000001030300 in ?? () #17 0x0000000000000000 in ?? () ----- kernel panic backtrace panic: VERIFY3(0 =3D=3D dsl_dataset_hold_obj(dp, dsl_dataset_phys(ds_next)->ds_next_snap_obj, FTAG, &ds_nextnext)) failed (0= =3D=3D 22) cpuid =3D 7 time =3D 1618658184 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00b55e2= 2b0 vpanic() at vpanic+0x181/frame 0xfffffe00b55e2300 spl_panic() at spl_panic+0x3a/frame 0xfffffe00b55e2360 dsl_destroy_snapshot_sync_impl() at dsl_destroy_snapshot_sync_impl+0xbf6/fr= ame 0xfffffe00b55e2440 dsl_destroy_snapshot_sync() at dsl_destroy_snapshot_sync+0x4e/frame 0xfffffe00b55e2480 zcp_synctask_destroy() at zcp_synctask_destroy+0xb0/frame 0xfffffe00b55e24c0 zcp_synctask_wrapper() at zcp_synctask_wrapper+0xee/frame 0xfffffe00b55e2510 luaD_precall() at luaD_precall+0x25f/frame 0xfffffe00b55e25e0 luaV_execute() at luaV_execute+0xf88/frame 0xfffffe00b55e2660 luaD_call() at luaD_call+0x1b3/frame 0xfffffe00b55e26a0 luaD_rawrunprotected() at luaD_rawrunprotected+0x53/frame 0xfffffe00b55e2740 luaD_pcall() at luaD_pcall+0x37/frame 0xfffffe00b55e2790 lua_pcallk() at lua_pcallk+0xa6/frame 0xfffffe00b55e27d0 zcp_eval_impl() at zcp_eval_impl+0xbc/frame 0xfffffe00b55e2800 dsl_sync_task_sync() at dsl_sync_task_sync+0xb4/frame 0xfffffe00b55e2830 dsl_pool_sync() at dsl_pool_sync+0x43b/frame 0xfffffe00b55e28b0 spa_sync() at spa_sync+0xafe/frame 0xfffffe00b55e2ae0 txg_sync_thread() at txg_sync_thread+0x3b3/frame 0xfffffe00b55e2bb0 fork_exit() at fork_exit+0x7d/frame 0xfffffe00b55e2bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00b55e2bf0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- KDB: enter: panic --=20 You are receiving this mail because: You are the assignee for the bug.=