From owner-freebsd-fs@freebsd.org Mon Jan 4 01:11:29 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 614C54C6D91 for ; Mon, 4 Jan 2021 01:11:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670046.outbound.protection.outlook.com [40.107.67.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8HdX3KnYz3JT1 for ; Mon, 4 Jan 2021 01:11:28 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F3TgMHTz1P3jZBjt4a56X2TQBF5FeuUljLHe651KsPuBFQtGfJcoLIckK0Cu+nUbCs8fdlK1Pm6ydY05WFOz/87XMmQtpfNV697ygoh+ADkr7PP6ZMkBu7BwByO4t9D57inc1CRJmTh02O9RahC0tREPKIseqtNkuLaaogFEGAA1KrlaMWOpwQlpmAkOh12bPV6yOVIYM0stR4RJ9jCN35jph7glSDpA4wQRPQMxdf3gUQ5P9jZwKQLwIisFmRrjd4hK7O5uy15eBgZ9JZWu39ZqhIzXmjajxjGKtNgVlPEoVioxFji86Q/e+HNSJrlWWROPwfs7DPXp/b8UL1SdxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xws5ZYt7HEF3V0eSqDcZ3UB8oRWuqOSYXlNGYS4fL/w=; b=L6PEd7WCbJ+xnVJ47zbcydLwp2IKwQnOXyqyXBxp50ip7EgF5dwL2R+NSBTS8x9Ts+bx3AYAUXKZsqNW8pm+rGBp7i9aY0dOj3u6u5xqEYuDfFahLykdeyCdy0sne++UCav1/5l6a6uIaICjWQtex+bjxU+pT0HRytaGf+zEEysZslVUU18Y5/LU+j/cSYFujAuX7u5lIQJfQAcfmQWooz/vy46LK/PE8cqTeU4YZJwj/hC1y4IDxefCHOv+7Y8kuZBeA4gCmolZjfHPgdhUvEMaBjji1x+uwm7uuS2sxBRbjS5PMyOyyZW3alLHRNj/FgH7pOjIFbiHz6Lf6cKV3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xws5ZYt7HEF3V0eSqDcZ3UB8oRWuqOSYXlNGYS4fL/w=; b=sK2VdFdw7f7jY5CptyUkN7VgWE/dzdItjo1nwPbLDB9yhHTFWDThg9CUMjuC4zB/+7oNtRrghB1AFJL/+64bBm6GGTTDXqJ/3YTKcpEY7RK8+RFUzg6KxSNpsZlrdef2EGOqgeQyHB1LjNM2O6ZQPMmz4/yX1tQnG2QDpOEcm2c/G5gCiDygeZMie9Q2jQlHkriiGz3bc0mUbs93uKaFTCHP7X1B7172w0KGrRVgPTfBZjPWyzHx69nu0f0qaMLydM98GQFMlWAmhSxY+HYQeI2+f6uxVV81HMHFkF15fKCcyFU+dL2X9EL4H1DCT75qcRP690cmF0yOgGj7r6YBxg== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR0101MB1478.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:17::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3721.23; Mon, 4 Jan 2021 01:11:26 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::3d86:c7f9:bc4c:40c0]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::3d86:c7f9:bc4c:40c0%6]) with mapi id 15.20.3721.024; Mon, 4 Jan 2021 01:11:26 +0000 From: Rick Macklem To: Konstantin Belousov CC: J David , "freebsd-fs@freebsd.org" Subject: Re: Major issues with nfsv4 Thread-Topic: Major issues with nfsv4 Thread-Index: AQHWzw/HDat+dHoH9kKG5K3Xpd53kqnxDteQgAFi0QCAABTa84AALLCAgAAVvcmAAAu0AIAA4wiAgAAI5gCAAO/IAIAAiDrmgABWOgCAAGv4AIAEv9nSgA9Z2ruAAR1dAIALkqgC Date: Mon, 4 Jan 2021 01:11:26 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 49bdf073-d853-4219-cfff-08d8b04da910 x-ms-traffictypediagnostic: YQXPR0101MB1478: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: uiGiQs1mPkqnN9Kh6Ce77mkGjR1ZqCAthyB8oCKvBnU2MdDEqtLP1Utr45XzUFfO2WmntGsykKFMhhv8CTBl1me0ihR4K+HlVSmVYtieCTT6jSJQ61pB3MApFTEkj8GUSpBBaBWjziXJQM3DmLn/m/HF89YfGZ/bQvWjCrem/BTezQ2TyQ4gYTT/XBaEJcpIIITgxNBYLozrqyT/cqXp4E0C6QwoId944+xFWFMngXR606d6PGjgmF0pF1RrucQC4ocFJFkVPGjkIJNJjqSVGeIX001lQKt426hY1XWwRW4bDVBTiPIqAnmPtFzZ/Nycwpxryg1tc4yD98+o1eXYl9I4Tec7sP4IN0NVA3gkjL8fynIx/UV/IAp4ffoCoh000BQ9h8Iaw/n0WONWKNDfEA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(396003)(39850400004)(376002)(366004)(136003)(346002)(9686003)(8936002)(66946007)(86362001)(2906002)(66476007)(91956017)(33656002)(76116006)(52536014)(478600001)(8676002)(5660300002)(6506007)(26005)(186003)(54906003)(316002)(786003)(55016002)(83380400001)(64756008)(66556008)(66446008)(4326008)(71200400001)(7696005)(6916009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?Ew4z3XsSh/NU+SQBdvNXTHeZjJF+51fbfQqSeVoKNiPgL527I+b/UDrh6z?= =?iso-8859-1?Q?/dYWoFhpaX5c1+jYcNj5++bFKHe2F2kyVtTSytA5Bz5LjUYQToDksl/4/z?= =?iso-8859-1?Q?0UzkoKF3C1cAJCgyNFnbXM/3qglH8nDBGw86rJ5nlS5Ebq4Ve0Yz3HgZeR?= =?iso-8859-1?Q?YrAC09c4i3gv55wVmIwHAr86oqM1cu/CNPWdnW5SDP/WEAvW/5MeQ2vQY7?= =?iso-8859-1?Q?TpINKwDlJHjB15KlMBX0mQj1v8gYXAvpNut5TS+pZVGyyTdqEVZqrOq9Pa?= =?iso-8859-1?Q?4EwQQfp7idus/Ug7qKDIpcL8nQeIKrFFZwYNn1BQO/G/xf5H0eMMUmvbJn?= =?iso-8859-1?Q?jWaRgImU89nMGDvSpCikdvxFeNa+8Kl7M0930Ljs33SKti09NuPJh35Osk?= =?iso-8859-1?Q?NYV9briJKn7bl+6l5X7nQKWxKHlWeIg/dGXVnqylW2DYN0iaZrkdmQLB7M?= =?iso-8859-1?Q?YRbBaBkkRmiuxKFjxhyFEwBgOx3lu0OS0oDJ3PRXN96h+6vCKMWxxTTVAz?= =?iso-8859-1?Q?c0aDNQTOakjP4jj2voQFMpRWlqMjyI4+CuP24+/EC5yz83UTtolLIofp2X?= =?iso-8859-1?Q?+loLJGdMJ0rkgIcHOXn6Z7PGkZK6IvJXzDi4R2svYiWB7y+VWIMNgAx6Kb?= =?iso-8859-1?Q?gK/u8WIPBQl1Csp/GUPscwIHCA0k00XwOTBcSSdlYpBPOzjVt54Ek3DLe1?= =?iso-8859-1?Q?neuSw8XKu4OMoqGkwhKjvXRhMaU9k+1RhvGZZiVfb0OtPuvS+T4TYr2qMy?= =?iso-8859-1?Q?87PHM9s9fjKpbo9rqSp7usqqKAFe0EqEkmCmbWNag5Dwe6Qu5IwMAJdpmv?= =?iso-8859-1?Q?T+8oaUzdKyHLQ/3uecx+wupAOcCmq8Si9OEnYLHJLbMMjITFgC7S/j0UAK?= =?iso-8859-1?Q?N6pGJUG2chp4TS8F9cIW6393VhtKdCQeRgpciylUePbQW12DBAWB71a+SX?= =?iso-8859-1?Q?zE/WapD+9zBPUHqvQbPBPXz3adLHFAkmRn68ubGQDA8WH5VHbCl8Ci7zdo?= =?iso-8859-1?Q?KZr1XmGATVmmdu8YU=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 49bdf073-d853-4219-cfff-08d8b04da910 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jan 2021 01:11:26.7157 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: WbR9eUVVIBwhc7ldF4aRwyyhtHwpD8qXR2kscuXSkvX3zkOIxydw9IofXIS3MqSuC5HNsR5fy/wYDfSIytwlPg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB1478 X-Rspamd-Queue-Id: 4D8HdX3KnYz3JT1 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=sK2VdFdw; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.46 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.46:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.46:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[40.107.67.46:from:127.0.2.255]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.46:from]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2021 01:11:29 -0000 Kostik wrote:=0A= >Rick Macklem wrote:=0A= [stuff snipped]=0A= >> I see vfs_deferred_inactive() in sys/kern/vfs_subr.c, but I do not=0A= >> know when/how it gets called?=0A= >Right, vfs_deferred_inactive() is one way which tries to handle missed=0A= >inactivations. If upon vput() the lock is only shared and upgrade=0A= >failed, vnode is marked as VI_OWEINACT and put onto 'lazy' list,=0A= >processed by vfs_sync(MNT_LAZY). It is typically called from syncer,=0A= >which means each 60 secs. There, if the vnode is still unreferenced, it=0A= >is inactivated.=0A= If I read the code correctly vfs_deferred_inactive() gets called with=0A= LK_EXCLUSIVE | LK_NOWAIT | LK_INTERLOCK.=0A= Then there is this code in nfs_lock():=0A= vp =3D ap->a_vp;=0A= 332 lktype =3D ap->a_flags & LK_TYPE_MASK;=0A= 333 error =3D VOP_LOCK1_APV(&default_vnodeops, ap);=0A= 334 if (error !=3D 0 || vp->v_op !=3D &newnfs_vnodeops)=0A= 335 return (error);=0A= 336 np =3D VTONFS(vp);=0A= 337 if (np =3D=3D NULL)=0A= 338 return (0);=0A= 339 NFSLOCKNODE(np);=0A= 340 if ((np->n_flag & NVNSETSZSKIP) =3D=3D 0 || (lktype !=3D LK_SHA= RED &&=0A= 341 lktype !=3D LK_EXCLUSIVE && lktype !=3D LK_UPGRADE &&=0A= 342 lktype !=3D LK_TRYUPGRADE)) {=0A= 343 NFSUNLOCKNODE(np);=0A= 344 return (0);=0A= 345 }=0A= 346 onfault =3D (ap->a_flags & LK_EATTR_MASK) =3D=3D LK_NOWAIT &&= =0A= 347 (ap->a_flags & LK_INIT_MASK) =3D=3D LK_CANRECURSE &&=0A= 348 (lktype =3D=3D LK_SHARED || lktype =3D=3D LK_EXCLUSIVE);=0A= 349 if (onfault && vp->v_vnlock->lk_recurse =3D=3D 0) {=0A= 350 /*=0A= 351 * Force retry in vm_fault(), to make the lock request= =0A= 352 * sleepable, which allows us to piggy-back the=0A= 353 * sleepable call to vnode_pager_setsize().=0A= 354 */=0A= 355 NFSUNLOCKNODE(np);=0A= 356 VOP_UNLOCK(vp);=0A= 357 return (EBUSY);=0A= 358 }=0A= Would the above code result in an EBUSY reply when the vn_lock(LK_NOWAIT |= =0A= LK_EXCLUSIVE) is made in vfs_deferred_inactive()?=0A= =0A= If so, it looks like vfs_periodic_inactive()->vfs_deferred_inactive() will = just=0A= call vdefer_inactive_unlocked()->vdefer_inactive() and it will end up=0A= on the lazy list with VI_DEFINACT set again, if I am reading the=0A= code correctly?=0A= =0A= >Another place where inactivation can occur is reclamation. There in=0A= >vgonel(), we call VOP_INACTIVE() if VI_OWEINACT is set. In principle,=0A= >this is redundand because correct filesystem must do the same cleanup=0A= >(and more) at reclamation as at the inactivation. But we also call=0A= >VOP_CLOSE(FNONBLOCK) before VOP_RECLAIM().=0A= And this would make all the NFSv4 Opens go away (get closed)=0A= when the nullfs mounts are unmounted, as reported by J. David.=0A= =0A= >Looking at this from another angle, if inactivation for NFSv4 vnodes=0A= >is not called longer than 2 minutes, perhaps there is a reference leak.=0A= >It is not due to VFS forgetting about due VOP_INACTIVE() call.=0A= Unless the nfs_lock(LK_NOWAIT) call keeps failing with EBUSY,=0A= I think?=0A= =0A= rick=0A= ps: I need to find a way to reproduce this.=0A= =0A= From owner-freebsd-fs@freebsd.org Mon Jan 4 08:56:55 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE7D94CF786 for ; Mon, 4 Jan 2021 08:56:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8TyZ68nSz4RJn for ; Mon, 4 Jan 2021 08:56:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1048ukER099201 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 4 Jan 2021 10:56:49 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1048ukER099201 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1048ukV0099200; Mon, 4 Jan 2021 10:56:46 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 4 Jan 2021 10:56:46 +0200 From: Konstantin Belousov To: Rick Macklem Cc: J David , "freebsd-fs@freebsd.org" Subject: Re: Major issues with nfsv4 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4D8TyZ68nSz4RJn X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2021 08:56:55 -0000 On Mon, Jan 04, 2021 at 01:11:26AM +0000, Rick Macklem wrote: > Kostik wrote: > >Rick Macklem wrote: > [stuff snipped] > >> I see vfs_deferred_inactive() in sys/kern/vfs_subr.c, but I do not > >> know when/how it gets called? > >Right, vfs_deferred_inactive() is one way which tries to handle missed > >inactivations. If upon vput() the lock is only shared and upgrade > >failed, vnode is marked as VI_OWEINACT and put onto 'lazy' list, > >processed by vfs_sync(MNT_LAZY). It is typically called from syncer, > >which means each 60 secs. There, if the vnode is still unreferenced, it > >is inactivated. > If I read the code correctly vfs_deferred_inactive() gets called with > LK_EXCLUSIVE | LK_NOWAIT | LK_INTERLOCK. > Then there is this code in nfs_lock(): > vp = ap->a_vp; > 332 lktype = ap->a_flags & LK_TYPE_MASK; > 333 error = VOP_LOCK1_APV(&default_vnodeops, ap); > 334 if (error != 0 || vp->v_op != &newnfs_vnodeops) > 335 return (error); > 336 np = VTONFS(vp); > 337 if (np == NULL) > 338 return (0); > 339 NFSLOCKNODE(np); > 340 if ((np->n_flag & NVNSETSZSKIP) == 0 || (lktype != LK_SHARED && > 341 lktype != LK_EXCLUSIVE && lktype != LK_UPGRADE && > 342 lktype != LK_TRYUPGRADE)) { > 343 NFSUNLOCKNODE(np); > 344 return (0); > 345 } > 346 onfault = (ap->a_flags & LK_EATTR_MASK) == LK_NOWAIT && > 347 (ap->a_flags & LK_INIT_MASK) == LK_CANRECURSE && > 348 (lktype == LK_SHARED || lktype == LK_EXCLUSIVE); > 349 if (onfault && vp->v_vnlock->lk_recurse == 0) { > 350 /* > 351 * Force retry in vm_fault(), to make the lock request > 352 * sleepable, which allows us to piggy-back the > 353 * sleepable call to vnode_pager_setsize(). > 354 */ > 355 NFSUNLOCKNODE(np); > 356 VOP_UNLOCK(vp); > 357 return (EBUSY); > 358 } > Would the above code result in an EBUSY reply when the vn_lock(LK_NOWAIT | > LK_EXCLUSIVE) is made in vfs_deferred_inactive()? I do not think so, there is no LK_CANRECURSE specified by vn_lock(). > > If so, it looks like vfs_periodic_inactive()->vfs_deferred_inactive() will just > call vdefer_inactive_unlocked()->vdefer_inactive() and it will end up > on the lazy list with VI_DEFINACT set again, if I am reading the > code correctly? If there is lock conflict then yes. Otherwise, I do not think so. > > >Another place where inactivation can occur is reclamation. There in > >vgonel(), we call VOP_INACTIVE() if VI_OWEINACT is set. In principle, > >this is redundand because correct filesystem must do the same cleanup > >(and more) at reclamation as at the inactivation. But we also call > >VOP_CLOSE(FNONBLOCK) before VOP_RECLAIM(). > And this would make all the NFSv4 Opens go away (get closed) > when the nullfs mounts are unmounted, as reported by J. David. > > >Looking at this from another angle, if inactivation for NFSv4 vnodes > >is not called longer than 2 minutes, perhaps there is a reference leak. > >It is not due to VFS forgetting about due VOP_INACTIVE() call. > Unless the nfs_lock(LK_NOWAIT) call keeps failing with EBUSY, > I think? > > rick > ps: I need to find a way to reproduce this. > From owner-freebsd-fs@freebsd.org Mon Jan 4 12:33:25 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 78D624D4966; Mon, 4 Jan 2021 12:33:25 +0000 (UTC) (envelope-from ali.abdallah@suse.com) Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp2.suse.de", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8ZmN5zBKz4fN0; Mon, 4 Jan 2021 12:33:24 +0000 (UTC) (envelope-from ali.abdallah@suse.com) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id AD724AD1E; Mon, 4 Jan 2021 12:33:22 +0000 (UTC) Date: Mon, 4 Jan 2021 13:33:21 +0100 From: Ali Abdallah To: techyNotes Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Consistency of pkg db on UFS Message-ID: <20210104123321.nvjambhslmvzmiac@Fryzen495> References: <20201211165713.syvzamtdtrbrgx44@frix230> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4D8ZmN5zBKz4fN0 X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.20 / 15.00]; MID_RHS_NOT_FQDN(0.50)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[suse.com:s=susede1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.135.220.15]; MIME_GOOD(-0.10)[text/plain]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DWL_DNSWL_MED(-2.00)[suse.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[195.135.220.15:from]; DKIM_TRACE(0.00)[suse.com:+]; DMARC_POLICY_ALLOW(-0.50)[suse.com,quarantine]; RWL_MAILSPIKE_GOOD(0.00)[195.135.220.15:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:29298, ipnet:195.135.220.0/22, country:DE]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-fs,freebsd-stable] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2021 12:33:25 -0000 On 12.12.2020 09:44, techyNotes wrote: > Hi Ali Hi > > I had a similar problem with my HP Elitebook couple of days ago with UFS option. I had been experimenting with various modes and setup options including the file system ZFS and UFS. I would have installed and reinstalled Freebsd 12.2 more than 20 times! > > I guess the issue lies in the Installation process. Just before the final step of rebooting it takes sometime (more than when installing with ZFS option) to complete the installation closure tasks. This was not completed or you did not wait until the point the message popped up to reboot the system! > > To verify if this is the case, check your loader.conf and/or rc.conf it would be empty. The options you selected during the install process would not be recorded in these files. > > SOLUTION: I just reinstalled again with the option of UFS and waited on the last before step patiently and then finally rebooted upon the alter message of REBOOT option from the installer. I'm no speaking about the installer, but an already installed system. I don't believe there is a solution to the described issue, the blocks of a newly installed package can make it to the disk and to the fs metadata, but the information about it written by the package manager to its database/plain file might not make it to the disk in case of power failure/panic, so you would end up with an installed package that the package manager itself doesn't know about! The issue can happen on any filesystem, not only on UFS. Regards, Ali From owner-freebsd-fs@freebsd.org Wed Jan 6 15:54:34 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 45A684D4D32 for ; Wed, 6 Jan 2021 15:54:34 +0000 (UTC) (envelope-from jdelisle@gmail.com) Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9v7Y2Kcbz4Tdx for ; Wed, 6 Jan 2021 15:54:33 +0000 (UTC) (envelope-from jdelisle@gmail.com) Received: by mail-pl1-x635.google.com with SMTP id q4so1740084plr.7 for ; Wed, 06 Jan 2021 07:54:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=JZvcNogseCbtdBa6HYzH67uBWHDdeHRiagnPA6xxWNM=; b=J9O+dZaCHuY8EOrWRIDd4+GleVwNoOHOPPGNtADdvOpEFAOG7t+1cG3ELph1oI5jQi F/gl8IzKymCC6qycUYESeR787MQU9BQcSeRdAObvvQG/TwOoR2qFmu3dHoFn9HDFkgYc sXO4Gtsy9J6JTgDk8+CI6AM27+fKWLJC+1mysXxjCvP5zlo0uCNXAvQXrjknki7qQo6W aDHVW9Y+zKzhGMrwjxUQ3weyUeZYTnULquZGnvUK/kRoEdqSyleKlGf9HOu5pQZ1mAQK lr7SCbNTF8vQlomEpXH7GjVXdH11yqQDUPp6DsjJgZqcE314HiB5Ktk035qrESePaWtn DYOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=JZvcNogseCbtdBa6HYzH67uBWHDdeHRiagnPA6xxWNM=; b=e3rpx5Gj+hZjxFuGTsouoOfbuNvE8vkN6QeKO28n7lMjbMt+9joLhVIuNrtMdLhr1U tjG107Ws7N6i+LWvubBl9ERT3jOhH+v6VRlum4O/TAa+68AJiAKzdk081RfRmAxxudHI 7LEsAcsgLY28g+LRG30Ednx42a9T8neJrjBAhz1zWGYjvpPvD7A1gGgKqwmFyXXpY2We 71MxPMNHUO6vV4Cm7uYUhciAA2F8MhE5a7ELYCcGx7FTidO7dWaApxlSJLqiR3ViiUua egRTfvBOuV/0PHiEH7YTL6tyyzGbvaP8ef77W5nAEzGmzpqqPkY0k4926USCyoyLVUmj L8FA== X-Gm-Message-State: AOAM530v6BKI585F3qT6d5aV3JNtNlyo7FGnpkHguewhdxjdwnRyosdV 9brp4EpfsW/InpnJ24zd/Ty5bBtywWUtxfsujvGS/XgbUexKCNps X-Google-Smtp-Source: ABdhPJxcRIbL33aUAI1WBp1CObPsaunl/WCyfo1B0E5s1LoHQhoD28vtKyy6elavssCLRKdtbXVi/5fbji8uAUP17/U= X-Received: by 2002:a17:90a:1f01:: with SMTP id u1mr4863839pja.62.1609948471308; Wed, 06 Jan 2021 07:54:31 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: John Delisle Date: Wed, 6 Jan 2021 09:54:19 -0600 Message-ID: Subject: Re: zpool remove not working for metadata special devices To: freebsd-fs X-Rspamd-Queue-Id: 4D9v7Y2Kcbz4Tdx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=J9O+dZaC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jdelisle@gmail.com designates 2607:f8b0:4864:20::635 as permitted sender) smtp.mailfrom=jdelisle@gmail.com X-Spamd-Result: default: False [-3.90 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::635:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::635:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::635:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2021 15:54:34 -0000 I'd really like to use this functionality on my production pool. I'm cautious, so I built it out in a lab setting first only to find removing the special device mirror doesn't work on FreeBSD, yet works fine with ZFS on Linux. Does anyone have any suggestions or insight into what might be going on? On Sun, Dec 27, 2020 at 9:04 PM John Delisle wrote: > To experiment, I've completed this same procedure using Ubuntu 20.10 (on > the same "hardware", both are identically configured VMs in Azure) . > > The steps above work fine with ZFS on Ubuntu, and I can successfully > remove both data vdev mirrors and the special vdev mirror. This makes me > think my syntax is correct at least.. but for whatever reason the exact > same procedure fails on FreeBSD (both removing a mirror and removing the > special mirror fail with the same error on FreeBSD). > > Thanks > > On Sun, Dec 27, 2020 at 6:55 PM John Delisle wrote: > >> I have a pool of mirrors, and added a mirrored special device. Although >> documentation suggests it should be removable, I cannot get zpool to do so. >> >> All top-level vdevs are mirrors, and all have the same sector size and >> ashift. No raidz. >> >> ## Current 12.2 p2 >> root@jmdtest:/ # freebsd-version >> 12.2-RELEASE-p2 >> >> ## Disk info >> root@jmdtest:/ # diskinfo /dev/da[2-9] /dev/da[0-9][0-9] >> /dev/da2 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da3 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da4 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da5 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da6 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da7 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da8 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da9 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da10 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da11 512 274877906944 536870912 4096 0 >> 33418 255 63 >> /dev/da12 512 68719476736 134217728 4096 0 >> 8354 255 63 >> /dev/da13 512 68719476736 134217728 4096 0 >> 8354 255 63 >> >> >> ## Create the pool: >> zpool create nebula mirror da2 da3 mirror da4 da5 mirror da6 da7 mirror >> da8 da9 mirror da10 da11 >> >> ## Add the special mirror >> zpool add nebula special mirror da12 da13 >> >> ## zpool status >> root@jmdtest:/ # zpool list -v nebula >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP >> HEALTH ALTROOT >> nebula 1.30T 2.57G 1.30T - - 0% 0% 1.00x >> ONLINE - >> mirror 254G 492M 254G - - 0% 0.18% >> da2 - - - - - - - >> da3 - - - - - - - >> mirror 254G 501M 254G - - 0% 0.19% >> da4 - - - - - - - >> da5 - - - - - - - >> mirror 254G 596M 253G - - 0% 0.22% >> da6 - - - - - - - >> da7 - - - - - - - >> mirror 254G 481M 254G - - 0% 0.18% >> da8 - - - - - - - >> da9 - - - - - - - >> mirror 254G 561M 253G - - 0% 0.21% >> da10 - - - - - - - >> da11 - - - - - - - >> special - - - - - - >> mirror 63.5G 2.62M 63.5G - - 0% 0.00% >> da12 - - - - - - - >> da13 - - - - - - - >> root@jmdtest:/ # >> >> ## Remove the special mirror >> root@jmdtest:/ # zpool remove nebula mirror-5 >> cannot remove mirror-5: invalid config; all top-level vdevs must have the >> same sector size and not be raidz. >> >> ## All mirrors, and all the same ashift: >> root@jmdtest:/ # zdb -C | grep -e child -e ashift >> vdev_children: 6 >> children[0]: >> ashift: 12 >> children[0]: >> children[1]: >> children[1]: >> ashift: 12 >> children[0]: >> children[1]: >> children[2]: >> ashift: 12 >> children[0]: >> children[1]: >> children[3]: >> ashift: 12 >> children[0]: >> children[1]: >> children[4]: >> ashift: 12 >> children[0]: >> children[1]: >> children[5]: >> ashift: 12 >> children[0]: >> children[1]: >> root@jmdtest:/ # >> >> What am I doing wrong? >> >> >> From owner-freebsd-fs@freebsd.org Fri Jan 8 23:41:22 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1FF8C4DA59C for ; Fri, 8 Jan 2021 23:41:22 +0000 (UTC) (envelope-from stevenschlansker@gmail.com) Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCKPF2J3Fz3hmc for ; Fri, 8 Jan 2021 23:41:21 +0000 (UTC) (envelope-from stevenschlansker@gmail.com) Received: by mail-io1-xd2c.google.com with SMTP id n4so11477373iow.12 for ; Fri, 08 Jan 2021 15:41:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=rEwxb7tNKtNtbN0hKOaLq6drKMER4jGL0ECVEmOxsZU=; b=UuISV/C/QTsJewkrY7oaj9vjoUzVpvvYaYuHLaxZCm4YDSQsSG/AA1Ao2ikJsYPuTI fc8gQY7Fl8GGEiZ3txby+YPuiLi6XYVICl0O5mR2r4S8K/WZpvDFlrpRfDmWCf9UlfzE PbOKzdgq5HlCmLv9AxR1bdf8sZFCGASFtO4gH7/+lUDnAi5SfWKNEqAtQCcHnLGckURA v1SO+yanVbQDjicL4ZNN2b+DcBS2RKv7LlJUU1jq2eir11ouPY97VOr3IHyTJdPknZsq HnG5kI1PZrlUqs9a7s+u12o455yExmwriSQvwYGRgYJaX+Kwd1sonlcFu91KxPnecFZc WdoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=rEwxb7tNKtNtbN0hKOaLq6drKMER4jGL0ECVEmOxsZU=; b=kMIgcUkpe7Muryq5oZBfoDhgSNrHj82Fuig2V+u4IB6LdMnkpEybq26cy3g6LqDsro Na10bcOyByj43WNUEGTsZnk2ctdmweytZwBwqAQMIzBP4Hz1RtUTGpjyBsvBt3GokZY5 pw0BOkbNywJtenDVD4K32/iwS8GG/m/W1r5hSsAAC6CyinbnEgGLb9/AXFkm19MTRgwG Mz+Tc06YTy+wcTgYKdAbrOGIGxs4wruQvkxiqkhaPw7bGEUNy5jp65Br84O/a78hHv62 hw5agwB34xnRxQfiayPA/4+I3fHN60vdr0eJ4DQZWeDBCz18e6+z1TNNwFp8OfD8aL7g PXRw== X-Gm-Message-State: AOAM533hd0IP1ia3lzEglT65Jw+oRjjOAnL0ZFJKuw2OOa7AIg1KFiQx rI9ArAnLk7N6G5WnB4srl2CcY/Ed91Pqx+prUKsDxWEbqN3Wfg== X-Google-Smtp-Source: ABdhPJzKE0Rop3AO1wLpZqVc2PD6eOmuzQAd80VMufAlWGhTZyRD67nlYO7w/QmKji7eN3KD47FlgWq/udTio8h03Vg= X-Received: by 2002:a6b:3115:: with SMTP id j21mr7091489ioa.55.1610149279793; Fri, 08 Jan 2021 15:41:19 -0800 (PST) MIME-Version: 1.0 From: Steven Schlansker Date: Fri, 8 Jan 2021 15:41:08 -0800 Message-ID: Subject: persistent integer divide fault panic in zfs_rmnode To: freebsd-fs@freebsd.org X-Rspamd-Queue-Id: 4DCKPF2J3Fz3hmc X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=UuISV/C/; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of stevenschlansker@gmail.com designates 2607:f8b0:4864:20::d2c as permitted sender) smtp.mailfrom=stevenschlansker@gmail.com X-Spamd-Result: default: False [-3.17 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.993]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::d2c:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.18)[-0.181]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::d2c:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d2c:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2021 23:41:22 -0000 Hi freebsd-fs, I have a 8-way raidz2 system running FreeBSD 12.2-RELEASE-p1 GENERIC Approximately since upgrading to FreeBSD 12.2-RELEASE, I receive a nasty panic when trying to unlink any of a large number of files. Fatal trap 18: integer divide fault while in kernel mode The pool reports as healthy: pool: universe state: ONLINE status: One or more devices are configured to use a non-native block size. Expect reduced performance. action: Replace affected devices with devices that support the configured block size, or migrate data to a properly configured pool. scan: resilvered 416M in 0 days 00:08:35 with 0 errors on Thu Jan 7 02:16:03 2021 When some files are unlinked, the system panics with a partial backtrace of: #6 0xffffffff82a148ce at zfs_rmnode+0x5e #7 0xffffffff82a35612 at zfs_freebsd_reclaim+0x42 #8 0xffffffff812482db at VOP_RECLAIM_APV+0x7b #9 0xffffffff80c8e376 at vgonel+0x216 #10 0xffffffff80c8e9c5 at vrecycle+0x45 I captured a dump, and using kgdb extracted a full backtrace, and filed it as https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250784 #8 0xffffffff82963725 in get_next_chunk (dn=0xfffff804325045c0, start=, minimum=0, l1blks=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:721 warning: Source file is more recent than executable. 721 (roundup(*start, iblkrange) - (minimum / iblkrange * iblkrange)) / (kgdb) list 716 * L1 blocks in this range have data. If we can, we use this 717 * worst case value as an estimate so we can avoid having to look 718 * at the object's actual data. 719 */ 720 uint64_t total_l1blks = 721 (roundup(*start, iblkrange) - (minimum / iblkrange * iblkrange)) / 722 iblkrange; 723 if (total_l1blks <= maxblks) { 724 *l1blks = total_l1blks; 725 *start = minimum; (kgdb) print iblkrange $1 = 0 (kgdb) print minimum $2 = 0 It looks like it is attempting to compute 0 / 0, causing the panic. How can I restore my zpool to a working state? Thank you for any assistance.