From owner-freebsd-fs@freebsd.org Mon Jan 4 01:11:29 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 614C54C6D91 for ; Mon, 4 Jan 2021 01:11:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670046.outbound.protection.outlook.com [40.107.67.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8HdX3KnYz3JT1 for ; Mon, 4 Jan 2021 01:11:28 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F3TgMHTz1P3jZBjt4a56X2TQBF5FeuUljLHe651KsPuBFQtGfJcoLIckK0Cu+nUbCs8fdlK1Pm6ydY05WFOz/87XMmQtpfNV697ygoh+ADkr7PP6ZMkBu7BwByO4t9D57inc1CRJmTh02O9RahC0tREPKIseqtNkuLaaogFEGAA1KrlaMWOpwQlpmAkOh12bPV6yOVIYM0stR4RJ9jCN35jph7glSDpA4wQRPQMxdf3gUQ5P9jZwKQLwIisFmRrjd4hK7O5uy15eBgZ9JZWu39ZqhIzXmjajxjGKtNgVlPEoVioxFji86Q/e+HNSJrlWWROPwfs7DPXp/b8UL1SdxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xws5ZYt7HEF3V0eSqDcZ3UB8oRWuqOSYXlNGYS4fL/w=; b=L6PEd7WCbJ+xnVJ47zbcydLwp2IKwQnOXyqyXBxp50ip7EgF5dwL2R+NSBTS8x9Ts+bx3AYAUXKZsqNW8pm+rGBp7i9aY0dOj3u6u5xqEYuDfFahLykdeyCdy0sne++UCav1/5l6a6uIaICjWQtex+bjxU+pT0HRytaGf+zEEysZslVUU18Y5/LU+j/cSYFujAuX7u5lIQJfQAcfmQWooz/vy46LK/PE8cqTeU4YZJwj/hC1y4IDxefCHOv+7Y8kuZBeA4gCmolZjfHPgdhUvEMaBjji1x+uwm7uuS2sxBRbjS5PMyOyyZW3alLHRNj/FgH7pOjIFbiHz6Lf6cKV3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xws5ZYt7HEF3V0eSqDcZ3UB8oRWuqOSYXlNGYS4fL/w=; b=sK2VdFdw7f7jY5CptyUkN7VgWE/dzdItjo1nwPbLDB9yhHTFWDThg9CUMjuC4zB/+7oNtRrghB1AFJL/+64bBm6GGTTDXqJ/3YTKcpEY7RK8+RFUzg6KxSNpsZlrdef2EGOqgeQyHB1LjNM2O6ZQPMmz4/yX1tQnG2QDpOEcm2c/G5gCiDygeZMie9Q2jQlHkriiGz3bc0mUbs93uKaFTCHP7X1B7172w0KGrRVgPTfBZjPWyzHx69nu0f0qaMLydM98GQFMlWAmhSxY+HYQeI2+f6uxVV81HMHFkF15fKCcyFU+dL2X9EL4H1DCT75qcRP690cmF0yOgGj7r6YBxg== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR0101MB1478.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:17::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3721.23; Mon, 4 Jan 2021 01:11:26 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::3d86:c7f9:bc4c:40c0]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::3d86:c7f9:bc4c:40c0%6]) with mapi id 15.20.3721.024; Mon, 4 Jan 2021 01:11:26 +0000 From: Rick Macklem To: Konstantin Belousov CC: J David , "freebsd-fs@freebsd.org" Subject: Re: Major issues with nfsv4 Thread-Topic: Major issues with nfsv4 Thread-Index: AQHWzw/HDat+dHoH9kKG5K3Xpd53kqnxDteQgAFi0QCAABTa84AALLCAgAAVvcmAAAu0AIAA4wiAgAAI5gCAAO/IAIAAiDrmgABWOgCAAGv4AIAEv9nSgA9Z2ruAAR1dAIALkqgC Date: Mon, 4 Jan 2021 01:11:26 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 49bdf073-d853-4219-cfff-08d8b04da910 x-ms-traffictypediagnostic: YQXPR0101MB1478: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: uiGiQs1mPkqnN9Kh6Ce77mkGjR1ZqCAthyB8oCKvBnU2MdDEqtLP1Utr45XzUFfO2WmntGsykKFMhhv8CTBl1me0ihR4K+HlVSmVYtieCTT6jSJQ61pB3MApFTEkj8GUSpBBaBWjziXJQM3DmLn/m/HF89YfGZ/bQvWjCrem/BTezQ2TyQ4gYTT/XBaEJcpIIITgxNBYLozrqyT/cqXp4E0C6QwoId944+xFWFMngXR606d6PGjgmF0pF1RrucQC4ocFJFkVPGjkIJNJjqSVGeIX001lQKt426hY1XWwRW4bDVBTiPIqAnmPtFzZ/Nycwpxryg1tc4yD98+o1eXYl9I4Tec7sP4IN0NVA3gkjL8fynIx/UV/IAp4ffoCoh000BQ9h8Iaw/n0WONWKNDfEA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(396003)(39850400004)(376002)(366004)(136003)(346002)(9686003)(8936002)(66946007)(86362001)(2906002)(66476007)(91956017)(33656002)(76116006)(52536014)(478600001)(8676002)(5660300002)(6506007)(26005)(186003)(54906003)(316002)(786003)(55016002)(83380400001)(64756008)(66556008)(66446008)(4326008)(71200400001)(7696005)(6916009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?Ew4z3XsSh/NU+SQBdvNXTHeZjJF+51fbfQqSeVoKNiPgL527I+b/UDrh6z?= =?iso-8859-1?Q?/dYWoFhpaX5c1+jYcNj5++bFKHe2F2kyVtTSytA5Bz5LjUYQToDksl/4/z?= =?iso-8859-1?Q?0UzkoKF3C1cAJCgyNFnbXM/3qglH8nDBGw86rJ5nlS5Ebq4Ve0Yz3HgZeR?= =?iso-8859-1?Q?YrAC09c4i3gv55wVmIwHAr86oqM1cu/CNPWdnW5SDP/WEAvW/5MeQ2vQY7?= =?iso-8859-1?Q?TpINKwDlJHjB15KlMBX0mQj1v8gYXAvpNut5TS+pZVGyyTdqEVZqrOq9Pa?= =?iso-8859-1?Q?4EwQQfp7idus/Ug7qKDIpcL8nQeIKrFFZwYNn1BQO/G/xf5H0eMMUmvbJn?= =?iso-8859-1?Q?jWaRgImU89nMGDvSpCikdvxFeNa+8Kl7M0930Ljs33SKti09NuPJh35Osk?= =?iso-8859-1?Q?NYV9briJKn7bl+6l5X7nQKWxKHlWeIg/dGXVnqylW2DYN0iaZrkdmQLB7M?= =?iso-8859-1?Q?YRbBaBkkRmiuxKFjxhyFEwBgOx3lu0OS0oDJ3PRXN96h+6vCKMWxxTTVAz?= =?iso-8859-1?Q?c0aDNQTOakjP4jj2voQFMpRWlqMjyI4+CuP24+/EC5yz83UTtolLIofp2X?= =?iso-8859-1?Q?+loLJGdMJ0rkgIcHOXn6Z7PGkZK6IvJXzDi4R2svYiWB7y+VWIMNgAx6Kb?= =?iso-8859-1?Q?gK/u8WIPBQl1Csp/GUPscwIHCA0k00XwOTBcSSdlYpBPOzjVt54Ek3DLe1?= =?iso-8859-1?Q?neuSw8XKu4OMoqGkwhKjvXRhMaU9k+1RhvGZZiVfb0OtPuvS+T4TYr2qMy?= =?iso-8859-1?Q?87PHM9s9fjKpbo9rqSp7usqqKAFe0EqEkmCmbWNag5Dwe6Qu5IwMAJdpmv?= =?iso-8859-1?Q?T+8oaUzdKyHLQ/3uecx+wupAOcCmq8Si9OEnYLHJLbMMjITFgC7S/j0UAK?= =?iso-8859-1?Q?N6pGJUG2chp4TS8F9cIW6393VhtKdCQeRgpciylUePbQW12DBAWB71a+SX?= =?iso-8859-1?Q?zE/WapD+9zBPUHqvQbPBPXz3adLHFAkmRn68ubGQDA8WH5VHbCl8Ci7zdo?= =?iso-8859-1?Q?KZr1XmGATVmmdu8YU=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 49bdf073-d853-4219-cfff-08d8b04da910 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jan 2021 01:11:26.7157 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: WbR9eUVVIBwhc7ldF4aRwyyhtHwpD8qXR2kscuXSkvX3zkOIxydw9IofXIS3MqSuC5HNsR5fy/wYDfSIytwlPg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB1478 X-Rspamd-Queue-Id: 4D8HdX3KnYz3JT1 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=sK2VdFdw; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.46 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.46:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.46:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[40.107.67.46:from:127.0.2.255]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.46:from]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2021 01:11:29 -0000 Kostik wrote:=0A= >Rick Macklem wrote:=0A= [stuff snipped]=0A= >> I see vfs_deferred_inactive() in sys/kern/vfs_subr.c, but I do not=0A= >> know when/how it gets called?=0A= >Right, vfs_deferred_inactive() is one way which tries to handle missed=0A= >inactivations. If upon vput() the lock is only shared and upgrade=0A= >failed, vnode is marked as VI_OWEINACT and put onto 'lazy' list,=0A= >processed by vfs_sync(MNT_LAZY). It is typically called from syncer,=0A= >which means each 60 secs. There, if the vnode is still unreferenced, it=0A= >is inactivated.=0A= If I read the code correctly vfs_deferred_inactive() gets called with=0A= LK_EXCLUSIVE | LK_NOWAIT | LK_INTERLOCK.=0A= Then there is this code in nfs_lock():=0A= vp =3D ap->a_vp;=0A= 332 lktype =3D ap->a_flags & LK_TYPE_MASK;=0A= 333 error =3D VOP_LOCK1_APV(&default_vnodeops, ap);=0A= 334 if (error !=3D 0 || vp->v_op !=3D &newnfs_vnodeops)=0A= 335 return (error);=0A= 336 np =3D VTONFS(vp);=0A= 337 if (np =3D=3D NULL)=0A= 338 return (0);=0A= 339 NFSLOCKNODE(np);=0A= 340 if ((np->n_flag & NVNSETSZSKIP) =3D=3D 0 || (lktype !=3D LK_SHA= RED &&=0A= 341 lktype !=3D LK_EXCLUSIVE && lktype !=3D LK_UPGRADE &&=0A= 342 lktype !=3D LK_TRYUPGRADE)) {=0A= 343 NFSUNLOCKNODE(np);=0A= 344 return (0);=0A= 345 }=0A= 346 onfault =3D (ap->a_flags & LK_EATTR_MASK) =3D=3D LK_NOWAIT &&= =0A= 347 (ap->a_flags & LK_INIT_MASK) =3D=3D LK_CANRECURSE &&=0A= 348 (lktype =3D=3D LK_SHARED || lktype =3D=3D LK_EXCLUSIVE);=0A= 349 if (onfault && vp->v_vnlock->lk_recurse =3D=3D 0) {=0A= 350 /*=0A= 351 * Force retry in vm_fault(), to make the lock request= =0A= 352 * sleepable, which allows us to piggy-back the=0A= 353 * sleepable call to vnode_pager_setsize().=0A= 354 */=0A= 355 NFSUNLOCKNODE(np);=0A= 356 VOP_UNLOCK(vp);=0A= 357 return (EBUSY);=0A= 358 }=0A= Would the above code result in an EBUSY reply when the vn_lock(LK_NOWAIT |= =0A= LK_EXCLUSIVE) is made in vfs_deferred_inactive()?=0A= =0A= If so, it looks like vfs_periodic_inactive()->vfs_deferred_inactive() will = just=0A= call vdefer_inactive_unlocked()->vdefer_inactive() and it will end up=0A= on the lazy list with VI_DEFINACT set again, if I am reading the=0A= code correctly?=0A= =0A= >Another place where inactivation can occur is reclamation. There in=0A= >vgonel(), we call VOP_INACTIVE() if VI_OWEINACT is set. In principle,=0A= >this is redundand because correct filesystem must do the same cleanup=0A= >(and more) at reclamation as at the inactivation. But we also call=0A= >VOP_CLOSE(FNONBLOCK) before VOP_RECLAIM().=0A= And this would make all the NFSv4 Opens go away (get closed)=0A= when the nullfs mounts are unmounted, as reported by J. David.=0A= =0A= >Looking at this from another angle, if inactivation for NFSv4 vnodes=0A= >is not called longer than 2 minutes, perhaps there is a reference leak.=0A= >It is not due to VFS forgetting about due VOP_INACTIVE() call.=0A= Unless the nfs_lock(LK_NOWAIT) call keeps failing with EBUSY,=0A= I think?=0A= =0A= rick=0A= ps: I need to find a way to reproduce this.=0A= =0A=