From owner-freebsd-fs@freebsd.org Sun Jan 17 14:49:18 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A16524EBD0A for ; Sun, 17 Jan 2021 14:49:18 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on0620.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::620]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DJd9948KSz3Ht7 for ; Sun, 17 Jan 2021 14:49:17 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b+0KvMyl72Ojq4/aaJTkMqvdxTSyI7+5BZC/8cxwUeCh+R4BOY6mXr/zk/QXrBnvUgCyWrfQvc8X1GzZI3uhl9yGcOVh3X2NQxKU/TXyLbtizYXI1R+p4yUW86Pd1Dp/NSTRNzFp5bkfXdd7pHbGnYXiDgk7eEt1JQu9+e2Qj3GLsKSs5RrPiCginDYNr9XHENN8jJY2qmPTNHMjK82sBaOjjWRNWU2Lgn5DJEqw+znOGUytGZ0V7qnFgCrZgv0nLkWlWPlGOReyzAa42/XxliPo01VnvIcm0dQj+479tHB+bfiforzNrV4wCCNE4lessBD4B1kUe5TthHbzOwSlPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rYPyiK98fDSCHVlpKhwECSPe6s4KII1Ilx2hn9mdpVM=; b=jRO3+UmY7jI8kmwTGGnYBlg+R+al+xQNZS78c6tKZTU9CyAJ2IerR4ZjcgQ/xxT3JZz6GuI+gtAY5dIKI20dN4xqgVQ9bZ1lnx7JmGW9IG1Z02/wZ8y2w1dH/6rF+Mftj63MHWyla4fRg33QHsg1PKW2H1iH0FIjZ8bvPcn9kkYMpOTp+zxhLiNcXcCxrCNr7DqcAfgnW3pSphUFXdrNfnsTw/T/+rBPWJ45v7vO1Vh12EFGcldV6BzuiUV4U674L9pdFy7ES1PUko2WCH3hTf7NHru9y57COTfYgU/1W6K1bUF9i76GwrNXQ54gSubgxBBTazo6mp8KaPhKb4gE4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rYPyiK98fDSCHVlpKhwECSPe6s4KII1Ilx2hn9mdpVM=; b=OtRWPBU8UwYM3x4TkwsB92h9u79j6kowZ4arNnGRQCVq2q3CGwGIBgBFExQaGalj4kH+iHOLh/GULdBVcNdNE4kin1sVnnnIwVaRQSc8bdzymvKxi5wRKiqTGuO/t1HClgZtM+Y2UTU4q37qz5Y1pQvB6XMaxSG6JIuEvSalbw1tGi399pECnVXtSuXEWJu6qvCRT0d7D4tLEhD7Xqf0xxRR3PXwHp2tqKFpukAvrWOOy14wE5jnPaSLdvm4FPApHEX943LwHyP/ozDRbjW+EzGu75LMNRndHnLSXrNqyZv4kEto+z+DVwMAGNh1qhYK80cwIfWaPu7HbXEM61Or5w== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR01MB3909.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:4e::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Sun, 17 Jan 2021 14:49:15 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::3d86:c7f9:bc4c:40c0]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::3d86:c7f9:bc4c:40c0%6]) with mapi id 15.20.3763.014; Sun, 17 Jan 2021 14:49:09 +0000 From: Rick Macklem To: J David CC: Konstantin Belousov , "freebsd-fs@freebsd.org" Subject: Re: Major issues with nfsv4 Thread-Topic: Major issues with nfsv4 Thread-Index: AQHWzw/HDat+dHoH9kKG5K3Xpd53kqnxDteQgAFi0QCAABTa84AALLCAgAAVvcmAAAu0AIAA4wiAgAAI5gCAAO/IAIAAiDrmgABWOgCAAGv4AIAEv9nSgCyfuACAAIn3C4ACDQ+AgAESerWAAHpKAIAAoS7J Date: Sun, 17 Jan 2021 14:49:09 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 98fca50c-401f-48b1-cd2c-08d8baf70c31 x-ms-traffictypediagnostic: YQXPR01MB3909: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: E+qv5Svy96/W++IN9Z9Uf+73EXADKVdHuUNaFMf2wgKasNq0sv42nvvh5GGel9segGyMihD26prZMKRIwXUsg0DkKO9b9p2m3jlQLBloJqHjKg40mA12EfiogH4YBrytdSCX/p24Q4nZUVDB/3E9QFzZ02vjSSj/8SDlnCF+okMVYLnvuzsaPOKvV2aX6KQgRDp9W9uECLBcLBvJvVgCY1lK7c9bD3LeTzyKbQZ8UlSaMDH5PxKmJLvw+lRC7v18AmiUXUaTS1iBfKxW6B6NnIOWUc+Cmb5ucQsu2o9c9KNEVKt1mdRhHqv3jTMUWRRBQFDNOCJAgVrrKqPyow2yJhrTbtJKGsnAT2/UR949Es3CYegAru2WnysLAAJzPlRbsDe+QfyqrRBvVZSf6YuyF1spexpNRaZx9oCT0zihwpfDbpSbUTERGg30eUzlwmRrczI49xHCZdRzUOS1+LYUEGdsITvzQS2sLv0BC54hI12qPf7wen2lWITYDA+Jc8nXSy1UwzlsStM1AdzBlzq5dA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(366004)(39850400004)(376002)(346002)(396003)(136003)(55016002)(33656002)(6916009)(7696005)(186003)(6506007)(86362001)(66556008)(4326008)(83380400001)(71200400001)(5660300002)(91956017)(66446008)(66946007)(66476007)(316002)(2906002)(8936002)(9686003)(478600001)(76116006)(64756008)(52536014)(54906003)(8676002)(786003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?TznECn8E6F+APV36mC3KzEAC3iLnjHw+Qyu6PNwK0ncuqClLs0axoBQKRn?= =?iso-8859-1?Q?cnZOSTbpZIGuP9m5Hy4/yBFSfIGEmICR6SZEBgLT6PnH7WZvcKvcMo/uPr?= =?iso-8859-1?Q?vim8MkZr+0xL1o6+jXs1OGM36R6a9ABKzT5zgq8Vvx6A/SzaokUzk0MNu9?= =?iso-8859-1?Q?XU+4V8W6EVU5pi7KRVWOO0MYxWp5SA+cJDjSSrwcuklFjKnFsGV+FmL5Gb?= =?iso-8859-1?Q?dz+xaYvMB1KaDYuE7fWenYCEzSRFYwWY7DLZE1L4hDnWoM0oamP/UPgTRw?= =?iso-8859-1?Q?g6gzlKrATmlOOlbQ762RQTRWcd0OJc3+S2E0TW0EauZArmse4MaaIpGgXa?= =?iso-8859-1?Q?4lTwl2nDS38VULZh6kvt8tG3k2TTCzZDA8s8/PJx1qKjtvnGdpO9/ivoJ1?= =?iso-8859-1?Q?hWMBP8yYCF8AJ1HtOaW39ipQEsnBcILm59bkirZ5UCyMqVgqG63ZLu/EHg?= =?iso-8859-1?Q?09KjpfRT0fMWa5zIEZXK6BA+6wDHPNaCJiqnxEdojTuispX0zxTEPvRBqZ?= =?iso-8859-1?Q?DLEvDsap3L2a2uac8hBk8UqPsDHeOjZYf5d6xXSIUTHGN9EMEiu9T6m4hx?= =?iso-8859-1?Q?krShIC/mLQxIubRpS+HTPJzWwKjCGCtZYhGd8fy62RfFwIfg3TMSNLRu/x?= =?iso-8859-1?Q?KHEqQOH3oLJOWu6Jpuv3Ap2ASDv5Wg/D6qLbUqlzz6jFEtUr2Fwvmqzmj4?= =?iso-8859-1?Q?V582fndD9CSsYJZ/boDwkNRY70gFPUi+A6v+JOqqm9B7dPcKmL1KEANAWT?= =?iso-8859-1?Q?+A++cBV461Sl/FXgokr2DtScPMY9KSVQ6Qut+oc7LJgNXwCsYIYNgdCKDe?= =?iso-8859-1?Q?hrQeLaXs5IvajYAtDHi0I1eRSWE87OGjqUTtZCa0gsnb/a33RECGTaxOqt?= =?iso-8859-1?Q?0FT6nxNob0L0f+kWIwZs86JPGiZH7giLhY8k2RCWtbcCELpvSViyJ3+7Y3?= =?iso-8859-1?Q?FHqf8+NqEf9SkILCewBS+YuwOB23dxVFYaKa0u0mMhTprMPIy8Y+wE/njK?= =?iso-8859-1?Q?xwTg3rUXW/UpkgCDdfkqzUq02d5a49rdi0s7dl3ToTokTSg4z+7bSGVUrP?= =?iso-8859-1?Q?o/aq+RnNyLCuUTPSTjhahv0=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 98fca50c-401f-48b1-cd2c-08d8baf70c31 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jan 2021 14:49:09.6654 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ktR6KPJcmDIRNknmieZA2rS3HK+YycFzhA+nsLsGF31peKqJALkvKrulthrjTgUMDIWBK5L7KHvXlA/jwWmIMw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB3909 X-Rspamd-Queue-Id: 4DJd9948KSz3Ht7 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=OtRWPBU8; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::620 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:111:f400:fe5d::620:from]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; MIME_TRACE(0.00)[0:+]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2a01:111:f400:fe5d::620:from:127.0.2.255]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Jan 2021 14:49:18 -0000 J David wrote:=0A= >On Sat, Jan 16, 2021 at 5:57 PM Rick Macklem wrote:= =0A= >> Opens can be closed until all FreeBSD open file descriptors for the file= =0A= >> are closed.=0A= >> --> Just the way it is. It is not an unintended leak. They go away once= =0A= >> all file descriptors get closed,=0A= >[...]=0A= >> However, none of the above seems unexpected, except maybe for why=0A= >> "ls" in the chroot opens 3 regular files each time. I don't know what=0A= >> chroot actually does for something like "ls"? I'll look.=0A= >=0A= >It probably isn't anything ls is doing. It's probably just the open=0A= >of whatever shared libraries ls has in common with the shell running=0A= >in the other window.=0A= >=0A= >And so, on a long-running system with frequently-accessed binaries (or=0A= >just shared libraries) mounted over NFS, where it's unlikely that all=0A= >descriptors will ever be closed at the same time, a hang/crash/failure=0A= >essentially becomes inevitable?=0A= Yes. If you have NFSv4 mounted shared libraries and an environment=0A= where there is at least one process running using a given shared library=0A= on the mount at any time, the NFSv4 Opens will accumulate and=0A= eventually constipate the mount.=0A= =0A= >Is this a solvable problem? (Outside of oneopenown.)=0A= For this case, "oneopenown" is your only option.=0A= =0A= I think there might have been a nullfs/NFSv4 interaction issue,=0A= but it appears to be resolved in FreeBSD13.=0A= =0A= rick=0A= =0A= Thanks!=0A=