From owner-freebsd-fs@freebsd.org Wed Mar 17 02:31:39 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 20FBC576C46 for ; Wed, 17 Mar 2021 02:31:39 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670051.outbound.protection.outlook.com [40.107.67.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0Z0j5Cbwz4WGX; Wed, 17 Mar 2021 02:31:32 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nYkIyxjWmIiWtM9EhJJneMkYUXukCA+PcZcXBrmchGMnp2tU/AKquVcr2ebTv54B0WBTZgQGgqYrZl1IxdPZaBFmhObl10ra8h3KFsBDnd57sHafCZjGNWvm3vttYOpOEFRvgiQ6sXtJLYZzxg8hcl8QlTPYCKWZCTmjhD8S/q2V+ZtUIbChl18A/4nEIiutVTDEjM6oVfIfRmhmxsyEPsl5sUbacH/dEaQQD01DLtLbhBsA31S0w1vSAfpJShg8B/TfQM0xRDjV7o14VwJzV9rNXfHkw5H7XEDPRjnbUb4YIIbyfzTAhAb/vcGnHnyTdkI5KolTQO5dxgkPBffPMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wLCP9g/PEdXdiOyqQqrfl32Sgu0Y8tOatQjHXXyPjZo=; b=g4MWHtFgukECLOUQ9hbel/TUKPX0imdyAokDIn1CxebzK6ECbZAVKJ0ZH1BlWDOW6JFjNRTsyl/JqPXeABI7kmmGYDn/G0w2M6KNDS0OBNrpQZtdbu9B0NLdMPZqrKC4XB9a9uG0zQ2HGIzbCGgTcyxVH1OErZ2F4YbXjwmWyqsmJ4lRniPeSDRyKoBL7dxPES7pLK4nKZdybb1KBdJ3IcT/3cc5tq7F3J3MN8WN1+QoBzHth8Y8UgMtHeIxubqw1NMquol0akCxm1z5l+a3wEFDjUFUH5yBKENp9Tw6LZw5Z06J2zntj+EfBnk7cHcG2941+e/ir48Y6fkTJo665w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wLCP9g/PEdXdiOyqQqrfl32Sgu0Y8tOatQjHXXyPjZo=; b=TZcAHbmwNB1Xpk31zXJG2dY8SiEWSrfcxlSsQlWEociYeCywP350vRCE8ambjQVxnmns9RjKIZN2e1Ix0P4NsAAYj/w0nYx/PfEbWJf0EVJR+JuVOZe8Dke81T71Y8f5Lj0cJXf5Z6YlmzzwAMnGE4A7YeyeSjIwshVFsl31X71nR76aTociT72HhAIeZT35DjXszSsNeUprsghz+xitph5xaYGe54m6IIEZI+ItxveGTe8WaUuwlMrqNoqJRHqF1WvFuxkoMitwZtfFQyk+qHk1b09EVtKVuulVdJFEwJJfqogft0NNGfDagxvmQGMhA+kXNNXrhExY/t9gapgdig== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR01MB3430.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:4d::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Wed, 17 Mar 2021 02:31:30 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a%7]) with mapi id 15.20.3933.032; Wed, 17 Mar 2021 02:31:30 +0000 From: Rick Macklem To: Alan Somers CC: freebsd-fs Subject: Re: Understanding the vfs.nfsd.request_space* sysctls Thread-Topic: Understanding the vfs.nfsd.request_space* sysctls Thread-Index: AQHXGom0SMOUlewkdkSbAUM6yshgqaqHSCtTgAAdewCAAAz0qQ== Date: Wed, 17 Mar 2021 02:31:30 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b53ed26f-ae04-4060-727c-08d8e8ecc627 x-ms-traffictypediagnostic: YQXPR01MB3430: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: J/0wBns632WIQavUurYRQBArDZ2H0tPeu0lS5q6yoj7xKoVGvFGDf6i1eCcLMCUyIJ6ChQGodiHJ3zTOkFlaKrLld026HGHXo2/C1Nvz6dndgKkeczEE7KRdHZmho3kIhD2+xT0qWC5xvcPuqfydCRjdSo+aU+HmnNhg0iTTrFwpRhafIY3ovhK/3DIRNFXRaRlcy9ybJXTCZVIuC5Mqy9BfVlPrucR0QABQPI3Wf1d8QGfx9c+FgptN5MlhvkXbE09j8Wyzb4DUvX6rYrGksMpFbn3/RLP+018lE/8TzS7uYldZj+0UWRp3Y8qHs04GU8a7ltFDJ/qMbbjfL8WqgNzdvqP0T13eIHAs4mViBW2eS1gwE4sKDPbpOvkvdkeTYQpMlkzW1k/RoaEvNRCfhqCVdFAcHbvD4JReUMk7AqtN7UOHUM+Rarvgd0VOKBzlstkWDxUpPAdP5X+s9mhWu6PVPvp+MhsrIs71joxAPcqbdTmxMvVTe7/wI4Kztblm1OZTDUeAjx9fJGD4cd09hoDx6Si0T6dvn1ba9cIrGu1/VVuXLf1+/dWkr7jgXOL69SxENsum91X/JI1rRME34sN8RH7B214WYn92d0Pj+j3yh9mTy7fA6Sca8cGVZVLdtxKH2yZCrX4oc6uWP57F2A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(376002)(396003)(136003)(346002)(39860400002)(366004)(66446008)(76116006)(66946007)(316002)(9686003)(8936002)(478600001)(64756008)(66476007)(6506007)(8676002)(6916009)(7696005)(450100002)(55016002)(4326008)(86362001)(5660300002)(66556008)(91956017)(52536014)(2906002)(83380400001)(33656002)(786003)(186003)(966005)(71200400001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?tQKcv5fggIBPb4QCrT+K4tWsMOeK+etSa/C1d7r6S8fkiZc0EUiHobjwyo?= =?iso-8859-1?Q?VWcfAH7JGKcCESITsDfyHDSTkRusiwD17NQCczPsMXzIsoCbCf9NGnXFGZ?= =?iso-8859-1?Q?K3Gx9WXbk3UBLIzUZR/tb91symuHsRQBKViUmvb8iZHhvb/L7XfkkFHOVK?= =?iso-8859-1?Q?d+BX7yq3ZrvnwZ8HYPg4fKNTh1hgG6amNm0oOviBOzFxfXoPLG1kJD/9mo?= =?iso-8859-1?Q?DGVNjwhE4JG9uWIOyMc0SLS8Lfl3gcAPExh4R0c7K/2Qi9eDgoxT2Th5Dq?= =?iso-8859-1?Q?HCjvqdp+NhOV23n8N3KfsCoklWIi6cYMGvCkTD/y5tgmlE5PdIprhN29rH?= =?iso-8859-1?Q?M1F/ek7yhYqe0NnllNDwdbsA/hRgA/Bn6Z3R27PLtBcwVbHF99YCa59zDg?= =?iso-8859-1?Q?tpx4GgvMk+8zZCXnzWRHqh+Lqga1skKQtWQc9InjCweYOO3ESB1ayvLlCj?= =?iso-8859-1?Q?fvJzfv5AlxqdifHw2mPRt9Vzh1nRkyQ+qid9Lak1hqlWG4/tyS+RmU9T53?= =?iso-8859-1?Q?7r04A1Cz3s8aV7Y5b5ldRFckHb+tkMsikOeqoHYvsh89+HherWoiN3o6Ii?= =?iso-8859-1?Q?2QSYaalOqVx8PzS+Ze327WtcQ9sjH3K4mwl3P61qWDjbOMxaprupRzBgUa?= =?iso-8859-1?Q?ed5At+6xBuGMkndWGFnSAUVOeBYIzG33796CmVb4ZeUJfwh57jJe5bVnoZ?= =?iso-8859-1?Q?JnZEVqE060oEFVKXg9rlP0Dhs6khxiDNVTBnGWXjVQ6xTRfbgGKZF368Ke?= =?iso-8859-1?Q?aw5bgApqlxPD+31EuZ0oeTDz1uQNpbY1796ewpV/uIyQt60QJcxW1SCGZ7?= =?iso-8859-1?Q?zwLwdsHyCsy6fOHs/lSTUOiDH2cUH2cCjhBMS5tSRG6WjoU7B25blXBEG8?= =?iso-8859-1?Q?SNqpCtRd3knDLWmpbopbVxxljhH/VzR9Aqn7BhssYrVJF3iHnnC0tbCOlL?= =?iso-8859-1?Q?gswx4dJaDllmb0OvM3TPNFOwzeyLflS+8NMe0OyMbEH6ZL4+EYdnQST2ZJ?= =?iso-8859-1?Q?T1QtbgW9pna49mfxXHt1yEfAn74muKWNjyhvp2uaAYvzkDbirEDE37dMZU?= =?iso-8859-1?Q?vg4mIb1TjiV6grU+NiUIuaMwO4tfvNDIiZE0ljj5+4uE+QoxciAdeTa5ij?= =?iso-8859-1?Q?a1zA53RS6io4cRwuX339c+F8i9/SK9hWJ7fvP5P3a/UK7Xh9ABhV1+7VQf?= =?iso-8859-1?Q?ThNFiJspCGsR50v8B8BWj5YQCi/qUxUDNcBsuvTDFiQbR3KayjmgrlpA1B?= =?iso-8859-1?Q?kyIlu+5epv0Gzyfu09/3qe5qBk+9yaJlpHBFvT/TPa3jqnQP6VFFO7R+aY?= =?iso-8859-1?Q?s6Bavi0iCCl21IIXgYBgrgIs4mdBvIsygRHWYBbCVFnE2j0mL4aSSLt3ao?= =?iso-8859-1?Q?nHZE39s+Is8a/WwkIQ9Xa/SErtzRSQ2ClOdIfsLjroESZV4bi9rSU=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: b53ed26f-ae04-4060-727c-08d8e8ecc627 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Mar 2021 02:31:30.6365 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: kwnvC7ROnPdKxrwvQ9TYwduUBbIbPTc3GSspDKXZ63nLfrFMP/+4o/5drui8J+OCOn32qPKcWg0dS4a5SpwXVA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB3430 X-Rspamd-Queue-Id: 4F0Z0j5Cbwz4WGX X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=TZcAHbmw; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=temperror (mx1.freebsd.org: error in processing during lookup of rmacklem@uoguelph.ca: DNS error) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-5.80 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.51:from]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; R_SPF_DNSFAIL(0.00)[temporary DNS error]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[40.107.67.51:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.107.67.51:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-fs]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.51:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 02:31:39 -0000 Alan Somers wrote:=0A= [stuff snipped]=0A= >That all makes sense. So having a high request_space_used basically just = means that >my storage is too slow.=0A= Since it appears that you have plenty of nfsd threads, that would imply=0A= that the bottleneck is probably the file system.=0A= (You might want to check that there is cpu available. ie. Some amount of=0A= idle cpu, just in case?)=0A= [more stuff snipped]=0A= >I've added an L2ARC since I sent that original email. We'll see how well = it works. I >expect it to take 48 hours before results are apparent.=0A= Others have tweaked something so that more of the arc is used for metadata.= =0A= [more stuff snipped]=0A= =0A= >Experimentally, I determined that 768 threads is sufficient. But your for= mula would >suggest > 2500, which is a lot of threads! I'll keep it in min= d if I ever reach 768, >though.=0A= 768 is probably plenty. My number would be an extreme upper bound.=0A= I did it because the default of 8 per cpu is very low (assumes a machine mo= stly=0A= doing other things).=0A= =0A= Good luck with it, rick=0A= =0A= =0A= rick=0A= =0A= -Alan=0A= _______________________________________________=0A= freebsd-fs@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-fs=0A= To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"=0A= =0A=