From owner-freebsd-fs@freebsd.org Tue Nov 28 14:12:40 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E039AE57BED; Tue, 28 Nov 2017 14:12:40 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0078.outbound.protection.outlook.com [104.47.40.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D16E662CF; Tue, 28 Nov 2017 14:12:39 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTOPR0101MB2172.CANPRD01.PROD.OUTLOOK.COM (52.132.46.161) by YTOPR0101MB2171.CANPRD01.PROD.OUTLOOK.COM (52.132.46.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.260.4; Tue, 28 Nov 2017 14:12:38 +0000 Received: from YTOPR0101MB2172.CANPRD01.PROD.OUTLOOK.COM ([fe80::f072:85d3:769:e6f8]) by YTOPR0101MB2172.CANPRD01.PROD.OUTLOOK.COM ([fe80::f072:85d3:769:e6f8%13]) with mapi id 15.20.0260.007; Tue, 28 Nov 2017 14:12:38 +0000 From: Rick Macklem To: Konstantin Belousov , Emmanuel Vadot CC: FreeBSD Current , "freebsd-fs@freebsd.org" , Rick Macklem Subject: Re: Switch vfs.nfsd.issue_delegations to TUNABLE ? Thread-Topic: Switch vfs.nfsd.issue_delegations to TUNABLE ? Thread-Index: AQHTaEyAfy6q2cCLHkW9s1ZyeaxyRaMpzDeAgAAEfyo= Date: Tue, 28 Nov 2017 14:12:38 +0000 Message-ID: References: <20171128114136.10e44d5bcfd563e9b4ab5453@bidouilliste.com> <20171128110428.GN2272@kib.kiev.ua> <20171128142610.6ab535b0b8c6b5d372cd3f27@bidouilliste.com>, <20171128134009.GQ2272@kib.kiev.ua> In-Reply-To: <20171128134009.GQ2272@kib.kiev.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YTOPR0101MB2171; 6:ix74uNI1SocEsUxRJU+8XuYUKYqxzilCzO3DXT4Si2RPKRvF5szOjimqYCmXzh+F3tNvXg+bd4bSma2EX5xaZFux6EEjvIrhhjXNbBJREY9f0FvUYIifJhB3/9EOP63pMnzEywSjSYzPzdtVjGgCCGJcKgGjObJcS3+ZnaDl9369XhO8i9bCPRMO1jB6GgRUy4WGE1OjWFC/bg8UmAErUchj1u/6zq/h/cALgUxDHKxsa2KvimBcOMAANq84XMPik3TBfGFI05Kj91pBcjvho4M1MHikY/3GACi3bFGhebRDvkv8rEwlGWA6yvr2HIURXUR+oeyi1u2Oc6jMLv2vIIHKCE+jEJXFwHKMjJ/OiL0=; 5:iERIKJKB16qwB2o3QhLxcAPrvcsdMeEXN8qsIEb52Fr+k7rPKuCxyg1Z9wLpAS5ji1pYnrBf0f8yhRy3WgjCWKlHvP8OzruOgltEjhhZWql3cu/FB0NRSLkkKQb+u/vJytD0VLAsSoQYjur3OaeQgCPrOp4MKwmRMUmJ8KZYLVU=; 24:R5LNBBN8LPw/vsNeo0pcl0+J/E0jJcxWluXtXrjVTYj0Jyjqs7PbW1EKIL5AzrUEJ355vn4CdyxK/RcqCmLzSxbElXqwDwf9OKDRo4x+gUs=; 7:vd8n8oukPMNBUqJ0hg+oEcW9F7YVzZvGVrGSzTzMuW9bg047GbxbtDKaOZRD8hvwJniQRT7/h7J47jWmln4gIa9y+cr928CPLhkc8i4CIT88KT0kkQXEMWp9BdHmaejU8MZ8341TiVWQhtjYpBlk9lu5sihJQ/c3smeTvY1WF+JLi7YRj2mQwoYUA7q7j+5IwtGr8fELfWqmfORS57Gdy1RqbERPHhbjCOsdxoXU6+xLqxffdYaR+mUZmAjRd8zO x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 5caea99f-8e08-4fdd-4d13-08d5366a14c3 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4602075)(4627115)(8989060)(201703031133081)(201702281549075)(8990040)(5600026)(4604075)(2017052603258); SRVR:YTOPR0101MB2171; x-ms-traffictypediagnostic: YTOPR0101MB2171: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(3231022)(10201501046)(93006095)(93001095)(3002001)(6041248)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(20161123555025)(20161123562025)(20161123560025)(6072148)(201708071742011); SRVR:YTOPR0101MB2171; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:YTOPR0101MB2171; x-forefront-prvs: 0505147DDB x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(376002)(346002)(366004)(24454002)(189002)(199003)(316002)(25786009)(786003)(5250100002)(478600001)(110136005)(86362001)(55016002)(54906003)(68736007)(97736004)(7696005)(74316002)(4326008)(8676002)(3280700002)(189998001)(305945005)(81156014)(9686003)(81166006)(5660300001)(3660700001)(2950100002)(93886005)(105586002)(2900100001)(14454004)(2906002)(39060400002)(8936002)(33656002)(76176999)(102836003)(229853002)(74482002)(54356999)(99286004)(6246003)(53936002)(106356001)(50986999)(6436002)(6506006)(101416001); DIR:OUT; SFP:1101; SCL:1; SRVR:YTOPR0101MB2171; H:YTOPR0101MB2172.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 5caea99f-8e08-4fdd-4d13-08d5366a14c3 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Nov 2017 14:12:38.4632 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTOPR0101MB2171 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Nov 2017 14:12:41 -0000 Konstantin Belousov wrote: >On Tue, Nov 28, 2017 at 02:26:10PM +0100, Emmanuel Vadot wrote: >> On Tue, 28 Nov 2017 13:04:28 +0200 >> Konstantin Belousov wrote: >> >> > On Tue, Nov 28, 2017 at 11:41:36AM +0100, Emmanuel Vadot wrote: >> > > >> > > Hello, >> > > >> > > I would like to switch the vfs.nfsd.issue_delegations sysctl to a >> > > tunable. Since it defaults to "disabled", I don't see why a tunable would be necessa= ry? (Just do nothing and delegations don't happen. If you want the server to issue delegations, then use the sysctl to turn them on. If you want to = turn them off again at the server, reboot the server without setting the sysctl= .) > > > The reason behind it is recent problem at work on some on our filer > > > related to NFS. > > > We use NFSv4 without delegation as we never been able to have good > > > performance with FreeBSD server and Linux client (we need to do test > > > again but that's for later). Delegations are almost never useful, especially with Linux clients. [stuff snipped] > > Perhaps make nodeleg per-mount flag ? The sysctl vfs.nfsd.issue_delegations only affects the server. If you have a FreeBSD client that is mounting a delegations enabled server = and does not want delegations, just don't run the "nfscbd" daemon on the client= . The only time you want the "nfscbd" daemon running is for delegations and pNFS layouts. (I suppose there is the case of a client using NFSv4.1/pNFS a= gainst a server with delegations enabled, but since delegations aren't very useful= , I'd just disable delegations on the server for this case.) Having a per-mount version of this would be overkill, I think. It would hav= e to disable callbacks on the mount point, since there is no way for a client to= say "don't give me delegations" except disabling callbacks, which the server requires for delegation issue. [stuff snipped] The case where there has never been delegations issued will result in an empty delegation queue. Maybe this case can be handled without acquiring the "global NFSv4 state lock", which is what sounds like is causing the performance issue. Maybe a global counter of how many delegations are issued that is handled by atomic ops. --> If it is 0, nfsrv_checkgetattr() can just return without acquiring the = lock. I'll take a look at this, rick=