From nobody Tue May 16 21:40:18 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QLV5Z4Vyhz4BTgL; Tue, 16 May 2023 21:40:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QLV5Z3jWrz4Pr6; Tue, 16 May 2023 21:40:18 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684273218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZiMM0lsLPaBF0VE90QpHRkYEUTJrqQCYMz01S6+1S90=; b=jpKcWj3y1cAz18RfDeCyUVoNZLZFforUxTLtaejeC7KDEauGXpZqp6MgPdaGz3SXyLfF3A DFreVa3piapeNI3NLwS36FYELMET9OHg89VUyJYU8gmOZlOXSDF3t8pjW6cG+yaE66vRfj PPbYS2yeCOI+5JA16xGwcyyDgATCnc9ru7sjXRYyvtwxL1qasa/7lq3gV7nz2LZLKopEBH QgaeD9KgodA4wyTdYfiktHO0odCjpWTcJM9+ZvQjzCHZtnNw1zQCfxOUDEV8/LDlvBtmSQ kPW2azL6Hmizy1aOrZzCr8gLKWb/qGW52q6f6NwohRDBWDnWD75ZY7Z0b/9rhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684273218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZiMM0lsLPaBF0VE90QpHRkYEUTJrqQCYMz01S6+1S90=; b=MleaP0pKSiyXApCkzScrzNAAsU/41s46CfBFlL0GXciSrJCqqOO26Tp3uXfkE32a/TeFNs qB7xwpjcrh3Zfy74Ob3SKPU5cBoLOHBBUHOglkXCqUjUmdqKLU6urWYeZmLlkGYNQZtjWr YfjXdcfngSXctrel2ZkGyWFHk0HzYRYORyq8RuovCPvzKTdQWuXnw1H+nTxRXK8UzM8gPo QDJiatcmGaW6Sln7eGuoyMGja+UwXHedPPzztJtEqpCvB/BrG0o0KN9/+d/J1KMX+J4zb8 IEHDAN/O7y+NHeGL5R9+hcTPu/MgSEFVW6FaaDUjHd+7nzaL3kwHzWVh3VViAA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1684273218; a=rsa-sha256; cv=none; b=I+21Lx4d2XXq2P/EsE8LtG+p3pYEFkEc9+gz2yCnh31tbyRNF5QrNVg93CnbpWZy1D6X0T sLzFVxdEec/63IeZIIODRfPiIFgRcsX8JtYFsOa3L2BuUQxk18OTrLG5L07GppKrvl3Z+E 5vd4RgueMjPtd+J5QKYzMZOZSsestPIZLxvcIvXsqZLw2bQBmNrm8Mn143HE6EhVj35u7W w/QKhcGAiQzMSeASjEVOdZeQlhp3fPtRNYjnUtsnxJ5uO4gM3fPrqFgIVCWJNu6l13UBcn A6zetggvVM3msHptzGfoMd9OS7A9SNIptbRUESebtVUh4uhYVQ+DHwv4o/m7yQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QLV5Z2jBczdSy; Tue, 16 May 2023 21:40:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 34GLeIGA029798; Tue, 16 May 2023 21:40:18 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 34GLeIf9029797; Tue, 16 May 2023 21:40:18 GMT (envelope-from git) Date: Tue, 16 May 2023 21:40:18 GMT Message-Id: <202305162140.34GLeIf9029797@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: d9ecb62798da - stable/13 - nfs_fha_new: Fix nfs_fha_new so that sysctls work in prisons List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d9ecb62798da799c7a391b05eabeb57e3082cf97 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=d9ecb62798da799c7a391b05eabeb57e3082cf97 commit d9ecb62798da799c7a391b05eabeb57e3082cf97 Author: Rick Macklem AuthorDate: 2023-03-01 23:25:35 +0000 Commit: Rick Macklem CommitDate: 2023-05-16 21:39:27 +0000 nfs_fha_new: Fix nfs_fha_new so that sysctls work in prisons The sysctls were in "struct fha_params", making it impractical to vnet them. This patch moves the sysctls out of "struct fha_params" and vnet's them, so that they are available in vnet prisons running nfsd(8). It also avoids the IS_DEFAULT_VNET() macros in the VNET_SYSINIT() function. (cherry picked from commit a90b47abcbdbd48c68fbf7c407546293479056de) --- sys/fs/nfsserver/nfs_fha_new.c | 118 ++++++++++++++++++----------------------- sys/fs/nfsserver/nfs_fha_new.h | 3 -- 2 files changed, 53 insertions(+), 68 deletions(-) diff --git a/sys/fs/nfsserver/nfs_fha_new.c b/sys/fs/nfsserver/nfs_fha_new.c index 203e98d3ac86..3d18083f7205 100644 --- a/sys/fs/nfsserver/nfs_fha_new.c +++ b/sys/fs/nfsserver/nfs_fha_new.c @@ -57,8 +57,46 @@ static void fha_extract_info(struct svc_req *req, struct fha_info *i); NFSD_VNET_DEFINE_STATIC(struct fha_params *, fhanew_softc); +NFSD_VNET_DEFINE_STATIC(struct fha_ctls, nfsfha_ctls); SYSCTL_DECL(_vfs_nfsd); +SYSCTL_NODE(_vfs_nfsd, OID_AUTO, fha, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, + "NFS File Handle Affinity (FHA)"); + +SYSCTL_UINT(_vfs_nfsd_fha, + OID_AUTO, enable, CTLFLAG_NFSD_VNET | CTLFLAG_RWTUN, + &NFSD_VNET_NAME(nfsfha_ctls).enable, 0, + "Enable NFS File Handle Affinity (FHA)"); + +SYSCTL_UINT(_vfs_nfsd_fha, + OID_AUTO, read, CTLFLAG_NFSD_VNET | CTLFLAG_RWTUN, + &NFSD_VNET_NAME(nfsfha_ctls).read, 0, + "Enable NFS FHA read locality"); + +SYSCTL_UINT(_vfs_nfsd_fha, + OID_AUTO, write, CTLFLAG_NFSD_VNET | CTLFLAG_RWTUN, + &NFSD_VNET_NAME(nfsfha_ctls).write, 0, + "Enable NFS FHA write locality"); + +SYSCTL_UINT(_vfs_nfsd_fha, + OID_AUTO, bin_shift, CTLFLAG_NFSD_VNET | CTLFLAG_RWTUN, + &NFSD_VNET_NAME(nfsfha_ctls).bin_shift, 0, + "Maximum locality distance 2^(bin_shift) bytes"); + +SYSCTL_UINT(_vfs_nfsd_fha, + OID_AUTO, max_nfsds_per_fh, CTLFLAG_NFSD_VNET | CTLFLAG_RWTUN, + &NFSD_VNET_NAME(nfsfha_ctls).max_nfsds_per_fh, 0, + "Maximum nfsd threads that " + "should be working on requests for the same file handle"); + +SYSCTL_UINT(_vfs_nfsd_fha, + OID_AUTO, max_reqs_per_nfsd, CTLFLAG_NFSD_VNET | CTLFLAG_RWTUN, + &NFSD_VNET_NAME(nfsfha_ctls).max_reqs_per_nfsd, 0, "Maximum requests that " + "single nfsd thread should be working on at any time"); + +SYSCTL_PROC(_vfs_nfsd_fha, OID_AUTO, fhe_stats, + CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, 0, 0, + fhenew_stats_sysctl, "A", ""); extern int newnfs_nfsv3_procid[]; @@ -78,68 +116,19 @@ fhanew_init(void *foo) snprintf(softc->server_name, sizeof(softc->server_name), FHANEW_SERVER_NAME); - /* - * Initialize the sysctl context list for the fha module. - */ - sysctl_ctx_init(&softc->sysctl_ctx); - if (IS_DEFAULT_VNET(curvnet)) { - softc->sysctl_tree = SYSCTL_ADD_NODE(&softc->sysctl_ctx, - SYSCTL_STATIC_CHILDREN(_vfs_nfsd), OID_AUTO, "fha", - CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "NFS File Handle Affinity (FHA)"); - if (softc->sysctl_tree == NULL) { - printf("%s: unable to allocate sysctl tree\n", __func__); - return; - } - } - for (i = 0; i < FHA_HASH_SIZE; i++) mtx_init(&softc->fha_hash[i].mtx, "fhalock", NULL, MTX_DEF); /* * Set the default tuning parameters. */ - softc->ctls.enable = FHA_DEF_ENABLE; - softc->ctls.read = FHA_DEF_READ; - softc->ctls.write = FHA_DEF_WRITE; - softc->ctls.bin_shift = FHA_DEF_BIN_SHIFT; - softc->ctls.max_nfsds_per_fh = FHA_DEF_MAX_NFSDS_PER_FH; - softc->ctls.max_reqs_per_nfsd = FHA_DEF_MAX_REQS_PER_NFSD; + NFSD_VNET(nfsfha_ctls).enable = FHA_DEF_ENABLE; + NFSD_VNET(nfsfha_ctls).read = FHA_DEF_READ; + NFSD_VNET(nfsfha_ctls).write = FHA_DEF_WRITE; + NFSD_VNET(nfsfha_ctls).bin_shift = FHA_DEF_BIN_SHIFT; + NFSD_VNET(nfsfha_ctls).max_nfsds_per_fh = FHA_DEF_MAX_NFSDS_PER_FH; + NFSD_VNET(nfsfha_ctls).max_reqs_per_nfsd = FHA_DEF_MAX_REQS_PER_NFSD; - /* - * Add sysctls so the user can change the tuning parameters. - */ - if (IS_DEFAULT_VNET(curvnet)) { - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "enable", CTLFLAG_RWTUN, - &softc->ctls.enable, 0, "Enable NFS File Handle Affinity (FHA)"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "read", CTLFLAG_RWTUN, - &softc->ctls.read, 0, "Enable NFS FHA read locality"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "write", CTLFLAG_RWTUN, - &softc->ctls.write, 0, "Enable NFS FHA write locality"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "bin_shift", CTLFLAG_RWTUN, - &softc->ctls.bin_shift, 0, - "Maximum locality distance 2^(bin_shift) bytes"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "max_nfsds_per_fh", CTLFLAG_RWTUN, - &softc->ctls.max_nfsds_per_fh, 0, "Maximum nfsd threads that " - "should be working on requests for the same file handle"); - - SYSCTL_ADD_UINT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "max_reqs_per_nfsd", CTLFLAG_RWTUN, - &softc->ctls.max_reqs_per_nfsd, 0, "Maximum requests that " - "single nfsd thread should be working on at any time"); - - SYSCTL_ADD_OID(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree), - OID_AUTO, "fhe_stats", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, - 0, 0, fhenew_stats_sysctl, "A", ""); - } } static void @@ -150,7 +139,6 @@ fhanew_uninit(void *foo) softc = NFSD_VNET(fhanew_softc); - sysctl_ctx_free(&softc->sysctl_ctx); for (i = 0; i < FHA_HASH_SIZE; i++) mtx_destroy(&softc->fha_hash[i].mtx); free(softc, M_TEMP); @@ -489,8 +477,8 @@ fha_hash_entry_choose_thread(struct fha_params *softc, } /* Check whether we should consider locality. */ - if ((i->read && !softc->ctls.read) || - (i->write && !softc->ctls.write)) + if ((i->read && !NFSD_VNET(nfsfha_ctls).read) || + (i->write && !NFSD_VNET(nfsfha_ctls).write)) goto noloc; /* @@ -501,11 +489,11 @@ fha_hash_entry_choose_thread(struct fha_params *softc, offset2 = thread->st_p3; if (((offset1 >= offset2) - && ((offset1 - offset2) < (1 << softc->ctls.bin_shift))) + && ((offset1 - offset2) < (1 << NFSD_VNET(nfsfha_ctls).bin_shift))) || ((offset2 > offset1) - && ((offset2 - offset1) < (1 << softc->ctls.bin_shift)))) { - if ((softc->ctls.max_reqs_per_nfsd == 0) || - (req_count < softc->ctls.max_reqs_per_nfsd)) { + && ((offset2 - offset1) < (1 << NFSD_VNET(nfsfha_ctls).bin_shift)))) { + if ((NFSD_VNET(nfsfha_ctls).max_reqs_per_nfsd == 0) || + (req_count < NFSD_VNET(nfsfha_ctls).max_reqs_per_nfsd)) { #if 0 ITRACE_CURPROC(ITRACE_NFS, ITRACE_INFO, "fha: %p(%d)r", thread, req_count); @@ -535,8 +523,8 @@ noloc: * We didn't find a good match yet. See if we can add * a new thread to this file handle entry's thread list. */ - if ((softc->ctls.max_nfsds_per_fh == 0) || - (fhe->num_threads < softc->ctls.max_nfsds_per_fh)) { + if ((NFSD_VNET(nfsfha_ctls).max_nfsds_per_fh == 0) || + (fhe->num_threads < NFSD_VNET(nfsfha_ctls).max_nfsds_per_fh)) { thread = this_thread; #if 0 ITRACE_CURPROC(ITRACE_NFS, ITRACE_INFO, @@ -569,7 +557,7 @@ fhanew_assign(SVCTHREAD *this_thread, struct svc_req *req) NFSD_CURVNET_SET(NFSD_TD_TO_VNET(curthread)); softc = NFSD_VNET(fhanew_softc); /* Check to see whether we're enabled. */ - if (softc->ctls.enable == 0) + if (NFSD_VNET(nfsfha_ctls).enable == 0) goto thist; /* diff --git a/sys/fs/nfsserver/nfs_fha_new.h b/sys/fs/nfsserver/nfs_fha_new.h index 08f52539450e..5510fa2891f1 100644 --- a/sys/fs/nfsserver/nfs_fha_new.h +++ b/sys/fs/nfsserver/nfs_fha_new.h @@ -95,9 +95,6 @@ struct fha_info { struct fha_params { struct fha_hash_slot fha_hash[FHA_HASH_SIZE]; - struct sysctl_ctx_list sysctl_ctx; - struct sysctl_oid *sysctl_tree; - struct fha_ctls ctls; char server_name[32]; };