From nobody Tue Jul 21 01:28:14 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h40Bq6230z6lGGs for ; Tue, 21 Jul 2026 01:28:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h40Bq58Pkz3dwD for ; Tue, 21 Jul 2026 01:28:19 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784597299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=X/nw/G82RHCDLhJfIl3BFPph8S1rvaNKeZMWllXM+Vk=; b=BfbbhpliD8+TnX7qUV3VYxq1+d7DvmMHkh/c3zF0zrbf49XkWKjcR+GIOwuMpPQFKfeGFL pRPs1SaWOyJktPKoclkqAphD1aDA2TejnVLTrC5JtIXVXgMawGEU6IC0Xs7zvFP7hzliNB iPpqnUtqgh4mUXkTT4oZGkiGi2wQKJtpu3GNvCwM3EiQ1Tye+LuAIXF2LC3vFAYcZcRTtA xZMrYk7O56zj40wSpSM26u/+dhXh4CWl6yZxd/xkxXM2jOxwA+bawcvjTa9FARisoElGP+ o+ryswJcNuEEa/Q4VCPzUxEHPic6WyX+fbTm9Tte6krUCuceV+egWpSJWVH8bg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1784597299; a=rsa-sha256; cv=none; b=MrDir/2O/kLHCR8wYQXeUeazEr8jmmxzZSrKmCsTSFSFLfTnNM9mE4TZkTHdYniC4yXLN3 f3ERGZqwGpaZqGIV1V7J1z1ijikE7d7REUcBROMktN7GIiixRbf6tIVIrqESi1QGMJVX5v 23PCyqyQsmJMT5+vqtzd6yWBZH5vHwhiruAjs8QIgr4kGYIsbbPf0Qbz2DqW9m8K411QRZ 6eT/r5vyjXVnqJa3iMLhGpeOnABtA1DY0dRVH/T+ImxfzGyDBpPrTggVA/1tpIy6q3sh0L wCsWnNGW5iKke0Ju6xXq48+eLxhlTc7gzvVtTofb6aJ4yk8oiCgD+UkYsrbH0w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784597299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=X/nw/G82RHCDLhJfIl3BFPph8S1rvaNKeZMWllXM+Vk=; b=HfQnKliUWAktXbUxohX5UEJHxWIkyGAkI47kSP6A/cXWNPxpVU6l7RKVHxEf3CLw2IkIMr husQT/LUQXGPHg33HO78RRgoNnTpFUYSsfXpiTaR7uuJSBiKbFi6U9CQlBswHpmSWY1Zh3 Ifx6mWc+2luePMruRn69cjjvcWKcY6smCKDEI2KrbWrGpZRZbEuDnbaTvwz1Uk2y2Bcvr5 vrmwtTtAbydzMPMJPTxPsHIIlFC9X9AvQsfSvUWY2pIfzILiKhInG0qoChQ0Oz7hE5NCf/ mFpqjHYiUhJSn8cQjuudUNLsSZUkwvHiFvUSZK7h4vG/K1zCcjm3KqaQcIT1FA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h40Bq40T4zBsm for ; Tue, 21 Jul 2026 01:28:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3f7e2 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 21 Jul 2026 01:28:14 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 94b21dade677 - stable/15 - nfsd: Optionally enable use of M_EXTPG mbufs for read replies List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 94b21dade6773c0b73261e2ffc69be3d3a13a9cb Auto-Submitted: auto-generated Date: Tue, 21 Jul 2026 01:28:14 +0000 Message-Id: <6a5ecb2e.3f7e2.57301f59@gitrepo.freebsd.org> The branch stable/15 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=94b21dade6773c0b73261e2ffc69be3d3a13a9cb commit 94b21dade6773c0b73261e2ffc69be3d3a13a9cb Author: Rick Macklem AuthorDate: 2026-07-07 00:10:11 +0000 Commit: Rick Macklem CommitDate: 2026-07-21 01:26:39 +0000 nfsd: Optionally enable use of M_EXTPG mbufs for read replies A test site determined that, for a Mellanox NIC which can handle M_EXTPG mbufs, an improvement of 5-15% for read rate could be achieved if the read reply was in M_EXTPG mbufs. A patch that tried to determine if the outbound NIC supported M_EXTPG mbufs (IFCAP_MEXTPG) did not pass review. However, it does appear that this can be useful for NFS-over-RDMA. (Which just happen to use NICs that do support M_EXTPG mbufs.) As such, this patch enables them is xp_extpg is set to true, which is never for now, but might be set true for RDMA or when vfs.nfsd.enable_mextpg is set non-zero. (It is 0 by default, so this is never enabled by default at this time.) (cherry picked from commit d516e52373e1768ea84bf1ca220671a44f413abe) --- sys/fs/nfs/nfs.h | 1 + sys/fs/nfsserver/nfs_nfsdkrpc.c | 18 +++++++++++++++++- sys/fs/nfsserver/nfs_nfsdport.c | 16 ++++++++++------ sys/fs/nfsserver/nfs_nfsdserv.c | 7 +++++-- sys/rpc/svc.h | 1 + 5 files changed, 34 insertions(+), 9 deletions(-) diff --git a/sys/fs/nfs/nfs.h b/sys/fs/nfs/nfs.h index b30e4f17b7c5..0afd97902039 100644 --- a/sys/fs/nfs/nfs.h +++ b/sys/fs/nfs/nfs.h @@ -743,6 +743,7 @@ struct nfsrv_descript { #define ND_EXTLSCERTUSER 0x20000000000 #define ND_ERELOOKUP 0x40000000000 #define ND_MACHCRED 0x80000000000 +#define ND_CANEXTPG 0x100000000000 /* * ND_GSS should be the "or" of all GSS type authentications. diff --git a/sys/fs/nfsserver/nfs_nfsdkrpc.c b/sys/fs/nfsserver/nfs_nfsdkrpc.c index 8c557143efbd..16c21b4e1607 100644 --- a/sys/fs/nfsserver/nfs_nfsdkrpc.c +++ b/sys/fs/nfsserver/nfs_nfsdkrpc.c @@ -97,6 +97,10 @@ SYSCTL_INT(_vfs_nfsd, OID_AUTO, server_max_nfsvers, CTLFLAG_VNET | CTLFLAG_RWTUN, &VNET_NAME(nfs_maxvers), 0, "The highest version of NFS handled by the server"); +static bool nfsrv_mextpg = false; +SYSCTL_BOOL(_vfs_nfsd, OID_AUTO, enable_mextpg, CTLFLAG_RW, + &nfsrv_mextpg, 0, "Enable use of M_EXTPG mbufs"); + static int nfs_proc(struct nfsrv_descript *, u_int32_t, SVCXPRT *xprt, struct nfsrvcache **); @@ -315,7 +319,19 @@ nfssvc_program(struct svc_req *rqst, SVCXPRT *xprt) if ((xprt->xp_tls & RPCTLS_FLAGS_CERTUSER) != 0) nd.nd_flag |= ND_TLSCERTUSER; } - nd.nd_maxextsiz = 16384; + nd.nd_maxextsiz = MBUF_PEXT_MAX_PGS * PAGE_SIZE; + /* + * If the NIC can handle M_EXTPG mbufs, they can be used + * only if the reply will not be copied into the DRC. + * This implies NFSv3 over TCP and NFSv4.n, but not NFSv4.0. + * (NFSv4.n will set ND_SAVEREPLY if the reply is going + * to be copied into the session slot.) + * Check for TCP transport (UDP uses the DRC) and a direct + * map. + */ + if ((nfsrv_mextpg || xprt->xp_extpg) && nd.nd_nam2 == NULL && + PMAP_HAS_DMAP != 0) + nd.nd_flag |= ND_CANEXTPG; #ifdef MAC mac_cred_associate_nfsd(nd.nd_cred); #endif diff --git a/sys/fs/nfsserver/nfs_nfsdport.c b/sys/fs/nfsserver/nfs_nfsdport.c index 51683855df22..21607337559a 100644 --- a/sys/fs/nfsserver/nfs_nfsdport.c +++ b/sys/fs/nfsserver/nfs_nfsdport.c @@ -2393,12 +2393,13 @@ again: /* * If cnt > MCLBYTES and the reply will not be saved, use - * ext_pgs mbufs for TLS. + * ext_pgs mbufs for TLS or if enabled via vfs.nfsd.enable_mextpg. * For NFSv4.0, we do not know for sure if the reply will * be saved, so do not use ext_pgs mbufs for NFSv4.0. */ if (cnt > MCLBYTES && siz > MCLBYTES && - (nd->nd_flag & (ND_TLS | ND_EXTPG | ND_SAVEREPLY)) == ND_TLS && + ((nd->nd_flag & (ND_TLS | ND_SAVEREPLY)) == ND_TLS || + (nd->nd_flag & (ND_CANEXTPG | ND_SAVEREPLY)) == ND_CANEXTPG) && (nd->nd_flag & (ND_NFSV4 | ND_NFSV41)) != ND_NFSV4) nd->nd_flag |= ND_EXTPG; @@ -2738,7 +2739,8 @@ ateof: /* * If the reply is likely to exceed MCLBYTES and the reply will - * not be saved, use ext_pgs mbufs for TLS. + * not be saved, use ext_pgs mbufs for TLS or if enabled via + * vfs.nfsd.enable_mextpg. * It is difficult to predict how large each entry will be and * how many entries have been read, so just assume the directory * entries grow by a factor of 4 when attributes are included. @@ -2746,7 +2748,8 @@ ateof: * be saved, so do not use ext_pgs mbufs for NFSv4.0. */ if (cnt > MCLBYTES && siz > MCLBYTES / 4 && - (nd->nd_flag & (ND_TLS | ND_EXTPG | ND_SAVEREPLY)) == ND_TLS && + ((nd->nd_flag & (ND_TLS | ND_SAVEREPLY)) == ND_TLS || + (nd->nd_flag & (ND_CANEXTPG | ND_SAVEREPLY)) == ND_CANEXTPG) && (nd->nd_flag & (ND_NFSV4 | ND_NFSV41)) != ND_NFSV4) nd->nd_flag |= ND_EXTPG; @@ -7090,13 +7093,14 @@ nfsvno_getxattr(struct vnode *vp, char *name, uint32_t maxresp, if (tlen > 0) { /* * If cnt > MCLBYTES and the reply will not be saved, use - * ext_pgs mbufs for TLS. + * ext_pgs mbufs for TLS or enabled via vfs.nfsd.enable_mextpg. * For NFSv4.0, we do not know for sure if the reply will * be saved, so do not use ext_pgs mbufs for NFSv4.0. * Always use ext_pgs mbufs if ND_EXTPG is set. */ if ((flag & ND_EXTPG) != 0 || (tlen > MCLBYTES && - (flag & (ND_TLS | ND_SAVEREPLY)) == ND_TLS && + ((flag & (ND_TLS | ND_SAVEREPLY)) == ND_TLS || + (flag & (ND_CANEXTPG | ND_SAVEREPLY)) == ND_CANEXTPG) && (flag & (ND_NFSV4 | ND_NFSV41)) != ND_NFSV4)) uiop->uio_iovcnt = nfsrv_createiovec_extpgs(tlen, maxextsiz, &m, &m2, &iv); diff --git a/sys/fs/nfsserver/nfs_nfsdserv.c b/sys/fs/nfsserver/nfs_nfsdserv.c index ef24f3362dce..6b617fedd3fb 100644 --- a/sys/fs/nfsserver/nfs_nfsdserv.c +++ b/sys/fs/nfsserver/nfs_nfsdserv.c @@ -1028,13 +1028,16 @@ nfsrvd_read(struct nfsrv_descript *nd, __unused int isdgram, if (cnt > 0) { /* * If cnt > MCLBYTES and the reply will not be saved, use - * ext_pgs mbufs for TLS. + * ext_pgs mbufs for TLS of if enabled via + * vfs.nfsd.enable_mextpg. * For NFSv4.0, we do not know for sure if the reply will * be saved, so do not use ext_pgs mbufs for NFSv4.0. * Always use ext_pgs mbufs if ND_EXTPG is set. */ if ((nd->nd_flag & ND_EXTPG) != 0 || (cnt > MCLBYTES && - (nd->nd_flag & (ND_TLS | ND_SAVEREPLY)) == ND_TLS && + ((nd->nd_flag & (ND_TLS | ND_SAVEREPLY)) == ND_TLS || + (nd->nd_flag & (ND_CANEXTPG | ND_SAVEREPLY)) == + ND_CANEXTPG) && (nd->nd_flag & (ND_NFSV4 | ND_NFSV41)) != ND_NFSV4)) nd->nd_repstat = nfsvno_read(vp, off, cnt, nd->nd_cred, nd->nd_maxextsiz, p, &m3, &m2); diff --git a/sys/rpc/svc.h b/sys/rpc/svc.h index b0fc78d4d044..e73e61e5ac93 100644 --- a/sys/rpc/svc.h +++ b/sys/rpc/svc.h @@ -155,6 +155,7 @@ typedef struct __rpc_svcxprt { uid_t xp_uid; gid_t *xp_gidp; int xp_doneddp; + bool_t xp_extpg; /* Can use M_EXTPG mbufs. */ } SVCXPRT; /*