From nobody Sat Apr 30 20:53:45 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 713FF199F2F5; Sat, 30 Apr 2022 20:53:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KrM5j2nZjz4YVZ; Sat, 30 Apr 2022 20:53:45 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1651352025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hA2HGPimFsJXjzMvbfs2OoHitKtwgEFis+Apb9ZxmN4=; b=T5/rgIrOWZmqtrJk7iJ4ZbWJZ+Bc6qHqLHYOL6qODUVBSw26Zejq5eUDhdGKEJJgX0LpgN ow0IC3vU1THzlnQ4ihILLdGbfCV+cOyp8f3jwkO4SvKMtJ0ZuQVCweBwwQHeoiPVGxktSs TovKJ8yONc/Yyo6Q0ygoxlFix2/fzD9cSpeGUCxSVQysT+HSalqlQOeB6vbvD0vQ0mKvkK gmNa2HHIuprqZ4JuWnzKQmcn3tTxffbIntfMhNeNGcWFC8ADvqz0Pej6Urwp0k9zKJi8wQ D7CNwG1XhjxsQdagXGDz5KGaFrq6UX3jswG1rDzCp7jVF1aSVDC7GAkWxPhSOg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 40A838F4; Sat, 30 Apr 2022 20:53:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 23UKrj6S005126; Sat, 30 Apr 2022 20:53:45 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 23UKrjeB005125; Sat, 30 Apr 2022 20:53:45 GMT (envelope-from git) Date: Sat, 30 Apr 2022 20:53:45 GMT Message-Id: <202204302053.23UKrjeB005125@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Rick Macklem Subject: git: 5218d82c81f9 - main - nfscl: Add support for a NFSv4 AppendWrite RPC List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 5218d82c81f987223054671b9821d39f08d87599 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1651352025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hA2HGPimFsJXjzMvbfs2OoHitKtwgEFis+Apb9ZxmN4=; b=Fy1pbkCqK/t2QKG08uwWMOAiy3f4qXl76wYXOBSGq/Vsj7ZEKdsz9UFzOIzcKbFvhtIvNk jNO34dfCofA/WnoHOvvsT+46iX+tLPjenYzCwKRYl79rdgRDFzox0JEWVMB894VosruaLG RoFIgABikCbhPPu+tLeo7WmO8uRrskThZRet4ku9odXTDgRGSJllS1KHFBHIvjQMqj7M+2 KD3bgBNo8dMTWqps6BDcsnYSlKLQVvbGDTxne1ir5wgL+rFeIc++1j1WQSAefX8c4EBTlC liMhm8owSJ2WKdmhUcepcrHHeccu8xf6KLo7X9j/cVpl5J2UrXMFx24wjRjI2Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1651352025; a=rsa-sha256; cv=none; b=MC94ghOLmwnn759QTjEdgM05N7qK3vkLfGkDsVQQ4RfLzrPXpYauJsesZAJJ9LpGJ9vINB yvk8VpciCqYtN/hjJJGf13pf0P5jTgwUy26ud+CMNOTVmfNypfLKEjvYn+hhY8k/ga2skD Lxzhq6nFwUlg0pMvnV7RARJGGtWzhBtSPsvVF7yvTxn4IKC3WntGSPSs5/Hp4AprlfgA4k TUfVCzkAuuQQDItgnWmGcDeqTaByT3MsqcBV+8Fk8H24dut3i7r8p4CYMXbqKyaZhAPmam 538t1pBP440NCW0aAlUZkDT32xIdloqTnh0DGg4G24Cui9/mfrWpXa9yylTSJQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=5218d82c81f987223054671b9821d39f08d87599 commit 5218d82c81f987223054671b9821d39f08d87599 Author: Rick Macklem AuthorDate: 2022-04-30 20:49:23 +0000 Commit: Rick Macklem CommitDate: 2022-04-30 20:49:23 +0000 nfscl: Add support for a NFSv4 AppendWrite RPC For IO_APPEND VOP_WRITE()s, the code first does a Getattr RPC to acquire the file's size, before it can do the Write RPC. Although NFS does not have an append write operation, an NFSv4 compound can use a Verify operation to check that the client's notion of the file's size is correct, followed by the Write operation. This patch modifies the NFSv4 client to use an Appendwrite RPC, which does a Verify to check the file's size before doing the Write. This avoids the need for a Getattr RPC to preceed this RPC and reduces the RPC count by half for IO_APPEND writes, so long as the client knows the file's size. The nfsd structure was moved from the stack to be malloc()'d, since the kernel stack limit was being exceeded. While here, fix the types of a few variables, although there should not be any semantics change caused by these type changes. --- sys/fs/nfs/nfs_var.h | 2 +- sys/fs/nfsclient/nfs.h | 2 +- sys/fs/nfsclient/nfs_clbio.c | 26 +++++++++++----- sys/fs/nfsclient/nfs_clrpcops.c | 67 +++++++++++++++++++++++++++++++++-------- sys/fs/nfsclient/nfs_clvnops.c | 6 ++-- 5 files changed, 78 insertions(+), 25 deletions(-) diff --git a/sys/fs/nfs/nfs_var.h b/sys/fs/nfs/nfs_var.h index 3a2a27890d7b..c4dc2451523e 100644 --- a/sys/fs/nfs/nfs_var.h +++ b/sys/fs/nfs/nfs_var.h @@ -471,7 +471,7 @@ int nfsrpc_readlink(vnode_t, struct uio *, struct ucred *, int nfsrpc_read(vnode_t, struct uio *, struct ucred *, NFSPROC_T *, struct nfsvattr *, int *, void *); int nfsrpc_write(vnode_t, struct uio *, int *, int *, - struct ucred *, NFSPROC_T *, struct nfsvattr *, int *, void *, int); + struct ucred *, NFSPROC_T *, struct nfsvattr *, int *, int, int); int nfsrpc_mknod(vnode_t, char *, int, struct vattr *, u_int32_t, enum vtype, struct ucred *, NFSPROC_T *, struct nfsvattr *, struct nfsvattr *, struct nfsfh **, int *, int *, void *); diff --git a/sys/fs/nfsclient/nfs.h b/sys/fs/nfsclient/nfs.h index ce1747a2ab6b..94faeb381ac0 100644 --- a/sys/fs/nfsclient/nfs.h +++ b/sys/fs/nfsclient/nfs.h @@ -107,7 +107,7 @@ void ncl_nodeunlock(struct nfsnode *); int ncl_getattrcache(struct vnode *, struct vattr *); int ncl_readrpc(struct vnode *, struct uio *, struct ucred *); int ncl_writerpc(struct vnode *, struct uio *, struct ucred *, int *, int *, - int); + int, int); int ncl_readlinkrpc(struct vnode *, struct uio *, struct ucred *); int ncl_readdirrpc(struct vnode *, struct uio *, struct ucred *, struct thread *); diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index 081aff6c0e7c..c89d6796ea00 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -793,7 +793,7 @@ do_sync: */ must_commit = 2; error = ncl_writerpc(vp, &uio, cred, &iomode, - &must_commit, 0); + &must_commit, 0, ioflag); KASSERT((must_commit == 2), ("ncl_directio_write: Updated write verifier")); if (error) @@ -986,11 +986,21 @@ ncl_write(struct vop_write_args *ap) * get the append lock. */ if (ioflag & IO_APPEND) { - np->n_attrstamp = 0; - KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); - error = VOP_GETATTR(vp, &vattr, cred); - if (error) - return (error); + /* + * For NFSv4, the AppendWrite will Verify the size against + * the file's size on the server. If not the same, the + * write will then be retried, using the file size returned + * by the AppendWrite. However, for NFSv2 and NFSv3, the + * size must be acquired here via a Getattr RPC. + * The AppendWrite is not done for a pNFS mount. + */ + if (!NFSHASNFSV4(nmp) || NFSHASPNFS(nmp)) { + np->n_attrstamp = 0; + KDTRACE_NFS_ATTRCACHE_FLUSH_DONE(vp); + error = VOP_GETATTR(vp, &vattr, cred); + if (error) + return (error); + } NFSLOCKNODE(np); uio->uio_offset = np->n_size; NFSUNLOCKNODE(np); @@ -1633,7 +1643,7 @@ ncl_doio_directwrite(struct buf *bp) * verifier on the mount point. */ must_commit = 2; - ncl_writerpc(bp->b_vp, uiop, bp->b_wcred, &iomode, &must_commit, 0); + ncl_writerpc(bp->b_vp, uiop, bp->b_wcred, &iomode, &must_commit, 0, 0); KASSERT((must_commit == 2), ("ncl_doio_directwrite: Updated write" " verifier")); if (iomode != NFSWRITE_FILESYNC) @@ -1818,7 +1828,7 @@ ncl_doio(struct vnode *vp, struct buf *bp, struct ucred *cr, struct thread *td, iomode = NFSWRITE_FILESYNC; error = ncl_writerpc(vp, uiop, cr, &iomode, &must_commit, - called_from_strategy); + called_from_strategy, 0); /* * When setting B_NEEDCOMMIT also set B_CLUSTEROK to try diff --git a/sys/fs/nfsclient/nfs_clrpcops.c b/sys/fs/nfsclient/nfs_clrpcops.c index 48cd6467c78f..9d19114ab07c 100644 --- a/sys/fs/nfsclient/nfs_clrpcops.c +++ b/sys/fs/nfsclient/nfs_clrpcops.c @@ -134,7 +134,7 @@ static int nfsrpc_readrpc(vnode_t , struct uio *, struct ucred *, nfsv4stateid_t *, NFSPROC_T *, struct nfsvattr *, int *); static int nfsrpc_writerpc(vnode_t , struct uio *, int *, int *, struct ucred *, nfsv4stateid_t *, NFSPROC_T *, struct nfsvattr *, int *, - void *); + int); static int nfsrpc_deallocaterpc(vnode_t, off_t, off_t, nfsv4stateid_t *, struct nfsvattr *, int *, struct ucred *, NFSPROC_T *); static int nfsrpc_createv23(vnode_t , char *, int, struct vattr *, @@ -1849,7 +1849,7 @@ nfsmout: int nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, struct ucred *cred, NFSPROC_T *p, struct nfsvattr *nap, int *attrflagp, - void *stuff, int called_from_strategy) + int called_from_strategy, int ioflag) { int error, expireret = 0, retrycnt, nostateid; u_int32_t clidrev = 0; @@ -1893,7 +1893,7 @@ nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, error = 0; else error = nfsrpc_writerpc(vp, uiop, iomode, must_commit, - newcred, &stateid, p, nap, attrflagp, stuff); + newcred, &stateid, p, nap, attrflagp, ioflag); if (error == NFSERR_STALESTATEID) nfscl_initiate_recovery(nmp->nm_clp); if (lckp != NULL) @@ -1928,18 +1928,19 @@ nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, static int nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, struct ucred *cred, nfsv4stateid_t *stateidp, - NFSPROC_T *p, struct nfsvattr *nap, int *attrflagp, void *stuff) + NFSPROC_T *p, struct nfsvattr *nap, int *attrflagp, int ioflag) { u_int32_t *tl; struct nfsmount *nmp = VFSTONFS(vp->v_mount); struct nfsnode *np = VTONFS(vp); - int error = 0, len, tsiz, rlen, commit, committed = NFSWRITE_FILESYNC; - int wccflag = 0, wsize; + int error = 0, len, rlen, commit, committed = NFSWRITE_FILESYNC; + int wccflag = 0; int32_t backup; - struct nfsrv_descript nfsd; - struct nfsrv_descript *nd = &nfsd; + struct nfsrv_descript *nd; nfsattrbit_t attrbits; - off_t tmp_off; + uint64_t tmp_off; + ssize_t tsiz, wsize; + bool do_append; KASSERT(uiop->uio_iovcnt == 1, ("nfs: writerpc iovcnt > 1")); *attrflagp = 0; @@ -1951,14 +1952,31 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, return (EFBIG); } wsize = nmp->nm_wsize; + do_append = false; + if ((ioflag & IO_APPEND) != 0 && NFSHASNFSV4(nmp) && !NFSHASPNFS(nmp)) + do_append = true; NFSUNLOCKMNT(nmp); + nd = malloc(sizeof(*nd), M_TEMP, M_WAITOK); nd->nd_mrep = NULL; /* NFSv2 sometimes does a write with */ nd->nd_repstat = 0; /* uio_resid == 0, so the while is not done */ while (tsiz > 0) { *attrflagp = 0; len = (tsiz > wsize) ? wsize : tsiz; - NFSCL_REQSTART(nd, NFSPROC_WRITE, vp); + if (do_append) + NFSCL_REQSTART(nd, NFSPROC_APPENDWRITE, vp); + else + NFSCL_REQSTART(nd, NFSPROC_WRITE, vp); if (nd->nd_flag & ND_NFSV4) { + if (do_append) { + NFSZERO_ATTRBIT(&attrbits); + NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_SIZE); + nfsrv_putattrbit(nd, &attrbits); + NFSM_BUILD(tl, uint32_t *, 2 * NFSX_UNSIGNED + + NFSX_HYPER); + *tl++ = txdr_unsigned(NFSX_HYPER); + txdr_hyper(uiop->uio_offset, tl); tl += 2; + *tl = txdr_unsigned(NFSV4OP_WRITE); + } nfsm_stateidtom(nd, stateidp, NFSSTATEID_PUTSTATEID); NFSM_BUILD(tl, u_int32_t *, NFSX_HYPER+2*NFSX_UNSIGNED); txdr_hyper(uiop->uio_offset, tl); @@ -2018,8 +2036,10 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, (void) nfsrv_putattrbit(nd, &attrbits); } error = nfscl_request(nd, vp, p, cred); - if (error) + if (error) { + free(nd, M_TEMP); return (error); + } if (nd->nd_repstat) { /* * In case the rpc gets retried, roll @@ -2034,11 +2054,33 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, } if (nd->nd_flag & (ND_NFSV3 | ND_NFSV4)) { error = nfscl_wcc_data(nd, vp, nap, attrflagp, - &wccflag, NULL); + &wccflag, &tmp_off); if (error) goto nfsmout; } + if ((nd->nd_flag & (ND_NFSV4 | ND_NOMOREDATA)) == + (ND_NFSV4 | ND_NOMOREDATA) && + nd->nd_repstat == NFSERR_NOTSAME && do_append) { + /* + * Verify of the file's size failed, so redo the + * write using the file's size as returned in + * the wcc attributes. + */ + if (tmp_off + tsiz <= nmp->nm_maxfilesize) { + do_append = false; + uiop->uio_offset = tmp_off; + m_freem(nd->nd_mrep); + nd->nd_mrep = NULL; + continue; + } else + nd->nd_repstat = EFBIG; + } if (!nd->nd_repstat) { + if (do_append) { + /* Strip off the Write reply status. */ + do_append = false; + NFSM_DISSECT(tl, uint32_t *, 2 * NFSX_UNSIGNED); + } if (nd->nd_flag & (ND_NFSV3 | ND_NFSV4)) { NFSM_DISSECT(tl, u_int32_t *, 2 * NFSX_UNSIGNED + NFSX_VERF); @@ -2103,6 +2145,7 @@ nfsmout: *iomode = committed; if (nd->nd_repstat && !error) error = nd->nd_repstat; + free(nd, M_TEMP); return (error); } diff --git a/sys/fs/nfsclient/nfs_clvnops.c b/sys/fs/nfsclient/nfs_clvnops.c index 4d549909f80a..3bc65f5136f4 100644 --- a/sys/fs/nfsclient/nfs_clvnops.c +++ b/sys/fs/nfsclient/nfs_clvnops.c @@ -1622,7 +1622,7 @@ ncl_readrpc(struct vnode *vp, struct uio *uiop, struct ucred *cred) */ int ncl_writerpc(struct vnode *vp, struct uio *uiop, struct ucred *cred, - int *iomode, int *must_commit, int called_from_strategy) + int *iomode, int *must_commit, int called_from_strategy, int ioflag) { struct nfsvattr nfsva; int error, attrflag, ret; @@ -1637,8 +1637,8 @@ ncl_writerpc(struct vnode *vp, struct uio *uiop, struct ucred *cred, NFSCL_DEBUG(4, "writerpc: aft doiods=%d\n", error); if (error != 0) error = nfsrpc_write(vp, uiop, iomode, must_commit, cred, - uiop->uio_td, &nfsva, &attrflag, NULL, - called_from_strategy); + uiop->uio_td, &nfsva, &attrflag, called_from_strategy, + ioflag); if (attrflag) { if (VTONFS(vp)->n_flag & ND_NFSV4) ret = nfscl_loadattrcache(&vp, &nfsva, NULL, 1, 1);