From owner-svn-src-projects@freebsd.org Mon Feb 17 20:58:34 2020 Return-Path: Delivered-To: svn-src-projects@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85392246F67 for ; Mon, 17 Feb 2020 20:58:34 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48LxCt3Pqzz3HxH; Mon, 17 Feb 2020 20:58:34 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1EF27D11F; Mon, 17 Feb 2020 20:58:34 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 01HKwXOU096535; Mon, 17 Feb 2020 20:58:33 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 01HKwXR8096531; Mon, 17 Feb 2020 20:58:33 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <202002172058.01HKwXR8096531@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 17 Feb 2020 20:58:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r358051 - projects/nfs-over-tls/sys/fs/nfsserver X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: projects/nfs-over-tls/sys/fs/nfsserver X-SVN-Commit-Revision: 358051 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2020 20:58:34 -0000 Author: rmacklem Date: Mon Feb 17 20:58:33 2020 New Revision: 358051 URL: https://svnweb.freebsd.org/changeset/base/358051 Log: Update sys/fs/nfsserver sources so that they handle ext_pgs mbufs. Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c Mon Feb 17 20:35:25 2020 (r358050) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdkrpc.c Mon Feb 17 20:58:33 2020 (r358051) @@ -38,6 +38,7 @@ __FBSDID("$FreeBSD$"); #include "opt_inet6.h" #include "opt_kgssapi.h" +#include "opt_kern_tls.h" #include @@ -110,6 +111,9 @@ extern time_t nfsdev_time; extern int nfsrv_writerpc[NFS_NPROCS]; extern volatile int nfsrv_devidcnt; extern struct nfsv4_opflag nfsv4_opflag[NFSV42_NOPS]; +#ifdef KERN_TLS +extern u_int ktls_maxlen; +#endif /* * NFS server system calls @@ -158,9 +162,11 @@ nfssvc_program(struct svc_req *rqst, SVCXPRT *xprt) */ nd.nd_mrep = rqst->rq_args; rqst->rq_args = NULL; +#ifdef notnow newnfs_realign(&nd.nd_mrep, M_WAITOK); +#endif nd.nd_md = nd.nd_mrep; - nd.nd_dpos = mtod(nd.nd_md, caddr_t); + nfsm_set(&nd, false); nd.nd_nam = svc_getrpccaller(rqst); nd.nd_nam2 = rqst->rq_addr; nd.nd_mreq = NULL; @@ -269,6 +275,14 @@ nfssvc_program(struct svc_req *rqst, SVCXPRT *xprt) } } + if (xprt->xp_tls) + nd.nd_flag |= ND_TLS; + nd.nd_maxextsiz = 16384; +#ifdef KERN_TLS + if (xprt->xp_tls) + nd.nd_maxextsiz = min(TLS_MAX_MSG_SIZE_V10_2, + ktls_maxlen); +#endif cacherep = nfs_proc(&nd, rqst->rq_xid, xprt, &rp); NFSLOCKV4ROOTMUTEX(); nfsv4_relref(&nfsd_suspend_lock); Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Mon Feb 17 20:35:25 2020 (r358050) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdport.c Mon Feb 17 20:58:33 2020 (r358051) @@ -76,6 +76,7 @@ extern struct nfsdontlisthead nfsrv_dontlisthead; extern volatile int nfsrv_dontlistlen; extern volatile int nfsrv_devidcnt; extern int nfsrv_maxpnfsmirror; +extern bool nfs_use_ext_pgs; struct vfsoptlist nfsv4root_opt, nfsv4root_newopt; NFSDLOCKMUTEX; NFSSTATESPINLOCK; @@ -108,8 +109,12 @@ extern struct nfsdevicehead nfsrv_devidhead; static int nfsrv_createiovec(int, struct mbuf **, struct mbuf **, struct iovec **); +static int nfsrv_createiovec_extpgs(int, int, struct mbuf **, + struct mbuf **, struct iovec **); static int nfsrv_createiovecw(int, struct mbuf *, char *, struct iovec **, int *); +static int nfsrv_createiovecw_extpgs(int, struct mbuf *, char *, int, + int, struct iovec **, int *); static void nfsrv_pnfscreate(struct vnode *, struct vattr *, struct ucred *, NFSPROC_T *); static void nfsrv_pnfsremovesetup(struct vnode *, NFSPROC_T *, struct vnode **, @@ -730,8 +735,8 @@ nfsvno_relpathbuf(struct nameidata *ndp) * Readlink vnode op into an mbuf list. */ int -nfsvno_readlink(struct vnode *vp, struct ucred *cred, struct thread *p, - struct mbuf **mpp, struct mbuf **mpendp, int *lenp) +nfsvno_readlink(struct vnode *vp, struct ucred *cred, int maxextsiz, + struct thread *p, struct mbuf **mpp, struct mbuf **mpendp, int *lenp) { struct iovec *iv; struct uio io, *uiop = &io; @@ -739,7 +744,11 @@ nfsvno_readlink(struct vnode *vp, struct ucred *cred, int len, tlen, error = 0; len = NFS_MAXPATHLEN; - uiop->uio_iovcnt = nfsrv_createiovec(len, &mp3, &mp, &iv); + if (maxextsiz > 0) + uiop->uio_iovcnt = nfsrv_createiovec_extpgs(len, maxextsiz, + &mp3, &mp, &iv); + else + uiop->uio_iovcnt = nfsrv_createiovec(len, &mp3, &mp, &iv); uiop->uio_iov = iv; uiop->uio_offset = 0; uiop->uio_resid = len; @@ -756,7 +765,12 @@ nfsvno_readlink(struct vnode *vp, struct ucred *cred, if (uiop->uio_resid > 0) { len -= uiop->uio_resid; tlen = NFSM_RNDUP(len); - nfsrv_adj(mp3, NFS_MAXPATHLEN - tlen, tlen - len); + if (tlen == 0) { + m_freem(mp3); + mp3 = mp = NULL; + } else if (tlen != NFS_MAXPATHLEN || tlen != len) + mp = nfsrv_adj(mp3, NFS_MAXPATHLEN - tlen, + tlen - len); } *lenp = len; *mpp = mp3; @@ -824,11 +838,80 @@ nfsrv_createiovec(int len, struct mbuf **mpp, struct m } /* + * Create an mbuf chain and an associated iovec that can be used to Read + * or Getextattr of data. + * Upon success, return pointers to the first and last mbufs in the chain + * plus the malloc'd iovec and its iovlen. + * Same as above, but creates ext_pgs mbuf(s). + */ +static int +nfsrv_createiovec_extpgs(int len, int maxextsiz, struct mbuf **mpp, + struct mbuf **mpendp, struct iovec **ivp) +{ + struct mbuf *m, *m2 = NULL, *m3; + struct mbuf_ext_pgs *pgs; + struct iovec *iv; + int i, left, pgno, siz; + + left = len; + m3 = NULL; + /* + * Generate the mbuf list with the uio_iov ref. to it. + */ + i = 0; + while (left > 0) { + siz = min(left, maxextsiz); + m = mb_alloc_ext_plus_pages(siz, M_WAITOK, false, + mb_free_mext_pgs); + left -= siz; + i += m->m_ext.ext_pgs->npgs; + if (m3 != NULL) + m2->m_next = m; + else + m3 = m; + m2 = m; + } + *ivp = iv = malloc(i * sizeof (struct iovec), M_TEMP, M_WAITOK); + m = m3; + left = len; + i = 0; + pgno = 0; + pgs = m->m_ext.ext_pgs; + while (left > 0) { + if (m == NULL) + panic("nfsvno_createiovec_extpgs iov"); + siz = min(PAGE_SIZE, left); + if (siz > 0) { + iv->iov_base = (void *)PHYS_TO_DMAP(pgs->pa[pgno]); + iv->iov_len = siz; + m->m_len += siz; + if (pgno == pgs->npgs - 1) + pgs->last_pg_len = siz; + left -= siz; + iv++; + i++; + pgno++; + } + if (pgno == pgs->npgs && left > 0) { + m = m->m_next; + if (m == NULL) + panic("nfsvno_createiovec_extpgs iov"); + pgs = m->m_ext.ext_pgs; + pgno = 0; + } + } + *mpp = m3; + *mpendp = m2; + return (i); +} + +/* * Read vnode op call into mbuf list. */ int nfsvno_read(struct vnode *vp, off_t off, int cnt, struct ucred *cred, - struct thread *p, struct mbuf **mpp, struct mbuf **mpendp) + int maxextsiz, struct thread *p, struct mbuf **mpp, + struct mbuf **mpendp) { struct mbuf *m; struct iovec *iv; @@ -847,7 +930,11 @@ nfsvno_read(struct vnode *vp, off_t off, int cnt, stru return (error); len = NFSM_RNDUP(cnt); - uiop->uio_iovcnt = nfsrv_createiovec(len, &m3, &m, &iv); + if (maxextsiz > 0) + uiop->uio_iovcnt = nfsrv_createiovec_extpgs(len, maxextsiz, + &m3, &m, &iv); + else + uiop->uio_iovcnt = nfsrv_createiovec(len, &m3, &m, &iv); uiop->uio_iov = iv; uiop->uio_offset = off; uiop->uio_resid = len; @@ -871,9 +958,9 @@ nfsvno_read(struct vnode *vp, off_t off, int cnt, stru tlen = NFSM_RNDUP(cnt); if (tlen == 0) { m_freem(m3); - m3 = NULL; + m3 = m = NULL; } else if (len != tlen || tlen != cnt) - nfsrv_adj(m3, len - tlen, tlen - cnt); + m = nfsrv_adj(m3, len - tlen, tlen - cnt); *mpp = m3; *mpendp = m; @@ -943,11 +1030,98 @@ nfsrv_createiovecw(int retlen, struct mbuf *m, char *c } /* + * Create the iovec for the mbuf chain passed in as an argument. + * The "cp" argument is where the data starts within the first mbuf in + * the chain. It returns the iovec and the iovcnt. + * Same as above, but for ext_pgs mbufs. + */ +static int +nfsrv_createiovecw_extpgs(int retlen, struct mbuf *m, char *cp, int dextpg, + int dextpgsiz, struct iovec **ivpp, int *iovcntp) +{ + struct mbuf *mp; + struct mbuf_ext_pgs *pgs; + struct iovec *ivp; + int cnt, i, len, pgno; + + /* + * Loop through the mbuf chain, counting how many pages are + * part of this write oepration, so the iovec size is known. + */ + cnt = 0; + len = retlen; + mp = m; + pgs = mp->m_ext.ext_pgs; + i = dextpgsiz; + pgno = dextpg; + while (len > 0) { + if (i > 0) { + len -= i; + cnt++; + } + if (len > 0) { + if (pgno == pgs->npgs - 1) { + mp = mp->m_next; + if (mp == NULL) + return (EBADRPC); + pgno = 0; + pgs = mp->m_ext.ext_pgs; + } else + pgno++; + if (pgno == 0) + i = mbuf_ext_pg_len(pgs, 0, + pgs->first_pg_off); + else + i = mbuf_ext_pg_len(pgs, pgno, 0); + } + } + + /* Now, create the iovec. */ + mp = m; + *ivpp = ivp = malloc(cnt * sizeof (struct iovec), M_TEMP, + M_WAITOK); + *iovcntp = cnt; + len = retlen; + pgs = mp->m_ext.ext_pgs; + i = dextpgsiz; + pgno = dextpg; + while (len > 0) { + if (i > 0) { + i = min(i, len); + ivp->iov_base = cp; + ivp->iov_len = i; + ivp++; + len -= i; + } + if (len > 0) { + if (pgno == pgs->npgs - 1) { + mp = mp->m_next; + if (mp == NULL) + return (EBADRPC); + pgno = 0; + pgs = mp->m_ext.ext_pgs; + } else + pgno++; + cp = (char *)(void *) + PHYS_TO_DMAP(pgs->pa[pgno]); + if (pgno == 0) { + cp += pgs->first_pg_off; + i = mbuf_ext_pg_len(pgs, 0, + pgs->first_pg_off); + } else + i = mbuf_ext_pg_len(pgs, pgno, 0); + } + } + return (0); +} + +/* * Write vnode op from an mbuf list. */ int nfsvno_write(struct vnode *vp, off_t off, int retlen, int *stable, - struct mbuf *mp, char *cp, struct ucred *cred, struct thread *p) + struct mbuf *mp, char *cp, int dextpg, int dextpgsiz, + struct ucred *cred, struct thread *p) { struct iovec *iv; int cnt, ioflags, error; @@ -970,7 +1144,11 @@ nfsvno_write(struct vnode *vp, off_t off, int retlen, ioflags = IO_NODELOCKED; else ioflags = (IO_SYNC | IO_NODELOCKED); - error = nfsrv_createiovecw(retlen, mp, cp, &iv, &cnt); + if ((mp->m_flags & (M_EXT | M_NOMAP)) == (M_EXT | M_NOMAP)) + error = nfsrv_createiovecw_extpgs(retlen, mp, cp, dextpg, + dextpgsiz, &iv, &cnt); + else + error = nfsrv_createiovecw(retlen, mp, cp, &iv, &cnt); if (error != 0) return (error); uiop->uio_iov = iv; @@ -2135,6 +2313,7 @@ nfsrvd_readdirplus(struct nfsrv_descript *nd, int isdg struct mount *mp, *new_mp; uint64_t mounted_on_fileno; struct thread *p = curthread; + int bextpg0, bextpg1, bextpgsiz0, bextpgsiz1; if (nd->nd_repstat) { nfsrv_postopattr(nd, getret, &at); @@ -2353,6 +2532,8 @@ again: */ mb0 = nd->nd_mb; bpos0 = nd->nd_bpos; + bextpg0 = nd->nd_bextpg; + bextpgsiz0 = nd->nd_bextpgsiz; /* * Fill in the first part of the reply. @@ -2374,6 +2555,8 @@ again: */ mb1 = nd->nd_mb; bpos1 = nd->nd_bpos; + bextpg1 = nd->nd_bextpg; + bextpgsiz1 = nd->nd_bextpgsiz; /* Loop through the records and build reply */ entrycnt = 0; @@ -2390,6 +2573,8 @@ again: */ mb1 = nd->nd_mb; bpos1 = nd->nd_bpos; + bextpg1 = nd->nd_bextpg; + bextpgsiz1 = nd->nd_bextpgsiz; /* * For readdir_and_lookup get the vnode using @@ -2615,11 +2800,11 @@ invalid: if (!nd->nd_repstat && entrycnt == 0) nd->nd_repstat = NFSERR_TOOSMALL; if (nd->nd_repstat) { - newnfs_trimtrailing(nd, mb0, bpos0); + nfsm_trimtrailing(nd, mb0, bpos0, bextpg0, bextpgsiz0); if (nd->nd_flag & ND_NFSV3) nfsrv_postopattr(nd, getret, &at); } else - newnfs_trimtrailing(nd, mb1, bpos1); + nfsm_trimtrailing(nd, mb1, bpos1, bextpg1, bextpgsiz1); eofflag = 0; } else if (cpos < cend) eofflag = 0; @@ -4928,7 +5113,7 @@ nfsrv_readdsrpc(fhandle_t *fhp, off_t off, int len, st st.other[2] = 0x55555555; st.seqid = 0xffffffff; nfscl_reqstart(nd, NFSPROC_READDS, nmp, (u_int8_t *)fhp, sizeof(*fhp), - NULL, NULL, 0, 0); + NULL, NULL, 0, 0, false); nfsm_stateidtom(nd, &st, NFSSTATEID_PUTSTATEID); NFSM_BUILD(tl, uint32_t *, NFSX_UNSIGNED * 3); txdr_hyper(off, tl); @@ -4963,10 +5148,15 @@ nfsrv_readdsrpc(fhandle_t *fhp, off_t off, int len, st * Now, adjust first mbuf so that any XDR before the * read data is skipped over. */ - trimlen = nd->nd_dpos - mtod(m, char *); - if (trimlen > 0) { - m->m_len -= trimlen; - NFSM_DATAP(m, trimlen); + if ((nd->nd_md->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) + nfsm_trimatpos_extpgs(nd); + else { + trimlen = nd->nd_dpos - mtod(m, char *); + if (trimlen > 0) { + m->m_len -= trimlen; + NFSM_DATAP(m, trimlen); + } } /* @@ -4977,7 +5167,11 @@ nfsrv_readdsrpc(fhandle_t *fhp, off_t off, int len, st tlen = NFSM_RNDUP(retlen); do { if (m->m_len >= tlen) { - m->m_len = tlen; + if ((m->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) + nfsm_trimback_extpgs(m, tlen); + else + m->m_len = tlen; tlen = 0; m2 = m->m_next; m->m_next = NULL; @@ -5036,7 +5230,7 @@ nfsrv_writedsdorpc(struct nfsmount *nmp, fhandle_t *fh nd = malloc(sizeof(*nd), M_TEMP, M_WAITOK | M_ZERO); nfscl_reqstart(nd, NFSPROC_WRITE, nmp, (u_int8_t *)fhp, - sizeof(fhandle_t), NULL, NULL, 0, 0); + sizeof(fhandle_t), NULL, NULL, 0, 0, false); /* * Use a stateid where other is an alternating 01010 pattern and @@ -5258,7 +5452,7 @@ nfsrv_allocatedsdorpc(struct nfsmount *nmp, fhandle_t nd = malloc(sizeof(*nd), M_TEMP, M_WAITOK | M_ZERO); nfscl_reqstart(nd, NFSPROC_ALLOCATE, nmp, (u_int8_t *)fhp, - sizeof(fhandle_t), NULL, NULL, 0, 0); + sizeof(fhandle_t), NULL, NULL, 0, 0, false); /* * Use a stateid where other is an alternating 01010 pattern and @@ -5412,7 +5606,7 @@ nfsrv_setattrdsdorpc(fhandle_t *fhp, struct ucred *cre st.other[2] = 0x55555555; st.seqid = 0xffffffff; nfscl_reqstart(nd, NFSPROC_SETATTR, nmp, (u_int8_t *)fhp, sizeof(*fhp), - NULL, NULL, 0, 0); + NULL, NULL, 0, 0, false); nfsm_stateidtom(nd, &st, NFSSTATEID_PUTSTATEID); nfscl_fillsattr(nd, &nap->na_vattr, vp, NFSSATTR_FULL, 0); @@ -5597,7 +5791,7 @@ nfsrv_setacldsdorpc(fhandle_t *fhp, struct ucred *cred st.other[2] = 0x55555555; st.seqid = 0xffffffff; nfscl_reqstart(nd, NFSPROC_SETACL, nmp, (u_int8_t *)fhp, sizeof(*fhp), - NULL, NULL, 0, 0); + NULL, NULL, 0, 0, false); nfsm_stateidtom(nd, &st, NFSSTATEID_PUTSTATEID); NFSZERO_ATTRBIT(&attrbits); NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_ACL); @@ -5732,7 +5926,7 @@ nfsrv_getattrdsrpc(fhandle_t *fhp, struct ucred *cred, NFSD_DEBUG(4, "in nfsrv_getattrdsrpc\n"); nd = malloc(sizeof(*nd), M_TEMP, M_WAITOK | M_ZERO); nfscl_reqstart(nd, NFSPROC_GETATTR, nmp, (u_int8_t *)fhp, - sizeof(fhandle_t), NULL, NULL, 0, 0); + sizeof(fhandle_t), NULL, NULL, 0, 0, false); NFSZERO_ATTRBIT(&attrbits); NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_SIZE); NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_CHANGE); @@ -5800,7 +5994,7 @@ nfsrv_seekdsrpc(fhandle_t *fhp, off_t *offp, int conte st.seqid = 0xffffffff; nd = malloc(sizeof(*nd), M_TEMP, M_WAITOK | M_ZERO); nfscl_reqstart(nd, NFSPROC_SEEKDS, nmp, (u_int8_t *)fhp, - sizeof(fhandle_t), NULL, NULL, 0, 0); + sizeof(fhandle_t), NULL, NULL, 0, 0, false); nfsm_stateidtom(nd, &st, NFSSTATEID_PUTSTATEID); NFSM_BUILD(tl, uint32_t *, NFSX_HYPER + NFSX_UNSIGNED); txdr_hyper(*offp, tl); tl += 2; @@ -6140,8 +6334,8 @@ nfsvno_allocate(struct vnode *vp, off_t off, off_t len */ int nfsvno_getxattr(struct vnode *vp, char *name, uint32_t maxresp, - struct ucred *cred, struct thread *p, struct mbuf **mpp, - struct mbuf **mpendp, int *lenp) + struct ucred *cred, uint64_t flag, int maxextsiz, struct thread *p, + struct mbuf **mpp, struct mbuf **mpendp, int *lenp) { struct iovec *iv; struct uio io, *uiop = &io; @@ -6158,7 +6352,17 @@ nfsvno_getxattr(struct vnode *vp, char *name, uint32_t return (NFSERR_XATTR2BIG); len = siz; tlen = NFSM_RNDUP(len); - uiop->uio_iovcnt = nfsrv_createiovec(tlen, &m, &m2, &iv); + /* + * If the cnt is larger than MCLBYTES, use ext_pgs if + * possible. + * Always use ext_pgs if ND_EXTPG is set. + */ + if ((flag & ND_EXTPG) != 0 || (tlen > MCLBYTES && + PMAP_HAS_DMAP != 0 && ((flag & ND_TLS) != 0 || nfs_use_ext_pgs))) + uiop->uio_iovcnt = nfsrv_createiovec_extpgs(tlen, maxextsiz, + &m, &m2, &iv); + else + uiop->uio_iovcnt = nfsrv_createiovec(tlen, &m, &m2, &iv); uiop->uio_iov = iv; uiop->uio_offset = 0; uiop->uio_resid = tlen; @@ -6182,7 +6386,11 @@ nfsvno_getxattr(struct vnode *vp, char *name, uint32_t tlen = NFSM_RNDUP(len); if (alen != tlen) printf("nfsvno_getxattr: weird size read\n"); - nfsrv_adj(m, alen - tlen, tlen - len); + if (tlen == 0) { + m_freem(m); + m = m2 = NULL; + } else if (alen != tlen || tlen != len) + m2 = nfsrv_adj(m, alen - tlen, tlen - len); } *lenp = len; *mpp = m; @@ -6203,7 +6411,8 @@ out: */ int nfsvno_setxattr(struct vnode *vp, char *name, int len, struct mbuf *m, - char *cp, struct ucred *cred, struct thread *p) + char *cp, int dextpg, int dextpgsiz, struct ucred *cred, + struct thread *p) { struct iovec *iv; struct uio uio, *uiop = &uio; @@ -6222,7 +6431,11 @@ nfsvno_setxattr(struct vnode *vp, char *name, int len, uiop->uio_td = p; uiop->uio_offset = 0; uiop->uio_resid = len; - error = nfsrv_createiovecw(len, m, cp, &iv, &cnt); + if ((m->m_flags & (M_EXT | M_NOMAP)) == (M_EXT | M_NOMAP)) + error = nfsrv_createiovecw_extpgs(len, m, cp, dextpg, + dextpgsiz, &iv, &cnt); + else + error = nfsrv_createiovecw(len, m, cp, &iv, &cnt); uiop->uio_iov = iv; uiop->uio_iovcnt = cnt; if (error == 0) { Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Mon Feb 17 20:35:25 2020 (r358050) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdserv.c Mon Feb 17 20:58:33 2020 (r358051) @@ -68,6 +68,7 @@ extern u_long sb_max_adj; extern int nfsrv_pnfsatime; extern int nfsrv_maxpnfsmirror; extern int nfs_maxcopyrange; +extern bool nfs_use_ext_pgs; #endif /* !APPLEKEXT */ static int nfs_async = 0; @@ -665,6 +666,8 @@ nfsrvd_readlink(struct nfsrv_descript *nd, __unused in int getret = 1, len; struct nfsvattr nva; struct thread *p = curthread; + struct mbuf_ext_pgs *pgs; + uint16_t off; if (nd->nd_repstat) { nfsrv_postopattr(nd, getret, &nva); @@ -676,9 +679,14 @@ nfsrvd_readlink(struct nfsrv_descript *nd, __unused in else nd->nd_repstat = EINVAL; } - if (!nd->nd_repstat) - nd->nd_repstat = nfsvno_readlink(vp, nd->nd_cred, p, - &mp, &mpend, &len); + if (nd->nd_repstat == 0) { + if ((nd->nd_flag & ND_EXTPG) != 0) + nd->nd_repstat = nfsvno_readlink(vp, nd->nd_cred, + nd->nd_maxextsiz, p, &mp, &mpend, &len); + else + nd->nd_repstat = nfsvno_readlink(vp, nd->nd_cred, + 0, p, &mp, &mpend, &len); + } if (nd->nd_flag & ND_NFSV3) getret = nfsvno_getattr(vp, &nva, nd, p, 1, NULL); vput(vp); @@ -688,9 +696,21 @@ nfsrvd_readlink(struct nfsrv_descript *nd, __unused in goto out; NFSM_BUILD(tl, u_int32_t *, NFSX_UNSIGNED); *tl = txdr_unsigned(len); - mbuf_setnext(nd->nd_mb, mp); - nd->nd_mb = mpend; - nd->nd_bpos = NFSMTOD(mpend, caddr_t) + mbuf_len(mpend); + if (mp != NULL) { + nd->nd_mb->m_next = mp; + nd->nd_mb = mpend; + if ((mpend->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) { + pgs = mpend->m_ext.ext_pgs; + nd->nd_bextpg = pgs->npgs - 1; + nd->nd_bpos = (char *)(void *) + PHYS_TO_DMAP(pgs->pa[nd->nd_bextpg]); + off = (nd->nd_bextpg == 0) ? pgs->first_pg_off : 0; + nd->nd_bpos += off + pgs->last_pg_len; + nd->nd_bextpgsiz = PAGE_SIZE - pgs->last_pg_len - off; + } else + nd->nd_bpos = mtod(mpend, char *) + mpend->m_len; + } out: NFSEXITCODE2(0, nd); @@ -714,6 +734,8 @@ nfsrvd_read(struct nfsrv_descript *nd, __unused int is nfsv4stateid_t stateid; nfsquad_t clientid; struct thread *p = curthread; + struct mbuf_ext_pgs *pgs; + uint16_t poff; if (nd->nd_repstat) { nfsrv_postopattr(nd, getret, &nva); @@ -835,8 +857,18 @@ nfsrvd_read(struct nfsrv_descript *nd, __unused int is cnt = reqlen; m3 = NULL; if (cnt > 0) { - nd->nd_repstat = nfsvno_read(vp, off, cnt, nd->nd_cred, p, - &m3, &m2); + /* + * If the cnt is larger than MCLBYTES, use ext_pgs if + * possible. + * Always use ext_pgs if ND_EXTPG is set. + */ + if ((nd->nd_flag & ND_EXTPG) != 0 || (PMAP_HAS_DMAP != 0 && + ((nd->nd_flag & ND_TLS) != 0 || nfs_use_ext_pgs))) + nd->nd_repstat = nfsvno_read(vp, off, cnt, nd->nd_cred, + nd->nd_maxextsiz, p, &m3, &m2); + else + nd->nd_repstat = nfsvno_read(vp, off, cnt, nd->nd_cred, + 0, p, &m3, &m2); if (!(nd->nd_flag & ND_NFSV4)) { getret = nfsvno_getattr(vp, &nva, nd, p, 1, NULL); if (!nd->nd_repstat) @@ -869,9 +901,20 @@ nfsrvd_read(struct nfsrv_descript *nd, __unused int is } *tl = txdr_unsigned(cnt); if (m3) { - mbuf_setnext(nd->nd_mb, m3); + nd->nd_mb->m_next = m3; nd->nd_mb = m2; - nd->nd_bpos = NFSMTOD(m2, caddr_t) + mbuf_len(m2); + if ((m2->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) { + nd->nd_flag |= ND_EXTPG; + pgs = m2->m_ext.ext_pgs; + nd->nd_bextpg = pgs->npgs - 1; + nd->nd_bpos = (char *)(void *) + PHYS_TO_DMAP(pgs->pa[nd->nd_bextpg]); + poff = (nd->nd_bextpg == 0) ? pgs->first_pg_off : 0; + nd->nd_bpos += poff + pgs->last_pg_len; + nd->nd_bextpgsiz = PAGE_SIZE - pgs->last_pg_len - poff; + } else + nd->nd_bpos = mtod(m2, char *) + m2->m_len; } out: @@ -1014,7 +1057,8 @@ nfsrvd_write(struct nfsrv_descript *nd, __unused int i */ if (retlen > 0) { nd->nd_repstat = nfsvno_write(vp, off, retlen, &stable, - nd->nd_md, nd->nd_dpos, nd->nd_cred, p); + nd->nd_md, nd->nd_dpos, nd->nd_dextpg, nd->nd_dextpgsiz, + nd->nd_cred, p); error = nfsm_advance(nd, NFSM_RNDUP(retlen), -1); if (error) goto nfsmout; @@ -5509,6 +5553,8 @@ nfsrvd_getxattr(struct nfsrv_descript *nd, __unused in int error, len; char *name; struct thread *p = curthread; + struct mbuf_ext_pgs *pgs; + uint16_t off; error = 0; if (nfs_rootfhset == 0 || nfsd_checkrootexp(nd) != 0) { @@ -5528,8 +5574,9 @@ nfsrvd_getxattr(struct nfsrv_descript *nd, __unused in name = malloc(len + 1, M_TEMP, M_WAITOK); nd->nd_repstat = nfsrv_mtostr(nd, name, len); if (nd->nd_repstat == 0) - nd->nd_repstat = nfsvno_getxattr(vp, name, nd->nd_maxresp, - nd->nd_cred, p, &mp, &mpend, &len); + nd->nd_repstat = nfsvno_getxattr(vp, name, + nd->nd_maxresp, nd->nd_cred, nd->nd_flag, + nd->nd_maxextsiz, p, &mp, &mpend, &len); if (nd->nd_repstat == ENOATTR) nd->nd_repstat = NFSERR_NOXATTR; else if (nd->nd_repstat == EOPNOTSUPP) @@ -5537,9 +5584,22 @@ nfsrvd_getxattr(struct nfsrv_descript *nd, __unused in if (nd->nd_repstat == 0) { NFSM_BUILD(tl, uint32_t *, NFSX_UNSIGNED); *tl = txdr_unsigned(len); - mbuf_setnext(nd->nd_mb, mp); - nd->nd_mb = mpend; - nd->nd_bpos = NFSMTOD(mpend, caddr_t) + mbuf_len(mpend); + if (mp != NULL) { + nd->nd_mb->m_next = mp; + nd->nd_mb = mpend; + if ((mpend->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) { + nd->nd_flag |= ND_EXTPG; + pgs = mpend->m_ext.ext_pgs; + nd->nd_bextpg = pgs->npgs - 1; + nd->nd_bpos = (char *)(void *) + PHYS_TO_DMAP(pgs->pa[nd->nd_bextpg]); + off = (nd->nd_bextpg == 0) ? pgs->first_pg_off : 0; + nd->nd_bpos += off + pgs->last_pg_len; + nd->nd_bextpgsiz = PAGE_SIZE - pgs->last_pg_len - off; + } else + nd->nd_bpos = mtod(mpend, char *) + mpend->m_len; + } } free(name, M_TEMP); @@ -5621,7 +5681,8 @@ nfsrvd_setxattr(struct nfsrv_descript *nd, __unused in nd->nd_repstat = nfsvno_getattr(vp, &ova, nd, p, 1, &attrbits); if (nd->nd_repstat == 0) { nd->nd_repstat = nfsvno_setxattr(vp, name, len, nd->nd_md, - nd->nd_dpos, nd->nd_cred, p); + nd->nd_dpos, nd->nd_dextpg, nd->nd_dextpgsiz, nd->nd_cred, + p); if (nd->nd_repstat == ENXIO) nd->nd_repstat = NFSERR_XATTR2BIG; } Modified: projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c ============================================================================== --- projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c Mon Feb 17 20:35:25 2020 (r358050) +++ projects/nfs-over-tls/sys/fs/nfsserver/nfs_nfsdsubs.c Mon Feb 17 20:58:33 2020 (r358051) @@ -1270,62 +1270,96 @@ static short *nfsrv_v4errmap[] = { }; /* - * A fiddled version of m_adj() that ensures null fill to a long - * boundary and only trims off the back end + * Trim tlen bytes off the end of the mbuf list and then ensure + * the end of the last mbuf is nul filled to a long boundary, + * as indicated by the value of "nul". + * Return the last mbuf in the updated list and free and mbufs + * that follow it in the original list. + * This is somewhat different than the old nfsrv_adj() with + * support for ext_pgs mbufs. It frees the remaining mbufs + * instead of setting them 0 length, since lists of ext_pgs + * mbufs are all expected to be non-empty. */ -APPLESTATIC void -nfsrv_adj(mbuf_t mp, int len, int nul) +struct mbuf * +nfsrv_adj(struct mbuf *mp, int len, int nul) { - mbuf_t m; - int count, i; + struct mbuf *m, *m2; + struct mbuf_ext_pgs *pgs; + vm_page_t pg; + int i, lastlen, pgno, plen, tlen, trim; + uint16_t off; char *cp; /* - * Trim from tail. Scan the mbuf chain, - * calculating its length and finding the last mbuf. - * If the adjustment only affects this mbuf, then just - * adjust and return. Otherwise, rescan and truncate - * after the remaining size. + * Find the last mbuf after adjustment and + * how much it needs to be adjusted by. */ - count = 0; + tlen = 0; m = mp; for (;;) { - count += mbuf_len(m); - if (mbuf_next(m) == NULL) + tlen += m->m_len; + if (m->m_next == NULL) break; - m = mbuf_next(m); + m = m->m_next; } - if (mbuf_len(m) > len) { - mbuf_setlen(m, mbuf_len(m) - len); - if (nul > 0) { - cp = NFSMTOD(m, caddr_t) + mbuf_len(m) - nul; - for (i = 0; i < nul; i++) - *cp++ = '\0'; + + /* m is now the last mbuf and tlen the total length. */ + if (len >= m->m_len) { + /* Need to trim away the last mbuf(s). */ + i = tlen - len; + m = mp; + for (;;) { + if (m->m_len >= i) + break; + i -= m->m_len; + m = m->m_next; } - return; - } - count -= len; - if (count < 0) - count = 0; - /* - * Correct length for chain is "count". - * Find the mbuf with last data, adjust its length, - * and toss data from remaining mbufs on chain. - */ - for (m = mp; m; m = mbuf_next(m)) { - if (mbuf_len(m) >= count) { - mbuf_setlen(m, count); - if (nul > 0) { - cp = NFSMTOD(m, caddr_t) + mbuf_len(m) - nul; - for (i = 0; i < nul; i++) - *cp++ = '\0'; + lastlen = i; + } else + lastlen = m->m_len - len; + + /* Adjust the last mbuf. */ + if ((m->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) { + pgs = m->m_ext.ext_pgs; + pgno = pgs->npgs - 1; + off = (pgno == 0) ? pgs->first_pg_off : 0; + plen = mbuf_ext_pg_len(pgs, pgno, off); + if (m->m_len > lastlen) { + /* Trim this mbuf. */ + trim = m->m_len - lastlen; + while (trim >= plen) { + /* Free page. */ + pg = PHYS_TO_VM_PAGE(pgs->pa[pgno]); + vm_page_unwire_noq(pg); + vm_page_free(pg); + trim -= plen; + pgs->npgs--; + pgno--; + off = (pgno == 0) ? pgs->first_pg_off : 0; + plen = mbuf_ext_pg_len(pgs, pgno, off); } - break; + plen -= trim; + pgs->last_pg_len = plen; + m->m_len = lastlen; } - count -= mbuf_len(m); + cp = (char *)(void *)PHYS_TO_DMAP(pgs->pa[pgno]); + cp += off + plen - nul; + } else { + m->m_len = lastlen; + cp = mtod(m, char *) + m->m_len - nul; } - for (m = mbuf_next(m); m; m = mbuf_next(m)) - mbuf_setlen(m, 0); + + /* Write the nul bytes. */ + for (i = 0; i < nul; i++) + *cp++ = '\0'; + + /* Free up any mbufs past "m". */ + m2 = m->m_next; + m->m_next = NULL; + if (m2 != NULL) + m_freem(m2); + return (m); } /* @@ -1837,8 +1871,9 @@ APPLESTATIC int nfsrv_parsename(struct nfsrv_descript *nd, char *bufp, u_long *hashp, NFSPATHLEN_T *outlenp) { + struct mbuf_ext_pgs *pgs; + vm_page_t pg; char *fromcp, *tocp, val = '\0'; - mbuf_t md; int i; int rem, len, error = 0, pubtype = 0, outlen = 0, percent = 0; char digit; @@ -1853,177 +1888,194 @@ nfsrv_parsename(struct nfsrv_descript *nd, char *bufp, * Otherwise, get the component name. */ if ((nd->nd_flag & ND_NFSV4) && nd->nd_procnum == NFSV4OP_LOOKUPP) { - *tocp++ = '.'; - hash += ((u_char)'.'); - *tocp++ = '.'; - hash += ((u_char)'.'); - outlen = 2; + *tocp++ = '.'; + hash += ((u_char)'.'); + *tocp++ = '.'; + hash += ((u_char)'.'); + outlen = 2; } else { - /* - * First, get the name length. - */ - NFSM_DISSECT(tl, u_int32_t *, NFSX_UNSIGNED); - len = fxdr_unsigned(int, *tl); - if (len > NFS_MAXNAMLEN) { - nd->nd_repstat = NFSERR_NAMETOL; - error = 0; - goto nfsmout; - } else if (len <= 0) { - nd->nd_repstat = NFSERR_INVAL; - error = 0; - goto nfsmout; - } - - /* - * Now, copy the component name into the buffer. - */ - fromcp = nd->nd_dpos; - md = nd->nd_md; - rem = NFSMTOD(md, caddr_t) + mbuf_len(md) - fromcp; - for (i = 0; i < len; i++) { - while (rem == 0) { - md = mbuf_next(md); - if (md == NULL) { - error = EBADRPC; - goto nfsmout; - } - fromcp = NFSMTOD(md, caddr_t); - rem = mbuf_len(md); - } - if (*fromcp == '\0') { - nd->nd_repstat = EACCES; + /* + * First, get the name length. + */ + NFSM_DISSECT(tl, u_int32_t *, NFSX_UNSIGNED); + len = fxdr_unsigned(int, *tl); + if (len > NFS_MAXNAMLEN) { + nd->nd_repstat = NFSERR_NAMETOL; error = 0; goto nfsmout; + } else if (len <= 0) { + nd->nd_repstat = NFSERR_INVAL; + error = 0; + goto nfsmout; } + /* - * For lookups on the public filehandle, do some special - * processing on the name. (The public file handle is the - * root of the public file system for this server.) + * Now, copy the component name into the buffer. */ - if (nd->nd_flag & ND_PUBLOOKUP) { - /* - * If the first char is ASCII, it is a canonical - * path, otherwise it is a native path. (RFC2054 - * doesn't actually state what it is if the first - * char isn't ASCII or 0x80, so I assume native.) - * pubtype == 1 -> native path - * pubtype == 2 -> canonical path - */ - if (i == 0) { - if (*fromcp & 0x80) { - /* - * Since RFC2054 doesn't indicate - * that a native path of just 0x80 - * isn't allowed, I'll replace the - * 0x80 with '/' instead of just - * throwing it away. - */ - *fromcp = '/'; - pubtype = 1; + fromcp = nd->nd_dpos; + if ((nd->nd_md->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP)) + rem = nd->nd_dextpgsiz; + else + rem = mtod(nd->nd_md, char *) + nd->nd_md->m_len - + fromcp; + for (i = 0; i < len; i++) { + while (rem == 0) { + if ((nd->nd_md->m_flags & (M_EXT | M_NOMAP)) == + (M_EXT | M_NOMAP) && nd->nd_dextpg < + nd->nd_md->m_ext.ext_pgs->npgs - 1) { + pgs = nd->nd_md->m_ext.ext_pgs; + pg = PHYS_TO_VM_PAGE( + pgs->pa[nd->nd_dextpg]); + vm_page_unwire_noq(pg); + vm_page_free(pg); + for (i = nd->nd_bextpg; + i < pgs->npgs - 1; i++) + pgs->pa[i] = pgs->pa[i + 1]; + pgs->npgs--; + if (nd->nd_dextpg == 0) + pgs->first_pg_off = 0; + fromcp = nd->nd_dpos = (char *)(void *) + PHYS_TO_DMAP(pgs->pa[nd->nd_dextpg]); + rem = nd->nd_dextpgsiz = + mbuf_ext_pg_len(pgs, nd->nd_dextpg, 0); } else { - pubtype = 2; + if (!nfsm_shiftnext(nd, &rem)) { + error = EBADRPC; + goto nfsmout; + } + fromcp = nd->nd_dpos; } } - /* *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***