From owner-svn-src-projects@freebsd.org Thu May 31 02:25:48 2018 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9AECBEF6F4C for ; Thu, 31 May 2018 02:25:48 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4976068DDD; Thu, 31 May 2018 02:25:48 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2A94913E06; Thu, 31 May 2018 02:25:48 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w4V2PmF9058045; Thu, 31 May 2018 02:25:48 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w4V2Pk1w058039; Thu, 31 May 2018 02:25:46 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201805310225.w4V2Pk1w058039@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Thu, 31 May 2018 02:25:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r334411 - in projects/pnfs-planb-server/sys/fs: nfs nfsclient nfsserver X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/pnfs-planb-server/sys/fs: nfs nfsclient nfsserver X-SVN-Commit-Revision: 334411 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2018 02:25:48 -0000 Author: rmacklem Date: Thu May 31 02:25:46 2018 New Revision: 334411 URL: https://svnweb.freebsd.org/changeset/base/334411 Log: Fix the handling of statfs so that the pNFS service reports the sum of the blocks for the DSs to the NFS clients instead of the block counts for the MDS file system. Reported by: james.rose@framestore.com Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_commonsubs.c projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_commonsubs.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfs/nfs_commonsubs.c Thu May 31 02:22:40 2018 (r334410) +++ projects/pnfs-planb-server/sys/fs/nfs/nfs_commonsubs.c Thu May 31 02:25:46 2018 (r334411) @@ -2068,7 +2068,8 @@ APPLESTATIC int nfsv4_fillattr(struct nfsrv_descript *nd, struct mount *mp, vnode_t vp, NFSACL_T *saclp, struct vattr *vap, fhandle_t *fhp, int rderror, nfsattrbit_t *attrbitp, struct ucred *cred, NFSPROC_T *p, int isdgram, - int reterr, int supports_nfsv4acls, int at_root, uint64_t mounted_on_fileno) + int reterr, int supports_nfsv4acls, int at_root, uint64_t mounted_on_fileno, + struct statfs *pnfssf) { int bitpos, retnum = 0; u_int32_t *tl; @@ -2470,25 +2471,45 @@ nfsv4_fillattr(struct nfsrv_descript *nd, struct mount break; case NFSATTRBIT_SPACEAVAIL: NFSM_BUILD(tl, u_int32_t *, NFSX_HYPER); - if (priv_check_cred(cred, PRIV_VFS_BLOCKRESERVE, 0)) - uquad = (u_int64_t)fs->f_bfree; + if (priv_check_cred(cred, PRIV_VFS_BLOCKRESERVE, 0)) { + if (pnfssf != NULL) + uquad = (u_int64_t)pnfssf->f_bfree; + else + uquad = (u_int64_t)fs->f_bfree; + } else { + if (pnfssf != NULL) + uquad = (u_int64_t)pnfssf->f_bavail; + else + uquad = (u_int64_t)fs->f_bavail; + } + if (pnfssf != NULL) + uquad *= pnfssf->f_bsize; else - uquad = (u_int64_t)fs->f_bavail; - uquad *= fs->f_bsize; + uquad *= fs->f_bsize; txdr_hyper(uquad, tl); retnum += NFSX_HYPER; break; case NFSATTRBIT_SPACEFREE: NFSM_BUILD(tl, u_int32_t *, NFSX_HYPER); - uquad = (u_int64_t)fs->f_bfree; - uquad *= fs->f_bsize; + if (pnfssf != NULL) { + uquad = (u_int64_t)pnfssf->f_bfree; + uquad *= pnfssf->f_bsize; + } else { + uquad = (u_int64_t)fs->f_bfree; + uquad *= fs->f_bsize; + } txdr_hyper(uquad, tl); retnum += NFSX_HYPER; break; case NFSATTRBIT_SPACETOTAL: NFSM_BUILD(tl, u_int32_t *, NFSX_HYPER); - uquad = (u_int64_t)fs->f_blocks; - uquad *= fs->f_bsize; + if (pnfssf != NULL) { + uquad = (u_int64_t)pnfssf->f_blocks; + uquad *= pnfssf->f_bsize; + } else { + uquad = (u_int64_t)fs->f_blocks; + uquad *= fs->f_bsize; + } txdr_hyper(uquad, tl); retnum += NFSX_HYPER; break; Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Thu May 31 02:22:40 2018 (r334410) +++ projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Thu May 31 02:25:46 2018 (r334411) @@ -372,7 +372,7 @@ void nfsrv_wcc(struct nfsrv_descript *, int, struct nf struct nfsvattr *); int nfsv4_fillattr(struct nfsrv_descript *, struct mount *, vnode_t, NFSACL_T *, struct vattr *, fhandle_t *, int, nfsattrbit_t *, - struct ucred *, NFSPROC_T *, int, int, int, int, uint64_t); + struct ucred *, NFSPROC_T *, int, int, int, int, uint64_t, struct statfs *); void nfsrv_fillattr(struct nfsrv_descript *, struct nfsvattr *); void nfsrv_adj(mbuf_t, int, int); void nfsrv_postopattr(struct nfsrv_descript *, int, struct nfsvattr *); Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c Thu May 31 02:22:40 2018 (r334410) +++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c Thu May 31 02:25:46 2018 (r334411) @@ -931,7 +931,7 @@ nfscl_fillsattr(struct nfsrv_descript *nd, struct vatt if (vap->va_mtime.tv_sec != VNOVAL) NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_TIMEMODIFYSET); (void) nfsv4_fillattr(nd, vp->v_mount, vp, NULL, vap, NULL, 0, - &attrbits, NULL, NULL, 0, 0, 0, 0, (uint64_t)0); + &attrbits, NULL, NULL, 0, 0, 0, 0, (uint64_t)0, NULL); break; } } Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c Thu May 31 02:22:40 2018 (r334410) +++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c Thu May 31 02:25:46 2018 (r334411) @@ -4619,7 +4619,7 @@ nfsrpc_setaclrpc(vnode_t vp, struct ucred *cred, NFSPR NFSZERO_ATTRBIT(&attrbits); NFSSETBIT_ATTRBIT(&attrbits, NFSATTRBIT_ACL); (void) nfsv4_fillattr(nd, vnode_mount(vp), vp, aclp, NULL, NULL, 0, - &attrbits, NULL, NULL, 0, 0, 0, 0, (uint64_t)0); + &attrbits, NULL, NULL, 0, 0, 0, 0, (uint64_t)0, NULL); error = nfscl_request(nd, vp, p, cred, stuff); if (error) return (error); Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Thu May 31 02:22:40 2018 (r334410) +++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Thu May 31 02:25:46 2018 (r334411) @@ -3374,7 +3374,7 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) if (!error) (void) nfsv4_fillattr(nd, NULL, NULL, NULL, &va, NULL, 0, &rattrbits, NULL, p, 0, 0, 0, 0, - (uint64_t)0); + (uint64_t)0, NULL); break; case NFSV4OP_CBRECALL: NFSCL_DEBUG(4, "cbrecall\n"); Modified: projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c Thu May 31 02:22:40 2018 (r334410) +++ projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c Thu May 31 02:25:46 2018 (r334411) @@ -131,6 +131,7 @@ static int nfsrv_pnfslookupds(struct vnode *, struct v static void nfsrv_pnfssetfh(struct vnode *, struct pnfsdsfile *, struct vnode *, NFSPROC_T *); static int nfsrv_dsremove(struct vnode *, char *, struct ucred *, NFSPROC_T *); +static int nfsrv_pnfsstatfs(struct statfs *); int nfs_pnfsio(task_fn_t *, void *); @@ -1567,10 +1568,27 @@ nfsvno_fsync(struct vnode *vp, u_int64_t off, int cnt, int nfsvno_statfs(struct vnode *vp, struct statfs *sf) { + struct statfs *tsf; int error; + tsf = NULL; + if (nfsrv_devidcnt > 0) { + /* For a pNFS service, get the DS numbers. */ + tsf = malloc(sizeof(*tsf), M_TEMP, M_WAITOK | M_ZERO); + error = nfsrv_pnfsstatfs(tsf); + if (error != 0) { + free(tsf, M_TEMP); + tsf = NULL; + } + } error = VFS_STATFS(vp->v_mount, sf); if (error == 0) { + if (tsf != NULL) { + sf->f_blocks = tsf->f_blocks; + sf->f_bavail = tsf->f_bavail; + sf->f_bfree = tsf->f_bfree; + sf->f_bsize = tsf->f_bsize; + } /* * Since NFS handles these values as unsigned on the * wire, there is no way to represent negative values, @@ -1583,6 +1601,7 @@ nfsvno_statfs(struct vnode *vp, struct statfs *sf) if (sf->f_ffree < 0) sf->f_ffree = 0; } + free(tsf, M_TEMP); NFSEXITCODE(error); return (error); } @@ -1728,11 +1747,25 @@ nfsvno_fillattr(struct nfsrv_descript *nd, struct moun struct ucred *cred, struct thread *p, int isdgram, int reterr, int supports_nfsv4acls, int at_root, uint64_t mounted_on_fileno) { + struct statfs *sf; int error; + sf = NULL; + if (nfsrv_devidcnt > 0 && + (NFSISSET_ATTRBIT(attrbitp, NFSATTRBIT_SPACEAVAIL) || + NFSISSET_ATTRBIT(attrbitp, NFSATTRBIT_SPACEFREE) || + NFSISSET_ATTRBIT(attrbitp, NFSATTRBIT_SPACETOTAL))) { + sf = malloc(sizeof(*sf), M_TEMP, M_WAITOK | M_ZERO); + error = nfsrv_pnfsstatfs(sf); + if (error != 0) { + free(sf, M_TEMP); + sf = NULL; + } + } error = nfsv4_fillattr(nd, mp, vp, NULL, &nvap->na_vattr, fhp, rderror, attrbitp, cred, p, isdgram, reterr, supports_nfsv4acls, at_root, - mounted_on_fileno); + mounted_on_fileno, sf); + free(sf, M_TEMP); NFSEXITCODE2(0, nd); return (error); } @@ -5168,7 +5201,7 @@ nfsrv_setacldsdorpc(fhandle_t *fhp, struct ucred *cred * the same type (VREG). */ nfsv4_fillattr(nd, NULL, vp, aclp, NULL, NULL, 0, &attrbits, NULL, - NULL, 0, 0, 0, 0, 0); + NULL, 0, 0, 0, 0, 0, NULL); error = newnfs_request(nd, nmp, NULL, &nmp->nm_sockreq, NULL, p, cred, NFS_PROG, NFS_VER4, NULL, 1, NULL, NULL); if (error != 0) { @@ -5436,6 +5469,78 @@ nfsrv_killrpcs(struct nfsmount *nmp) * called. */ newnfs_nmcancelreqs(nmp); +} + +/* + * Sum up the statfs info for each of the DSs, so that the client will + * receive the total for all DSs. + */ +static int +nfsrv_pnfsstatfs(struct statfs *sf) +{ + struct statfs *tsf; + struct nfsdevice *ds; + struct vnode **dvpp, **tdvpp, *dvp; + uint64_t tot; + int cnt, error = 0, i; + + if (nfsrv_devidcnt <= 0) + return (ENXIO); + dvpp = mallocarray(nfsrv_devidcnt, sizeof(*dvpp), M_TEMP, M_WAITOK); + tsf = malloc(sizeof(*tsf), M_TEMP, M_WAITOK); + + /* Get an array of the dvps for the DSs. */ + tdvpp = dvpp; + i = 0; + NFSDDSLOCK(); + TAILQ_FOREACH(ds, &nfsrv_devidhead, nfsdev_list) { + if (ds->nfsdev_nmp != NULL) { + if (++i > nfsrv_devidcnt) + break; + *tdvpp++ = ds->nfsdev_dvp; + } + } + NFSDDSUNLOCK(); + cnt = i; + + /* Do a VFS_STATFS() for each of the DSs and sum them up. */ + tdvpp = dvpp; + for (i = 0; i < cnt && error == 0; i++) { + dvp = *tdvpp++; + error = VFS_STATFS(dvp->v_mount, tsf); + if (error == 0) { + if (sf->f_bsize == 0) { + if (tsf->f_bsize > 0) + sf->f_bsize = tsf->f_bsize; + else + sf->f_bsize = 8192; + } + if (tsf->f_blocks > 0) { + if (sf->f_bsize != tsf->f_bsize) { + tot = tsf->f_blocks * tsf->f_bsize; + sf->f_blocks += (tot / sf->f_bsize); + } else + sf->f_blocks += tsf->f_blocks; + } + if (tsf->f_bfree > 0) { + if (sf->f_bsize != tsf->f_bsize) { + tot = tsf->f_bfree * tsf->f_bsize; + sf->f_bfree += (tot / sf->f_bsize); + } else + sf->f_bfree += tsf->f_bfree; + } + if (tsf->f_bavail > 0) { + if (sf->f_bsize != tsf->f_bsize) { + tot = tsf->f_bavail * tsf->f_bsize; + sf->f_bavail += (tot / sf->f_bsize); + } else + sf->f_bavail += tsf->f_bavail; + } + } + } + free(tsf, M_TEMP); + free(dvpp, M_TEMP); + return (error); } extern int (*nfsd_call_nfsd)(struct thread *, struct nfssvc_args *);