From owner-svn-src-projects@freebsd.org Sat Apr 21 19:54:07 2018 Return-Path: Delivered-To: svn-src-projects@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC83EF92D76 for ; Sat, 21 Apr 2018 19:54:07 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6221084966; Sat, 21 Apr 2018 19:54:07 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5CF7B398D; Sat, 21 Apr 2018 19:54:07 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w3LJs7Cm040360; Sat, 21 Apr 2018 19:54:07 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w3LJs6De040354; Sat, 21 Apr 2018 19:54:06 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201804211954.w3LJs6De040354@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 21 Apr 2018 19:54:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-projects@freebsd.org Subject: svn commit: r332862 - in projects/pnfs-planb-server/sys/fs: nfs nfsclient X-SVN-Group: projects X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in projects/pnfs-planb-server/sys/fs: nfs nfsclient X-SVN-Commit-Revision: 332862 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Apr 2018 19:54:07 -0000 Author: rmacklem Date: Sat Apr 21 19:54:06 2018 New Revision: 332862 URL: https://svnweb.freebsd.org/changeset/base/332862 Log: Add NFSv4.1 client support for Flexible File Layout callbacks and the generation of Ioerr stats for the LayoutReturn reply. These changes are needed by the client to do testing of the mirrored pNFS server configuration. Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clvfsops.c Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Sat Apr 21 18:43:09 2018 (r332861) +++ projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Sat Apr 21 19:54:06 2018 (r332862) @@ -525,7 +525,7 @@ int nfsrpc_layoutcommit(struct nfsmount *, uint8_t *, NFSPROC_T *, void *); int nfsrpc_layoutreturn(struct nfsmount *, uint8_t *, int, int, int, uint32_t, int, uint64_t, uint64_t, nfsv4stateid_t *, struct ucred *, NFSPROC_T *, - void *); + uint32_t, uint32_t, char *); int nfsrpc_reclaimcomplete(struct nfsmount *, struct ucred *, NFSPROC_T *); int nfscl_doiods(vnode_t, struct uio *, int *, int *, uint32_t, int, struct ucred *, NFSPROC_T *); @@ -598,6 +598,7 @@ int nfscl_layout(struct nfsmount *, vnode_t, u_int8_t NFSPROC_T *); struct nfscllayout *nfscl_getlayout(struct nfsclclient *, uint8_t *, int, uint64_t, struct nfsclflayout **, int *); +void nfscl_dserr(uint32_t, struct nfscldevinfo *, struct nfscllayout *); void nfscl_rellayout(struct nfscllayout *, int); struct nfscldevinfo *nfscl_getdevinfo(struct nfsclclient *, uint8_t *, struct nfscldevinfo *); Modified: projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h Sat Apr 21 18:43:09 2018 (r332861) +++ projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h Sat Apr 21 19:54:06 2018 (r332862) @@ -340,6 +340,9 @@ struct nfsclrecalllayout { int nfsrecly_recalltype; uint32_t nfsrecly_iomode; uint32_t nfsrecly_stateseqid; + uint32_t nfsrecly_stat; + uint32_t nfsrecly_op; + char nfsrecly_devid[NFSX_V4DEVICEID]; }; /* Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c Sat Apr 21 18:43:09 2018 (r332861) +++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clport.c Sat Apr 21 19:54:06 2018 (r332862) @@ -86,6 +86,7 @@ extern int nfs_numnfscbd; extern int nfscl_inited; struct mtx ncl_iod_mutex; NFSDLOCKMUTEX; +extern struct mtx nfsrv_dslock_mtx; extern void (*ncl_call_invalcaches)(struct vnode *); @@ -1383,6 +1384,14 @@ nfssvc_nfscl(struct thread *td, struct nfssvc_args *ua 0 && strcmp(mp->mnt_stat.f_fstypename, "nfs") == 0 && mp->mnt_data != NULL) { nmp = VFSTONFS(mp); + NFSDDSLOCK(); + if (nfsv4_findmirror(nmp, NULL) != NULL + ) { + NFSDDSUNLOCK(); + error = ENXIO; + nmp = NULL; + break; + } mtx_lock(&nmp->nm_mtx); if ((nmp->nm_privflag & NFSMNTP_FORCEDISM) == 0) { @@ -1391,9 +1400,10 @@ nfssvc_nfscl(struct thread *td, struct nfssvc_args *ua NFSMNTP_CANCELRPCS); mtx_unlock(&nmp->nm_mtx); } else { - nmp = NULL; mtx_unlock(&nmp->nm_mtx); + nmp = NULL; } + NFSDDSUNLOCK(); break; } } @@ -1418,7 +1428,7 @@ nfssvc_nfscl(struct thread *td, struct nfssvc_args *ua nmp->nm_privflag &= ~NFSMNTP_CANCELRPCS; wakeup(nmp); mtx_unlock(&nmp->nm_mtx); - } else + } else if (error == 0) error = EINVAL; } free(buf, M_TEMP); Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Sat Apr 21 18:43:09 2018 (r332861) +++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Sat Apr 21 19:54:06 2018 (r332862) @@ -164,7 +164,7 @@ static void nfscl_emptylockowner(struct nfscllockowner static void nfscl_mergeflayouts(struct nfsclflayouthead *, struct nfsclflayouthead *); static int nfscl_layoutrecall(int, struct nfscllayout *, uint32_t, uint64_t, - uint64_t, uint32_t, struct nfsclrecalllayout *); + uint64_t, uint32_t, uint32_t, uint32_t, char *, struct nfsclrecalllayout *); static int nfscl_seq(uint32_t, uint32_t); static void nfscl_layoutreturn(struct nfsmount *, struct nfscllayout *, struct ucred *, NFSPROC_T *); @@ -2762,7 +2762,8 @@ tryagain2: break; (void)nfscl_layoutrecall(NFSLAYOUTRETURN_FILE, lyp, NFSLAYOUTIOMODE_ANY, 0, UINT64_MAX, - lyp->nfsly_stateid.seqid, recallp); + lyp->nfsly_stateid.seqid, 0, 0, NULL, + recallp); } lyp = nlyp; } @@ -3422,9 +3423,12 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) else changed = 0; recalltype = fxdr_unsigned(int, *tl); + NFSCL_DEBUG(4, "layt=%d iom=%d ch=%d rectyp=%d\n", + laytype, iomode, changed, recalltype); recallp = malloc(sizeof(*recallp), M_NFSLAYRECALL, M_WAITOK); - if (laytype != NFSLAYOUT_NFSV4_1_FILES) + if (laytype != NFSLAYOUT_NFSV4_1_FILES && + laytype != NFSLAYOUT_FLEXFILE) error = NFSERR_NOMATCHLAYOUT; else if (recalltype == NFSLAYOUTRETURN_FILE) { error = nfsm_getfh(nd, &nfhp); @@ -3441,6 +3445,9 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) error = NFSERR_NOTSUPP; else if (i == 0) error = NFSERR_OPNOTINSESS; + NFSCL_DEBUG(4, "off=%ju len=%ju sq=%u err=%d\n", + (uintmax_t)off, (uintmax_t)len, + stateid.seqid, error); if (error == 0) { NFSLOCKCLSTATE(); clp = nfscl_getclntsess(sessionid); @@ -3453,7 +3460,8 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) lyp); if (lyp != NULL && (lyp->nfsly_flags & - NFSLY_FILES) != 0 && + (NFSLY_FILES | + NFSLY_FLEXFILE)) != 0 && !NFSBCMP(stateid.other, lyp->nfsly_stateid.other, NFSX_STATEIDOTHER)) { @@ -3462,6 +3470,7 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) recalltype, lyp, iomode, off, len, stateid.seqid, + 0, 0, NULL, recallp); recallp = NULL; wakeup(clp); @@ -3496,6 +3505,7 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) lyp, iomode, 0, UINT64_MAX, lyp->nfsly_stateid.seqid, + 0, 0, NULL, recallp); recallp = NULL; gotone = 1; @@ -3519,7 +3529,7 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) recalltype, lyp, iomode, 0, UINT64_MAX, lyp->nfsly_stateid.seqid, - recallp); + 0, 0, NULL, recallp); recallp = NULL; gotone = 1; } @@ -4952,13 +4962,42 @@ nfscl_retoncloselayout(vnode_t vp, struct nfsclclient if (!LIST_EMPTY(&lyp->nfsly_flayrw)) iomode |= NFSLAYOUTIOMODE_RW; (void)nfscl_layoutrecall(NFSLAYOUTRETURN_FILE, lyp, iomode, - 0, UINT64_MAX, lyp->nfsly_stateid.seqid, *recallpp); + 0, UINT64_MAX, lyp->nfsly_stateid.seqid, 0, 0, NULL, + *recallpp); NFSCL_DEBUG(4, "retoncls recall iomode=%d\n", iomode); *recallpp = NULL; } } /* + * Mark the layout to be recalled and with an error. + */ +void +nfscl_dserr(uint32_t op, struct nfscldevinfo *dp, struct nfscllayout *lyp) +{ + struct nfsclrecalllayout *recallp; + uint32_t iomode; + + recallp = malloc(sizeof(*recallp), M_NFSLAYRECALL, M_WAITOK); + iomode = 0; + NFSLOCKCLSTATE(); + if ((lyp->nfsly_flags & NFSLY_RECALL) == 0) { + if (!LIST_EMPTY(&lyp->nfsly_flayread)) + iomode |= NFSLAYOUTIOMODE_READ; + if (!LIST_EMPTY(&lyp->nfsly_flayrw)) + iomode |= NFSLAYOUTIOMODE_RW; + (void)nfscl_layoutrecall(NFSLAYOUTRETURN_FILE, lyp, iomode, + 0, UINT64_MAX, lyp->nfsly_stateid.seqid, NFSERR_IO, op, + dp->nfsdi_deviceid, recallp); + NFSUNLOCKCLSTATE(); + NFSCL_DEBUG(4, "retoncls recall iomode=%d\n", iomode); + } else { + NFSUNLOCKCLSTATE(); + free(recallp, M_NFSLAYRECALL); + } +} + +/* * Dereference a layout. */ void @@ -5175,8 +5214,8 @@ nfscl_freedevinfo(struct nfscldevinfo *dip) */ static int nfscl_layoutrecall(int recalltype, struct nfscllayout *lyp, uint32_t iomode, - uint64_t off, uint64_t len, uint32_t stateseqid, - struct nfsclrecalllayout *recallp) + uint64_t off, uint64_t len, uint32_t stateseqid, uint32_t stat, uint32_t op, + char *devid, struct nfsclrecalllayout *recallp) { struct nfsclrecalllayout *rp, *orp; @@ -5185,6 +5224,10 @@ nfscl_layoutrecall(int recalltype, struct nfscllayout recallp->nfsrecly_stateseqid = stateseqid; recallp->nfsrecly_off = off; recallp->nfsrecly_len = len; + recallp->nfsrecly_stat = stat; + recallp->nfsrecly_op = op; + if (devid != NULL) + NFSBCOPY(devid, recallp->nfsrecly_devid, NFSX_V4DEVICEID); /* * Order the list as file returns first, followed by fsid and any * returns, both in increasing stateseqid order. @@ -5259,7 +5302,8 @@ nfscl_layoutreturn(struct nfsmount *nmp, struct nfscll lyp->nfsly_fhlen, 0, layouttype, rp->nfsrecly_iomode, rp->nfsrecly_recalltype, rp->nfsrecly_off, rp->nfsrecly_len, - &stateid, cred, p, NULL); + &stateid, cred, p, rp->nfsrecly_stat, rp->nfsrecly_op, + rp->nfsrecly_devid); } } Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clvfsops.c ============================================================================== --- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clvfsops.c Sat Apr 21 18:43:09 2018 (r332861) +++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clvfsops.c Sat Apr 21 19:54:06 2018 (r332862) @@ -86,6 +86,7 @@ extern enum nfsiod_state ncl_iodwant[NFS_MAXASYNCDAEMO extern struct nfsmount *ncl_iodmount[NFS_MAXASYNCDAEMON]; extern struct mtx ncl_iod_mutex; NFSCLSTATEMUTEX; +extern struct mtx nfsrv_dslock_mtx; MALLOC_DEFINE(M_NEWNFSREQ, "newnfsclient_req", "NFS request header"); MALLOC_DEFINE(M_NEWNFSMNT, "newnfsmnt", "NFS mount struct"); @@ -1672,6 +1673,7 @@ nfs_unmount(struct mount *mp, int mntflags) if (mntflags & MNT_FORCE) flags |= FORCECLOSE; nmp = VFSTONFS(mp); + error = 0; /* * Goes something like this.. * - Call vflush() to clear out vnodes for this filesystem @@ -1680,6 +1682,12 @@ nfs_unmount(struct mount *mp, int mntflags) */ /* In the forced case, cancel any outstanding requests. */ if (mntflags & MNT_FORCE) { + NFSDDSLOCK(); + if (nfsv4_findmirror(nmp, NULL) != NULL) + error = ENXIO; + NFSDDSUNLOCK(); + if (error) + goto out; error = newnfs_nmcancelreqs(nmp); if (error) goto out;