From owner-svn-src-head@freebsd.org Sat Apr 22 22:37:46 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FDF5D4B96D; Sat, 22 Apr 2017 22:37:46 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02B1C136C; Sat, 22 Apr 2017 22:37:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v3MMbjiX025732; Sat, 22 Apr 2017 22:37:45 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v3MMbjcL025731; Sat, 22 Apr 2017 22:37:45 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201704222237.v3MMbjcL025731@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sat, 22 Apr 2017 22:37:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r317305 - head/sys/fs/nfsclient X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Apr 2017 22:37:46 -0000 Author: rmacklem Date: Sat Apr 22 22:37:44 2017 New Revision: 317305 URL: https://svnweb.freebsd.org/changeset/base/317305 Log: Fix the NFSv4.1/pNFS client return layout on close. The "return layout on close" case in the pNFS client was badly broken. Fortunately, extant pNFS servers that I have tested against do not do this. This patch fixes it. It also changes the way the layout stateid.seqid is set for LayoutReturn. I think this change is correct w.r.t. the RFC, but I am not 100% sure. This was found during recent testing of the pNFS server under development. MFC after: 2 weeks Modified: head/sys/fs/nfsclient/nfs_clstate.c Modified: head/sys/fs/nfsclient/nfs_clstate.c ============================================================================== --- head/sys/fs/nfsclient/nfs_clstate.c Sat Apr 22 22:34:45 2017 (r317304) +++ head/sys/fs/nfsclient/nfs_clstate.c Sat Apr 22 22:37:44 2017 (r317305) @@ -88,6 +88,8 @@ extern struct nfsstatsv1 nfsstatsv1; extern struct nfsreqhead nfsd_reqq; extern u_int32_t newnfs_false, newnfs_true; extern int nfscl_debuglevel; +extern int nfscl_enablecallb; +extern int nfs_numnfscbd; NFSREQSPINLOCK; NFSCLSTATEMUTEX; int nfscl_inited = 0; @@ -118,7 +120,8 @@ static struct nfsclclient *nfscl_getclnt static struct nfsclclient *nfscl_getclntsess(uint8_t *); static struct nfscldeleg *nfscl_finddeleg(struct nfsclclient *, u_int8_t *, int); -static void nfscl_retoncloselayout(struct nfsclclient *, uint8_t *, int); +static void nfscl_retoncloselayout(vnode_t, struct nfsclclient *, uint8_t *, + int, struct nfsclrecalllayout **); static void nfscl_reldevinfo_locked(struct nfscldevinfo *); static struct nfscllayout *nfscl_findlayout(struct nfsclclient *, u_int8_t *, int); @@ -3121,6 +3124,7 @@ nfscl_doclose(vnode_t vp, struct nfsclcl struct nfsclopen *op; struct nfscldeleg *dp; struct nfsfh *nfhp; + struct nfsclrecalllayout *recallp; int error; error = nfscl_getcl(vnode_mount(vp), NULL, NULL, 1, &clp); @@ -3129,6 +3133,7 @@ nfscl_doclose(vnode_t vp, struct nfsclcl *clpp = clp; nfhp = VTONFS(vp)->n_fhp; + recallp = malloc(sizeof(*recallp), M_NFSLAYRECALL, M_WAITOK); NFSLOCKCLSTATE(); /* * First get rid of the local Open structures, which should be no @@ -3148,7 +3153,7 @@ nfscl_doclose(vnode_t vp, struct nfsclcl } /* Return any layouts marked return on close. */ - nfscl_retoncloselayout(clp, nfhp->nfh_fh, nfhp->nfh_len); + nfscl_retoncloselayout(vp, clp, nfhp->nfh_fh, nfhp->nfh_len, &recallp); /* Now process the opens against the server. */ lookformore: @@ -3171,6 +3176,11 @@ lookformore: } } NFSUNLOCKCLSTATE(); + /* + * recallp has been set NULL by nfscl_retoncloselayout() if it was + * used by the function, but calling free() with a NULL pointer is ok. + */ + free(recallp, M_NFSLAYRECALL); return (0); } @@ -4890,28 +4900,32 @@ nfscl_getlayout(struct nfsclclient *clp, } /* - * Search for a layout by MDS file handle. If one is found that is marked - * "return on close", delete it, since it should now be forgotten. + * Search for a layout by MDS file handle. If one is found, mark in to be + * recalled, if it already marked "return on close". */ static void -nfscl_retoncloselayout(struct nfsclclient *clp, uint8_t *fhp, int fhlen) +nfscl_retoncloselayout(vnode_t vp, struct nfsclclient *clp, uint8_t *fhp, + int fhlen, struct nfsclrecalllayout **recallpp) { struct nfscllayout *lyp; + uint32_t iomode; -tryagain: + if (vp->v_type != VREG || !NFSHASPNFS(VFSTONFS(vnode_mount(vp))) || + nfscl_enablecallb == 0 || nfs_numnfscbd == 0 || + (VTONFS(vp)->n_flag & NNOLAYOUT) != 0) + return; lyp = nfscl_findlayout(clp, fhp, fhlen); - if (lyp != NULL && (lyp->nfsly_flags & NFSLY_RETONCLOSE) != 0) { - /* - * Wait for outstanding I/O ops to be done. - */ - if (lyp->nfsly_lock.nfslock_usecnt != 0 || - lyp->nfsly_lock.nfslock_lock != 0) { - lyp->nfsly_lock.nfslock_lock |= NFSV4LOCK_WANTED; - (void)mtx_sleep(&lyp->nfsly_lock, - NFSCLSTATEMUTEXPTR, PZERO, "nfslyc", 0); - goto tryagain; - } - nfscl_freelayout(lyp); + if (lyp != NULL && (lyp->nfsly_flags & (NFSLY_RETONCLOSE | + NFSLY_RECALL)) == NFSLY_RETONCLOSE) { + iomode = 0; + if (!LIST_EMPTY(&lyp->nfsly_flayread)) + iomode |= NFSLAYOUTIOMODE_READ; + if (!LIST_EMPTY(&lyp->nfsly_flayrw)) + iomode |= NFSLAYOUTIOMODE_RW; + (void)nfscl_layoutrecall(NFSLAYOUTRETURN_FILE, lyp, iomode, + 0, UINT64_MAX, lyp->nfsly_stateid.seqid, *recallpp); + NFSCL_DEBUG(4, "retoncls recall iomode=%d\n", iomode); + *recallpp = NULL; } } @@ -5195,8 +5209,8 @@ nfscl_layoutreturn(struct nfsmount *nmp, nfsv4stateid_t stateid; NFSBCOPY(lyp->nfsly_stateid.other, stateid.other, NFSX_STATEIDOTHER); + stateid.seqid = lyp->nfsly_stateid.seqid; LIST_FOREACH(rp, &lyp->nfsly_recall, nfsrecly_list) { - stateid.seqid = rp->nfsrecly_stateseqid; (void)nfsrpc_layoutreturn(nmp, lyp->nfsly_fh, lyp->nfsly_fhlen, 0, NFSLAYOUT_NFSV4_1_FILES, rp->nfsrecly_iomode, rp->nfsrecly_recalltype,