From owner-svn-src-projects@freebsd.org Mon Jul 9 00:38:40 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F57110269A0
for <svn-src-projects@mailman.ysv.freebsd.org>;
Mon, 9 Jul 2018 00:38:40 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4F42B75D1C;
Mon, 9 Jul 2018 00:38:40 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 308255BC8;
Mon, 9 Jul 2018 00:38:40 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w690ceLW014976;
Mon, 9 Jul 2018 00:38:40 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w690cd2W014972;
Mon, 9 Jul 2018 00:38:39 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807090038.w690cd2W014972@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Mon, 9 Jul 2018 00:38:39 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336116 - in projects/pnfs-planb-server/sys/fs: nfs
nfsserver
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: in projects/pnfs-planb-server/sys/fs: nfs nfsserver
X-SVN-Commit-Revision: 336116
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Jul 2018 00:38:41 -0000
Author: rmacklem
Date: Mon Jul 9 00:38:39 2018
New Revision: 336116
URL: https://svnweb.freebsd.org/changeset/base/336116
Log:
Modify the pNFS server kernel code so that a "force" option can be implemented
for pnfsdskill(8).
pnfsdskill(8) will fail if there isn't a mirror for the DS that is being
disabled. However, if the DS has failed, it may be necessary to do a
pnfsdskill on the DS even when there is no valid mirror, so that the
nfsd threads can be killed.
This patch modifies the kernel to add a PNFSDOP_FORCEDELDS option that
pnfsdkill(8) can use to force the disable.
Modified:
projects/pnfs-planb-server/sys/fs/nfs/nfs.h
projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h
projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c
projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdstate.c
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs.h
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfs.h Sun Jul 8 23:22:04 2018 (r336115)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfs.h Mon Jul 9 00:38:39 2018 (r336116)
@@ -207,6 +207,7 @@ struct nfsd_pnfsd_args {
#define PNFSDOP_DELDSSERVER 1
#define PNFSDOP_COPYMR 2
+#define PNFSDOP_FORCEDELDS 3
/* Old version. */
struct nfsd_nfsd_oargs {
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Sun Jul 8 23:22:04 2018 (r336115)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Mon Jul 9 00:38:39 2018 (r336116)
@@ -159,8 +159,8 @@ int nfsrv_getdevinfo(char *, int, uint32_t *, uint32_t
void nfsrv_freeonedevid(struct nfsdevice *);
void nfsrv_freealllayoutsanddevids(void);
void nfsrv_freefilelayouts(fhandle_t *);
-int nfsrv_deldsserver(char *, NFSPROC_T *);
-struct nfsdevice *nfsrv_deldsnmp(struct nfsmount *, NFSPROC_T *);
+int nfsrv_deldsserver(int, char *, NFSPROC_T *);
+struct nfsdevice *nfsrv_deldsnmp(int, struct nfsmount *, NFSPROC_T *);
int nfsrv_createdevids(struct nfsd_nfsd_args *, NFSPROC_T *);
int nfsrv_checkdsattr(struct nfsrv_descript *, vnode_t, NFSPROC_T *);
int nfsrv_copymr(vnode_t, vnode_t, vnode_t, struct nfsdevice *,
Modified: projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c Sun Jul 8 23:22:04 2018 (r336115)
+++ projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdport.c Mon Jul 9 00:38:39 2018 (r336116)
@@ -3433,12 +3433,13 @@ nfssvc_nfsd(struct thread *td, struct nfssvc_args *uap
free(nfsdarg.mdspath, M_TEMP);
} else if (uap->flag & NFSSVC_PNFSDS) {
error = copyin(uap->argp, &pnfsdarg, sizeof(pnfsdarg));
- if (error == 0 && pnfsdarg.op == PNFSDOP_DELDSSERVER) {
+ if (error == 0 && (pnfsdarg.op == PNFSDOP_DELDSSERVER ||
+ pnfsdarg.op == PNFSDOP_FORCEDELDS)) {
cp = malloc(PATH_MAX + 1, M_TEMP, M_WAITOK);
error = copyinstr(pnfsdarg.dspath, cp, PATH_MAX + 1,
NULL);
if (error == 0)
- error = nfsrv_deldsserver(cp, td);
+ error = nfsrv_deldsserver(pnfsdarg.op, cp, td);
free(cp, M_TEMP);
} else if (error == 0 && pnfsdarg.op == PNFSDOP_COPYMR) {
cp = malloc(PATH_MAX + 1, M_TEMP, M_WAITOK);
@@ -4014,7 +4015,7 @@ nfsrv_pnfscreate(struct vnode *vp, struct vattr *vap,
NFSMNTP_CANCELRPCS)) == 0) {
nmp->nm_privflag |= NFSMNTP_CANCELRPCS;
NFSUNLOCKMNT(nmp);
- ds = nfsrv_deldsnmp(nmp, p);
+ ds = nfsrv_deldsnmp(PNFSDOP_DELDSSERVER, nmp, p);
NFSD_DEBUG(4, "dscreatfail fail=%d ds=%p\n", failpos,
ds);
if (ds != NULL)
@@ -4248,7 +4249,7 @@ nfsrv_pnfsremove(struct vnode **dvp, int mirrorcnt, ch
NFSMNTP_CANCELRPCS)) == 0) {
nmp->nm_privflag |= NFSMNTP_CANCELRPCS;
NFSUNLOCKMNT(nmp);
- ds = nfsrv_deldsnmp(nmp, p);
+ ds = nfsrv_deldsnmp(PNFSDOP_DELDSSERVER, nmp, p);
NFSD_DEBUG(4, "dsremovefail fail=%d ds=%p\n", failpos,
ds);
if (ds != NULL)
@@ -4463,9 +4464,10 @@ tryagain:
NFSMNTP_CANCELRPCS)) == 0) {
failnmp->nm_privflag |= NFSMNTP_CANCELRPCS;
NFSUNLOCKMNT(failnmp);
- ds = nfsrv_deldsnmp(failnmp, p);
- NFSD_DEBUG(4, "dsldsnmp fail=%d ds=%p\n",
- failpos, ds);
+ ds = nfsrv_deldsnmp(PNFSDOP_DELDSSERVER,
+ failnmp, p);
+ NFSD_DEBUG(4, "nfsrv_proxyds: deldsnmp fail=%d "
+ "ds=%p\n", failpos, ds);
if (ds != NULL)
nfsrv_killrpcs(failnmp);
NFSLOCKMNT(failnmp);
Modified: projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdstate.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdstate.c Sun Jul 8 23:22:04 2018 (r336115)
+++ projects/pnfs-planb-server/sys/fs/nfsserver/nfs_nfsdstate.c Mon Jul 9 00:38:39 2018 (r336116)
@@ -7482,7 +7482,7 @@ out:
* Look up the mount path for the DS server and delete it.
*/
int
-nfsrv_deldsserver(char *dspathp, NFSPROC_T *p)
+nfsrv_deldsserver(int op, char *dspathp, NFSPROC_T *p)
{
struct mount *mp;
struct nfsmount *nmp;
@@ -7524,7 +7524,7 @@ nfsrv_deldsserver(char *dspathp, NFSPROC_T *p)
mtx_unlock(&mountlist_mtx);
if (nmp != NULL) {
- ds = nfsrv_deldsnmp(nmp, p);
+ ds = nfsrv_deldsnmp(op, nmp, p);
NFSD_DEBUG(4, "deldsnmp=%p\n", ds);
if (ds != NULL) {
nfsrv_killrpcs(nmp);
@@ -7544,20 +7544,26 @@ nfsrv_deldsserver(char *dspathp, NFSPROC_T *p)
* Search for and remove a DS entry which matches the "nmp" argument.
* The nfsdevice structure pointer is returned so that the caller can
* free it via nfsrv_freeonedevid().
+ * For the forced case, do not try to do LayoutRecalls, since the server
+ * must be shut down now anyhow.
*/
struct nfsdevice *
-nfsrv_deldsnmp(struct nfsmount *nmp, NFSPROC_T *p)
+nfsrv_deldsnmp(int op, struct nfsmount *nmp, NFSPROC_T *p)
{
struct nfsdevice *fndds;
NFSD_DEBUG(4, "deldsdvp\n");
NFSDDSLOCK();
- fndds = nfsrv_findmirroredds(nmp);
+ if (op == PNFSDOP_FORCEDELDS)
+ fndds = nfsv4_findmirror(nmp);
+ else
+ fndds = nfsrv_findmirroredds(nmp);
if (fndds != NULL)
nfsrv_deleteds(fndds);
NFSDDSUNLOCK();
if (fndds != NULL) {
- nfsrv_flexmirrordel(fndds->nfsdev_deviceid, p);
+ if (op != PNFSDOP_FORCEDELDS)
+ nfsrv_flexmirrordel(fndds->nfsdev_deviceid, p);
printf("pNFS server: mirror %s failed\n", fndds->nfsdev_host);
}
return (fndds);
From owner-svn-src-projects@freebsd.org Mon Jul 9 00:47:37 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59D121027A95
for <svn-src-projects@mailman.ysv.freebsd.org>;
Mon, 9 Jul 2018 00:47:37 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 0A45B76332;
Mon, 9 Jul 2018 00:47:37 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DA8755D58;
Mon, 9 Jul 2018 00:47:36 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w690laFL020029;
Mon, 9 Jul 2018 00:47:36 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w690laLX020028;
Mon, 9 Jul 2018 00:47:36 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807090047.w690laLX020028@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Mon, 9 Jul 2018 00:47:36 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336117 - projects/pnfs-planb-server/usr.bin/pnfsdskill
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: projects/pnfs-planb-server/usr.bin/pnfsdskill
X-SVN-Commit-Revision: 336117
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Jul 2018 00:47:37 -0000
Author: rmacklem
Date: Mon Jul 9 00:47:36 2018
New Revision: 336117
URL: https://svnweb.freebsd.org/changeset/base/336117
Log:
Add a -f/--force option to pnfsdskill, so that it disable a DS even if there
is no valid mirror for it. This should only be used to facilitate shutting
down of the nfsd when DSs have failed.
Modified:
projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.c
Modified: projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.c
==============================================================================
--- projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.c Mon Jul 9 00:38:39 2018 (r336116)
+++ projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.c Mon Jul 9 00:47:36 2018 (r336117)
@@ -30,6 +30,7 @@
__FBSDID("$FreeBSD$");
#include <err.h>
+#include <getopt.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
@@ -44,6 +45,11 @@ __FBSDID("$FreeBSD$");
static void usage(void);
+static struct option longopts[] = {
+ { "force", no_argument, NULL, 'f' },
+ { NULL, 0, NULL, 0 }
+};
+
/*
* This program disables use of a DS mirror. The "dspath" command line
* argument must be an exact match for the mounted-on path of the DS.
@@ -54,23 +60,39 @@ int
main(int argc, char *argv[])
{
struct nfsd_pnfsd_args pnfsdarg;
+ int ch, force;
- if (argc != 2)
- usage();
if (geteuid() != 0)
errx(1, "Must be run as root/su");
+ force = 0;
+ while ((ch = getopt_long(argc, argv, "f", longopts, NULL)) != -1) {
+ switch (ch) {
+ case 'f':
+ force = 1;
+ break;
+ default:
+ usage();
+ }
+ }
+ argc -= optind;
+ argv += optind;
+ if (argc != 1)
+ usage();
- pnfsdarg.op = PNFSDOP_DELDSSERVER;
- pnfsdarg.dspath = argv[1];
+ if (force != 0)
+ pnfsdarg.op = PNFSDOP_FORCEDELDS;
+ else
+ pnfsdarg.op = PNFSDOP_DELDSSERVER;
+ pnfsdarg.dspath = *argv;
if (nfssvc(NFSSVC_PNFSDS, &pnfsdarg) < 0)
- err(1, "Can't kill %s", argv[1]);
+ err(1, "Can't kill %s", *argv);
}
static void
usage(void)
{
- fprintf(stderr, "pnfsdsfile [filepath]\n");
+ fprintf(stderr, "pnfsdsfile [-f] mounted-on-DS-dir\n");
exit(1);
}
From owner-svn-src-projects@freebsd.org Mon Jul 9 21:29:24 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4191A1032552
for <svn-src-projects@mailman.ysv.freebsd.org>;
Mon, 9 Jul 2018 21:29:24 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id E68FF88AD4;
Mon, 9 Jul 2018 21:29:23 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C2F7A1A754;
Mon, 9 Jul 2018 21:29:23 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w69LTNMg071535;
Mon, 9 Jul 2018 21:29:23 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w69LTN93071534;
Mon, 9 Jul 2018 21:29:23 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807092129.w69LTN93071534@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Mon, 9 Jul 2018 21:29:23 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336147 - projects/pnfs-planb-server/usr.bin/pnfsdskill
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: projects/pnfs-planb-server/usr.bin/pnfsdskill
X-SVN-Commit-Revision: 336147
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Jul 2018 21:29:24 -0000
Author: rmacklem
Date: Mon Jul 9 21:29:23 2018
New Revision: 336147
URL: https://svnweb.freebsd.org/changeset/base/336147
Log:
Document the "-f" flag for pnfsdskill(1).
Modified:
projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.1
Modified: projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.1
==============================================================================
--- projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.1 Mon Jul 9 21:13:21 2018 (r336146)
+++ projects/pnfs-planb-server/usr.bin/pnfsdskill/pnfsdskill.1 Mon Jul 9 21:29:23 2018 (r336147)
@@ -23,7 +23,7 @@
.\"
.\" $FreeBSD$
.\"
-.Dd March 9, 2018
+.Dd July 9, 2018
.Dt PNFSDSKILL 1
.Os
.Sh NAME
@@ -32,6 +32,7 @@
disables a pNFS data storage server (DS)
.Sh SYNOPSIS
.Nm
+.Op Fl f
.Ar mounted-on-DS-dir
.Sh DESCRIPTION
The
@@ -42,7 +43,19 @@ may use this command on the metadata server (MDS) to d
This command must be used on the MDS and the
.Ar mounted-on-DS-dir
must be the exact pathname used when mounting the DS on the MDS.
+Normally this command will fail if there is no valid mirror for the DS
+available.
.Pp
+The following option is available:
+.Bl -tag -width Ds
+.It Fl f
+This option forces the DS to be disabled even if there is no valid mirror
+for the DS available.
+It should only be used to allow the
+.Xr nfsd 8
+threads to be terminated.
+.El
+.Pp
If this command fails with
.Dq Device not configured
(ENXIO), it probably
@@ -61,4 +74,5 @@ operational DSs is at least as many as the level of mi
.Sh HISTORY
The
.Nm
-command appeared in FreeBSD12.
+command first appeared in
+.Fx 12.0 .
From owner-svn-src-projects@freebsd.org Thu Jul 12 20:28:54 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71D981041F3E
for <svn-src-projects@mailman.ysv.freebsd.org>;
Thu, 12 Jul 2018 20:28:54 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 2511A7E329;
Thu, 12 Jul 2018 20:28:54 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0585D26440;
Thu, 12 Jul 2018 20:28:54 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6CKSrQ8064468;
Thu, 12 Jul 2018 20:28:53 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6CKSrhM064463;
Thu, 12 Jul 2018 20:28:53 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807122028.w6CKSrhM064463@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Thu, 12 Jul 2018 20:28:53 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336228 - in projects/pnfs-planb-server/sys/fs: nfs
nfsclient
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: in projects/pnfs-planb-server/sys/fs: nfs nfsclient
X-SVN-Commit-Revision: 336228
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Jul 2018 20:28:54 -0000
Author: rmacklem
Date: Thu Jul 12 20:28:52 2018
New Revision: 336228
URL: https://svnweb.freebsd.org/changeset/base/336228
Log:
Fix the NFSv4.1 pNFS client for the case of mirrored DSs on separate machines.
My test environment has all the DSs colocated on the same machine, so I did
not spot this bug until recently. The client was using the TCP connection
for the first DS when doing writes/commits to subsequent mirrored DSs.
This worked fine for my test setup, but would have failed when the DSs are
on separate machines.
This patch fixes the code so that it uses the correct TCP connection for
the writes/commits to each of the mirrored DSs.
Modified:
projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h
projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h
projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c
projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Thu Jul 12 19:02:59 2018 (r336227)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Thu Jul 12 20:28:52 2018 (r336228)
@@ -607,7 +607,7 @@ void nfscl_rellayout(struct nfscllayout *, int);
struct nfscldevinfo *nfscl_getdevinfo(struct nfsclclient *, uint8_t *,
struct nfscldevinfo *);
void nfscl_reldevinfo(struct nfscldevinfo *);
-int nfscl_adddevinfo(struct nfsmount *, struct nfscldevinfo *,
+int nfscl_adddevinfo(struct nfsmount *, struct nfscldevinfo *, int,
struct nfsclflayout *);
void nfscl_freelayout(struct nfscllayout *);
void nfscl_freeflayout(struct nfsclflayout *);
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h Thu Jul 12 19:02:59 2018 (r336227)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfsclstate.h Thu Jul 12 20:28:52 2018 (r336228)
@@ -269,6 +269,7 @@ struct nfscllayout {
*/
struct nfsffm {
nfsv4stateid_t st;
+ struct nfscldevinfo *devp;
char dev[NFSX_V4DEVICEID];
uint32_t eff;
uid_t user;
@@ -290,7 +291,6 @@ struct nfsclflayout {
uint64_t nfsfl_off;
uint64_t nfsfl_end;
uint32_t nfsfl_iomode;
- struct nfscldevinfo *nfsfl_devp;
uint16_t nfsfl_flags;
union {
struct {
@@ -299,6 +299,7 @@ struct nfsclflayout {
uint32_t stripe1;
uint8_t dev[NFSX_V4DEVICEID];
uint16_t fhcnt;
+ struct nfscldevinfo *devp;
} fl;
struct {
uint64_t stripeunit;
@@ -317,6 +318,7 @@ struct nfsclflayout {
#define nfsfl_stripe1 nfsfl_un.fl.stripe1
#define nfsfl_dev nfsfl_un.fl.dev
#define nfsfl_fhcnt nfsfl_un.fl.fhcnt
+#define nfsfl_devp nfsfl_un.fl.devp
#define nfsfl_stripeunit nfsfl_un.ff.stripeunit
#define nfsfl_fflags nfsfl_un.ff.fflags
#define nfsfl_statshint nfsfl_un.ff.statshint
Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c Thu Jul 12 19:02:59 2018 (r336227)
+++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c Thu Jul 12 20:28:52 2018 (r336228)
@@ -5722,12 +5722,15 @@ nfscl_doiods(vnode_t vp, struct uio *uiop, int *iomode
}
}
for (i = firstmirror; i < mirrorcnt && error == 0; i++){
- if ((layp->nfsly_flags & NFSLY_FLEXFILE) != 0)
+ if ((layp->nfsly_flags & NFSLY_FLEXFILE) != 0) {
dev = rflp->nfsfl_ffm[i].dev;
- else
+ dip = nfscl_getdevinfo(nmp->nm_clp, dev,
+ rflp->nfsfl_ffm[i].devp);
+ } else {
dev = rflp->nfsfl_dev;
- dip = nfscl_getdevinfo(nmp->nm_clp, dev,
- rflp->nfsfl_devp);
+ dip = nfscl_getdevinfo(nmp->nm_clp, dev,
+ rflp->nfsfl_devp);
+ }
if (dip != NULL) {
if ((rflp->nfsfl_flags & NFSFL_FLEXFILE)
!= 0)
@@ -6828,7 +6831,8 @@ nfsrv_parselayoutget(struct nfsrv_descript *nd, nfsv4s
sizeof(struct nfsffm), M_NFSFLAYOUT, M_WAITOK);
flp->nfsfl_flags = NFSFL_FLEXFILE;
flp->nfsfl_mirrorcnt = mirrorcnt;
- flp->nfsfl_devp = NULL;
+ for (j = 0; j < mirrorcnt; j++)
+ flp->nfsfl_ffm[j].devp = NULL;
flp->nfsfl_off = off;
if (flp->nfsfl_off + retlen < flp->nfsfl_off)
flp->nfsfl_end = UINT64_MAX - flp->nfsfl_off;
@@ -7588,6 +7592,7 @@ nfsrpc_layoutgetres(struct nfsmount *nmp, vnode_t vp,
struct nfsclflayout *tflp;
struct nfscldevinfo *dip;
uint8_t *dev;
+ int i, mirrorcnt;
if (laystat == NFSERR_UNKNLAYOUTTYPE) {
NFSLOCKMNT(nmp);
@@ -7604,25 +7609,35 @@ nfsrpc_layoutgetres(struct nfsmount *nmp, vnode_t vp,
if (laystat == 0) {
NFSCL_DEBUG(4, "nfsrpc_layoutgetres at FOREACH\n");
LIST_FOREACH(tflp, flhp, nfsfl_list) {
- laystat = nfscl_adddevinfo(nmp, NULL, tflp);
- NFSCL_DEBUG(4, "aft adddev=%d\n", laystat);
- if (laystat != 0) {
- if (layouttype == NFSLAYOUT_FLEXFILE)
- dev = tflp->nfsfl_ffm[0].dev;
- else
- dev = tflp->nfsfl_dev;
- laystat = nfsrpc_getdeviceinfo(nmp, dev,
- layouttype, notifybit, &dip, cred, p);
- NFSCL_DEBUG(4, "aft nfsrpc_gdi=%d\n",
- laystat);
- if (laystat != 0)
- break;
- laystat = nfscl_adddevinfo(nmp, dip, tflp);
- if (laystat != 0)
- printf("getlayout: cannot add\n");
+ if (layouttype == NFSLAYOUT_FLEXFILE)
+ mirrorcnt = tflp->nfsfl_mirrorcnt;
+ else
+ mirrorcnt = 1;
+ for (i = 0; i < mirrorcnt; i++) {
+ laystat = nfscl_adddevinfo(nmp, NULL, i, tflp);
+ NFSCL_DEBUG(4, "aft adddev=%d\n", laystat);
+ if (laystat != 0) {
+ if (layouttype == NFSLAYOUT_FLEXFILE)
+ dev = tflp->nfsfl_ffm[i].dev;
+ else
+ dev = tflp->nfsfl_dev;
+ laystat = nfsrpc_getdeviceinfo(nmp, dev,
+ layouttype, notifybit, &dip, cred,
+ p);
+ NFSCL_DEBUG(4, "aft nfsrpc_gdi=%d\n",
+ laystat);
+ if (laystat != 0)
+ goto out;
+ laystat = nfscl_adddevinfo(nmp, dip, i,
+ tflp);
+ if (laystat != 0)
+ printf("nfsrpc_layoutgetresout"
+ ": cannot add\n");
+ }
}
}
}
+out:
if (laystat == 0) {
/*
* nfscl_layout() always returns with the nfsly_lock
Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Thu Jul 12 19:02:59 2018 (r336227)
+++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Thu Jul 12 20:28:52 2018 (r336228)
@@ -5026,8 +5026,8 @@ nfscl_cancelreqs(struct nfsclds *dsp)
NFSUNLOCKDS(dsp);
CLNT_CLOSE(cl);
/*
- * This 1sec sleep is done to give the MDS time to recall
- * layouts before this client attempts further I/O.
+ * This 1sec sleep is done to reduce the number of reconnect
+ * attempts made on the DS while it has failed.
*/
tsleep(&non_event, PVFS, "ndscls", hz);
return;
@@ -5152,7 +5152,7 @@ nfscl_mergeflayouts(struct nfsclflayouthead *fhlp,
* This function consumes the structure pointed at by dip, if not NULL.
*/
APPLESTATIC int
-nfscl_adddevinfo(struct nfsmount *nmp, struct nfscldevinfo *dip,
+nfscl_adddevinfo(struct nfsmount *nmp, struct nfscldevinfo *dip, int ind,
struct nfsclflayout *flp)
{
struct nfsclclient *clp;
@@ -5170,11 +5170,14 @@ nfscl_adddevinfo(struct nfsmount *nmp, struct nfscldev
if ((flp->nfsfl_flags & NFSFL_FILE) != 0)
dev = flp->nfsfl_dev;
else
- dev = flp->nfsfl_ffm[0].dev;
+ dev = flp->nfsfl_ffm[ind].dev;
tdip = nfscl_finddevinfo(clp, dev);
if (tdip != NULL) {
tdip->nfsdi_layoutrefs++;
- flp->nfsfl_devp = tdip;
+ if ((flp->nfsfl_flags & NFSFL_FILE) != 0)
+ flp->nfsfl_devp = tdip;
+ else
+ flp->nfsfl_ffm[ind].devp = tdip;
nfscl_reldevinfo_locked(tdip);
NFSUNLOCKCLSTATE();
if (dip != NULL)
@@ -5184,7 +5187,10 @@ nfscl_adddevinfo(struct nfsmount *nmp, struct nfscldev
if (dip != NULL) {
LIST_INSERT_HEAD(&clp->nfsc_devinfo, dip, nfsdi_list);
dip->nfsdi_layoutrefs = 1;
- flp->nfsfl_devp = dip;
+ if ((flp->nfsfl_flags & NFSFL_FILE) != 0)
+ flp->nfsfl_devp = dip;
+ else
+ flp->nfsfl_ffm[ind].devp = dip;
}
NFSUNLOCKCLSTATE();
if (dip == NULL)
@@ -5225,15 +5231,19 @@ nfscl_freeflayout(struct nfsclflayout *flp)
{
int i, j;
- if ((flp->nfsfl_flags & NFSFL_FILE) != 0)
+ if ((flp->nfsfl_flags & NFSFL_FILE) != 0) {
for (i = 0; i < flp->nfsfl_fhcnt; i++)
free(flp->nfsfl_fh[i], M_NFSFH);
+ if (flp->nfsfl_devp != NULL)
+ flp->nfsfl_devp->nfsdi_layoutrefs--;
+ }
if ((flp->nfsfl_flags & NFSFL_FLEXFILE) != 0)
- for (i = 0; i < flp->nfsfl_mirrorcnt; i++)
+ for (i = 0; i < flp->nfsfl_mirrorcnt; i++) {
for (j = 0; j < flp->nfsfl_ffm[i].fhcnt; j++)
free(flp->nfsfl_ffm[i].fh[j], M_NFSFH);
- if (flp->nfsfl_devp != NULL)
- flp->nfsfl_devp->nfsdi_layoutrefs--;
+ if (flp->nfsfl_ffm[i].devp != NULL)
+ flp->nfsfl_ffm[i].devp->nfsdi_layoutrefs--;
+ }
free(flp, M_NFSFLAYOUT);
}
From owner-svn-src-projects@freebsd.org Sat Jul 14 13:39:23 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1AB7102B5A3
for <svn-src-projects@mailman.ysv.freebsd.org>;
Sat, 14 Jul 2018 13:39:22 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id A11E07A1C3;
Sat, 14 Jul 2018 13:39:22 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 82B8E1FAE1;
Sat, 14 Jul 2018 13:39:22 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6EDdMq8034722;
Sat, 14 Jul 2018 13:39:22 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6EDdMD0034720;
Sat, 14 Jul 2018 13:39:22 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807141339.w6EDdMD0034720@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Sat, 14 Jul 2018 13:39:22 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336276 - projects/pnfs-planb-server/sys/rpc
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: projects/pnfs-planb-server/sys/rpc
X-SVN-Commit-Revision: 336276
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Jul 2018 13:39:23 -0000
Author: rmacklem
Date: Sat Jul 14 13:39:21 2018
New Revision: 336276
URL: https://svnweb.freebsd.org/changeset/base/336276
Log:
Add a timeout to the msleep() waiting for a socket connection for the pNFS
client.
Without this patch, the client TCP connection code can wait a long time
for a connection attempt to fail when a DS is not operational.
This patch adds a timeout for this msleep() if the CLSET_TIMEOUT has been
set for the connection. This is only set for TCP connections to DSs.
Modified:
projects/pnfs-planb-server/sys/rpc/clnt_rc.c
projects/pnfs-planb-server/sys/rpc/clnt_vc.c
Modified: projects/pnfs-planb-server/sys/rpc/clnt_rc.c
==============================================================================
--- projects/pnfs-planb-server/sys/rpc/clnt_rc.c Sat Jul 14 10:14:59 2018 (r336275)
+++ projects/pnfs-planb-server/sys/rpc/clnt_rc.c Sat Jul 14 13:39:21 2018 (r336276)
@@ -175,7 +175,7 @@ clnt_reconnect_connect(CLIENT *cl)
(struct sockaddr *) &rc->rc_addr, rc->rc_prog, rc->rc_vers,
rc->rc_sendsz, rc->rc_recvsz);
else {
- if (rc->rc_timeout.tv_sec > 0 && rc->rc_timeout.tv_usec != -1) {
+ if (rc->rc_timeout.tv_sec > 0 && rc->rc_timeout.tv_usec >= 0) {
error = so_setsockopt(so, SOL_SOCKET, SO_SNDTIMEO,
&rc->rc_timeout, sizeof(struct timeval));
if (error != 0) {
Modified: projects/pnfs-planb-server/sys/rpc/clnt_vc.c
==============================================================================
--- projects/pnfs-planb-server/sys/rpc/clnt_vc.c Sat Jul 14 10:14:59 2018 (r336275)
+++ projects/pnfs-planb-server/sys/rpc/clnt_vc.c Sat Jul 14 13:39:21 2018 (r336276)
@@ -142,6 +142,8 @@ clnt_vc_create(
XDR xdrs;
int error, interrupted, one = 1, sleep_flag;
struct sockopt sopt;
+ struct timeval tv;
+ int timo;
if (disrupt == 0)
disrupt = (uint32_t)(long)raddr;
@@ -156,6 +158,18 @@ clnt_vc_create(
ct->ct_upcallrefs = 0;
if ((so->so_state & (SS_ISCONNECTED|SS_ISCONFIRMING)) == 0) {
+ /* Check for a timeout on the soconnect(). */
+ bzero(&sopt, sizeof(sopt));
+ sopt.sopt_dir = SOPT_GET;
+ sopt.sopt_level = SOL_SOCKET;
+ sopt.sopt_name = SO_SNDTIMEO;
+ sopt.sopt_val = &tv;
+ sopt.sopt_valsize = sizeof(tv);
+ error = sogetopt(so, &sopt);
+ timo = 0;
+ if (error == 0 && tv.tv_sec > 0 && tv.tv_sec <= 600 &&
+ tv.tv_usec >= 0)
+ timo = hz * tv.tv_sec;
error = soconnect(so, raddr, curthread);
SOCK_LOCK(so);
interrupted = 0;
@@ -165,9 +179,10 @@ clnt_vc_create(
while ((so->so_state & SS_ISCONNECTING)
&& so->so_error == 0) {
error = msleep(&so->so_timeo, SOCK_MTX(so),
- sleep_flag, "connec", 0);
+ sleep_flag, "connec", timo);
if (error) {
- if (error == EINTR || error == ERESTART)
+ if (error == EINTR || error == ERESTART ||
+ error == EAGAIN)
interrupted = 1;
break;
}
From owner-svn-src-projects@freebsd.org Sat Jul 14 13:45:32 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75952102B980
for <svn-src-projects@mailman.ysv.freebsd.org>;
Sat, 14 Jul 2018 13:45:32 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 2730B7A5AF;
Sat, 14 Jul 2018 13:45:32 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 087061FC73;
Sat, 14 Jul 2018 13:45:32 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6EDjV6Q039647;
Sat, 14 Jul 2018 13:45:31 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6EDjVKq039646;
Sat, 14 Jul 2018 13:45:31 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807141345.w6EDjVKq039646@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Sat, 14 Jul 2018 13:45:31 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336277 - projects/pnfs-planb-server/sys/fs/nfs
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: projects/pnfs-planb-server/sys/fs/nfs
X-SVN-Commit-Revision: 336277
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Jul 2018 13:45:32 -0000
Author: rmacklem
Date: Sat Jul 14 13:45:31 2018
New Revision: 336277
URL: https://svnweb.freebsd.org/changeset/base/336277
Log:
Set CLSET_TIMEOUT() on a TCP connection to a DS to lease_duration/4.
Setting the value this way results in the DS being disabled after approximately
the lease duration, due to retries within the krpc.
At least this is what I am seeing during testing and making it a fraction of
the lease duration seemed better than just picking an arbitrary timeout.
Modified:
projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c Sat Jul 14 13:39:21 2018 (r336276)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c Sat Jul 14 13:45:31 2018 (r336277)
@@ -98,6 +98,7 @@ extern int nfscl_ticks;
extern void (*ncl_call_invalcaches)(struct vnode *);
extern int nfs_numnfscbd;
extern int nfscl_debuglevel;
+extern int nfsrv_lease;
SVCPOOL *nfscbd_pool;
static int nfsrv_gsscallbackson = 0;
@@ -195,8 +196,10 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq
*/
if (nrp->nr_cred != NULL)
td->td_ucred = nrp->nr_cred;
- else if (cred != NULL)
+ else {
+ KASSERT(cred != NULL, ("newnfs_request: null cred"));
td->td_ucred = cred;
+ }
saddr = nrp->nr_nam;
if (saddr->sa_family == AF_INET)
@@ -300,7 +303,9 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq
if (cred != NULL) {
if (NFSHASSOFT(nmp)) {
/* This should be a DS mount. */
- timo.tv_sec = 15;
+ timo.tv_sec = nfsrv_lease / 4;
+ if (timo.tv_sec < 10)
+ timo.tv_sec = 10;
timo.tv_usec = 0;
CLNT_CONTROL(client, CLSET_TIMEOUT,
&timo);
@@ -333,7 +338,9 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq
* only case where using a "soft" mount is
* recommended for NFSv4.
*/
- timo.tv_sec = 15;
+ timo.tv_sec = nfsrv_lease / 4;
+ if (timo.tv_sec < 10)
+ timo.tv_sec = 10;
timo.tv_usec = 0;
CLNT_CONTROL(client, CLSET_TIMEOUT, &timo);
retries = 2;
From owner-svn-src-projects@freebsd.org Sat Jul 14 20:39:11 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 905B5103D3FE
for <svn-src-projects@mailman.ysv.freebsd.org>;
Sat, 14 Jul 2018 20:39:11 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 319E088A52;
Sat, 14 Jul 2018 20:39:11 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0D1BA23F11;
Sat, 14 Jul 2018 20:39:11 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6EKdAoN049000;
Sat, 14 Jul 2018 20:39:10 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6EKdA9i048998;
Sat, 14 Jul 2018 20:39:10 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807142039.w6EKdA9i048998@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Sat, 14 Jul 2018 20:39:10 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336295 - in projects/pnfs-planb-server/sys/fs: nfs
nfsclient
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: in projects/pnfs-planb-server/sys/fs: nfs nfsclient
X-SVN-Commit-Revision: 336295
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Jul 2018 20:39:11 -0000
Author: rmacklem
Date: Sat Jul 14 20:39:10 2018
New Revision: 336295
URL: https://svnweb.freebsd.org/changeset/base/336295
Log:
Add code to the pNFS client so that it shuts down a TCP connection to a
DS when a renew operation (actually just a Sequence Op) fails due to ENXIO.
Not particularily critical, but without this patch the client just does
a TCP reconnect attempt every 30sec.
Also, make the call to nfscl_cancelreqs() unconditional, since it checks
for NFSCLDS_SAMECONN and it does so with the lock held.
Modified:
projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h
projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c
projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Sat Jul 14 20:14:00 2018 (r336294)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfs_var.h Sat Jul 14 20:39:10 2018 (r336295)
@@ -603,6 +603,7 @@ struct nfscllayout *nfscl_getlayout(struct nfsclclient
uint64_t, struct nfsclflayout **, int *);
void nfscl_dserr(uint32_t, uint32_t, struct nfscldevinfo *,
struct nfscllayout *, struct nfsclds *);
+void nfscl_cancelreqs(struct nfsclds *);
void nfscl_rellayout(struct nfscllayout *, int);
struct nfscldevinfo *nfscl_getdevinfo(struct nfsclclient *, uint8_t *,
struct nfscldevinfo *);
Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c Sat Jul 14 20:14:00 2018 (r336294)
+++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clrpcops.c Sat Jul 14 20:39:10 2018 (r336295)
@@ -4408,9 +4408,12 @@ nfsrpc_renew(struct nfsclclient *clp, struct nfsclds *
if (dsp == NULL)
error = newnfs_request(nd, nmp, NULL, nrp, NULL, p, cred,
NFS_PROG, NFS_VER4, NULL, 1, NULL, NULL);
- else
+ else {
error = newnfs_request(nd, nmp, NULL, nrp, NULL, p, cred,
NFS_PROG, NFS_VER4, NULL, 1, NULL, &dsp->nfsclds_sess);
+ if (error == ENXIO)
+ nfscl_cancelreqs(dsp);
+ }
if (error)
return (error);
error = nd->nd_repstat;
Modified: projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Sat Jul 14 20:14:00 2018 (r336294)
+++ projects/pnfs-planb-server/sys/fs/nfsclient/nfs_clstate.c Sat Jul 14 20:39:10 2018 (r336295)
@@ -125,7 +125,6 @@ static struct nfscldeleg *nfscl_finddeleg(struct nfscl
static void nfscl_retoncloselayout(vnode_t, struct nfsclclient *, uint8_t *,
int, struct nfsclrecalllayout **);
static void nfscl_reldevinfo_locked(struct nfscldevinfo *);
-static void nfscl_cancelreqs(struct nfsclds *);
static struct nfscllayout *nfscl_findlayout(struct nfsclclient *, u_int8_t *,
int);
static struct nfscldevinfo *nfscl_finddevinfo(struct nfsclclient *, uint8_t *);
@@ -5001,16 +5000,17 @@ nfscl_dserr(uint32_t op, uint32_t stat, struct nfsclde
free(recallp, M_NFSLAYRECALL);
}
- /* If the connection isn't used for other DSs, we can shut it down. */
- if ((dsp->nfsclds_flags & NFSCLDS_SAMECONN) == 0)
- nfscl_cancelreqs(dsp);
+ /* And shut the TCP connection down. */
+ nfscl_cancelreqs(dsp);
}
/*
* Cancel all RPCs for this "dsp" by closing the connection.
* Also, mark the session as defunct.
+ * If NFSCLDS_SAMECONN is set, the connection is shared with other DSs and
+ * cannot be shut down.
*/
-static void
+APPLESTATIC void
nfscl_cancelreqs(struct nfsclds *dsp)
{
struct __rpc_client *cl;
From owner-svn-src-projects@freebsd.org Sat Jul 14 20:44:26 2018
Return-Path: <owner-svn-src-projects@freebsd.org>
Delivered-To: svn-src-projects@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58F2A103D813
for <svn-src-projects@mailman.ysv.freebsd.org>;
Sat, 14 Jul 2018 20:44:26 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mxrelay.nyi.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id F26EB89092;
Sat, 14 Jul 2018 20:44:25 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:0])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CDFFE240A3;
Sat, 14 Jul 2018 20:44:25 +0000 (UTC)
(envelope-from rmacklem@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6EKiPBK054002;
Sat, 14 Jul 2018 20:44:25 GMT (envelope-from rmacklem@FreeBSD.org)
Received: (from rmacklem@localhost)
by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6EKiPfS054001;
Sat, 14 Jul 2018 20:44:25 GMT (envelope-from rmacklem@FreeBSD.org)
Message-Id: <201807142044.w6EKiPfS054001@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to
rmacklem@FreeBSD.org using -f
From: Rick Macklem <rmacklem@FreeBSD.org>
Date: Sat, 14 Jul 2018 20:44:25 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-projects@freebsd.org
Subject: svn commit: r336296 - projects/pnfs-planb-server/sys/fs/nfs
X-SVN-Group: projects
X-SVN-Commit-Author: rmacklem
X-SVN-Commit-Paths: projects/pnfs-planb-server/sys/fs/nfs
X-SVN-Commit-Revision: 336296
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-projects@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: "SVN commit messages for the src " projects"
tree" <svn-src-projects.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-projects/>
List-Post: <mailto:svn-src-projects@freebsd.org>
List-Help: <mailto:svn-src-projects-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-projects>,
<mailto:svn-src-projects-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Jul 2018 20:44:26 -0000
Author: rmacklem
Date: Sat Jul 14 20:44:25 2018
New Revision: 336296
URL: https://svnweb.freebsd.org/changeset/base/336296
Log:
Add a comment explaining why the timeout for DS connections is set to
lease_duration/4.
Modified:
projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c
Modified: projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c
==============================================================================
--- projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c Sat Jul 14 20:39:10 2018 (r336295)
+++ projects/pnfs-planb-server/sys/fs/nfs/nfs_commonkrpc.c Sat Jul 14 20:44:25 2018 (r336296)
@@ -302,7 +302,16 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq
if (NFSHASNFSV4N(nmp)) {
if (cred != NULL) {
if (NFSHASSOFT(nmp)) {
- /* This should be a DS mount. */
+ /*
+ * This should be a DS mount. If the
+ * timeout is set to 1/4th of the lease
+ * duration, the DS should be disabled
+ * at approximately 1/2 lease duration,
+ * due to a retry count of 2.
+ * This will hopefully cause the client
+ * to continue without the disabled
+ * DS in less than the lease duration.
+ */
timo.tv_sec = nfsrv_lease / 4;
if (timo.tv_sec < 10)
timo.tv_sec = 10;
@@ -337,6 +346,7 @@ newnfs_connect(struct nfsmount *nmp, struct nfssockreq
* not maintain open/lock state and is the
* only case where using a "soft" mount is
* recommended for NFSv4.
+ * See the comment above w.r.t. timeout.
*/
timo.tv_sec = nfsrv_lease / 4;
if (timo.tv_sec < 10)