Date: Fri, 15 Jun 2018 19:35:08 +0000 (UTC) From: Rick Macklem <rmacklem@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r335234 - in head/usr.sbin: . pnfsdsfile Message-ID: <201806151935.w5FJZ80v011822@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: rmacklem Date: Fri Jun 15 19:35:08 2018 New Revision: 335234 URL: https://svnweb.freebsd.org/changeset/base/335234 Log: Add a command the displays and modifies the pNFS server's extended attribute. This command allows a sysadmin to display or modify the pnfsd.dsfile extended attribute used by the pNFS MDS server in various ways. Its main use is to set a DS's IP address to 0.0.0.0 when that DS has failed, so that it will not be used for the file when brought back online after being repaired. Added: head/usr.sbin/pnfsdsfile/ head/usr.sbin/pnfsdsfile/Makefile (contents, props changed) head/usr.sbin/pnfsdsfile/pnfsdsfile.8 (contents, props changed) head/usr.sbin/pnfsdsfile/pnfsdsfile.c (contents, props changed) Modified: head/usr.sbin/Makefile Modified: head/usr.sbin/Makefile ============================================================================== --- head/usr.sbin/Makefile Fri Jun 15 19:19:36 2018 (r335233) +++ head/usr.sbin/Makefile Fri Jun 15 19:35:08 2018 (r335234) @@ -59,6 +59,7 @@ SUBDIR= adduser \ nologin \ pciconf \ periodic \ + pnfsdsfile \ pnfsdskill \ powerd \ prometheus_sysctl_exporter \ Added: head/usr.sbin/pnfsdsfile/Makefile ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/pnfsdsfile/Makefile Fri Jun 15 19:35:08 2018 (r335234) @@ -0,0 +1,6 @@ +# $FreeBSD$ + +PROG= pnfsdsfile +MAN= pnfsdsfile.8 + +.include <bsd.prog.mk> Added: head/usr.sbin/pnfsdsfile/pnfsdsfile.8 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/pnfsdsfile/pnfsdsfile.8 Fri Jun 15 19:35:08 2018 (r335234) @@ -0,0 +1,133 @@ +.\" Copyright (c) 2017 Rick Macklem +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd March 11, 2018 +.Dt PNFSDSFILE 8 +.Os +.Sh NAME +.Nm pnfsdsfile +.Nd display +a pNFS data storage file's location(s) and/or modify the +.Dq pnfsd.dsfile +extended attribute for them +.Sh SYNOPSIS +.Nm +.Op Fl qz +.Op Fl s Ar dshostname +.Op Fl c Ar old-dshostname,new-dshostname +.Op Fl r Ar dshostname +.Ar mdsfile +.Sh DESCRIPTION +The +.Nm +command displays the data storage file's location(s) for a pNFS service and/or +modifies the +.Dq pnfsd.dsfile +extended attribute on the +.Ar mdsfile . +A pNFS service maintains a data storage file for each regular file on +the MetaData Server (MDS) on one or more of the Data Servers (DS). +If mirroring is enabled, the data storage file will be on more that one of the DSs. +Unless command options are specified, this command displays the location(s) +of the data storage file for the MDS file +.Ar mdsfile . +It must be used on the MDS and the +.Ar mdsfile +must be a file on the exported local file system and not an NFSv4.1 mount. +This information is stored in the +.Dq pnfsd.dsfile +extended attribute for this +.Ar mdsfile . +The command line options allow the information in the +.Dq pnfsd.dsfile +extended attribute to be changed. +.Pp +The following options are available: +.Bl -tag -width Ds +.It Fl q +This option suppresses printing of the DS file's location(s). +.It Fl z +This option specifies that the file handle field of the pnfsd.dsfile +extended attribute is to filled with all zero bits. +This forces the pNFS MDS to do a Lookup RPC against the DS to acquire the file +handle to update it. +Normally this will only be necessary after the DS file has been recovered +from a backup, causing the file handle to change. +.It Fl s Ar dshostname +This option can be used with +.Fl z +so that the zeroing out of the file handle is only done if the DS server +is the one specified by this option. +.It Fl c Ar old-dshostname,new-dshostname +This option allows a sysadmin to replace the host IP# for the DS in the +pnfsd.dsfile extended attribute. +The old-hostname must resolve to the IP# already in the pnfsd.dsfile extended +attribute or the replacement will not be done. +If the old-dshostname matches, then the IP# is replaced by the first AF_INET +or AF_INET6 address that +.Xr getaddrinfo 3 +returns for the new-dshostname. +Changing a DS server's host IP# should be avoided, but this option will +allow it to be changed, if the change is unavoidable. +.It Fl r Ar dshostname +This option sets the IP address of the extended attribute entry for the +.Ar dshostname +to 0.0.0.0 so that it will no longer be used. +.Pp +This is meant to be used when mirroring is enabled and the +.Ar dshostname +DS is disabled, so that it can be re-enabled once it is repaired. +This needs to be done for all files in the exported MDS tree where +the data may not be up-to-date on the repaired DS when it is re-enabled. +After being re-enabled, the command +.Xr pnfsdscopymr 1 +with the +.Dq -r +option +will be used to copy the the file's data to this repaired DS and then update the +extended attribute to use it. +.Pp +A typical use of this will be within a +.Xr find 1 +for all regular files in the MDS's exported tree. +.sp +For example, if the disabled DS is nfsv4-data3: +.br +# cd <top-level-exported-directory-on-MDS> +.br +# find . -type f -exec pnfsdsfile -q -r nfsv4-data3 {} \\; +.El +.Sh SEE ALSO +.Xr find 1 , +.Xr getaddrinfo 3 , +.Xr nfsv4 4 , +.Xr pnfs 4 , +.Xr nfsd 8 , +.Xr pnfsdscopymr 8 , +.Xr pnfsdskill 8 +.Sh HISTORY +The +.Nm +command appeared in FreeBSD12. Added: head/usr.sbin/pnfsdsfile/pnfsdsfile.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/pnfsdsfile/pnfsdsfile.c Fri Jun 15 19:35:08 2018 (r335234) @@ -0,0 +1,330 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 2017 Rick Macklem + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD$"); + +#include <err.h> +#include <getopt.h> +#include <netdb.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <sys/param.h> +#include <sys/extattr.h> +#include <sys/mount.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <fs/nfs/nfskpiport.h> +#include <fs/nfs/nfsproto.h> +#include <fs/nfs/nfs.h> +#include <fs/nfs/nfsrvstate.h> + +static void usage(void); + +static struct option longopts[] = { + { "changeds", required_argument, NULL, 'c' }, + { "quiet", no_argument, NULL, 'q' }, + { "zerods", required_argument, NULL, 'r' }, + { "ds", required_argument, NULL, 's' }, + { "zerofh", no_argument, NULL, 'z' }, + { NULL, 0, NULL, 0 } +}; + +/* + * This program displays the location information of a data storage file + * for a given file on a MetaData Server (MDS) in a pNFS service. This program + * must be run on the MDS and the file argument must be a file in a local + * file system that has been exported for the pNFS service. + */ +int +main(int argc, char *argv[]) +{ + struct addrinfo *res, *ad, *newres; + struct sockaddr_in *sin, adsin; + struct sockaddr_in6 *sin6, adsin6; + char hostn[2 * NI_MAXHOST + 2], *cp; + struct pnfsdsfile dsfile[NFSDEV_MAXMIRRORS]; + int ch, dosetxattr, i, mirrorcnt, quiet, zerods, zerofh; + in_port_t tport; + ssize_t xattrsize, xattrsize2; + + zerods = 0; + zerofh = 0; + quiet = 0; + dosetxattr = 0; + res = NULL; + newres = NULL; + cp = NULL; + while ((ch = getopt_long(argc, argv, "c:qr:s:z", longopts, NULL)) != -1) + { + switch (ch) { + case 'c': + /* Replace the first DS server with the second one. */ + if (zerofh != 0 || zerods != 0) + errx(1, "-c, -r and -z are mutually " + "exclusive"); + if (res != NULL) + errx(1, "-c and -s are mutually exclusive"); + strlcpy(hostn, optarg, 2 * NI_MAXHOST + 2); + cp = strchr(hostn, ','); + if (cp == NULL) + errx(1, "Bad -c argument %s", hostn); + *cp = '\0'; + if (getaddrinfo(hostn, NULL, NULL, &res) != 0) + errx(1, "Can't get IP# for %s", hostn); + *cp++ = ','; + if (getaddrinfo(cp, NULL, NULL, &newres) != 0) + errx(1, "Can't get IP# for %s", cp); + break; + case 'q': + quiet = 1; + break; + case 'r': + /* Reset the DS server in a mirror with 0.0.0.0. */ + if (zerofh != 0 || res != NULL || newres != NULL) + errx(1, "-r and -s, -z or -c are mutually " + "exclusive"); + zerods = 1; + /* Translate the server name to an IP address. */ + if (getaddrinfo(optarg, NULL, NULL, &res) != 0) + errx(1, "Can't get IP# for %s", optarg); + break; + case 's': + if (res != NULL) + errx(1, "-s, -c and -r are mutually " + "exclusive"); + /* Translate the server name to an IP address. */ + if (getaddrinfo(optarg, NULL, NULL, &res) != 0) + errx(1, "Can't get IP# for %s", optarg); + break; + case 'z': + if (newres != NULL || zerods != 0) + errx(1, "-c, -r and -z are mutually " + "exclusive"); + zerofh = 1; + break; + default: + usage(); + } + } + argc -= optind; + if (argc != 1) + usage(); + argv += optind; + + /* + * The host address and directory where the data storage file is + * located is in the extended attribute "pnfsd.dsfile". + */ + xattrsize = extattr_get_file(*argv, EXTATTR_NAMESPACE_SYSTEM, + "pnfsd.dsfile", dsfile, sizeof(dsfile)); + mirrorcnt = xattrsize / sizeof(struct pnfsdsfile); + xattrsize2 = mirrorcnt * sizeof(struct pnfsdsfile); + if (mirrorcnt < 1 || xattrsize != xattrsize2) + err(1, "Can't get extattr pnfsd.dsfile for %s", *argv); + + if (quiet == 0) + printf("%s:\t", *argv); + for (i = 0; i < mirrorcnt; i++) { + if (i > 0 && quiet == 0) + printf("\t"); + /* Do the zerofh option. You must be root. */ + if (zerofh != 0) { + if (geteuid() != 0) + errx(1, "Must be root/su to zerofh"); + + /* + * Do it for the server specified by -s/--ds or all + * servers, if -s/--ds was not specified. + */ + sin = &dsfile[i].dsf_sin; + sin6 = &dsfile[i].dsf_sin6; + ad = res; + while (ad != NULL) { + if (ad->ai_addr->sa_family == AF_INET && + sin->sin_family == AF_INET && + ad->ai_addrlen >= sizeof(adsin)) { + memcpy(&adsin, ad->ai_addr, + sizeof(adsin)); + if (sin->sin_addr.s_addr == + adsin.sin_addr.s_addr) + break; + } + if (ad->ai_addr->sa_family == AF_INET6 && + sin6->sin6_family == AF_INET6 && + ad->ai_addrlen >= sizeof(adsin6)) { + memcpy(&adsin6, ad->ai_addr, + sizeof(adsin6)); + if (IN6_ARE_ADDR_EQUAL(&sin6->sin6_addr, + &adsin6.sin6_addr)) + break; + } + ad = ad->ai_next; + } + if (res == NULL || ad != NULL) { + memset(&dsfile[i].dsf_fh, 0, sizeof(fhandle_t)); + dosetxattr = 1; + } + } + + /* Do the zerods option. You must be root. */ + if (zerods != 0 && mirrorcnt > 1) { + if (geteuid() != 0) + errx(1, "Must be root/su to zerods"); + + /* + * Do it for the server specified. + */ + sin = &dsfile[i].dsf_sin; + sin6 = &dsfile[i].dsf_sin6; + ad = res; + while (ad != NULL) { + if (ad->ai_addr->sa_family == AF_INET && + sin->sin_family == AF_INET && + ad->ai_addrlen >= sizeof(adsin)) { + memcpy(&adsin, ad->ai_addr, + sizeof(adsin)); + if (sin->sin_addr.s_addr == + adsin.sin_addr.s_addr) + break; + } + if (ad->ai_addr->sa_family == AF_INET6 && + sin6->sin6_family == AF_INET6 && + ad->ai_addrlen >= sizeof(adsin6)) { + memcpy(&adsin6, ad->ai_addr, + sizeof(adsin6)); + if (IN6_ARE_ADDR_EQUAL(&sin6->sin6_addr, + &adsin6.sin6_addr)) + break; + } + ad = ad->ai_next; + } + if (ad != NULL) { + sin->sin_family = AF_INET; + sin->sin_len = sizeof(*sin); + sin->sin_port = 0; + sin->sin_addr.s_addr = 0; + dosetxattr = 1; + } + } + + /* Do the -c option to replace the DS host address. */ + if (newres != NULL) { + if (geteuid() != 0) + errx(1, "Must be root/su to replace the host" + " addr"); + + /* + * Check that the old host address matches. + */ + sin = &dsfile[i].dsf_sin; + sin6 = &dsfile[i].dsf_sin6; + ad = res; + while (ad != NULL) { + if (ad->ai_addr->sa_family == AF_INET && + sin->sin_family == AF_INET && + ad->ai_addrlen >= sizeof(adsin)) { + memcpy(&adsin, ad->ai_addr, + sizeof(adsin)); + if (sin->sin_addr.s_addr == + adsin.sin_addr.s_addr) + break; + } + if (ad->ai_addr->sa_family == AF_INET6 && + sin6->sin6_family == AF_INET6 && + ad->ai_addrlen >= sizeof(adsin6)) { + memcpy(&adsin6, ad->ai_addr, + sizeof(adsin6)); + if (IN6_ARE_ADDR_EQUAL(&sin6->sin6_addr, + &adsin6.sin6_addr)) + break; + } + ad = ad->ai_next; + } + if (ad != NULL) { + if (sin->sin_family == AF_INET) + tport = sin->sin_port; + else + tport = sin6->sin6_port; + /* + * We have a match, so replace it with the first + * AF_INET or AF_INET6 address in the newres + * list. + */ + while (newres->ai_addr->sa_family != AF_INET && + newres->ai_addr->sa_family != AF_INET6) { + newres = newres->ai_next; + if (newres == NULL) + errx(1, "Hostname %s has no" + " IP#", cp); + } + if (newres->ai_addr->sa_family == AF_INET) { + memcpy(sin, newres->ai_addr, + sizeof(*sin)); + sin->sin_port = tport; + } else if (newres->ai_addr->sa_family == + AF_INET6) { + memcpy(sin6, newres->ai_addr, + sizeof(*sin6)); + sin6->sin6_port = tport; + } + dosetxattr = 1; + } + } + + if (quiet == 0) { + /* Translate the IP address to a hostname. */ + if (getnameinfo((struct sockaddr *)&dsfile[i].dsf_sin, + dsfile[i].dsf_sin.sin_len, hostn, sizeof(hostn), + NULL, 0, 0) < 0) + err(1, "Can't get hostname"); + printf("%s\tds%d/%s", hostn, dsfile[i].dsf_dir, + dsfile[i].dsf_filename); + } + } + if (quiet == 0) + printf("\n"); + if (dosetxattr != 0 && extattr_set_file(*argv, EXTATTR_NAMESPACE_SYSTEM, + "pnfsd.dsfile", dsfile, xattrsize) != xattrsize) + err(1, "Can't set pnfsd.dsfile"); +} + +static void +usage(void) +{ + + fprintf(stderr, "pnfsdsfile [-q/--quiet] [-z/--zerofh] " + "[-c/--changeds <old dshostname> <new dshostname>] " + "[-r/--zerods <dshostname>] " + "[-s/--ds <dshostname>] " + "<filename>\n"); + exit(1); +} +
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201806151935.w5FJZ80v011822>