Date: Fri, 1 Sep 2017 17:43:09 +0000 (UTC) From: Andriy Gapon <avg@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r323107 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/... Message-ID: <201709011743.v81Hh99N029459@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: avg Date: Fri Sep 1 17:43:08 2017 New Revision: 323107 URL: https://svnweb.freebsd.org/changeset/base/323107 Log: 8414 Implemented zpool scrub pause/resume illumos/illumos-gate@1702cce751c5cb7ead878d0205a6c90b027e3de8 https://github.com/illumos/illumos-gate/commit/1702cce751c5cb7ead878d0205a6c90b027e3de8 https://www.illumos.org/issues/8414 This issue tracks the port of scrub pause from ZoL: https://github.com/zfsonlinux/zfs/pull/6167 Currently, there is no way to pause a scrub. Pausing may be useful when the pool is busy with other I/O to preserve bandwidth. Description This patch adds the ability to pause and resume scrubbing. This is achieved by maintaining a persistent on-disk scrub state. While the state is 'paused' we do not scrub any more blocks. We do however perform regular scan housekeeping such as freeing async destroyed and deadlist blocks while paused. If you're testing this change, you probably want to include the patch from #6164 Motivation and Context Scrub pausing can be an I/O intensive operation and people have been asking for the ability to pause a scrub for a while. This allows one to preserve scrub progress while freeing up bandwidth for other I/O. How Has This Been Tested? Unit testing and zfs-tests. to the pool. This patch will also include the patch from https://github.com/zfsonlinux/zfs/ pull/6164 In certain cases (dsl_scan_sync() is one), we may end up calling Reviewed by: George Melikov <mail@gmelikov.ru> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed by: Brad Lewis <brad.lewis@delphix.com> Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: Alek Pinchuk <apinchuk@datto.com> Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c vendor/illumos/dist/lib/libzfs/common/libzfs_util.c vendor/illumos/dist/man/man1m/zpool.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_scan.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Fri Sep 1 17:38:49 2017 (r323106) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Fri Sep 1 17:43:08 2017 (r323107) @@ -26,6 +26,7 @@ * Copyright (c) 2013 by Prasad Joshi (sTec). All rights reserved. * Copyright 2016 Igor Kozhukhov <ikozhukhov@gmail.com>. * Copyright 2016 Nexenta Systems, Inc. + * Copyright (c) 2017 Datto Inc. */ #include <assert.h> @@ -250,7 +251,7 @@ get_usage(zpool_help_t idx) case HELP_REOPEN: return (gettext("\treopen <pool>\n")); case HELP_SCRUB: - return (gettext("\tscrub [-s] <pool> ...\n")); + return (gettext("\tscrub [-s | -p] <pool> ...\n")); case HELP_STATUS: return (gettext("\tstatus [-vx] [-T d|u] [pool] ... [interval " "[count]]\n")); @@ -3879,6 +3880,7 @@ typedef struct scrub_cbdata { int cb_type; int cb_argc; char **cb_argv; + pool_scrub_cmd_t cb_scrub_cmd; } scrub_cbdata_t; int @@ -3896,15 +3898,16 @@ scrub_callback(zpool_handle_t *zhp, void *data) return (1); } - err = zpool_scan(zhp, cb->cb_type); + err = zpool_scan(zhp, cb->cb_type, cb->cb_scrub_cmd); return (err != 0); } /* - * zpool scrub [-s] <pool> ... + * zpool scrub [-s | -p] <pool> ... * * -s Stop. Stops any in-progress scrub. + * -p Pause. Pause in-progress scrub. */ int zpool_do_scrub(int argc, char **argv) @@ -3913,13 +3916,17 @@ zpool_do_scrub(int argc, char **argv) scrub_cbdata_t cb; cb.cb_type = POOL_SCAN_SCRUB; + cb.cb_scrub_cmd = POOL_SCRUB_NORMAL; /* check options */ - while ((c = getopt(argc, argv, "s")) != -1) { + while ((c = getopt(argc, argv, "sp")) != -1) { switch (c) { case 's': cb.cb_type = POOL_SCAN_NONE; break; + case 'p': + cb.cb_scrub_cmd = POOL_SCRUB_PAUSE; + break; case '?': (void) fprintf(stderr, gettext("invalid option '%c'\n"), optopt); @@ -3927,6 +3934,13 @@ zpool_do_scrub(int argc, char **argv) } } + if (cb.cb_type == POOL_SCAN_NONE && + cb.cb_scrub_cmd == POOL_SCRUB_PAUSE) { + (void) fprintf(stderr, gettext("invalid option combination: " + "-s and -p are mutually exclusive\n")); + usage(B_FALSE); + } + cb.cb_argc = argc; cb.cb_argv = argv; argc -= optind; @@ -3955,7 +3969,7 @@ typedef struct status_cbdata { void print_scan_status(pool_scan_stat_t *ps) { - time_t start, end; + time_t start, end, pause; uint64_t elapsed, mins_left, hours_left; uint64_t pass_exam, examined, total; uint_t rate; @@ -3973,6 +3987,7 @@ print_scan_status(pool_scan_stat_t *ps) start = ps->pss_start_time; end = ps->pss_end_time; + pause = ps->pss_pass_scrub_pause; zfs_nicenum(ps->pss_processed, processed_buf, sizeof (processed_buf)); assert(ps->pss_func == POOL_SCAN_SCRUB || @@ -4015,8 +4030,17 @@ print_scan_status(pool_scan_stat_t *ps) * Scan is in progress. */ if (ps->pss_func == POOL_SCAN_SCRUB) { - (void) printf(gettext("scrub in progress since %s"), - ctime(&start)); + if (pause == 0) { + (void) printf(gettext("scrub in progress since %s"), + ctime(&start)); + } else { + char buf[32]; + struct tm *p = localtime(&pause); + (void) strftime(buf, sizeof (buf), "%a %b %e %T %Y", p); + (void) printf(gettext("scrub paused since %s\n"), buf); + (void) printf(gettext("\tscrub started on %s"), + ctime(&start)); + } } else if (ps->pss_func == POOL_SCAN_RESILVER) { (void) printf(gettext("resilver in progress since %s"), ctime(&start)); @@ -4028,6 +4052,7 @@ print_scan_status(pool_scan_stat_t *ps) /* elapsed time for this pass */ elapsed = time(NULL) - ps->pss_pass_start; + elapsed -= ps->pss_pass_scrub_spent_paused; elapsed = elapsed ? elapsed : 1; pass_exam = ps->pss_pass_exam ? ps->pss_pass_exam : 1; rate = pass_exam / elapsed; @@ -4037,19 +4062,25 @@ print_scan_status(pool_scan_stat_t *ps) zfs_nicenum(examined, examined_buf, sizeof (examined_buf)); zfs_nicenum(total, total_buf, sizeof (total_buf)); - zfs_nicenum(rate, rate_buf, sizeof (rate_buf)); /* * do not print estimated time if hours_left is more than 30 days + * or we have a paused scrub */ - (void) printf(gettext(" %s scanned out of %s at %s/s"), - examined_buf, total_buf, rate_buf); - if (hours_left < (30 * 24)) { - (void) printf(gettext(", %lluh%um to go\n"), - (u_longlong_t)hours_left, (uint_t)(mins_left % 60)); + if (pause == 0) { + zfs_nicenum(rate, rate_buf, sizeof (rate_buf)); + (void) printf(gettext("\t%s scanned out of %s at %s/s"), + examined_buf, total_buf, rate_buf); + if (hours_left < (30 * 24)) { + (void) printf(gettext(", %lluh%um to go\n"), + (u_longlong_t)hours_left, (uint_t)(mins_left % 60)); + } else { + (void) printf(gettext( + ", (scan is slow, no estimated time)\n")); + } } else { - (void) printf(gettext( - ", (scan is slow, no estimated time)\n")); + (void) printf(gettext("\t%s scanned out of %s\n"), + examined_buf, total_buf); } if (ps->pss_func == POOL_SCAN_RESILVER) { Modified: vendor/illumos/dist/lib/libzfs/common/libzfs.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs.h Fri Sep 1 17:38:49 2017 (r323106) +++ vendor/illumos/dist/lib/libzfs/common/libzfs.h Fri Sep 1 17:43:08 2017 (r323107) @@ -27,6 +27,7 @@ * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2016 Nexenta Systems, Inc. + * Copyright (c) 2017 Datto Inc. */ #ifndef _LIBZFS_H @@ -128,6 +129,7 @@ typedef enum zfs_error { EZFS_DIFF, /* general failure of zfs diff */ EZFS_DIFFDATA, /* bad zfs diff data */ EZFS_POOLREADONLY, /* pool is in read-only mode */ + EZFS_SCRUB_PAUSED, /* scrub currently paused */ EZFS_UNKNOWN } zfs_error_t; @@ -251,7 +253,7 @@ typedef struct splitflags { /* * Functions to manipulate pool and vdev state */ -extern int zpool_scan(zpool_handle_t *, pool_scan_func_t); +extern int zpool_scan(zpool_handle_t *, pool_scan_func_t, pool_scrub_cmd_t); extern int zpool_clear(zpool_handle_t *, const char *, nvlist_t *); extern int zpool_reguid(zpool_handle_t *); extern int zpool_reopen(zpool_handle_t *); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Fri Sep 1 17:38:49 2017 (r323106) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Fri Sep 1 17:43:08 2017 (r323107) @@ -25,6 +25,7 @@ * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright 2016 Nexenta Systems, Inc. * Copyright 2016 Igor Kozhukhov <ikozhukhov@gmail.com> + * Copyright (c) 2017 Datto Inc. */ #include <ctype.h> @@ -1851,22 +1852,39 @@ zpool_import_props(libzfs_handle_t *hdl, nvlist_t *con * Scan the pool. */ int -zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func) +zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func, pool_scrub_cmd_t cmd) { zfs_cmd_t zc = { 0 }; char msg[1024]; + int err; libzfs_handle_t *hdl = zhp->zpool_hdl; (void) strlcpy(zc.zc_name, zhp->zpool_name, sizeof (zc.zc_name)); zc.zc_cookie = func; + zc.zc_flags = cmd; - if (zfs_ioctl(hdl, ZFS_IOC_POOL_SCAN, &zc) == 0 || - (errno == ENOENT && func != POOL_SCAN_NONE)) + if (zfs_ioctl(hdl, ZFS_IOC_POOL_SCAN, &zc) == 0) return (0); + err = errno; + + /* ECANCELED on a scrub means we resumed a paused scrub */ + if (err == ECANCELED && func == POOL_SCAN_SCRUB && + cmd == POOL_SCRUB_NORMAL) + return (0); + + if (err == ENOENT && func != POOL_SCAN_NONE && cmd == POOL_SCRUB_NORMAL) + return (0); + if (func == POOL_SCAN_SCRUB) { - (void) snprintf(msg, sizeof (msg), - dgettext(TEXT_DOMAIN, "cannot scrub %s"), zc.zc_name); + if (cmd == POOL_SCRUB_PAUSE) { + (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + "cannot pause scrubbing %s"), zc.zc_name); + } else { + assert(cmd == POOL_SCRUB_NORMAL); + (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, + "cannot scrub %s"), zc.zc_name); + } } else if (func == POOL_SCAN_NONE) { (void) snprintf(msg, sizeof (msg), dgettext(TEXT_DOMAIN, "cannot cancel scrubbing %s"), @@ -1875,7 +1893,7 @@ zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func) assert(!"unexpected result"); } - if (errno == EBUSY) { + if (err == EBUSY) { nvlist_t *nvroot; pool_scan_stat_t *ps = NULL; uint_t psc; @@ -1884,14 +1902,18 @@ zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func) ZPOOL_CONFIG_VDEV_TREE, &nvroot) == 0); (void) nvlist_lookup_uint64_array(nvroot, ZPOOL_CONFIG_SCAN_STATS, (uint64_t **)&ps, &psc); - if (ps && ps->pss_func == POOL_SCAN_SCRUB) - return (zfs_error(hdl, EZFS_SCRUBBING, msg)); - else + if (ps && ps->pss_func == POOL_SCAN_SCRUB) { + if (cmd == POOL_SCRUB_PAUSE) + return (zfs_error(hdl, EZFS_SCRUB_PAUSED, msg)); + else + return (zfs_error(hdl, EZFS_SCRUBBING, msg)); + } else { return (zfs_error(hdl, EZFS_RESILVERING, msg)); - } else if (errno == ENOENT) { + } + } else if (err == ENOENT) { return (zfs_error(hdl, EZFS_NO_SCRUB, msg)); } else { - return (zpool_standard_error(hdl, errno, msg)); + return (zpool_standard_error(hdl, err, msg)); } } Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_util.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Fri Sep 1 17:38:49 2017 (r323106) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Fri Sep 1 17:43:08 2017 (r323107) @@ -24,6 +24,7 @@ * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2011, 2015 by Delphix. All rights reserved. * Copyright 2016 Igor Kozhukhov <ikozhukhov@gmail.com> + * Copyright (c) 2017 Datto Inc. */ /* @@ -219,6 +220,9 @@ libzfs_error_description(libzfs_handle_t *hdl) case EZFS_POSTSPLIT_ONLINE: return (dgettext(TEXT_DOMAIN, "disk was split from this pool " "into a new one")); + case EZFS_SCRUB_PAUSED: + return (dgettext(TEXT_DOMAIN, "scrub is paused; " + "use 'zpool scrub' to resume")); case EZFS_SCRUBBING: return (dgettext(TEXT_DOMAIN, "currently scrubbing; " "use 'zpool scrub -s' to cancel current scrub")); Modified: vendor/illumos/dist/man/man1m/zpool.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zpool.1m Fri Sep 1 17:38:49 2017 (r323106) +++ vendor/illumos/dist/man/man1m/zpool.1m Fri Sep 1 17:43:08 2017 (r323107) @@ -22,8 +22,9 @@ .\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved. .\" Copyright (c) 2013 by Delphix. All rights reserved. .\" Copyright 2017 Nexenta Systems, Inc. +.\" Copyright (c) 2017 Datto Inc. .\" -.Dd Oct 2, 2016 +.Dd June 21, 2017 .Dt ZPOOL 1M .Os .Sh NAME @@ -137,7 +138,7 @@ .Ar pool Ar device Op Ar new_device .Nm .Cm scrub -.Op Fl s +.Op Fl s | Fl p .Ar pool Ns ... .Nm .Cm set @@ -1445,10 +1446,10 @@ Not all devices can be overridden in this manner. .It Xo .Nm .Cm scrub -.Op Fl s +.Op Fl s | Fl p .Ar pool Ns ... .Xc -Begins a scrub. +Begins a scrub or resumes a paused scrub. The scrub examines all data in the specified pools to verify that it checksums correctly. For replicated @@ -1471,14 +1472,26 @@ faults or disk failure. .Pp Because scrubbing and resilvering are I/O-intensive operations, ZFS only allows one at a time. -If a scrub is already in progress, the +If a scrub is paused, the .Nm zpool Cm scrub -command terminates it and starts a new scrub. +resumes it. If a resilver is in progress, ZFS does not allow a scrub to be started until the resilver completes. .Bl -tag -width Ds .It Fl s Stop scrubbing. +.El +.Bl -tag -width Ds +.It Fl p +Pause scrubbing. +Scrub pause state and progress are periodically synced to disk. +If the system is restarted or pool is exported during a paused scrub, +even after import, scrub will remain paused until it is resumed. +Once resumed the scrub will pick up from the place where it was last +checkpointed to disk. +To resume a paused scrub issue +.Nm zpool Cm scrub +again. .El .It Xo .Nm
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201709011743.v81Hh99N029459>