From owner-svn-src-vendor@freebsd.org Wed Nov 6 08:44:37 2019 Return-Path: Delivered-To: svn-src-vendor@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AFD9D1AE703; Wed, 6 Nov 2019 08:44:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 477KpY473lz3HZW; Wed, 6 Nov 2019 08:44:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 72725227F3; Wed, 6 Nov 2019 08:44:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xA68ibtS040509; Wed, 6 Nov 2019 08:44:37 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xA68ia9d040503; Wed, 6 Nov 2019 08:44:36 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201911060844.xA68ia9d040503@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Wed, 6 Nov 2019 08:44:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r354377 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/lib... X-SVN-Group: vendor X-SVN-Commit-Author: avg X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/lib/libzfs_core/common vendo... X-SVN-Commit-Revision: 354377 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 08:44:37 -0000 Author: avg Date: Wed Nov 6 08:44:35 2019 New Revision: 354377 URL: https://svnweb.freebsd.org/changeset/base/354377 Log: 10554 Implemented zpool sync command illumos/illumos-gate@9c2acf00e275b6b2125a306f33cdddcc58393220 https://github.com/illumos/illumos-gate/commit/9c2acf00e275b6b2125a306f33cdddcc58393220 https://www.illumos.org/issues/10554 During the port of MMP (illumos bug 10499) from ZoL, I found this earlier ZoL project is a prerequisite. Here is the original description. This addition will enable us to sync an open TXG to the main pool on demand. The functionality is similar to 'sync(2)' but 'zpool sync' will return when data has hit the main storage instead of potentially just the ZIL as is the case with the 'sync(2)' cmd. Portions contributed by: Jerry Jelinek Author: Alek Pinchuk Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h vendor/illumos/dist/man/man1m/zpool.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Nov 6 08:37:55 2019 (r354376) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Nov 6 08:44:35 2019 (r354377) @@ -97,6 +97,8 @@ static int zpool_do_history(int, char **); static int zpool_do_get(int, char **); static int zpool_do_set(int, char **); +static int zpool_do_sync(int, char **); + /* * These libumem hooks provide a reasonable set of defaults for the allocator's * debugging facilities. @@ -141,6 +143,7 @@ typedef enum { HELP_GET, HELP_SET, HELP_SPLIT, + HELP_SYNC, HELP_REGUID, HELP_REOPEN } zpool_help_t; @@ -197,6 +200,7 @@ static zpool_command_t command_table[] = { { "history", zpool_do_history, HELP_HISTORY }, { "get", zpool_do_get, HELP_GET }, { "set", zpool_do_set, HELP_SET }, + { "sync", zpool_do_sync, HELP_SYNC }, }; #define NCOMMAND (sizeof (command_table) / sizeof (command_table[0])) @@ -283,6 +287,8 @@ get_usage(zpool_help_t idx) "[ ...]\n")); case HELP_REGUID: return (gettext("\treguid \n")); + case HELP_SYNC: + return (gettext("\tsync [pool] ...\n")); } abort(); @@ -2614,6 +2620,45 @@ error: free(searchdirs); return (err ? 1 : 0); +} + +/* + * zpool sync [-f] [pool] ... + * + * -f (undocumented) force uberblock (and config including zpool cache file) + * update. + * + * Sync the specified pool(s). + * Without arguments "zpool sync" will sync all pools. + * This command initiates TXG sync(s) and will return after the TXG(s) commit. + * + */ +static int +zpool_do_sync(int argc, char **argv) +{ + int ret; + boolean_t force = B_FALSE; + + /* check options */ + while ((ret = getopt(argc, argv, "f")) != -1) { + switch (ret) { + case 'f': + force = B_TRUE; + break; + case '?': + (void) fprintf(stderr, gettext("invalid option '%c'\n"), + optopt); + usage(B_FALSE); + } + } + + argc -= optind; + argv += optind; + + /* if argc == 0 we will execute zpool_sync_one on all pools */ + ret = for_each_pool(argc, argv, B_FALSE, NULL, zpool_sync_one, &force); + + return (ret); } typedef struct iostat_cbdata { Modified: vendor/illumos/dist/lib/libzfs/common/libzfs.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Nov 6 08:37:55 2019 (r354376) +++ vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Nov 6 08:44:35 2019 (r354377) @@ -269,6 +269,8 @@ extern int zpool_clear(zpool_handle_t *, const char *, extern int zpool_reguid(zpool_handle_t *); extern int zpool_reopen(zpool_handle_t *); +extern int zpool_sync_one(zpool_handle_t *, void *); + extern int zpool_vdev_online(zpool_handle_t *, const char *, int, vdev_state_t *); extern int zpool_vdev_offline(zpool_handle_t *, const char *, boolean_t); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Wed Nov 6 08:37:55 2019 (r354376) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Wed Nov 6 08:44:35 2019 (r354377) @@ -3504,6 +3504,27 @@ zpool_reopen(zpool_handle_t *zhp) return (zpool_standard_error(hdl, errno, msg)); } +/* call into libzfs_core to execute the sync IOCTL per pool */ +int +zpool_sync_one(zpool_handle_t *zhp, void *data) +{ + int ret; + libzfs_handle_t *hdl = zpool_get_handle(zhp); + const char *pool_name = zpool_get_name(zhp); + boolean_t *force = data; + nvlist_t *innvl = fnvlist_alloc(); + + fnvlist_add_boolean_value(innvl, "force", *force); + if ((ret = lzc_sync(pool_name, innvl, NULL)) != 0) { + nvlist_free(innvl); + return (zpool_standard_error_fmt(hdl, ret, + dgettext(TEXT_DOMAIN, "sync '%s' failed"), pool_name)); + } + nvlist_free(innvl); + + return (0); +} + /* * Convert from a devid string to a path. */ Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Wed Nov 6 08:37:55 2019 (r354376) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Wed Nov 6 08:44:35 2019 (r354377) @@ -24,6 +24,7 @@ * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2017 RackTop Systems. + * Copyright (c) 2017 Datto Inc. */ /* @@ -128,17 +129,20 @@ lzc_ioctl(zfs_ioc_t ioc, const char *name, { zfs_cmd_t zc = { 0 }; int error = 0; - char *packed; - size_t size; + char *packed = NULL; + size_t size = 0; ASSERT3S(g_refcount, >, 0); VERIFY3S(g_fd, !=, -1); - (void) strlcpy(zc.zc_name, name, sizeof (zc.zc_name)); + if (name != NULL) + (void) strlcpy(zc.zc_name, name, sizeof (zc.zc_name)); - packed = fnvlist_pack(source, &size); - zc.zc_nvlist_src = (uint64_t)(uintptr_t)packed; - zc.zc_nvlist_src_size = size; + if (source != NULL) { + packed = fnvlist_pack(source, &size); + zc.zc_nvlist_src = (uint64_t)(uintptr_t)packed; + zc.zc_nvlist_src_size = size; + } if (resultp != NULL) { *resultp = NULL; @@ -416,6 +420,18 @@ lzc_exists(const char *dataset) } /* + * outnvl is unused. + * It was added to preserve the function signature in case it is + * needed in the future. + */ +/*ARGSUSED*/ +int +lzc_sync(const char *pool_name, nvlist_t *innvl, nvlist_t **outnvl) +{ + return (lzc_ioctl(ZFS_IOC_POOL_SYNC, pool_name, innvl, NULL)); +} + +/* * Create "user holds" on snapshots. If there is a hold on a snapshot, * the snapshot can not be destroyed. (However, it can be marked for deletion * by lzc_destroy_snaps(defer=B_TRUE).) @@ -515,11 +531,7 @@ lzc_release(nvlist_t *holds, nvlist_t **errlist) int lzc_get_holds(const char *snapname, nvlist_t **holdsp) { - int error; - nvlist_t *innvl = fnvlist_alloc(); - error = lzc_ioctl(ZFS_IOC_GET_HOLDS, snapname, innvl, holdsp); - fnvlist_free(innvl); - return (error); + return (lzc_ioctl(ZFS_IOC_GET_HOLDS, snapname, NULL, holdsp)); } /* Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Wed Nov 6 08:37:55 2019 (r354376) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Wed Nov 6 08:44:35 2019 (r354377) @@ -23,6 +23,7 @@ * Copyright (c) 2012, 2016 by Delphix. All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2017 RackTop Systems. + * Copyright (c) 2017 Datto Inc. */ #ifndef _LIBZFS_CORE_H @@ -90,6 +91,8 @@ boolean_t lzc_exists(const char *); int lzc_rollback(const char *, char *, int); int lzc_rollback_to(const char *, const char *); + +int lzc_sync(const char *, nvlist_t *, nvlist_t **); int lzc_rename(const char *, const char *); int lzc_destroy(const char *); Modified: vendor/illumos/dist/man/man1m/zpool.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zpool.1m Wed Nov 6 08:37:55 2019 (r354376) +++ vendor/illumos/dist/man/man1m/zpool.1m Wed Nov 6 08:44:35 2019 (r354377) @@ -174,6 +174,9 @@ .Oo Ar pool Oc Ns ... .Op Ar interval Op Ar count .Nm +.Cm sync +.Oo Ar pool Oc Ns ... +.Nm .Cm upgrade .Nm .Cm upgrade @@ -1780,6 +1783,19 @@ Only display status for pools that are exhibiting erro unavailable. Warnings about pools not using the latest on-disk format will not be included. .El +.It Xo +.Nm +.Cm sync +.Oo Ar pool Oc Ns ... +.Xc +Forces all in-core dirty data to be written to the primary pool storage and +not the ZIL. +It will also update administrative information including quota reporting. +Without arguments, +.Nm zpool Cm sync +will sync all pools on the system. +Otherwise, it will only sync the specified +.Ar pool . .It Xo .Nm .Cm upgrade