From owner-svn-src-vendor@freebsd.org Sun Jan 21 19:26:39 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBB7BEB62F0; Sun, 21 Jan 2018 19:26:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A348378A60; Sun, 21 Jan 2018 19:26:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EE4967E83; Sun, 21 Jan 2018 19:26:38 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LJQcOi099070; Sun, 21 Jan 2018 19:26:38 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LJQcF0099064; Sun, 21 Jan 2018 19:26:38 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801211926.w0LJQcF0099064@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 19:26:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328220 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zfs vendor/illumos/di... X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zfs vendor/illumos/dist/lib/libzfs/common vend... X-SVN-Commit-Revision: 328220 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 19:26:40 -0000 Author: mav Date: Sun Jan 21 19:26:38 2018 New Revision: 328220 URL: https://svnweb.freebsd.org/changeset/base/328220 Log: 8677 Open-Context Channel Programs illumos/illumos-gate@a3b2868063897ff0083dea538f55f9873eec981f https://www.illumos.org/issues/8677 We want to be able to run channel programs outside of synching context. This would greatly improve performance of channel program that just gather information, as we won't have to wait for synching context anymore. This feature should introduce the following: - A new command line flag in "zfs program" to specify our intention to run in open context. - A new flag/option within the channel program ioctl which selects the context. - Appropriate error handling whenever we try a channel program in open-context that contains zfs.sync* expressions. - Documentation for the new feature in the manual pages. Reviewed by: Matt Ahrens Reviewed by: Chris Williamson Reviewed by: Pavel Zakharov Approved by: Robert Mustacchi Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h vendor/illumos/dist/man/man1m/zfs-program.1m vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c Sun Jan 21 19:26:38 2018 (r328220) @@ -542,6 +542,7 @@ dsl_destroy_snapshots_nvl(nvlist_t *snaps, boolean_t d nvlist_t *result = fnvlist_alloc(); int error = zcp_eval(nvpair_name(nvlist_next_nvpair(snaps, NULL)), program, + B_TRUE, 0, zfs_lua_max_memlimit, nvlist_next_nvpair(wrapper, NULL), result); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h Sun Jan 21 17:31:31 2018 (r328219) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h Sun Jan 21 19:26:38 2018 (r328220) @@ -38,20 +38,25 @@ extern uint64_t zfs_lua_max_memlimit; int zcp_argerror(lua_State *, int, const char *, ...); -int zcp_eval(const char *, const char *, uint64_t, uint64_t, nvpair_t *, - nvlist_t *); +int zcp_eval(const char *, const char *, boolean_t, uint64_t, uint64_t, + nvpair_t *, nvlist_t *); int zcp_load_list_lib(lua_State *); int zcp_load_synctask_lib(lua_State *, boolean_t); typedef void (zcp_cleanup_t)(void *); +typedef struct zcp_cleanup_handler { + zcp_cleanup_t *zch_cleanup_func; + void *zch_cleanup_arg; + list_node_t zch_node; +} zcp_cleanup_handler_t; typedef struct zcp_run_info { dsl_pool_t *zri_pool; /* - * An estimate of the total ammount of space consumed by all + * An estimate of the total amount of space consumed by all * synctasks we have successfully performed so far in this * channel program. Used to generate ENOSPC errors for syncfuncs. */ @@ -89,16 +94,21 @@ typedef struct zcp_run_info { boolean_t zri_timed_out; /* - * The currently registered cleanup function, which will be called - * with the stored argument if a fatal error occurs. + * Boolean indicating whether or not we are running in syncing + * context. */ - zcp_cleanup_t *zri_cleanup; - void *zri_cleanup_arg; + boolean_t zri_sync; + + /* + * List of currently registered cleanup handlers, which will be + * triggered in the event of a fatal error. + */ + list_t zri_cleanup_handlers; } zcp_run_info_t; zcp_run_info_t *zcp_run_info(lua_State *); -void zcp_register_cleanup(lua_State *, zcp_cleanup_t, void *); -void zcp_clear_cleanup(lua_State *); +zcp_cleanup_handler_t *zcp_register_cleanup(lua_State *, zcp_cleanup_t, void *); +void zcp_deregister_cleanup(lua_State *, zcp_cleanup_handler_t *); void zcp_cleanup(lua_State *); /* Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Sun Jan 21 19:26:38 2018 (r328220) @@ -130,13 +130,6 @@ typedef struct zcp_eval_arg { uint64_t ea_instrlimit; } zcp_eval_arg_t; -/*ARGSUSED*/ -static int -zcp_eval_check(void *arg, dmu_tx_t *tx) -{ - return (0); -} - /* * The outer-most error callback handler for use with lua_pcall(). On * error Lua will call this callback with a single argument that @@ -180,41 +173,45 @@ zcp_argerror(lua_State *state, int narg, const char *m * * If an error occurs, the cleanup function will be invoked exactly once and * then unreigstered. + * + * Returns the registered cleanup handler so the caller can deregister it + * if no error occurs. */ -void +zcp_cleanup_handler_t * zcp_register_cleanup(lua_State *state, zcp_cleanup_t cleanfunc, void *cleanarg) { zcp_run_info_t *ri = zcp_run_info(state); - /* - * A cleanup function should always be explicitly removed before - * installing a new one to avoid accidental clobbering. - */ - ASSERT3P(ri->zri_cleanup, ==, NULL); - ri->zri_cleanup = cleanfunc; - ri->zri_cleanup_arg = cleanarg; + zcp_cleanup_handler_t *zch = kmem_alloc(sizeof (*zch), KM_SLEEP); + zch->zch_cleanup_func = cleanfunc; + zch->zch_cleanup_arg = cleanarg; + list_insert_head(&ri->zri_cleanup_handlers, zch); + + return (zch); } void -zcp_clear_cleanup(lua_State *state) +zcp_deregister_cleanup(lua_State *state, zcp_cleanup_handler_t *zch) { zcp_run_info_t *ri = zcp_run_info(state); - - ri->zri_cleanup = NULL; - ri->zri_cleanup_arg = NULL; + list_remove(&ri->zri_cleanup_handlers, zch); + kmem_free(zch, sizeof (*zch)); } /* - * If it exists, execute the currently set cleanup function then unregister it. + * Execute the currently registered cleanup handlers then free them and + * destroy the handler list. */ void zcp_cleanup(lua_State *state) { zcp_run_info_t *ri = zcp_run_info(state); - if (ri->zri_cleanup != NULL) { - ri->zri_cleanup(ri->zri_cleanup_arg); - zcp_clear_cleanup(state); + for (zcp_cleanup_handler_t *zch = + list_remove_head(&ri->zri_cleanup_handlers); zch != NULL; + zch = list_remove_head(&ri->zri_cleanup_handlers)) { + zch->zch_cleanup_func(zch->zch_cleanup_arg); + kmem_free(zch, sizeof (*zch)); } } @@ -815,19 +812,12 @@ zcp_panic_cb(lua_State *state) } static void -zcp_eval_sync(void *arg, dmu_tx_t *tx) +zcp_eval_impl(dmu_tx_t *tx, boolean_t sync, zcp_eval_arg_t *evalargs) { int err; zcp_run_info_t ri; - zcp_eval_arg_t *evalargs = arg; lua_State *state = evalargs->ea_state; - /* - * Open context should have setup the stack to contain: - * 1: Error handler callback - * 2: Script to run (converted to a Lua function) - * 3: nvlist input to function (converted to Lua table or nil) - */ VERIFY3U(3, ==, lua_gettop(state)); /* @@ -840,8 +830,9 @@ zcp_eval_sync(void *arg, dmu_tx_t *tx) ri.zri_cred = evalargs->ea_cred; ri.zri_tx = tx; ri.zri_timed_out = B_FALSE; - ri.zri_cleanup = NULL; - ri.zri_cleanup_arg = NULL; + ri.zri_sync = sync; + list_create(&ri.zri_cleanup_handlers, sizeof (zcp_cleanup_handler_t), + offsetof(zcp_cleanup_handler_t, zch_node)); ri.zri_curinstrs = 0; ri.zri_maxinstrs = evalargs->ea_instrlimit; @@ -878,10 +869,10 @@ zcp_eval_sync(void *arg, dmu_tx_t *tx) /* * Remove the error handler callback from the stack. At this point, - * if there is a cleanup function registered, then it was registered - * but never run or removed, which should never occur. + * there shouldn't be any cleanup handler registered in the handler + * list (zri_cleanup_handlers), regardless of whether it ran or not. */ - ASSERT3P(ri.zri_cleanup, ==, NULL); + list_destroy(&ri.zri_cleanup_handlers); lua_remove(state, 1); switch (err) { @@ -963,9 +954,73 @@ zcp_eval_sync(void *arg, dmu_tx_t *tx) } } +static void +zcp_pool_error(zcp_eval_arg_t *evalargs, const char *poolname) +{ + evalargs->ea_result = SET_ERROR(ECHRNG); + (void) lua_pushfstring(evalargs->ea_state, "Could not open pool: %s", + poolname); + zcp_convert_return_values(evalargs->ea_state, evalargs->ea_outnvl, + ZCP_RET_ERROR, evalargs); + +} + +static void +zcp_eval_sync(void *arg, dmu_tx_t *tx) +{ + zcp_eval_arg_t *evalargs = arg; + + /* + * Open context should have setup the stack to contain: + * 1: Error handler callback + * 2: Script to run (converted to a Lua function) + * 3: nvlist input to function (converted to Lua table or nil) + */ + VERIFY3U(3, ==, lua_gettop(evalargs->ea_state)); + + zcp_eval_impl(tx, B_TRUE, evalargs); +} + +static void +zcp_eval_open(zcp_eval_arg_t *evalargs, const char *poolname) +{ + + int error; + dsl_pool_t *dp; + dmu_tx_t *tx; + + /* + * See comment from the same assertion in zcp_eval_sync(). + */ + VERIFY3U(3, ==, lua_gettop(evalargs->ea_state)); + + error = dsl_pool_hold(poolname, FTAG, &dp); + if (error != 0) { + zcp_pool_error(evalargs, poolname); + return; + } + + /* + * As we are running in open-context, we have no transaction associated + * with the channel program. At the same time, functions from the + * zfs.check submodule need to be associated with a transaction as + * they are basically dry-runs of their counterparts in the zfs.sync + * submodule. These functions should be able to run in open-context. + * Therefore we create a new transaction that we later abort once + * the channel program has been evaluated. + */ + tx = dmu_tx_create_dd(dp->dp_mos_dir); + + zcp_eval_impl(tx, B_FALSE, evalargs); + + dmu_tx_abort(tx); + + dsl_pool_rele(dp, FTAG); +} + int -zcp_eval(const char *poolname, const char *program, uint64_t instrlimit, - uint64_t memlimit, nvpair_t *nvarg, nvlist_t *outnvl) +zcp_eval(const char *poolname, const char *program, boolean_t sync, + uint64_t instrlimit, uint64_t memlimit, nvpair_t *nvarg, nvlist_t *outnvl) { int err; lua_State *state; @@ -1076,9 +1131,14 @@ zcp_eval(const char *poolname, const char *program, ui evalargs.ea_outnvl = outnvl; evalargs.ea_result = 0; - VERIFY0(dsl_sync_task(poolname, zcp_eval_check, - zcp_eval_sync, &evalargs, 0, ZFS_SPACE_CHECK_NONE)); - + if (sync) { + err = dsl_sync_task(poolname, NULL, + zcp_eval_sync, &evalargs, 0, ZFS_SPACE_CHECK_NONE); + if (err != 0) + zcp_pool_error(&evalargs, poolname); + } else { + zcp_eval_open(&evalargs, poolname); + } lua_close(state); return (evalargs.ea_result); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c Sun Jan 21 19:26:38 2018 (r328220) @@ -55,6 +55,10 @@ typedef struct zcp_synctask_info { * * If 'sync' is false, executes a dry run and returns the error code. * + * If we are not running in syncing context and we are not doing a dry run + * (meaning we are running a zfs.sync function in open-context) then we + * return a Lua error. + * * This function also handles common fatal error cases for channel program * library functions. If a fatal error occurs, err_dsname will be the dataset * name reported in error messages, if supplied. @@ -70,6 +74,13 @@ zcp_sync_task(lua_State *state, dsl_checkfunc_t *check if (!sync) return (err); + if (!ri->zri_sync) { + return (luaL_error(state, "running functions from the zfs.sync " + "submodule requires passing sync=TRUE to " + "lzc_channel_program() (i.e. do not specify the \"-n\" " + "command line argument)")); + } + if (err == 0) { syncfunc(arg, ri->zri_tx); } else if (err == EIO) { @@ -234,6 +245,15 @@ zcp_synctask_snapshot(lua_State *state, boolean_t sync zcp_run_info_t *ri = zcp_run_info(state); /* + * On old pools, the ZIL must not be active when a snapshot is created, + * but we can't suspend the ZIL because we're already in syncing + * context. + */ + if (spa_version(ri->zri_pool->dp_spa) < SPA_VERSION_FAST_SNAP) { + return (ENOTSUP); + } + + /* * We only allow for a single snapshot rather than a list, so the * error list output is unnecessary. */ @@ -243,33 +263,23 @@ zcp_synctask_snapshot(lua_State *state, boolean_t sync ddsa.ddsa_snaps = fnvlist_alloc(); fnvlist_add_boolean(ddsa.ddsa_snaps, dsname); - /* - * On old pools, the ZIL must not be active when a snapshot is created, - * but we can't suspend the ZIL because we're already in syncing - * context. - */ - if (spa_version(ri->zri_pool->dp_spa) < SPA_VERSION_FAST_SNAP) { - return (ENOTSUP); - } + zcp_cleanup_handler_t *zch = zcp_register_cleanup(state, + (zcp_cleanup_t *)&fnvlist_free, ddsa.ddsa_snaps); err = zcp_sync_task(state, dsl_dataset_snapshot_check, dsl_dataset_snapshot_sync, &ddsa, sync, dsname); + zcp_deregister_cleanup(state, zch); fnvlist_free(ddsa.ddsa_snaps); return (err); } -void -zcp_synctask_wrapper_cleanup(void *arg) -{ - fnvlist_free(arg); -} - static int zcp_synctask_wrapper(lua_State *state) { int err; + zcp_cleanup_handler_t *zch; int num_ret = 1; nvlist_t *err_details = fnvlist_alloc(); @@ -277,7 +287,8 @@ zcp_synctask_wrapper(lua_State *state) * Make sure err_details is properly freed, even if a fatal error is * thrown during the synctask. */ - zcp_register_cleanup(state, &zcp_synctask_wrapper_cleanup, err_details); + zch = zcp_register_cleanup(state, + (zcp_cleanup_t *)&fnvlist_free, err_details); zcp_synctask_info_t *info = lua_touserdata(state, lua_upvalueindex(1)); boolean_t sync = lua_toboolean(state, lua_upvalueindex(2)); @@ -317,7 +328,7 @@ zcp_synctask_wrapper(lua_State *state) num_ret++; } - zcp_clear_cleanup(state); + zcp_deregister_cleanup(state, zch); fnvlist_free(err_details); return (num_ret); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Sun Jan 21 19:26:38 2018 (r328220) @@ -3606,11 +3606,15 @@ zfs_ioc_channel_program(const char *poolname, nvlist_t { char *program; uint64_t instrlimit, memlimit; + boolean_t sync_flag; nvpair_t *nvarg = NULL; if (0 != nvlist_lookup_string(innvl, ZCP_ARG_PROGRAM, &program)) { return (EINVAL); } + if (0 != nvlist_lookup_boolean_value(innvl, ZCP_ARG_SYNC, &sync_flag)) { + sync_flag = B_TRUE; + } if (0 != nvlist_lookup_uint64(innvl, ZCP_ARG_INSTRLIMIT, &instrlimit)) { instrlimit = ZCP_DEFAULT_INSTRLIMIT; } @@ -3626,7 +3630,7 @@ zfs_ioc_channel_program(const char *poolname, nvlist_t if (memlimit == 0 || memlimit > zfs_lua_max_memlimit) return (EINVAL); - return (zcp_eval(poolname, program, instrlimit, memlimit, + return (zcp_eval(poolname, program, sync_flag, instrlimit, memlimit, nvarg, outnvl)); } Modified: vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Sun Jan 21 17:31:31 2018 (r328219) +++ vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Sun Jan 21 19:26:38 2018 (r328220) @@ -963,6 +963,7 @@ typedef enum { */ #define ZCP_ARG_PROGRAM "program" #define ZCP_ARG_ARGLIST "arg" +#define ZCP_ARG_SYNC "sync" #define ZCP_ARG_INSTRLIMIT "instrlimit" #define ZCP_ARG_MEMLIMIT "memlimit" From owner-svn-src-vendor@freebsd.org Sun Jan 21 19:26:40 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6FCDEB62F5; Sun, 21 Jan 2018 19:26:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9718878A61; Sun, 21 Jan 2018 19:26:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CBD9A7E84; Sun, 21 Jan 2018 19:26:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LJQdt4099082; Sun, 21 Jan 2018 19:26:39 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LJQdad099075; Sun, 21 Jan 2018 19:26:39 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801211926.w0LJQdad099075@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 19:26:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328220 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zfs vendor/illumos/di... X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zfs vendor/illumos/dist/lib/libzfs/common vend... X-SVN-Commit-Revision: 328220 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 19:26:40 -0000 Author: mav Date: Sun Jan 21 19:26:38 2018 New Revision: 328220 URL: https://svnweb.freebsd.org/changeset/base/328220 Log: 8677 Open-Context Channel Programs illumos/illumos-gate@a3b2868063897ff0083dea538f55f9873eec981f https://www.illumos.org/issues/8677 We want to be able to run channel programs outside of synching context. This would greatly improve performance of channel program that just gather information, as we won't have to wait for synching context anymore. This feature should introduce the following: - A new command line flag in "zfs program" to specify our intention to run in open context. - A new flag/option within the channel program ioctl which selects the context. - Appropriate error handling whenever we try a channel program in open-context that contains zfs.sync* expressions. - Documentation for the new feature in the manual pages. Reviewed by: Matt Ahrens Reviewed by: Chris Williamson Reviewed by: Pavel Zakharov Approved by: Robert Mustacchi Author: Serapheim Dimitropoulos Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h vendor/illumos/dist/man/man1m/zfs-program.1m vendor/illumos/dist/man/man1m/zfs.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zcp.h vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c vendor-sys/illumos/dist/uts/common/fs/zfs/zcp_synctask.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_main.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor/illumos/dist/cmd/zfs/zfs_main.c Sun Jan 21 19:26:38 2018 (r328220) @@ -329,7 +329,7 @@ get_usage(zfs_help_t idx) case HELP_BOOKMARK: return (gettext("\tbookmark \n")); case HELP_CHANNEL_PROGRAM: - return (gettext("\tprogram [-t ] " + return (gettext("\tprogram [-n] [-t ] " "[-m ] " "[lua args...]\n")); } @@ -6982,11 +6982,12 @@ zfs_do_channel_program(int argc, char **argv) nvlist_t *outnvl; uint64_t instrlimit = ZCP_DEFAULT_INSTRLIMIT; uint64_t memlimit = ZCP_DEFAULT_MEMLIMIT; + boolean_t sync_flag = B_TRUE; zpool_handle_t *zhp; /* check options */ while (-1 != - (c = getopt(argc, argv, "t:(instr-limit)m:(memory-limit)"))) { + (c = getopt(argc, argv, "nt:(instr-limit)m:(memory-limit)"))) { switch (c) { case 't': case 'm': { @@ -7024,6 +7025,10 @@ zfs_do_channel_program(int argc, char **argv) } break; } + case 'n': { + sync_flag = B_FALSE; + break; + } case '?': (void) fprintf(stderr, gettext("invalid option '%c'\n"), optopt); @@ -7095,8 +7100,13 @@ zfs_do_channel_program(int argc, char **argv) nvlist_t *argnvl = fnvlist_alloc(); fnvlist_add_string_array(argnvl, ZCP_ARG_CLIARGV, argv + 2, argc - 2); - ret = lzc_channel_program(poolname, progbuf, instrlimit, memlimit, - argnvl, &outnvl); + if (sync_flag) { + ret = lzc_channel_program(poolname, progbuf, + instrlimit, memlimit, argnvl, &outnvl); + } else { + ret = lzc_channel_program_nosync(poolname, progbuf, + instrlimit, memlimit, argnvl, &outnvl); + } if (ret != 0) { /* Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Sun Jan 21 19:26:38 2018 (r328220) @@ -2348,7 +2348,7 @@ zcp_check(zfs_handle_t *zhp, zfs_prop_t prop, uint64_t fnvlist_add_string(argnvl, "dataset", zhp->zfs_name); fnvlist_add_string(argnvl, "property", zfs_prop_to_name(prop)); - error = lzc_channel_program(poolname, program, + error = lzc_channel_program_nosync(poolname, program, 10 * 1000 * 1000, 10 * 1024 * 1024, argnvl, &outnvl); if (error == 0) { Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Sun Jan 21 17:31:31 2018 (r328219) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Sun Jan 21 19:26:38 2018 (r328220) @@ -881,6 +881,25 @@ lzc_destroy_bookmarks(nvlist_t *bmarks, nvlist_t **err return (error); } +static int +lzc_channel_program_impl(const char *pool, const char *program, boolean_t sync, + uint64_t instrlimit, uint64_t memlimit, nvlist_t *argnvl, nvlist_t **outnvl) +{ + int error; + nvlist_t *args; + + args = fnvlist_alloc(); + fnvlist_add_string(args, ZCP_ARG_PROGRAM, program); + fnvlist_add_nvlist(args, ZCP_ARG_ARGLIST, argnvl); + fnvlist_add_boolean_value(args, ZCP_ARG_SYNC, sync); + fnvlist_add_uint64(args, ZCP_ARG_INSTRLIMIT, instrlimit); + fnvlist_add_uint64(args, ZCP_ARG_MEMLIMIT, memlimit); + error = lzc_ioctl(ZFS_IOC_CHANNEL_PROGRAM, pool, args, outnvl); + fnvlist_free(args); + + return (error); +} + /* * Executes a channel program. * @@ -918,16 +937,26 @@ int lzc_channel_program(const char *pool, const char *program, uint64_t instrlimit, uint64_t memlimit, nvlist_t *argnvl, nvlist_t **outnvl) { - int error; - nvlist_t *args; + return (lzc_channel_program_impl(pool, program, B_TRUE, instrlimit, + memlimit, argnvl, outnvl)); +} - args = fnvlist_alloc(); - fnvlist_add_string(args, ZCP_ARG_PROGRAM, program); - fnvlist_add_nvlist(args, ZCP_ARG_ARGLIST, argnvl); - fnvlist_add_uint64(args, ZCP_ARG_INSTRLIMIT, instrlimit); - fnvlist_add_uint64(args, ZCP_ARG_MEMLIMIT, memlimit); - error = lzc_ioctl(ZFS_IOC_CHANNEL_PROGRAM, pool, args, outnvl); - fnvlist_free(args); - - return (error); +/* + * Executes a read-only channel program. + * + * A read-only channel program works programmatically the same way as a + * normal channel program executed with lzc_channel_program(). The only + * difference is it runs exclusively in open-context and therefore can + * return faster. The downside to that, is that the program cannot change + * on-disk state by calling functions from the zfs.sync submodule. + * + * The return values of this function (and their meaning) are exactly the + * same as the ones described in lzc_channel_program(). + */ +int +lzc_channel_program_nosync(const char *pool, const char *program, + uint64_t timeout, uint64_t memlimit, nvlist_t *argnvl, nvlist_t **outnvl) +{ + return (lzc_channel_program_impl(pool, program, B_FALSE, timeout, + memlimit, argnvl, outnvl)); } Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Sun Jan 21 17:31:31 2018 (r328219) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Sun Jan 21 19:26:38 2018 (r328220) @@ -86,8 +86,10 @@ boolean_t lzc_exists(const char *); int lzc_rollback(const char *, char *, int); int lzc_rollback_to(const char *, const char *); -int lzc_channel_program(const char *, const char *, uint64_t, uint64_t, - nvlist_t *, nvlist_t **); +int lzc_channel_program(const char *, const char *, uint64_t, + uint64_t, nvlist_t *, nvlist_t **); +int lzc_channel_program_nosync(const char *, const char *, uint64_t, + uint64_t, nvlist_t *, nvlist_t **); #ifdef __cplusplus } Modified: vendor/illumos/dist/man/man1m/zfs-program.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs-program.1m Sun Jan 21 17:31:31 2018 (r328219) +++ vendor/illumos/dist/man/man1m/zfs-program.1m Sun Jan 21 19:26:38 2018 (r328220) @@ -14,10 +14,11 @@ .Dt ZFS-PROGRAM 1M .Os .Sh NAME -.Nm zfs program +.Nm "zfs program" .Nd executes ZFS channel programs .Sh SYNOPSIS -.Cm zfs program +.Cm "zfs program" +.Op Fl n .Op Fl t Ar instruction-limit .Op Fl m Ar memory-limit .Ar pool @@ -45,6 +46,14 @@ will be run on and any attempts to access or modify other pools will cause an error. .Sh OPTIONS .Bl -tag -width "-t" +.It Fl n +Executes a read-only channel program, which runs faster. +The program cannot change on-disk state by calling functions from the +zfs.sync submodule. +The program can be used to gather information such as properties and +determining if changes would succeed (zfs.check.*). +Without this flag, all pending changes must be synced to disk before a +channel program can complete. .It Fl t Ar instruction-limit Execution time limit, in number of Lua instructions to execute. If a channel program executes more than the specified number of instructions, Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Sun Jan 21 17:31:31 2018 (r328219) +++ vendor/illumos/dist/man/man1m/zfs.1m Sun Jan 21 19:26:38 2018 (r328220) @@ -267,6 +267,7 @@ .Ar snapshot Ar snapshot Ns | Ns Ar filesystem .Nm .Cm program +.Op Fl n .Op Fl t Ar timeout .Op Fl m Ar memory_limit .Ar pool script @@ -3408,6 +3409,7 @@ Display the path's inode change time as the first colu .It Xo .Nm .Cm program +.Op Fl n .Op Fl t Ar timeout .Op Fl m Ar memory_limit .Ar pool script @@ -3427,8 +3429,15 @@ Channel programs may only be run with root privileges. .sp For full documentation of the ZFS channel program interface, see the manual page for -.Xr zfs-program 1M . .Bl -tag -width "" +.It Fl n +Executes a read-only channel program, which runs faster. +The program cannot change on-disk state by calling functions from +the zfs.sync submodule. +The program can be used to gather information such as properties and +determining if changes would succeed (zfs.check.*). +Without this flag, all pending changes must be synced to disk before +a channel program can complete. .It Fl t Ar timeout Execution time limit, in milliseconds. If a channel program executes for longer than the provided timeout, it will From owner-svn-src-vendor@freebsd.org Sun Jan 21 23:04:26 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC4A0EC0598; Sun, 21 Jan 2018 23:04:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A6E9580004; Sun, 21 Jan 2018 23:04:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CE52B121B1; Sun, 21 Jan 2018 23:04:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LN4PW0090811; Sun, 21 Jan 2018 23:04:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LN4PKO090808; Sun, 21 Jan 2018 23:04:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801212304.w0LN4PKO090808@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 23:04:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328225 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 328225 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 23:04:26 -0000 Author: mav Date: Sun Jan 21 23:04:25 2018 New Revision: 328225 URL: https://svnweb.freebsd.org/changeset/base/328225 Log: 8603 rename zilog's "zl_writer_lock" to "zl_issuer_lock" illumos/illumos-gate@cf07d3da9915c0d22da8f59e991639f819463cef https://www.illumos.org/issues/8603: To help make the ZIL's code more understandable, it was suggested that the zilog_t's "zl_writer_lock" field should be renamed to "zl_issuer_lock". Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h Sun Jan 21 23:02:05 2018 (r328224) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h Sun Jan 21 23:04:25 2018 (r328225) @@ -57,9 +57,9 @@ typedef enum { * Log write block (lwb) * * Prior to an lwb being issued to disk via zil_lwb_write_issue(), it - * will be protected by the zilog's "zl_writer_lock". Basically, prior + * will be protected by the zilog's "zl_issuer_lock". Basically, prior * to it being issued, it will only be accessed by the thread that's - * holding the "zl_writer_lock". After the lwb is issued, the zilog's + * holding the "zl_issuer_lock". After the lwb is issued, the zilog's * "zl_lock" is used to protect the lwb against concurrent access. */ typedef struct lwb { @@ -91,10 +91,10 @@ typedef struct lwb { * * The "zcw_lock" field is used to protect the commit waiter against * concurrent access. This lock is often acquired while already holding - * the zilog's "zl_writer_lock" or "zl_lock"; see the functions + * the zilog's "zl_issuer_lock" or "zl_lock"; see the functions * zil_process_commit_list() and zil_lwb_flush_vdevs_done() as examples * of this. Thus, one must be careful not to acquire the - * "zl_writer_lock" or "zl_lock" when already holding the "zcw_lock"; + * "zl_issuer_lock" or "zl_lock" when already holding the "zcw_lock"; * e.g. see the zil_commit_waiter_timeout() function. */ typedef struct zil_commit_waiter { @@ -161,7 +161,7 @@ struct zilog { uint8_t zl_keep_first; /* keep first log block in destroy */ uint8_t zl_replay; /* replaying records while set */ uint8_t zl_stop_sync; /* for debugging */ - kmutex_t zl_writer_lock; /* single writer, per ZIL, at a time */ + kmutex_t zl_issuer_lock; /* single writer, per ZIL, at a time */ uint8_t zl_logbias; /* latency or throughput */ uint8_t zl_sync; /* synchronous or asynchronous */ int zl_parse_error; /* last zil_parse() error */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Sun Jan 21 23:02:05 2018 (r328224) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Sun Jan 21 23:04:25 2018 (r328225) @@ -1095,7 +1095,7 @@ zil_lwb_write_open(zilog_t *zilog, lwb_t *lwb) zbookmark_phys_t zb; zio_priority_t prio; - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); ASSERT3P(lwb, !=, NULL); EQUIV(lwb->lwb_root_zio == NULL, lwb->lwb_state == LWB_STATE_CLOSED); EQUIV(lwb->lwb_root_zio != NULL, lwb->lwb_state == LWB_STATE_OPENED); @@ -1192,7 +1192,7 @@ zil_lwb_write_issue(zilog_t *zilog, lwb_t *lwb) int i, error; boolean_t slog; - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); ASSERT3P(lwb->lwb_root_zio, !=, NULL); ASSERT3P(lwb->lwb_write_zio, !=, NULL); ASSERT3S(lwb->lwb_state, ==, LWB_STATE_OPENED); @@ -1326,7 +1326,7 @@ zil_lwb_commit(zilog_t *zilog, itx_t *itx, lwb_t *lwb) char *lr_buf; uint64_t dlen, dnow, lwb_sp, reclen, txg; - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); ASSERT3P(lwb, !=, NULL); ASSERT3P(lwb->lwb_buf, !=, NULL); @@ -1742,7 +1742,7 @@ zil_get_commit_list(zilog_t *zilog) uint64_t otxg, txg; list_t *commit_list = &zilog->zl_itx_commit_list; - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); if (spa_freeze_txg(zilog->zl_spa) != UINT64_MAX) /* ziltest support */ otxg = ZILTEST_TXG; @@ -1849,7 +1849,7 @@ zil_prune_commit_list(zilog_t *zilog) { itx_t *itx; - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); while (itx = list_head(&zilog->zl_itx_commit_list)) { lr_t *lrc = &itx->itx_lr; @@ -1899,12 +1899,12 @@ zil_commit_writer_stall(zilog_t *zilog) * crash (because the previous lwb on-disk would not point to * it). * - * We must hold the zilog's zl_writer_lock while we do this, to + * We must hold the zilog's zl_issuer_lock while we do this, to * ensure no new threads enter zil_process_commit_list() until * all lwb's in the zl_lwb_list have been synced and freed * (which is achieved via the txg_wait_synced() call). */ - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); txg_wait_synced(zilog->zl_dmu_pool, 0); ASSERT3P(list_tail(&zilog->zl_lwb_list), ==, NULL); } @@ -1923,7 +1923,7 @@ zil_process_commit_list(zilog_t *zilog) lwb_t *lwb; itx_t *itx; - ASSERT(MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(MUTEX_HELD(&zilog->zl_issuer_lock)); /* * Return if there's nothing to commit before we dirty the fs by @@ -2093,17 +2093,17 @@ zil_commit_writer(zilog_t *zilog, zil_commit_waiter_t ASSERT(spa_writeable(zilog->zl_spa)); ASSERT0(zilog->zl_suspend); - mutex_enter(&zilog->zl_writer_lock); + mutex_enter(&zilog->zl_issuer_lock); if (zcw->zcw_lwb != NULL || zcw->zcw_done) { /* * It's possible that, while we were waiting to acquire - * the "zl_writer_lock", another thread committed this + * the "zl_issuer_lock", another thread committed this * waiter to an lwb. If that occurs, we bail out early, * without processing any of the zilog's queue of itxs. * * On certain workloads and system configurations, the - * "zl_writer_lock" can become highly contended. In an + * "zl_issuer_lock" can become highly contended. In an * attempt to reduce this contention, we immediately drop * the lock if the waiter has already been processed. * @@ -2120,13 +2120,13 @@ zil_commit_writer(zilog_t *zilog, zil_commit_waiter_t zil_process_commit_list(zilog); out: - mutex_exit(&zilog->zl_writer_lock); + mutex_exit(&zilog->zl_issuer_lock); } static void zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_waiter_t *zcw) { - ASSERT(!MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(!MUTEX_HELD(&zilog->zl_issuer_lock)); ASSERT(MUTEX_HELD(&zcw->zcw_lock)); ASSERT3B(zcw->zcw_done, ==, B_FALSE); @@ -2138,7 +2138,7 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w * If the lwb has already been issued by another thread, we can * immediately return since there's no work to be done (the * point of this function is to issue the lwb). Additionally, we - * do this prior to acquiring the zl_writer_lock, to avoid + * do this prior to acquiring the zl_issuer_lock, to avoid * acquiring it when it's not necessary to do so. */ if (lwb->lwb_state == LWB_STATE_ISSUED || @@ -2147,13 +2147,13 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w /* * In order to call zil_lwb_write_issue() we must hold the - * zilog's "zl_writer_lock". We can't simply acquire that lock, + * zilog's "zl_issuer_lock". We can't simply acquire that lock, * since we're already holding the commit waiter's "zcw_lock", * and those two locks are aquired in the opposite order * elsewhere. */ mutex_exit(&zcw->zcw_lock); - mutex_enter(&zilog->zl_writer_lock); + mutex_enter(&zilog->zl_issuer_lock); mutex_enter(&zcw->zcw_lock); /* @@ -2171,7 +2171,7 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w /* * We've already checked this above, but since we hadn't - * acquired the zilog's zl_writer_lock, we have to perform this + * acquired the zilog's zl_issuer_lock, we have to perform this * check a second time while holding the lock. We can't call * zil_lwb_write_issue() if the lwb had already been issued. */ @@ -2234,7 +2234,7 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w } out: - mutex_exit(&zilog->zl_writer_lock); + mutex_exit(&zilog->zl_issuer_lock); ASSERT(MUTEX_HELD(&zcw->zcw_lock)); } @@ -2261,7 +2261,7 @@ static void zil_commit_waiter(zilog_t *zilog, zil_commit_waiter_t *zcw) { ASSERT(!MUTEX_HELD(&zilog->zl_lock)); - ASSERT(!MUTEX_HELD(&zilog->zl_writer_lock)); + ASSERT(!MUTEX_HELD(&zilog->zl_issuer_lock)); ASSERT(spa_writeable(zilog->zl_spa)); ASSERT0(zilog->zl_suspend); @@ -2510,7 +2510,7 @@ zil_commit_itx_assign(zilog_t *zilog, zil_commit_waite * on two fundamental concepts: * * 1. The creation and issuance of lwb zio's is protected by - * the zilog's "zl_writer_lock", which ensures only a single + * the zilog's "zl_issuer_lock", which ensures only a single * thread is creating and/or issuing lwb's at a time * 2. The "previous" lwb is a child of the "current" lwb * (leveraging the zio parent-child depenency graph) @@ -2759,7 +2759,7 @@ zil_alloc(objset_t *os, zil_header_t *zh_phys) zilog->zl_last_lwb_latency = 0; mutex_init(&zilog->zl_lock, NULL, MUTEX_DEFAULT, NULL); - mutex_init(&zilog->zl_writer_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&zilog->zl_issuer_lock, NULL, MUTEX_DEFAULT, NULL); for (int i = 0; i < TXG_SIZE; i++) { mutex_init(&zilog->zl_itxg[i].itxg_lock, NULL, @@ -2804,7 +2804,7 @@ zil_free(zilog_t *zilog) mutex_destroy(&zilog->zl_itxg[i].itxg_lock); } - mutex_destroy(&zilog->zl_writer_lock); + mutex_destroy(&zilog->zl_issuer_lock); mutex_destroy(&zilog->zl_lock); cv_destroy(&zilog->zl_cv_suspend); From owner-svn-src-vendor@freebsd.org Sun Jan 21 23:12:40 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F124EC0B1F; Sun, 21 Jan 2018 23:12:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E8DC80775; Sun, 21 Jan 2018 23:12:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8C26E1233F; Sun, 21 Jan 2018 23:12:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LNCdag095070; Sun, 21 Jan 2018 23:12:39 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LNCd33095065; Sun, 21 Jan 2018 23:12:39 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801212312.w0LNCd33095065@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 23:12:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328227 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 328227 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 23:12:40 -0000 Author: mav Date: Sun Jan 21 23:12:38 2018 New Revision: 328227 URL: https://svnweb.freebsd.org/changeset/base/328227 Log: 8909 8585 can cause a use-after-free kernel panic illumos/illumos-gate@94ddd0900a8838f62bba15e270649a42f4ef9f81 https://www.illumos.org/issues/8909: There's a race condition that exists if `zil_free_lwb` races with either `zil_commit_waiter_timeout` and/or `zil_lwb_flush_vdevs_done`. Here's an example panic due to this bug: > ::status debugging crash dump vmcore.0 (64-bit) from ip-10-110-205-40 operating system: 5.11 dlpx-5.2.2.0_2017-12-04-17-28-32b6ba51fb (i86pc) image uuid: 4af0edfb-e58e-6ed8-cafc-d3e9167c7513 panic message: BAD TRAP: type=e (#pf Page fault) rp=ffffff0010555970 addr=60 occurred in module "zfs" due to a NULL pointer dereference dump content: kernel pages only > $c zio_shrink+0x12() zil_lwb_write_issue+0x30d(ffffff03dcd15cc0, ffffff03e0730e20) zil_commit_waiter_timeout+0xa2(ffffff03dcd15cc0, ffffff03d97ffcf8) zil_commit_waiter+0xf3(ffffff03dcd15cc0, ffffff03d97ffcf8) zil_commit+0x80(ffffff03dcd15cc0, 9a9) zfs_write+0xc34(ffffff03dc38b140, ffffff0010555e60, 40, ffffff03e00fb758, 0) fop_write+0x5b(ffffff03dc38b140, ffffff0010555e60, 40, ffffff03e00fb758, 0) write+0x250(42, fffffd7ff4832000, 2000) sys_syscall+0x177() If there's an outstanding lwb that's in `zil_commit_waiter_timeout` waiting to timeout, waiting on it's waiter's CV, we must be sure not to call `zil_free_lwb`. If we end up calling `zil_free_lwb`, then that LWB may be freed and can result in a use-after-free situation where the stale lwb pointer stored in the `zil_commit_waiter_t` structure of the thread waiting on the waiter's CV is used. A similar situation can occur if an lwb is issued to disk, and thus in the `LWB_STATE_ISSUED` state, and `zil_free_lwb` is called while the disk is servicing that lwb. In this situation, the lwb will be freed by `zil_free_lwb`, which will result in a use-after-free situation when the lwb's zio completes, and `zil_lwb_flush_vdevs_done` is called. This race condition is prevented in `zil_close` by calling `zil_commit` before `zil_free_lwb` is called, which will ensure all outstanding (i.e. all lwb's in the `LWB_STATE_OPEN` and/or `LWB_STATE_ISSUED` states) reach the `LWB_STATE_DONE` state before the lwb's are freed (`zil_commit` will not return untill all the lwb's are `LWB_STATE_DONE`). Further, this race condition is prevented in `zil_sync` by only calling `zil_free_lwb` for lwb's that do not have their `lwb_buf` pointer set. All lwb's not in the `LWB_STATE_DONE` state will have a non-null value for this pointer; the pointer is only cleared in `zil_lwb_flush_vdevs_done`, at which point the lwb's state will be changed to `LWB_STATE_DONE`. This race is present in `zil_suspend`, leading to this bug. At first glance, it would appear as though this would not be true because `zil_suspend` will call `zil_commit`, just like `zil_close`, but the problem is that `zil_suspend` will set the zilog's `zl_suspend` field prior to calling `zil_commit`. Further, in `zil_commit`, if `zl_suspend` is set, `zil_commit` will take a special branch of logic and use `txg_wait_synced` instead of performing the normal `zil_commit` logic. This call to `txg_wait_synced` might be good enough for the data to reach disk safely before it returns, but it does not ensure that all outstanding lwb's reach the `LWB_STATE_DONE` state before it returns. This is because, if there's an lwb "stuck" in `zil_commit_waiter_timeout`, waiting for it's lwb to timeout, it will maintain a non-null value for it's `lwb_buf` field and thus `zil_sync` will not free that lwb. Thus, even though the lwb's data is already on disk, the lwb will be left lingering, waiting on the CV, and will eventually timeout and be issued to disk even though the write is unnesseary. So, after `zil_commit` is called from `zil_suspend`, we incorrectly assume that there are not outstanding lwb's, and proceed to free all lwb's found on the zilog's lwb list. As a result, we free the lwb that will later be used `zil_commit_waiter_timeout`. Reviewed by: John Kennedy Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: Brad Lewis Reviewed by: Igor Kozhukhov Approved by: Robert Mustacchi Author: Prakash Surya Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil.h Sun Jan 21 23:11:20 2018 (r328226) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil.h Sun Jan 21 23:12:38 2018 (r328227) @@ -417,6 +417,7 @@ extern void zil_itx_destroy(itx_t *itx); extern void zil_itx_assign(zilog_t *zilog, itx_t *itx, dmu_tx_t *tx); extern void zil_commit(zilog_t *zilog, uint64_t oid); +extern void zil_commit_impl(zilog_t *zilog, uint64_t oid); extern int zil_vdev_offline(const char *osname, void *txarg); extern int zil_claim(struct dsl_pool *dp, Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h Sun Jan 21 23:11:20 2018 (r328226) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zil_impl.h Sun Jan 21 23:12:38 2018 (r328227) @@ -37,13 +37,38 @@ extern "C" { #endif /* - * Possbile states for a given lwb structure. An lwb will start out in - * the "closed" state, and then transition to the "opened" state via a - * call to zil_lwb_write_open(). After the lwb is "open", it can - * transition into the "issued" state via zil_lwb_write_issue(). After - * the lwb's zio completes, and the vdev's are flushed, the lwb will - * transition into the "done" state via zil_lwb_write_done(), and the - * structure eventually freed. + * Possbile states for a given lwb structure. + * + * An lwb will start out in the "closed" state, and then transition to + * the "opened" state via a call to zil_lwb_write_open(). When + * transitioning from "closed" to "opened" the zilog's "zl_issuer_lock" + * must be held. + * + * After the lwb is "opened", it can transition into the "issued" state + * via zil_lwb_write_issue(). Again, the zilog's "zl_issuer_lock" must + * be held when making this transition. + * + * After the lwb's zio completes, and the vdev's are flushed, the lwb + * will transition into the "done" state via zil_lwb_write_done(). When + * transitioning from "issued" to "done", the zilog's "zl_lock" must be + * held, *not* the "zl_issuer_lock". + * + * The zilog's "zl_issuer_lock" can become heavily contended in certain + * workloads, so we specifically avoid acquiring that lock when + * transitioning an lwb from "issued" to "done". This allows us to avoid + * having to acquire the "zl_issuer_lock" for each lwb ZIO completion, + * which would have added more lock contention on an already heavily + * contended lock. + * + * Additionally, correctness when reading an lwb's state is often + * acheived by exploiting the fact that these state transitions occur in + * this specific order; i.e. "closed" to "opened" to "issued" to "done". + * + * Thus, if an lwb is in the "closed" or "opened" state, holding the + * "zl_issuer_lock" will prevent a concurrent thread from transitioning + * that lwb to the "issued" state. Likewise, if an lwb is already in the + * "issued" state, holding the "zl_lock" will prevent a concurrent + * thread from transitioning that lwb to the "done" state. */ typedef enum { LWB_STATE_CLOSED, Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Sun Jan 21 23:11:20 2018 (r328226) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Sun Jan 21 23:12:38 2018 (r328227) @@ -556,7 +556,6 @@ extern enum zio_checksum zio_checksum_dedup_select(spa extern enum zio_compress zio_compress_select(spa_t *spa, enum zio_compress child, enum zio_compress parent); -extern void zio_cancel(zio_t *zio); extern void zio_suspend(spa_t *spa, zio_t *zio); extern int zio_resume(spa_t *spa); extern void zio_resume_wait(spa_t *spa); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Sun Jan 21 23:11:20 2018 (r328226) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Sun Jan 21 23:12:38 2018 (r328227) @@ -502,7 +502,7 @@ zil_alloc_lwb(zilog_t *zilog, blkptr_t *bp, boolean_t ASSERT(!MUTEX_HELD(&lwb->lwb_vdev_lock)); ASSERT(avl_is_empty(&lwb->lwb_vdev_tree)); - ASSERT(list_is_empty(&lwb->lwb_waiters)); + VERIFY(list_is_empty(&lwb->lwb_waiters)); return (lwb); } @@ -512,31 +512,13 @@ zil_free_lwb(zilog_t *zilog, lwb_t *lwb) { ASSERT(MUTEX_HELD(&zilog->zl_lock)); ASSERT(!MUTEX_HELD(&lwb->lwb_vdev_lock)); - ASSERT(list_is_empty(&lwb->lwb_waiters)); - - if (lwb->lwb_state == LWB_STATE_OPENED) { - avl_tree_t *t = &lwb->lwb_vdev_tree; - void *cookie = NULL; - zil_vdev_node_t *zv; - - while ((zv = avl_destroy_nodes(t, &cookie)) != NULL) - kmem_free(zv, sizeof (*zv)); - - ASSERT3P(lwb->lwb_root_zio, !=, NULL); - ASSERT3P(lwb->lwb_write_zio, !=, NULL); - - zio_cancel(lwb->lwb_root_zio); - zio_cancel(lwb->lwb_write_zio); - - lwb->lwb_root_zio = NULL; - lwb->lwb_write_zio = NULL; - } else { - ASSERT3S(lwb->lwb_state, !=, LWB_STATE_ISSUED); - } - + VERIFY(list_is_empty(&lwb->lwb_waiters)); ASSERT(avl_is_empty(&lwb->lwb_vdev_tree)); ASSERT3P(lwb->lwb_write_zio, ==, NULL); ASSERT3P(lwb->lwb_root_zio, ==, NULL); + ASSERT3U(lwb->lwb_max_txg, <=, spa_syncing_txg(zilog->zl_spa)); + ASSERT(lwb->lwb_state == LWB_STATE_CLOSED || + lwb->lwb_state == LWB_STATE_DONE); /* * Clear the zilog's field to indicate this lwb is no longer @@ -883,6 +865,12 @@ zil_commit_waiter_skip(zil_commit_waiter_t *zcw) static void zil_commit_waiter_link_lwb(zil_commit_waiter_t *zcw, lwb_t *lwb) { + /* + * The lwb_waiters field of the lwb is protected by the zilog's + * zl_lock, thus it must be held when calling this function. + */ + ASSERT(MUTEX_HELD(&lwb->lwb_zilog->zl_lock)); + mutex_enter(&zcw->zcw_lock); ASSERT(!list_link_active(&zcw->zcw_node)); ASSERT3P(zcw->zcw_lwb, ==, NULL); @@ -1348,8 +1336,10 @@ zil_lwb_commit(zilog_t *zilog, itx_t *itx, lwb_t *lwb) * For more details, see the comment above zil_commit(). */ if (lrc->lrc_txtype == TX_COMMIT) { + mutex_enter(&zilog->zl_lock); zil_commit_waiter_link_lwb(itx->itx_private, lwb); itx->itx_private = NULL; + mutex_exit(&zilog->zl_lock); return (lwb); } @@ -1957,16 +1947,54 @@ zil_process_commit_list(zilog_t *zilog) zilog_t *, zilog, itx_t *, itx); } - /* - * This is inherently racy and may result in us writing - * out a log block for a txg that was just synced. This - * is ok since we'll end cleaning up that log block the - * next time we call zil_sync(). - */ boolean_t synced = txg <= spa_last_synced_txg(spa); boolean_t frozen = txg > spa_freeze_txg(spa); - if (!synced || frozen) { + /* + * If the txg of this itx has already been synced out, then + * we don't need to commit this itx to an lwb. This is + * because the data of this itx will have already been + * written to the main pool. This is inherently racy, and + * it's still ok to commit an itx whose txg has already + * been synced; this will result in a write that's + * unnecessary, but will do no harm. + * + * With that said, we always want to commit TX_COMMIT itxs + * to an lwb, regardless of whether or not that itx's txg + * has been synced out. We do this to ensure any OPENED lwb + * will always have at least one zil_commit_waiter_t linked + * to the lwb. + * + * As a counter-example, if we skipped TX_COMMIT itx's + * whose txg had already been synced, the following + * situation could occur if we happened to be racing with + * spa_sync: + * + * 1. we commit a non-TX_COMMIT itx to an lwb, where the + * itx's txg is 10 and the last synced txg is 9. + * 2. spa_sync finishes syncing out txg 10. + * 3. we move to the next itx in the list, it's a TX_COMMIT + * whose txg is 10, so we skip it rather than committing + * it to the lwb used in (1). + * + * If the itx that is skipped in (3) is the last TX_COMMIT + * itx in the commit list, than it's possible for the lwb + * used in (1) to remain in the OPENED state indefinitely. + * + * To prevent the above scenario from occuring, ensuring + * that once an lwb is OPENED it will transition to ISSUED + * and eventually DONE, we always commit TX_COMMIT itx's to + * an lwb here, even if that itx's txg has already been + * synced. + * + * Finally, if the pool is frozen, we _always_ commit the + * itx. The point of freezing the pool is to prevent data + * from being written to the main pool via spa_sync, and + * instead rely solely on the ZIL to persistently store the + * data; i.e. when the pool is frozen, the last synced txg + * value can't be trusted. + */ + if (frozen || !synced || lrc->lrc_txtype == TX_COMMIT) { if (lwb != NULL) { lwb = zil_lwb_commit(zilog, itx, lwb); } else if (lrc->lrc_txtype == TX_COMMIT) { @@ -1974,22 +2002,6 @@ zil_process_commit_list(zilog_t *zilog) zil_commit_waiter_link_nolwb( itx->itx_private, &nolwb_waiters); } - } else if (lrc->lrc_txtype == TX_COMMIT) { - ASSERT3B(synced, ==, B_TRUE); - ASSERT3B(frozen, ==, B_FALSE); - - /* - * If this is a commit itx, then there will be a - * thread that is either: already waiting for - * it, or soon will be waiting. - * - * This itx has already been committed to disk - * via spa_sync() so we don't bother committing - * it to an lwb. As a result, we cannot use the - * lwb zio callback to signal the waiter and - * mark it as done, so we must do that here. - */ - zil_commit_waiter_skip(itx->itx_private); } list_remove(&zilog->zl_itx_commit_list, itx); @@ -2091,7 +2103,6 @@ zil_commit_writer(zilog_t *zilog, zil_commit_waiter_t { ASSERT(!MUTEX_HELD(&zilog->zl_lock)); ASSERT(spa_writeable(zilog->zl_spa)); - ASSERT0(zilog->zl_suspend); mutex_enter(&zilog->zl_issuer_lock); @@ -2170,10 +2181,24 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w ASSERT3P(lwb, ==, zcw->zcw_lwb); /* - * We've already checked this above, but since we hadn't - * acquired the zilog's zl_issuer_lock, we have to perform this - * check a second time while holding the lock. We can't call + * We've already checked this above, but since we hadn't acquired + * the zilog's zl_issuer_lock, we have to perform this check a + * second time while holding the lock. + * + * We don't need to hold the zl_lock since the lwb cannot transition + * from OPENED to ISSUED while we hold the zl_issuer_lock. The lwb + * _can_ transition from ISSUED to DONE, but it's OK to race with + * that transition since we treat the lwb the same, whether it's in + * the ISSUED or DONE states. + * + * The important thing, is we treat the lwb differently depending on + * if it's ISSUED or OPENED, and block any other threads that might + * attempt to issue this lwb. For that reason we hold the + * zl_issuer_lock when checking the lwb_state; we must not call * zil_lwb_write_issue() if the lwb had already been issued. + * + * See the comment above the lwb_state_t structure definition for + * more details on the lwb states, and locking requirements. */ if (lwb->lwb_state == LWB_STATE_ISSUED || lwb->lwb_state == LWB_STATE_DONE) @@ -2263,7 +2288,6 @@ zil_commit_waiter(zilog_t *zilog, zil_commit_waiter_t ASSERT(!MUTEX_HELD(&zilog->zl_lock)); ASSERT(!MUTEX_HELD(&zilog->zl_issuer_lock)); ASSERT(spa_writeable(zilog->zl_spa)); - ASSERT0(zilog->zl_suspend); mutex_enter(&zcw->zcw_lock); @@ -2568,6 +2592,12 @@ zil_commit(zilog_t *zilog, uint64_t foid) return; } + zil_commit_impl(zilog, foid); +} + +void +zil_commit_impl(zilog_t *zilog, uint64_t foid) +{ /* * Move the "async" itxs for the specified foid to the "sync" * queues, such that they will be later committed (or skipped) @@ -2980,7 +3010,22 @@ zil_suspend(const char *osname, void **cookiep) zilog->zl_suspending = B_TRUE; mutex_exit(&zilog->zl_lock); - zil_commit(zilog, 0); + /* + * We need to use zil_commit_impl to ensure we wait for all + * LWB_STATE_OPENED and LWB_STATE_ISSUED lwb's to be committed + * to disk before proceeding. If we used zil_commit instead, it + * would just call txg_wait_synced(), because zl_suspend is set. + * txg_wait_synced() doesn't wait for these lwb's to be + * LWB_STATE_DONE before returning. + */ + zil_commit_impl(zilog, 0); + + /* + * Now that we've ensured all lwb's are LWB_STATE_DONE, we use + * txg_wait_synced() to ensure the data from the zilog has + * migrated to the main pool before calling zil_destroy(). + */ + txg_wait_synced(zilog->zl_dmu_pool, 0); zil_destroy(zilog, B_FALSE); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Sun Jan 21 23:11:20 2018 (r328226) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Sun Jan 21 23:12:38 2018 (r328227) @@ -1716,20 +1716,6 @@ zio_reexecute(zio_t *pio) } void -zio_cancel(zio_t *zio) -{ - /* - * Disallow cancellation of a zio that's already been issued. - */ - VERIFY3P(zio->io_executor, ==, NULL); - - zio->io_pipeline = ZIO_INTERLOCK_PIPELINE; - zio->io_done = NULL; - - zio_nowait(zio); -} - -void zio_suspend(spa_t *spa, zio_t *zio) { if (spa_get_failmode(spa) == ZIO_FAILURE_MODE_PANIC) From owner-svn-src-vendor@freebsd.org Sun Jan 21 23:42:46 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2897EC1F8D; Sun, 21 Jan 2018 23:42:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 84AD9818CA; Sun, 21 Jan 2018 23:42:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A86EA1281B; Sun, 21 Jan 2018 23:42:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LNgjsi007464; Sun, 21 Jan 2018 23:42:45 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LNgjCv007463; Sun, 21 Jan 2018 23:42:45 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801212342.w0LNgjCv007463@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 23:42:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328229 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 328229 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 23:42:46 -0000 Author: mav Date: Sun Jan 21 23:42:45 2018 New Revision: 328229 URL: https://svnweb.freebsd.org/changeset/base/328229 Log: 8930 zfs_zinactive: do not remove the node if the filesystem is readonly illumos/illumos-gate@93c618e0f4932dc0bb9a9c90d8c4a5d029de5797 https://www.illumos.org/issues/8930: We normally remove an unlinked node when its last user goes away and the node becomes inactive. However, we should not do that if the filesystem is mounted read-only including the case where it has its readonly property set. The node will remain on the unlinked queue, so it will not be leaked. One particular scenario is when we receive an incremental stream into a mounted read-only filesystem and that stream contains an unlinked file (still on the unlinked queue). If that file is opened before the receive and some time later after the receive it becomes inactive we would remove it and, thus, modify the read-only filesystem. As a result, the filesystem would diverge from its source and further incremental receives would not be possible (without forcing a rollback). Another related scenario, that may or may not be possible depending on an OS / VFS policy, is when an open file is unlinked, then the filesystem is remounted read-only, and then the file is closed. Reviewed by: Matthew Ahrens Approved by: Gordon Ross Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Sun Jan 21 23:18:42 2018 (r328228) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Sun Jan 21 23:42:45 2018 (r328229) @@ -1250,9 +1250,21 @@ zfs_rezget(znode_t *zp) return (SET_ERROR(EIO)); } - zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; + /* + * If the file has zero links, then it has been unlinked on the send + * side and it must be in the received unlinked set. + * We call zfs_znode_dmu_fini() now to prevent any accesses to the + * stale data and to prevent automatical removal of the file in + * zfs_zinactive(). The file will be removed either when it is removed + * on the send side and the next incremental stream is received or + * when the unlinked set gets processed. + */ + zp->z_unlinked = (zp->z_links == 0); + if (zp->z_unlinked) + zfs_znode_dmu_fini(zp); + ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); return (0); @@ -1314,14 +1326,21 @@ zfs_zinactive(znode_t *zp) mutex_exit(&vp->v_lock); /* - * If this was the last reference to a file with no links, - * remove the file from the file system. + * If this was the last reference to a file with no links, remove + * the file from the file system unless the file system is mounted + * read-only. That can happen, for example, if the file system was + * originally read-write, the file was opened, then unlinked and + * the file system was made read-only before the file was finally + * closed. The file will remain in the unlinked set. */ if (zp->z_unlinked) { - mutex_exit(&zp->z_lock); - ZFS_OBJ_HOLD_EXIT(zfsvfs, z_id); - zfs_rmnode(zp); - return; + ASSERT(!zfsvfs->z_issnap); + if ((zfsvfs->z_vfs->vfs_flag & VFS_RDONLY) == 0) { + mutex_exit(&zp->z_lock); + ZFS_OBJ_HOLD_EXIT(zfsvfs, z_id); + zfs_rmnode(zp); + return; + } } mutex_exit(&zp->z_lock); From owner-svn-src-vendor@freebsd.org Sun Jan 21 23:52:38 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70152EC2531; Sun, 21 Jan 2018 23:52:38 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A07481EBA; Sun, 21 Jan 2018 23:52:38 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7C9C2129A8; Sun, 21 Jan 2018 23:52:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LNqbJW011736; Sun, 21 Jan 2018 23:52:37 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LNqbf3011735; Sun, 21 Jan 2018 23:52:37 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801212352.w0LNqbf3011735@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 23:52:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328231 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 328231 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 23:52:38 -0000 Author: mav Date: Sun Jan 21 23:52:37 2018 New Revision: 328231 URL: https://svnweb.freebsd.org/changeset/base/328231 Log: 8897 zpool online -e fails assertion when run on non-leaf vdevs illumos/illumos-gate@9a551dd645b478816cb11251b19f5034d885bf01 https://www.illumos.org/issues/8897: # zpool online -e test mirror-1 Assertion failed: nvlist_lookup_string(tgt, "path", &pathname) == 0, file ../common/libzfs_pool.c, line 2558, function zpool_vdev_online Abort (core dumped) Not a big deal per se, but should be handled gracefully, same way as 'offline' and 'online' without '-e'. Also reported as: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221408 Reviewed by: Toomas Soome Reviewed by: Igor Kozhukhov Approved by: Dan McDonald Author: Yuri Pankov Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Sun Jan 21 23:49:17 2018 (r328230) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Sun Jan 21 23:52:37 2018 (r328231) @@ -2425,6 +2425,7 @@ zpool_vdev_online(zpool_handle_t *zhp, const char *pat { zfs_cmd_t zc = { 0 }; char msg[1024]; + char *pathname; nvlist_t *tgt; boolean_t avail_spare, l2cache, islog; libzfs_handle_t *hdl = zhp->zpool_hdl; @@ -2447,15 +2448,13 @@ zpool_vdev_online(zpool_handle_t *zhp, const char *pat if (avail_spare) return (zfs_error(hdl, EZFS_ISSPARE, msg)); - if (flags & ZFS_ONLINE_EXPAND || - zpool_get_prop_int(zhp, ZPOOL_PROP_AUTOEXPAND, NULL)) { - char *pathname = NULL; + if ((flags & ZFS_ONLINE_EXPAND || + zpool_get_prop_int(zhp, ZPOOL_PROP_AUTOEXPAND, NULL)) && + nvlist_lookup_string(tgt, ZPOOL_CONFIG_PATH, &pathname) == 0) { uint64_t wholedisk = 0; (void) nvlist_lookup_uint64(tgt, ZPOOL_CONFIG_WHOLE_DISK, &wholedisk); - verify(nvlist_lookup_string(tgt, ZPOOL_CONFIG_PATH, - &pathname) == 0); /* * XXX - L2ARC 1.0 devices can't support expansion. From owner-svn-src-vendor@freebsd.org Sun Jan 21 23:57:43 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25A59EC29E7; Sun, 21 Jan 2018 23:57:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0079B821B5; Sun, 21 Jan 2018 23:57:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 39D16129AC; Sun, 21 Jan 2018 23:57:42 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0LNvgEB012021; Sun, 21 Jan 2018 23:57:42 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0LNvfxX012019; Sun, 21 Jan 2018 23:57:41 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801212357.w0LNvfxX012019@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sun, 21 Jan 2018 23:57:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328233 - in vendor/illumos/dist: lib/libzfs/common man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor/illumos/dist: lib/libzfs/common man/man1m X-SVN-Commit-Revision: 328233 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 23:57:43 -0000 Author: mav Date: Sun Jan 21 23:57:41 2018 New Revision: 328233 URL: https://svnweb.freebsd.org/changeset/base/328233 Log: 8898 creating fs with checksum=skein on the boot pools fails ungracefully illumos/illumos-gate@9fa2266d9a78b8366e1cd2d5f050e8b5e37d558c https://www.illumos.org/issues/8898: # zfs create -o checksum=skein rpool/test internal error: Result too large Abort (core dumped) Not a big deal per se, but should be handled correctly. Also reported as: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222199 Reviewed by: Toomas Soome Reviewed by: Andy Stormont Approved by: Dan McDonald Author: Yuri Pankov Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Sun Jan 21 23:53:56 2018 (r328232) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Sun Jan 21 23:57:41 2018 (r328233) @@ -28,7 +28,7 @@ * Copyright (c) 2013 Martin Matuska. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Integros [integros.com] - * Copyright 2016 Nexenta Systems, Inc. + * Copyright 2017 Nexenta Systems, Inc. * Copyright 2016 Igor Kozhukhov * Copyright 2017 RackTop Systems. */ @@ -3476,6 +3476,10 @@ zfs_create(libzfs_handle_t *hdl, const char *path, zfs "pool must be upgraded to set this " "property or value")); return (zfs_error(hdl, EZFS_BADVERSION, errbuf)); + case ERANGE: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "invalid property value(s) specified")); + return (zfs_error(hdl, EZFS_BADPROP, errbuf)); #ifdef _ILP32 case EOVERFLOW: /* Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Sun Jan 21 23:53:56 2018 (r328232) +++ vendor/illumos/dist/man/man1m/zfs.1m Sun Jan 21 23:57:41 2018 (r328233) @@ -28,7 +28,7 @@ .\" Copyright (c) 2014 Integros [integros.com] .\" Copyright 2017 Nexenta Systems, Inc. .\" -.Dd September 16, 2016 +.Dd December 6, 2017 .Dt ZFS 1M .Os .Sh NAME @@ -997,6 +997,10 @@ Please see for more information on these algorithms. .Pp Changing this property affects only newly-written data. +.Pp +Salted checksum algorithms +.Pq Cm edonr , skein +are currently not supported for any filesystem on the boot pools. .It Xo .Sy compression Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy gzip Ns | Ns .Sy gzip- Ns Em N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle From owner-svn-src-vendor@freebsd.org Mon Jan 22 04:21:56 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40D33ECCECE; Mon, 22 Jan 2018 04:21:56 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AEA369D00; Mon, 22 Jan 2018 04:21:56 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 582151543C; Mon, 22 Jan 2018 04:21:55 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M4Lt4U021901; Mon, 22 Jan 2018 04:21:55 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M4LtYE021900; Mon, 22 Jan 2018 04:21:55 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220421.w0M4LtYE021900@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 04:21:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328245 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 328245 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 04:21:56 -0000 Author: mav Date: Mon Jan 22 04:21:55 2018 New Revision: 328245 URL: https://svnweb.freebsd.org/changeset/base/328245 Log: 8856 arc_cksum_is_equal() doesn't take into account ABD-logic illumos/illumos-gate@01a059ee0cdece49f47fd4d70086dd5bc7d0b0ff https://www.illumos.org/issues/8856: arc_cksum_is_equal() calls zio_push_transform() that requires abd_t* (second arg), but a void* is passed. Reviewed by: Matthew Ahrens Reviewed by: George Wilson Approved by: Gordon Ross Author: Roman Strashkin Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Mon Jan 22 04:15:12 2018 (r328244) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Mon Jan 22 04:21:55 2018 (r328245) @@ -23,7 +23,7 @@ * Copyright (c) 2012, Joyent, Inc. All rights reserved. * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright (c) 2014 by Saso Kiselkov. All rights reserved. - * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. All rights reserved. */ /* @@ -1547,8 +1547,9 @@ arc_cksum_is_equal(arc_buf_hdr_t *hdr, zio_t *zio) uint64_t lsize = HDR_GET_LSIZE(hdr); uint64_t csize; - void *cbuf = zio_buf_alloc(HDR_GET_PSIZE(hdr)); - csize = zio_compress_data(compress, zio->io_abd, cbuf, lsize); + abd_t *cdata = abd_alloc_linear(HDR_GET_PSIZE(hdr), B_TRUE); + csize = zio_compress_data(compress, zio->io_abd, + abd_to_buf(cdata), lsize); ASSERT3U(csize, <=, HDR_GET_PSIZE(hdr)); if (csize < HDR_GET_PSIZE(hdr)) { @@ -1564,10 +1565,10 @@ arc_cksum_is_equal(arc_buf_hdr_t *hdr, zio_t *zio) * and zero out any part that should not contain * data. */ - bzero((char *)cbuf + csize, HDR_GET_PSIZE(hdr) - csize); + abd_zero_off(cdata, csize, HDR_GET_PSIZE(hdr) - csize); csize = HDR_GET_PSIZE(hdr); } - zio_push_transform(zio, cbuf, csize, HDR_GET_PSIZE(hdr), NULL); + zio_push_transform(zio, cdata, csize, HDR_GET_PSIZE(hdr), NULL); } /* From owner-svn-src-vendor@freebsd.org Mon Jan 22 04:27:06 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FCE2ECD1F9; Mon, 22 Jan 2018 04:27:06 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FD8A6A19C; Mon, 22 Jan 2018 04:27:06 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 881DB15576; Mon, 22 Jan 2018 04:27:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M4R5ra025275; Mon, 22 Jan 2018 04:27:05 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M4R57H025273; Mon, 22 Jan 2018 04:27:05 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220427.w0M4R57H025273@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 04:27:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328247 - in vendor-sys/illumos/dist/uts/common: fs/zfs sys/sysevent X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common: fs/zfs sys/sysevent X-SVN-Commit-Revision: 328247 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 04:27:06 -0000 Author: mav Date: Mon Jan 22 04:27:05 2018 New Revision: 328247 URL: https://svnweb.freebsd.org/changeset/base/328247 Log: 8959 Add notifications when a scrub is paused or resumed illumos/illumos-gate@301fd1d6f25595cd8c6d6795f39c72d97aff8cd9 Reviewed by: Alek Pinchuk Reviewed by: Matthew Ahrens Approved by: Gordon Ross Author: Sean Eric Fagan Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c vendor-sys/illumos/dist/uts/common/sys/sysevent/eventdefs.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Mon Jan 22 04:23:48 2018 (r328246) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Mon Jan 22 04:27:05 2018 (r328247) @@ -363,6 +363,7 @@ dsl_scan_cancel_sync(void *arg, dmu_tx_t *tx) dsl_scan_done(scn, B_FALSE, tx); dsl_scan_sync_state(scn, tx); + spa_event_notify(scn->scn_dp->dp_spa, NULL, NULL, ESC_ZFS_SCRUB_ABORT); } int @@ -417,6 +418,7 @@ dsl_scrub_pause_resume_sync(void *arg, dmu_tx_t *tx) spa->spa_scan_pass_scrub_pause = gethrestime_sec(); scn->scn_phys.scn_flags |= DSF_SCRUB_PAUSED; dsl_scan_sync_state(scn, tx); + spa_event_notify(spa, NULL, NULL, ESC_ZFS_SCRUB_PAUSED); } else { ASSERT3U(*cmd, ==, POOL_SCRUB_NORMAL); if (dsl_scan_is_paused_scrub(scn)) { @@ -1986,8 +1988,10 @@ dsl_scan(dsl_pool_t *dp, pool_scan_func_t func) /* got scrub start cmd, resume paused scrub */ int err = dsl_scrub_set_pause_resume(scn->scn_dp, POOL_SCRUB_NORMAL); - if (err == 0) + if (err == 0) { + spa_event_notify(spa, NULL, NULL, ESC_ZFS_SCRUB_RESUME); return (ECANCELED); + } return (SET_ERROR(err)); } Modified: vendor-sys/illumos/dist/uts/common/sys/sysevent/eventdefs.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/sysevent/eventdefs.h Mon Jan 22 04:23:48 2018 (r328246) +++ vendor-sys/illumos/dist/uts/common/sys/sysevent/eventdefs.h Mon Jan 22 04:27:05 2018 (r328247) @@ -203,6 +203,9 @@ extern "C" { #define ESC_ZFS_CONFIG_SYNC "ESC_ZFS_config_sync" #define ESC_ZFS_SCRUB_START "ESC_ZFS_scrub_start" #define ESC_ZFS_SCRUB_FINISH "ESC_ZFS_scrub_finish" +#define ESC_ZFS_SCRUB_ABORT "ESC_ZFS_scrub_abort" +#define ESC_ZFS_SCRUB_RESUME "ESC_ZFS_scrub_resume" +#define ESC_ZFS_SCRUB_PAUSED "ESC_ZFS_scrub_paused" #define ESC_ZFS_VDEV_SPARE "ESC_ZFS_vdev_spare" #define ESC_ZFS_BOOTFS_VDEV_ATTACH "ESC_ZFS_bootfs_vdev_attach" #define ESC_ZFS_POOL_REGUID "ESC_ZFS_pool_reguid" From owner-svn-src-vendor@freebsd.org Mon Jan 22 04:35:18 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96E37ECD6D1; Mon, 22 Jan 2018 04:35:18 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70CA86A749; Mon, 22 Jan 2018 04:35:18 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A528B15712; Mon, 22 Jan 2018 04:35:17 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M4ZHxS029608; Mon, 22 Jan 2018 04:35:17 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M4ZH4s029607; Mon, 22 Jan 2018 04:35:17 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220435.w0M4ZH4s029607@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 04:35:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328249 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 328249 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 04:35:18 -0000 Author: mav Date: Mon Jan 22 04:35:17 2018 New Revision: 328249 URL: https://svnweb.freebsd.org/changeset/base/328249 Log: 8641 "zpool clear" and "zinject" don't work on "spare" or "replacing" vdevs illumos/illumos-gate@2ba5f978a4f9b02da9db1b8cdd9ea5498eb00ad9 https://www.illumos.org/issues/8641: "zpool clear" and "zinject -d" can both operate on specific vdevs, either leaf or interior. However, due to an oversight, neither works on a "spare" or "replacing" vdev. For example: sudo zpool create foo raidz1 c1t5000CCA000081D61d0 c1t5000CCA000186235d0 spare c1t5000CCA000094115d0 sudo zpool replace foo c1t5000CCA000186235d0 c1t5000CCA000094115d0 $ zpool status foo pool: foo state: ONLINE scan: resilvered 81.5K in 0h0m with 0 errors on Fri Sep 8 10:53:03 2017 config: NAME STATE READ WRITE CKSUM foo ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 c1t5000CCA000081D61d0 ONLINE 0 0 0 spare-1 ONLINE 0 0 0 c1t5000CCA000186235d0 ONLINE 0 0 0 c1t5000CCA000094115d0 ONLINE 0 0 0 spares c1t5000CCA000094115d0 INUSE currently in use $ sudo zinject -d spare-1 -A degrade foo cannot find device 'spare-1' in pool 'foo' $ sudo zpool clear foo spare-1 cannot clear errors for spare-1: no such device in pool Even though there was nothing to clear, those commands shouldn't have reported an error. by contrast, trying to clear "raidz1-0" works just fine: $ sudo zpool clear foo raidz1-0 Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Mon Jan 22 04:31:48 2018 (r328248) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Mon Jan 22 04:35:17 2018 (r328249) @@ -50,6 +50,7 @@ #include "zfeature_common.h" static int read_efi_label(nvlist_t *, diskaddr_t *, boolean_t *); +static boolean_t zpool_vdev_is_interior(const char *name); #define BACKUP_SLICE "s2" @@ -2072,10 +2073,7 @@ vdev_to_nvlist_iter(nvlist_t *nv, nvlist_t *search, bo break; } - verify(strncmp(type, VDEV_TYPE_RAIDZ, - strlen(VDEV_TYPE_RAIDZ)) == 0 || - strncmp(type, VDEV_TYPE_MIRROR, - strlen(VDEV_TYPE_MIRROR)) == 0); + verify(zpool_vdev_is_interior(type)); verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_ID, &id) == 0); @@ -2182,10 +2180,13 @@ zpool_find_vdev_by_physpath(zpool_handle_t *zhp, const /* * Determine if we have an "interior" top-level vdev (i.e mirror/raidz). */ -boolean_t +static boolean_t zpool_vdev_is_interior(const char *name) { if (strncmp(name, VDEV_TYPE_RAIDZ, strlen(VDEV_TYPE_RAIDZ)) == 0 || + strncmp(name, VDEV_TYPE_SPARE, strlen(VDEV_TYPE_SPARE)) == 0 || + strncmp(name, + VDEV_TYPE_REPLACING, strlen(VDEV_TYPE_REPLACING)) == 0 || strncmp(name, VDEV_TYPE_MIRROR, strlen(VDEV_TYPE_MIRROR)) == 0) return (B_TRUE); return (B_FALSE); From owner-svn-src-vendor@freebsd.org Mon Jan 22 04:48:16 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A878ECDEBD; Mon, 22 Jan 2018 04:48:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 014956AE37; Mon, 22 Jan 2018 04:48:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2C181158A3; Mon, 22 Jan 2018 04:48:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M4mFNC033775; Mon, 22 Jan 2018 04:48:15 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M4mEjB033773; Mon, 22 Jan 2018 04:48:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220448.w0M4mEjB033773@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 04:48:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328251 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 328251 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 04:48:16 -0000 Author: mav Date: Mon Jan 22 04:48:14 2018 New Revision: 328251 URL: https://svnweb.freebsd.org/changeset/base/328251 Log: 8652 Tautological comparisons with ZPROP_INVAL illumos/illumos-gate@4ae5f5f06c6c2d1db8167480f7d9e3b5378ba2f2 https://www.illumos.org/issues/8652: Clang and GCC prefer to use unsigned ints to store enums. With Clang, that causes tautological comparison warnings when comparing a zfs_prop_t or zpool_prop_t variable to the macro ZPROP_INVAL. It's likely that error handling code is being silently removed as a result. Reviewed by: Matthew Ahrens Reviewed by: Igor Kozhukhov Approved by: Gordon Ross Author: Alan Somers Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Mon Jan 22 04:37:04 2018 (r328250) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Mon Jan 22 04:48:14 2018 (r328251) @@ -440,7 +440,7 @@ add_prop_list(const char *propname, char *propval, nvl * feature@ properties and version should not be specified * at the same time. */ - if ((prop == ZPROP_INVAL && zpool_prop_feature(propname) && + if ((prop == ZPOOL_PROP_INVAL && zpool_prop_feature(propname) && nvlist_exists(proplist, vname)) || (prop == ZPOOL_PROP_VERSION && prop_list_contains_feature(proplist))) { Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Mon Jan 22 04:37:04 2018 (r328250) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Mon Jan 22 04:48:14 2018 (r328251) @@ -445,7 +445,7 @@ zpool_valid_proplist(libzfs_handle_t *hdl, const char const char *propname = nvpair_name(elem); prop = zpool_name_to_prop(propname); - if (prop == ZPROP_INVAL && zpool_prop_feature(propname)) { + if (prop == ZPOOL_PROP_INVAL && zpool_prop_feature(propname)) { int err; char *fname = strchr(propname, '@') + 1; @@ -484,7 +484,7 @@ zpool_valid_proplist(libzfs_handle_t *hdl, const char /* * Make sure this property is valid and applies to this type. */ - if (prop == ZPROP_INVAL) { + if (prop == ZPOOL_PROP_INVAL) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "invalid property '%s'"), propname); (void) zfs_error(hdl, EZFS_BADPROP, errbuf); From owner-svn-src-vendor@freebsd.org Mon Jan 22 04:48:15 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8C39ECDEBA; Mon, 22 Jan 2018 04:48:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B11886AE36; Mon, 22 Jan 2018 04:48:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E64E0158A2; Mon, 22 Jan 2018 04:48:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M4mEGb033768; Mon, 22 Jan 2018 04:48:14 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M4mEV1033766; Mon, 22 Jan 2018 04:48:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220448.w0M4mEV1033766@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 04:48:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328251 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 328251 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 04:48:15 -0000 Author: mav Date: Mon Jan 22 04:48:14 2018 New Revision: 328251 URL: https://svnweb.freebsd.org/changeset/base/328251 Log: 8652 Tautological comparisons with ZPROP_INVAL illumos/illumos-gate@4ae5f5f06c6c2d1db8167480f7d9e3b5378ba2f2 https://www.illumos.org/issues/8652: Clang and GCC prefer to use unsigned ints to store enums. With Clang, that causes tautological comparison warnings when comparing a zfs_prop_t or zpool_prop_t variable to the macro ZPROP_INVAL. It's likely that error handling code is being silently removed as a result. Reviewed by: Matthew Ahrens Reviewed by: Igor Kozhukhov Approved by: Gordon Ross Author: Alan Somers Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Mon Jan 22 04:37:04 2018 (r328250) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Mon Jan 22 04:48:14 2018 (r328251) @@ -335,7 +335,7 @@ spa_prop_get(spa_t *spa, nvlist_t **nvp) zprop_source_t src = ZPROP_SRC_DEFAULT; zpool_prop_t prop; - if ((prop = zpool_name_to_prop(za.za_name)) == ZPROP_INVAL) + if ((prop = zpool_name_to_prop(za.za_name)) == ZPOOL_PROP_INVAL) continue; switch (za.za_integer_length) { @@ -423,7 +423,7 @@ spa_prop_validate(spa_t *spa, nvlist_t *props) zpool_prop_t prop = zpool_name_to_prop(propname); switch (prop) { - case ZPROP_INVAL: + case ZPOOL_PROP_INVAL: if (!zpool_prop_feature(propname)) { error = SET_ERROR(EINVAL); break; @@ -666,7 +666,7 @@ spa_prop_set(spa_t *spa, nvlist_t *nvp) prop == ZPOOL_PROP_READONLY) continue; - if (prop == ZPOOL_PROP_VERSION || prop == ZPROP_INVAL) { + if (prop == ZPOOL_PROP_VERSION || prop == ZPOOL_PROP_INVAL) { uint64_t ver; if (prop == ZPOOL_PROP_VERSION) { @@ -6313,7 +6313,7 @@ spa_sync_props(void *arg, dmu_tx_t *tx) spa_feature_t fid; switch (prop = zpool_name_to_prop(nvpair_name(elem))) { - case ZPROP_INVAL: + case ZPOOL_PROP_INVAL: /* * We checked this earlier in spa_prop_validate(). */ Modified: vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Mon Jan 22 04:37:04 2018 (r328250) +++ vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Mon Jan 22 04:48:14 2018 (r328251) @@ -87,7 +87,9 @@ typedef enum dmu_objset_type { * the property table in usr/src/common/zfs/zfs_prop.c. */ typedef enum { - ZFS_PROP_TYPE, + ZPROP_CONT = -2, + ZPROP_INVAL = -1, + ZFS_PROP_TYPE = 0, ZFS_PROP_CREATION, ZFS_PROP_USED, ZFS_PROP_AVAILABLE, @@ -179,6 +181,7 @@ extern const char *zfs_userquota_prop_prefixes[ZFS_NUM * the property table in usr/src/common/zfs/zpool_prop.c. */ typedef enum { + ZPOOL_PROP_INVAL = -1, ZPOOL_PROP_NAME, ZPOOL_PROP_SIZE, ZPOOL_PROP_CAPACITY, @@ -210,9 +213,6 @@ typedef enum { /* Small enough to not hog a whole line of printout in zpool(1M). */ #define ZPROP_MAX_COMMENT 32 - -#define ZPROP_CONT -2 -#define ZPROP_INVAL -1 #define ZPROP_VALUE "value" #define ZPROP_SOURCE "source" From owner-svn-src-vendor@freebsd.org Mon Jan 22 05:55:44 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9CD9ED03BD; Mon, 22 Jan 2018 05:55:44 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2E766CA94; Mon, 22 Jan 2018 05:55:44 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B47B7163E2; Mon, 22 Jan 2018 05:55:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M5thh8062501; Mon, 22 Jan 2018 05:55:43 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M5th55062500; Mon, 22 Jan 2018 05:55:43 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220555.w0M5th55062500@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 05:55:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328253 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 328253 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 05:55:44 -0000 Author: mav Date: Mon Jan 22 05:55:43 2018 New Revision: 328253 URL: https://svnweb.freebsd.org/changeset/base/328253 Log: 8835 Speculative prefetch in ZFS not working for misaligned reads illumos/illumos-gate@5cb8d943bc8513c6230589aad5a409d58b0297cb https://www.illumos.org/issues/8835: Sequential reads not aligned to block size are not detected by ZFS prefetcher as sequential, killing prefetch and severely hurting performance. It is caused by dmu_zfetch() in case of misaligned sequential accesses being called with overlap of one block. Reviewed by: Matthew Ahrens Reviewed by: Allan Jude Approved by: Gordon Ross Author: Alexander Motin Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c Mon Jan 22 05:52:39 2018 (r328252) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c Mon Jan 22 05:55:43 2018 (r328253) @@ -224,19 +224,33 @@ dmu_zfetch(zfetch_t *zf, uint64_t blkid, uint64_t nblk rw_enter(&zf->zf_rwlock, RW_READER); + /* + * Find matching prefetch stream. Depending on whether the accesses + * are block-aligned, first block of the new access may either follow + * the last block of the previous access, or be equal to it. + */ for (zs = list_head(&zf->zf_stream); zs != NULL; zs = list_next(&zf->zf_stream, zs)) { - if (blkid == zs->zs_blkid) { + if (blkid == zs->zs_blkid || blkid + 1 == zs->zs_blkid) { mutex_enter(&zs->zs_lock); /* * zs_blkid could have changed before we * acquired zs_lock; re-check them here. */ - if (blkid != zs->zs_blkid) { - mutex_exit(&zs->zs_lock); - continue; + if (blkid == zs->zs_blkid) { + break; + } else if (blkid + 1 == zs->zs_blkid) { + blkid++; + nblks--; + if (nblks == 0) { + /* Already prefetched this before. */ + mutex_exit(&zs->zs_lock); + rw_exit(&zf->zf_rwlock); + return; + } + break; } - break; + mutex_exit(&zs->zs_lock); } } From owner-svn-src-vendor@freebsd.org Mon Jan 22 05:59:50 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F3F6ED0675; Mon, 22 Jan 2018 05:59:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFE146CDA3; Mon, 22 Jan 2018 05:59:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3C844163F7; Mon, 22 Jan 2018 05:59:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0M5xn1T062737; Mon, 22 Jan 2018 05:59:49 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0M5xnbs062736; Mon, 22 Jan 2018 05:59:49 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201801220559.w0M5xnbs062736@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 22 Jan 2018 05:59:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328255 - vendor/illumos/dist/cmd/zfs X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor/illumos/dist/cmd/zfs X-SVN-Commit-Revision: 328255 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 05:59:50 -0000 Author: mav Date: Mon Jan 22 05:59:48 2018 New Revision: 328255 URL: https://svnweb.freebsd.org/changeset/base/328255 Log: 8972 zfs holds: In scripted mode, do not pad columns with spaces illumos/illumos-gate@e9b7d6e7f7a6477679a35b73eb3934b096b3dd39 https://www.illumos.org/issues/8972: 'zfs holds -H' does not properly output content in scripted mode. It uses a tab instead of two spaces, but it still pads column widths with spaces when it should not. Reviewed by: Matthew Ahrens Approved by: Dan McDonald Author: Allan Jude Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_main.c Mon Jan 22 05:57:14 2018 (r328254) +++ vendor/illumos/dist/cmd/zfs/zfs_main.c Mon Jan 22 05:59:48 2018 (r328255) @@ -5569,8 +5569,6 @@ print_holds(boolean_t scripted, size_t nwidth, size_t uint64_t val = 0; time_t time; struct tm t; - char sep = scripted ? '\t' : ' '; - size_t sepnum = scripted ? 1 : 2; (void) nvpair_value_uint64(nvp2, &val); time = (time_t)val; @@ -5578,8 +5576,13 @@ print_holds(boolean_t scripted, size_t nwidth, size_t (void) strftime(tsbuf, DATETIME_BUF_LEN, gettext(STRFTIME_FMT_STR), &t); - (void) printf("%-*s%*c%-*s%*c%s\n", nwidth, zname, - sepnum, sep, tagwidth, tagname, sepnum, sep, tsbuf); + if (scripted) { + (void) printf("%s\t%s\t%s\n", zname, + tagname, tsbuf); + } else { + (void) printf("%-*s %-*s %s\n", nwidth, + zname, tagwidth, tagname, tsbuf); + } } } } From owner-svn-src-vendor@freebsd.org Mon Jan 22 09:25:36 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EEF5EB581B; Mon, 22 Jan 2018 09:25:36 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id 3F5B77398E; Mon, 22 Jan 2018 09:25:32 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 2B5D3D6E52D; Mon, 22 Jan 2018 20:25:21 +1100 (AEDT) Date: Mon, 22 Jan 2018 20:25:20 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alexander Motin cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: Re: svn commit: r328251 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzfs/common In-Reply-To: <201801220448.w0M4mEjB033773@repo.freebsd.org> Message-ID: <20180122201244.M1317@besplex.bde.org> References: <201801220448.w0M4mEjB033773@repo.freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=DIX/22Fb c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=DSHT9BU3AAAA:8 a=ltDKUUI1AAAA:8 a=XWMXhlxvoWOi9kvSVkMA:9 a=CjuIK1q_8ugA:10 a=8ljxv5-TN0tg3ptMfEC3:22 a=zPa3363zVR52QbuIqfxu:22 X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 09:25:36 -0000 On Mon, 22 Jan 2018, Alexander Motin wrote: > Log: > 8652 Tautological comparisons with ZPROP_INVAL > > illumos/illumos-gate@4ae5f5f06c6c2d1db8167480f7d9e3b5378ba2f2 > > https://www.illumos.org/issues/8652: > Clang and GCC prefer to use unsigned ints to store enums. With Clang, that > causes tautological comparison warnings when comparing a zfs_prop_t or > zpool_prop_t variable to the macro ZPROP_INVAL. It's likely that error > handling code is being silently removed as a result. This shows another reason to never use enums. emum constants have type int, but enum variables have an implementation-defined type capable of respresenting the values of all of the members of the enumeration. Inexperienced programmers use unsigned types to implement unsign extension bugs and bugs like this. If you use basic types or even your own typedefs, then unsigned types are easy to avoid. APIs tend to have too many unsigned types, but only especially badly designed ones allow the signedness to vary with the implementation or don't document what it is. The implementation's definition of this is hard to find. gcc.info has a second that clearly documents that it is required to document this (C99 6.7.2.2), but I couldn't find where it actually documents this. clang has no useful installed documentation. The type of an enum can be forced to be signed by putting a negative enum value in it. ZPROP_INVAL = -1 as an enum value would do this. This wouldn't work with a generic INVAL value used with different enums. This is the fix used here -- another recent commit added ZPOOL_PROP_INVAL as an enum value and this commit uses it. I think this changes the type of the enum from u_int to int, so it risks sign extension bugs instead of unsign extension bugs. Unsigned types might be used to pack 256 enum values in 8 bits, but gcc and clang on x86 use uint32_t for even 1-bit enums like "enum foo { ZERO, ONE, };", just like inexperienced programmers. gcc and clang have an option -fshort-enums which makes unsigned types give better packing in a few cases -- "enum foo { N = 256 };" can be fitted in 1 byte, and "enum foo { N = 65536 };" can be fitted in 2 byte. With 32-bit ints, uint32_t enums just give the unsign extension bugs since enum values have type int so "enum foo { N = 0xffffffff };" is invalid. A similar class of unsign extension bugs augmented by wrong fixes is when you have an opaque type with careful range checking of the form: opaque_t v = foo(); if (v < MIN || v > MAX) error(); First v is misimplemented an unsigned type. When MIN == 0, (v < MIN) becomes tautologous. Then the warning is "fixed" by removing this check. Then when v is correctly implemented by fixing its type in this implemenation or never breaking it in other implementations, the code is broken. Changing a single typedef or enum in a header file can uncover many latent sign extension, unsign extension, spurious warning, or missing range checking bugs. Bruce From owner-svn-src-vendor@freebsd.org Wed Jan 24 06:45:04 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9878EC1F01; Wed, 24 Jan 2018 06:45:04 +0000 (UTC) (envelope-from philip@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 694627A5D5; Wed, 24 Jan 2018 06:45:04 +0000 (UTC) (envelope-from philip@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5EF231A361; Wed, 24 Jan 2018 06:45:04 +0000 (UTC) (envelope-from philip@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0O6j4CO027928; Wed, 24 Jan 2018 06:45:04 GMT (envelope-from philip@FreeBSD.org) Received: (from philip@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0O6j3Xm027917; Wed, 24 Jan 2018 06:45:03 GMT (envelope-from philip@FreeBSD.org) Message-Id: <201801240645.w0O6j3Xm027917@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: philip set sender to philip@FreeBSD.org using -f From: Philip Paeps Date: Wed, 24 Jan 2018 06:45:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328316 - vendor/tzdata/dist X-SVN-Group: vendor X-SVN-Commit-Author: philip X-SVN-Commit-Paths: vendor/tzdata/dist X-SVN-Commit-Revision: 328316 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 06:45:05 -0000 Author: philip Date: Wed Jan 24 06:45:03 2018 New Revision: 328316 URL: https://svnweb.freebsd.org/changeset/base/328316 Log: Import tzdata 2018c Added: vendor/tzdata/dist/pacificnew Modified: vendor/tzdata/dist/Makefile vendor/tzdata/dist/NEWS vendor/tzdata/dist/README vendor/tzdata/dist/asia vendor/tzdata/dist/europe vendor/tzdata/dist/leap-seconds.list vendor/tzdata/dist/leapseconds vendor/tzdata/dist/theory.html vendor/tzdata/dist/version vendor/tzdata/dist/zishrink.awk Modified: vendor/tzdata/dist/Makefile ============================================================================== --- vendor/tzdata/dist/Makefile Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/Makefile Wed Jan 24 06:45:03 2018 (r328316) @@ -455,14 +455,15 @@ WEB_PAGES= tz-art.html tz-how-to.html tz-link.html DOCS= $(MANS) date.1 $(MANTXTS) $(WEB_PAGES) PRIMARY_YDATA= africa antarctica asia australasia \ europe northamerica southamerica -YDATA= $(PRIMARY_YDATA) etcetera $(BACKWARD) +YDATA= $(PRIMARY_YDATA) etcetera NDATA= systemv factory -TDATA= $(YDATA) $(NDATA) +TDATA_TO_CHECK= $(YDATA) $(NDATA) backward pacificnew +TDATA= $(YDATA) $(NDATA) $(BACKWARD) ZONETABLES= zone1970.tab zone.tab TABDATA= iso3166.tab $(TZDATA_TEXT) $(ZONETABLES) LEAP_DEPS= leapseconds.awk leap-seconds.list TZDATA_ZI_DEPS= zishrink.awk version $(TDATA) $(PACKRATDATA) -DATA= $(YDATA) $(NDATA) backzone iso3166.tab leap-seconds.list \ +DATA= $(TDATA_TO_CHECK) backzone iso3166.tab leap-seconds.list \ leapseconds yearistype.sh $(ZONETABLES) AWK_SCRIPTS= checklinks.awk checktab.awk leapseconds.awk zishrink.awk MISC= $(AWK_SCRIPTS) zoneinfo2tdf.pl @@ -670,7 +671,7 @@ check_character_set: $(ENCHILADA) $(MISC) $(SOURCES) $(WEB_PAGES) \ CONTRIBUTING LICENSE Makefile README \ version tzdata.zi && \ - ! grep -Env $(SAFE_SHARP_LINE) $(TDATA) backzone \ + ! grep -Env $(SAFE_SHARP_LINE) $(TDATA_TO_CHECK) backzone \ leapseconds yearistype.sh zone.tab && \ ! grep -Env $(OK_LINE) $(ENCHILADA); \ } @@ -678,14 +679,16 @@ check_character_set: $(ENCHILADA) check_white_space: $(ENCHILADA) patfmt=' \t|[\f\r\v]' && pat=`printf "$$patfmt\\n"` && \ ! grep -En "$$pat" $(ENCHILADA) - ! grep -n '[[:space:]]$$' $(ENCHILADA) + ! grep -n '[[:space:]]$$' \ + $$(ls $(ENCHILADA) | grep -Fvx leap-seconds.list) PRECEDES_FILE_NAME = ^(Zone|Link[[:space:]]+[^[:space:]]+)[[:space:]]+ FILE_NAME_COMPONENT_TOO_LONG = \ $(PRECEDES_FILE_NAME)[^[:space:]]*[^/[:space:]]{15} -check_name_lengths: $(TDATA) backzone - ! grep -En '$(FILE_NAME_COMPONENT_TOO_LONG)' $(TDATA) backzone +check_name_lengths: $(TDATA_TO_CHECK) backzone + ! grep -En '$(FILE_NAME_COMPONENT_TOO_LONG)' \ + $(TDATA_TO_CHECK) backzone CHECK_CC_LIST = { n = split($$1,a,/,/); for (i=2; i<=n; i++) print a[1], a[i]; } @@ -699,8 +702,8 @@ check_sorted: backward backzone iso3166.tab zone.tab z $(AWK) '/^[^#]/ $(CHECK_CC_LIST)' zone1970.tab | \ LC_ALL=C sort -cu -check_links: checklinks.awk $(TDATA) - $(AWK) -f checklinks.awk $(TDATA) +check_links: checklinks.awk $(TDATA_TO_CHECK) + $(AWK) -f checklinks.awk $(TDATA_TO_CHECK) $(AWK) -f checklinks.awk tzdata.zi check_tables: checktab.awk $(PRIMARY_YDATA) $(ZONETABLES) @@ -803,10 +806,10 @@ check_public: $(MAKE) maintainer-clean $(MAKE) CFLAGS='$(GCC_DEBUG_FLAGS)' ALL mkdir -p public.dir - for i in $(TDATA) tzdata.zi; do \ + for i in $(TDATA_TO_CHECK) tzdata.zi; do \ $(zic) -v -d public.dir $$i 2>&1 || exit; \ done - $(zic) -v -d public.dir $(TDATA) + $(zic) -v -d public.dir $(TDATA_TO_CHECK) rm -fr public.dir # Check that the code works under various alternative Modified: vendor/tzdata/dist/NEWS ============================================================================== --- vendor/tzdata/dist/NEWS Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/NEWS Wed Jan 24 06:45:03 2018 (r328316) @@ -1,5 +1,51 @@ News for the tz database +Release 2018c - 2018-01-22 23:00:44 -0800 + + Briefly: + Revert Irish changes that relied on negative DST offsets. + + Changes to tm_isdst + + Revert the 2018a change to Europe/Dublin. As before, this change + does not affect UT offsets or abbreviations; it affects only + whether timestamps are considered to be standard time or + daylight-saving time, as expressed in the tm_isdst flag of C's + struct tm type. This reversion is intended to be a temporary + workaround for problems discovered with downstream uses of + releases 2018a and 2018b, which implemented Irish time by using + negative DST offsets in the Eire rules of the 'europe' file. + Although negative DST offsets have been part of tzcode for many + years and are supported by many platforms, they were not + documented before 2018a and ICU and OpenJDK do not currently + support them. A mechanism to export data to platforms lacking + support for negative DST is planned to be developed before the + change is reapplied. (Problems reported by Deborah Goldsmith and + Stephen Colebourne.) + + Changes to past time stamps + + Japanese DST transitions (1948-1951) were Sundays at 00:00, not + Saturdays or Sundays at 02:00. (Thanks to Takayuki Nikai.) + + Changes to build procedure + + The build procedure now works around mawk 1.3.3's lack of support + for character class expressions. (Problem reported by Ohyama.) + + +Release 2018b - 2018-01-17 23:24:48 -0800 + + Briefly: + Fix a packaging problem in tz2018a, which was missing 'pacificnew'. + + Changes to build procedure + + The distribution now contains the file 'pacificnew' again. + This file was inadvertantly omitted in the 2018a distribution. + (Problem reported by Matias Fonzo.) + + Release 2018a - 2018-01-12 22:29:21 -0800 Briefly: Modified: vendor/tzdata/dist/README ============================================================================== --- vendor/tzdata/dist/README Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/README Wed Jan 24 06:45:03 2018 (r328316) @@ -18,7 +18,7 @@ platform other than GNU/Linux. Then run the following substituting your desired installation directory for "$HOME/tzdir": make TOPDIR=$HOME/tzdir install - $HOME/tzdir/etc/zdump -v America/Los_Angeles + $HOME/tzdir/usr/bin/zdump -v America/Los_Angeles Historical local time information has been included here to: Modified: vendor/tzdata/dist/asia ============================================================================== --- vendor/tzdata/dist/asia Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/asia Wed Jan 24 06:45:03 2018 (r328316) @@ -1464,17 +1464,17 @@ Zone Asia/Jerusalem 2:20:54 - LMT 1880 # of the Japanese wanted to scrap daylight-saving time, as opposed to 30% who # wanted to keep it.) -# From Paul Eggert (2006-03-22): -# Shanks & Pottenger write that DST in Japan during those years was as follows: +# From Takayuki Nikai (2018-01-19): +# The source of information is Japanese law. +# http://www.shugiin.go.jp/internet/itdb_housei.nsf/html/houritsu/00219480428029.htm +# http://www.shugiin.go.jp/internet/itdb_housei.nsf/html/houritsu/00719500331039.htm +# ... In summary, it is written as follows. From 24:00 on the first Saturday +# in May, until 0:00 on the day after the second Saturday in September. # Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S -Rule Japan 1948 only - May Sun>=1 2:00 1:00 D -Rule Japan 1948 1951 - Sep Sat>=8 2:00 0 S -Rule Japan 1949 only - Apr Sun>=1 2:00 1:00 D -Rule Japan 1950 1951 - May Sun>=1 2:00 1:00 D -# but the only locations using it (for birth certificates, presumably, since -# their audience is astrologers) were US military bases. For now, assume -# that for most purposes daylight-saving time was observed; otherwise, what -# would have been the point of the 1951 poll? +Rule Japan 1948 only - May Sat>=1 24:00 1:00 D +Rule Japan 1948 1951 - Sep Sun>=9 0:00 0 S +Rule Japan 1949 only - Apr Sat>=1 24:00 1:00 D +Rule Japan 1950 1951 - May Sat>=1 24:00 1:00 D # From Hideyuki Suzuki (1998-11-09): # 'Tokyo' usually stands for the former location of Tokyo Astronomical Modified: vendor/tzdata/dist/europe ============================================================================== --- vendor/tzdata/dist/europe Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/europe Wed Jan 24 06:45:03 2018 (r328316) @@ -508,16 +508,19 @@ Link Europe/London Europe/Jersey Link Europe/London Europe/Guernsey Link Europe/London Europe/Isle_of_Man +# From Paul Eggert (2018-01-19): # The following is like GB-Eire and EU, except with standard time in # summer and negative daylight saving time in winter. +# Although currently commented out, this will need to become uncommented +# once the ICU/OpenJDK workaround is removed; see below. # Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S -Rule Eire 1971 only - Oct 31 2:00u -1:00 GMT -Rule Eire 1972 1980 - Mar Sun>=16 2:00u 0 IST -Rule Eire 1972 1980 - Oct Sun>=23 2:00u -1:00 GMT -Rule Eire 1981 max - Mar lastSun 1:00u 0 IST -Rule Eire 1981 1989 - Oct Sun>=23 1:00u -1:00 GMT -Rule Eire 1990 1995 - Oct Sun>=22 1:00u -1:00 GMT -Rule Eire 1996 max - Oct lastSun 1:00u -1:00 GMT +#Rule Eire 1971 only - Oct 31 2:00u -1:00 GMT +#Rule Eire 1972 1980 - Mar Sun>=16 2:00u 0 IST +#Rule Eire 1972 1980 - Oct Sun>=23 2:00u -1:00 GMT +#Rule Eire 1981 max - Mar lastSun 1:00u 0 IST +#Rule Eire 1981 1989 - Oct Sun>=23 1:00u -1:00 GMT +#Rule Eire 1990 1995 - Oct Sun>=22 1:00u -1:00 GMT +#Rule Eire 1996 max - Oct lastSun 1:00u -1:00 GMT # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Europe/Dublin -0:25:00 - LMT 1880 Aug 2 @@ -530,7 +533,25 @@ Zone Europe/Dublin -0:25:00 - LMT 1880 Aug 2 0:00 1:00 IST 1947 Nov 2 2:00s 0:00 - GMT 1948 Apr 18 2:00s 0:00 GB-Eire GMT/IST 1968 Oct 27 - 1:00 Eire IST/GMT +# From Paul Eggert (2018-01-18): +# The next line should look like this: +# 1:00 Eire IST/GMT +# However, in January 2018 we discovered that the Eire rules cause +# problems with tests for ICU: +# https://mm.icann.org/pipermail/tz/2018-January/025825.html +# and with tests for OpenJDK: +# https://mm.icann.org/pipermail/tz/2018-January/025822.html +# To work around this problem, use a traditional approximation for +# time stamps after 1971-10-31 02:00 UTC, to give ICU and OpenJDK +# developers breathing room to fix bugs. This approximation has +# correct UTC offsets, but results in tm_isdst flags are the reverse +# of what they should be. This workaround is temporary and should be +# removed reasonably soon. + 1:00 - IST 1971 Oct 31 2:00u + 0:00 GB-Eire GMT/IST 1996 + 0:00 EU GMT/IST +# End of workaround for ICU and OpenJDK bugs. + ############################################################################### Modified: vendor/tzdata/dist/leap-seconds.list ============================================================================== --- vendor/tzdata/dist/leap-seconds.list Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/leap-seconds.list Wed Jan 24 06:45:03 2018 (r328316) @@ -61,7 +61,12 @@ # or # Terry Quinn, "The BIPM and the Accurate Measurement # of Time," Proc. of the IEEE, Vol. 79, pp. 894-905, -# July, 1991. +# July, 1991. +# reprinted in: +# Christine Hackman and Donald B Sullivan (eds.) +# Time and Frequency Measurement +# American Association of Physics Teachers (1996) +# , pp. 75-86 # # 4. The decision to insert a leap second into UTC is currently # the responsibility of the International Earth Rotation and @@ -199,10 +204,10 @@ # current -- the update time stamp, the data and the name of the file # will not change. # -# Updated through IERS Bulletin C54 -# File expires on: 28 June 2018 +# Updated through IERS Bulletin C55 +# File expires on: 28 December 2018 # -#@ 3739132800 +#@ 3754944000 # 2272060800 10 # 1 Jan 1972 2287785600 11 # 1 Jul 1972 @@ -247,4 +252,4 @@ # the hash line is also ignored in the # computation. # -#h 5101445a 69948b51 9153e2b 2086e3d8 d54561a3 +#h 44dcf58c e28d25aa b36612c8 f3d3e8b5 a8fdf478 Modified: vendor/tzdata/dist/leapseconds ============================================================================== --- vendor/tzdata/dist/leapseconds Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/leapseconds Wed Jan 24 06:45:03 2018 (r328316) @@ -57,5 +57,5 @@ Leap 2012 Jun 30 23:59:60 + S Leap 2015 Jun 30 23:59:60 + S Leap 2016 Dec 31 23:59:60 + S -# Updated through IERS Bulletin C54 -# File expires on: 28 June 2018 +# Updated through IERS Bulletin C55 +# File expires on: 28 December 2018 Added: vendor/tzdata/dist/pacificnew ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/tzdata/dist/pacificnew Wed Jan 24 06:45:03 2018 (r328316) @@ -0,0 +1,27 @@ +# This file is in the public domain, so clarified as of +# 2009-05-17 by Arthur David Olson. + +# From Arthur David Olson (1989-04-05): +# On 1989-04-05, the U. S. House of Representatives passed (238-154) a bill +# establishing "Pacific Presidential Election Time"; it was not acted on +# by the Senate or signed into law by the President. +# You might want to change the "PE" (Presidential Election) below to +# "Q" (Quadrennial) to maintain three-character zone abbreviations. +# If you're really conservative, you might want to change it to "D". +# Avoid "L" (Leap Year), which won't be true in 2100. + +# If Presidential Election Time is ever established, replace "XXXX" below +# with the year the law takes effect and uncomment the "##" lines. + +# Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S +## Rule Twilite XXXX max - Apr Sun>=1 2:00 1:00 D +## Rule Twilite XXXX max uspres Oct lastSun 2:00 1:00 PE +## Rule Twilite XXXX max uspres Nov Sun>=7 2:00 0 S +## Rule Twilite XXXX max nonpres Oct lastSun 2:00 0 S + +# Zone NAME GMTOFF RULES/SAVE FORMAT [UNTIL] +## Zone America/Los_Angeles-PET -8:00 US P%sT XXXX +## -8:00 Twilite P%sT + +# For now... +Link America/Los_Angeles US/Pacific-New ## Modified: vendor/tzdata/dist/theory.html ============================================================================== --- vendor/tzdata/dist/theory.html Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/theory.html Wed Jan 24 06:45:03 2018 (r328316) @@ -1065,7 +1065,7 @@ Sources: Michael Allison and Robert Schmunk, "Technical Notes on Mars Solar Time as Adopted by the Mars24 Sunclock" -(2012-08-08). +(2015-06-30).
  • Jia-Rui Chong, Modified: vendor/tzdata/dist/version ============================================================================== --- vendor/tzdata/dist/version Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/version Wed Jan 24 06:45:03 2018 (r328316) @@ -1 +1 @@ -2018a +2018c Modified: vendor/tzdata/dist/zishrink.awk ============================================================================== --- vendor/tzdata/dist/zishrink.awk Wed Jan 24 05:16:11 2018 (r328315) +++ vendor/tzdata/dist/zishrink.awk Wed Jan 24 06:45:03 2018 (r328316) @@ -37,7 +37,7 @@ function process_input_line(line, field, end, i, n, st # Remove comments, normalize spaces, and append a space to each line. sub(/#.*/, "", line) line = line " " - gsub(/[[:space:]]+/, " ", line) + gsub(/[\f\r\t\v ]+/, " ", line) # Abbreviate keywords. Do not abbreviate "Link" to just "L", # as pre-2017c zic erroneously diagnoses "Li" as ambiguous. @@ -94,7 +94,7 @@ function process_input_line(line, field, end, i, n, st sub(/ 0+$/, "", line) # Remove unnecessary trailing days-of-month "1". - if (match(line, /[[:alpha:]] 1$/)) + if (match(line, /[A-Za-z] 1$/)) line = substr(line, 1, RSTART) # Remove unnecessary trailing " Ja" (for January). @@ -148,7 +148,7 @@ BEGIN { print "# This zic input file is in the public domain." } -/^[[:space:]]*[^#[:space:]]/ { +/^[\f\r\t\v ]*[^#\f\r\t\v ]/ { process_input_line($0) } From owner-svn-src-vendor@freebsd.org Wed Jan 24 06:45:53 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2809CEC1F82; Wed, 24 Jan 2018 06:45:53 +0000 (UTC) (envelope-from philip@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF43F7A6FA; Wed, 24 Jan 2018 06:45:52 +0000 (UTC) (envelope-from philip@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B114B1A362; Wed, 24 Jan 2018 06:45:52 +0000 (UTC) (envelope-from philip@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0O6jqfv027994; Wed, 24 Jan 2018 06:45:52 GMT (envelope-from philip@FreeBSD.org) Received: (from philip@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0O6jqC8027993; Wed, 24 Jan 2018 06:45:52 GMT (envelope-from philip@FreeBSD.org) Message-Id: <201801240645.w0O6jqC8027993@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: philip set sender to philip@FreeBSD.org using -f From: Philip Paeps Date: Wed, 24 Jan 2018 06:45:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328317 - vendor/tzdata/tzdata2018c X-SVN-Group: vendor X-SVN-Commit-Author: philip X-SVN-Commit-Paths: vendor/tzdata/tzdata2018c X-SVN-Commit-Revision: 328317 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 06:45:53 -0000 Author: philip Date: Wed Jan 24 06:45:52 2018 New Revision: 328317 URL: https://svnweb.freebsd.org/changeset/base/328317 Log: Tag import of tzdata 2018c Added: vendor/tzdata/tzdata2018c/ - copied from r328316, vendor/tzdata/dist/ From owner-svn-src-vendor@freebsd.org Wed Jan 24 11:01:16 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67833ECD07A; Wed, 24 Jan 2018 11:01:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1040483475; Wed, 24 Jan 2018 11:01:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0B0B51CCBA; Wed, 24 Jan 2018 11:01:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OB1Geo053666; Wed, 24 Jan 2018 11:01:16 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OB1Cf0053628; Wed, 24 Jan 2018 11:01:12 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201801241101.w0OB1Cf0053628@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Wed, 24 Jan 2018 11:01:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328323 - in vendor/libarchive/dist: . cat cat/test libarchive libarchive/test tar/test X-SVN-Group: vendor X-SVN-Commit-Author: mm X-SVN-Commit-Paths: in vendor/libarchive/dist: . cat cat/test libarchive libarchive/test tar/test X-SVN-Commit-Revision: 328323 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 11:01:16 -0000 Author: mm Date: Wed Jan 24 11:01:12 2018 New Revision: 328323 URL: https://svnweb.freebsd.org/changeset/base/328323 Log: Update libarchive to git 2d9156bb3ad92c43c1e81546f4a2c21549ba248f Relevant vendor changes: PR #893: delete dead ppmd7 alloc callbacks PR #904: Fix archive freeing bug in bsdcat PR #961: Fix ZIP format names PR #962: Don't modify attributes for existing directories when ARCHIVE_EXTRACT_NO_OVERWRITE is set PR #964: Fix -Werror=implicit-fallthrough= for GCC 7 PR #970: zip: Allow backslash as path separator Modified: vendor/libarchive/dist/CMakeLists.txt vendor/libarchive/dist/Makefile.am vendor/libarchive/dist/cat/bsdcat.c vendor/libarchive/dist/cat/test/CMakeLists.txt vendor/libarchive/dist/libarchive/archive_acl.c vendor/libarchive/dist/libarchive/archive_disk_acl_freebsd.c vendor/libarchive/dist/libarchive/archive_match.c vendor/libarchive/dist/libarchive/archive_platform.h vendor/libarchive/dist/libarchive/archive_ppmd7.c vendor/libarchive/dist/libarchive/archive_ppmd7_private.h vendor/libarchive/dist/libarchive/archive_ppmd_private.h vendor/libarchive/dist/libarchive/archive_read.c vendor/libarchive/dist/libarchive/archive_read_disk_posix.c vendor/libarchive/dist/libarchive/archive_read_disk_windows.c vendor/libarchive/dist/libarchive/archive_read_support_format_7zip.c vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c vendor/libarchive/dist/libarchive/archive_read_support_format_rar.c vendor/libarchive/dist/libarchive/archive_read_support_format_tar.c vendor/libarchive/dist/libarchive/archive_read_support_format_zip.c vendor/libarchive/dist/libarchive/archive_util.c vendor/libarchive/dist/libarchive/archive_virtual.c vendor/libarchive/dist/libarchive/archive_write.c vendor/libarchive/dist/libarchive/archive_write_disk_posix.c vendor/libarchive/dist/libarchive/archive_write_disk_windows.c vendor/libarchive/dist/libarchive/archive_write_set_format_7zip.c vendor/libarchive/dist/libarchive/test/read_open_memory.c vendor/libarchive/dist/libarchive/test/test.h vendor/libarchive/dist/libarchive/test/test_acl_platform_nfs4.c vendor/libarchive/dist/libarchive/test/test_compat_zip.c vendor/libarchive/dist/libarchive/test/test_read_format_zip.c vendor/libarchive/dist/libarchive/test/test_write_disk_perms.c vendor/libarchive/dist/tar/test/test_option_acls.c Modified: vendor/libarchive/dist/CMakeLists.txt ============================================================================== --- vendor/libarchive/dist/CMakeLists.txt Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/CMakeLists.txt Wed Jan 24 11:01:12 2018 (r328323) @@ -179,6 +179,7 @@ include(CTest) OPTION(ENABLE_NETTLE "Enable use of Nettle" ON) OPTION(ENABLE_OPENSSL "Enable use of OpenSSL" ON) +OPTION(ENABLE_LZ4 "Enable the use of the system LZ4 library if found" ON) OPTION(ENABLE_LZO "Enable the use of the system LZO library if found" OFF) OPTION(ENABLE_LZMA "Enable the use of the system LZMA library if found" ON) @@ -509,15 +510,19 @@ MARK_AS_ADVANCED(CLEAR LZO2_LIBRARY) # # Find LZ4 # -IF (LZ4_INCLUDE_DIR) - # Already in cache, be silent - SET(LZ4_FIND_QUIETLY TRUE) -ENDIF (LZ4_INCLUDE_DIR) +IF(ENABLE_LZ4) + IF (LZ4_INCLUDE_DIR) + # Already in cache, be silent + SET(LZ4_FIND_QUIETLY TRUE) + ENDIF (LZ4_INCLUDE_DIR) -FIND_PATH(LZ4_INCLUDE_DIR lz4.h) -FIND_LIBRARY(LZ4_LIBRARY NAMES lz4 liblz4) -INCLUDE(FindPackageHandleStandardArgs) -FIND_PACKAGE_HANDLE_STANDARD_ARGS(LZ4 DEFAULT_MSG LZ4_LIBRARY LZ4_INCLUDE_DIR) + FIND_PATH(LZ4_INCLUDE_DIR lz4.h) + FIND_LIBRARY(LZ4_LIBRARY NAMES lz4 liblz4) + INCLUDE(FindPackageHandleStandardArgs) + FIND_PACKAGE_HANDLE_STANDARD_ARGS(LZ4 DEFAULT_MSG LZ4_LIBRARY LZ4_INCLUDE_DIR) +ELSE(ENABLE_LZ4) + SET(LZ4_FOUND FALSE) # Override cached value +ENDIF(ENABLE_LZ4) IF(LZ4_FOUND) SET(HAVE_LIBLZ4 1) SET(HAVE_LZ4_H 1) Modified: vendor/libarchive/dist/Makefile.am ============================================================================== --- vendor/libarchive/dist/Makefile.am Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/Makefile.am Wed Jan 24 11:01:12 2018 (r328323) @@ -681,6 +681,7 @@ libarchive_test_EXTRA_DIST=\ libarchive/test/test_compat_zip_5.zip.uu \ libarchive/test/test_compat_zip_6.zip.uu \ libarchive/test/test_compat_zip_7.xps.uu \ + libarchive/test/test_compat_zip_8.zip.uu \ libarchive/test/test_compat_zstd_1.tar.zst.uu \ libarchive/test/test_fuzz.cab.uu \ libarchive/test/test_fuzz.lzh.uu \ @@ -1282,6 +1283,7 @@ bsdcat_test_SOURCES= \ cat/test/test_expand_xz.c \ cat/test/test_expand_zstd.c \ cat/test/test_help.c \ + cat/test/test_stdin.c \ cat/test/test_version.c bsdcat_test_CPPFLAGS= \ Modified: vendor/libarchive/dist/cat/bsdcat.c ============================================================================== --- vendor/libarchive/dist/cat/bsdcat.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/cat/bsdcat.c Wed Jan 24 11:01:12 2018 (r328323) @@ -70,6 +70,12 @@ version(void) void bsdcat_next(void) { + if (a != NULL) { + if (archive_read_close(a) != ARCHIVE_OK) + bsdcat_print_error(); + archive_read_free(a); + } + a = archive_read_new(); archive_read_support_filter_all(a); archive_read_support_format_empty(a); @@ -100,8 +106,10 @@ bsdcat_read_to_stdout(const char* filename) ; else if (archive_read_data_into_fd(a, 1) != ARCHIVE_OK) bsdcat_print_error(); - if (archive_read_free(a) != ARCHIVE_OK) + if (archive_read_close(a) != ARCHIVE_OK) bsdcat_print_error(); + archive_read_free(a); + a = NULL; } int @@ -135,15 +143,14 @@ main(int argc, char **argv) if (*bsdcat->argv == NULL) { bsdcat_current_path = ""; bsdcat_read_to_stdout(NULL); - } else + } else { while (*bsdcat->argv) { bsdcat_current_path = *bsdcat->argv++; bsdcat_read_to_stdout(bsdcat_current_path); bsdcat_next(); } - - if (a != NULL) - archive_read_free(a); + archive_read_free(a); /* Help valgrind & friends */ + } exit(exit_status); } Modified: vendor/libarchive/dist/cat/test/CMakeLists.txt ============================================================================== --- vendor/libarchive/dist/cat/test/CMakeLists.txt Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/cat/test/CMakeLists.txt Wed Jan 24 11:01:12 2018 (r328323) @@ -24,6 +24,7 @@ IF(ENABLE_CAT AND ENABLE_TEST) test_expand_xz.c test_expand_zstd.c test_help.c + test_stdin.c test_version.c ) Modified: vendor/libarchive/dist/libarchive/archive_acl.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_acl.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_acl.c Wed Jan 24 11:01:12 2018 (r328323) @@ -1159,6 +1159,7 @@ archive_acl_from_text_w(struct archive_acl *acl, const switch (want_type) { case ARCHIVE_ENTRY_ACL_TYPE_POSIX1E: want_type = ARCHIVE_ENTRY_ACL_TYPE_ACCESS; + __LA_FALLTHROUGH; case ARCHIVE_ENTRY_ACL_TYPE_ACCESS: case ARCHIVE_ENTRY_ACL_TYPE_DEFAULT: numfields = 5; @@ -1626,6 +1627,7 @@ archive_acl_from_text_l(struct archive_acl *acl, const switch (want_type) { case ARCHIVE_ENTRY_ACL_TYPE_POSIX1E: want_type = ARCHIVE_ENTRY_ACL_TYPE_ACCESS; + __LA_FALLTHROUGH; case ARCHIVE_ENTRY_ACL_TYPE_ACCESS: case ARCHIVE_ENTRY_ACL_TYPE_DEFAULT: numfields = 5; Modified: vendor/libarchive/dist/libarchive/archive_disk_acl_freebsd.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_disk_acl_freebsd.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_disk_acl_freebsd.c Wed Jan 24 11:01:12 2018 (r328323) @@ -93,7 +93,9 @@ static const acl_perm_map_t acl_nfs4_flag_map[] = { {ARCHIVE_ENTRY_ACL_ENTRY_INHERIT_ONLY, ACL_ENTRY_INHERIT_ONLY}, {ARCHIVE_ENTRY_ACL_ENTRY_SUCCESSFUL_ACCESS, ACL_ENTRY_SUCCESSFUL_ACCESS}, {ARCHIVE_ENTRY_ACL_ENTRY_FAILED_ACCESS, ACL_ENTRY_FAILED_ACCESS}, +#ifdef ACL_ENTRY_INHERITED {ARCHIVE_ENTRY_ACL_ENTRY_INHERITED, ACL_ENTRY_INHERITED} +#endif }; static const int acl_nfs4_flag_map_size = Modified: vendor/libarchive/dist/libarchive/archive_match.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_match.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_match.c Wed Jan 24 11:01:12 2018 (r328323) @@ -1582,7 +1582,7 @@ time_excluded(struct archive_match *a, struct archive_ */ int -archive_match_include_uid(struct archive *_a, int64_t uid) +archive_match_include_uid(struct archive *_a, la_int64_t uid) { struct archive_match *a; @@ -1593,7 +1593,7 @@ archive_match_include_uid(struct archive *_a, int64_t } int -archive_match_include_gid(struct archive *_a, int64_t gid) +archive_match_include_gid(struct archive *_a, la_int64_t gid) { struct archive_match *a; Modified: vendor/libarchive/dist/libarchive/archive_platform.h ============================================================================== --- vendor/libarchive/dist/libarchive/archive_platform.h Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_platform.h Wed Jan 24 11:01:12 2018 (r328323) @@ -191,4 +191,10 @@ #define ARCHIVE_ERRNO_MISC (-1) #endif +#if defined(__GNUC__) && (__GNUC__ >= 7) +#define __LA_FALLTHROUGH __attribute__((fallthrough)) +#else +#define __LA_FALLTHROUGH +#endif + #endif /* !ARCHIVE_PLATFORM_H_INCLUDED */ Modified: vendor/libarchive/dist/libarchive/archive_ppmd7.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_ppmd7.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_ppmd7.c Wed Jan 24 11:01:12 2018 (r328323) @@ -115,14 +115,14 @@ static void Ppmd7_Construct(CPpmd7 *p) memset(p->HB2Flag + 0x40, 8, 0x100 - 0x40); } -static void Ppmd7_Free(CPpmd7 *p, ISzAlloc *alloc) +static void Ppmd7_Free(CPpmd7 *p) { - alloc->Free(alloc, p->Base); + free(p->Base); p->Size = 0; p->Base = 0; } -static Bool Ppmd7_Alloc(CPpmd7 *p, UInt32 size, ISzAlloc *alloc) +static Bool Ppmd7_Alloc(CPpmd7 *p, UInt32 size) { if (p->Base == 0 || p->Size != size) { @@ -131,14 +131,14 @@ static Bool Ppmd7_Alloc(CPpmd7 *p, UInt32 size, ISzAll if (size < UNIT_SIZE) { return False; } - Ppmd7_Free(p, alloc); + Ppmd7_Free(p); p->AlignOffset = #ifdef PPMD_32BIT (4 - size) & 3; #else 4 - (size & 3); #endif - if ((p->Base = (Byte *)alloc->Alloc(alloc, p->AlignOffset + size + if ((p->Base = (Byte *)malloc(p->AlignOffset + size #ifndef PPMD_32BIT + UNIT_SIZE #endif Modified: vendor/libarchive/dist/libarchive/archive_ppmd7_private.h ============================================================================== --- vendor/libarchive/dist/libarchive/archive_ppmd7_private.h Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_ppmd7_private.h Wed Jan 24 11:01:12 2018 (r328323) @@ -95,8 +95,8 @@ typedef struct { /* Base Functions */ void (*Ppmd7_Construct)(CPpmd7 *p); - Bool (*Ppmd7_Alloc)(CPpmd7 *p, UInt32 size, ISzAlloc *alloc); - void (*Ppmd7_Free)(CPpmd7 *p, ISzAlloc *alloc); + Bool (*Ppmd7_Alloc)(CPpmd7 *p, UInt32 size); + void (*Ppmd7_Free)(CPpmd7 *p); void (*Ppmd7_Init)(CPpmd7 *p, unsigned maxOrder); #define Ppmd7_WasAllocated(p) ((p)->Base != NULL) Modified: vendor/libarchive/dist/libarchive/archive_ppmd_private.h ============================================================================== --- vendor/libarchive/dist/libarchive/archive_ppmd_private.h Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_ppmd_private.h Wed Jan 24 11:01:12 2018 (r328323) @@ -69,13 +69,6 @@ typedef struct void (*Write)(void *p, Byte b); } IByteOut; - -typedef struct -{ - void *(*Alloc)(void *p, size_t size); - void (*Free)(void *p, void *address); /* address can be 0 */ -} ISzAlloc; - /*** End defined in Types.h ***/ /*** Begin defined in CpuArch.h ***/ Modified: vendor/libarchive/dist/libarchive/archive_read.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read.c Wed Jan 24 11:01:12 2018 (r328323) @@ -120,7 +120,8 @@ archive_read_new(void) * Record the do-not-extract-to file. This belongs in archive_read_extract.c. */ void -archive_read_extract_set_skip_file(struct archive *_a, int64_t d, int64_t i) +archive_read_extract_set_skip_file(struct archive *_a, la_int64_t d, + la_int64_t i) { struct archive_read *a = (struct archive_read *)_a; @@ -747,7 +748,7 @@ choose_format(struct archive_read *a) * Return the file offset (within the uncompressed data stream) where * the last header started. */ -int64_t +la_int64_t archive_read_header_position(struct archive *_a) { struct archive_read *a = (struct archive_read *)_a; @@ -943,7 +944,7 @@ archive_read_data_skip(struct archive *_a) return (r); } -int64_t +la_int64_t archive_seek_data(struct archive *_a, int64_t offset, int whence) { struct archive_read *a = (struct archive_read *)_a; @@ -1626,7 +1627,8 @@ __archive_read_filter_seek(struct archive_read_filter switch (whence) { case SEEK_CUR: /* Adjust the offset and use SEEK_SET instead */ - offset += filter->position; + offset += filter->position; + __LA_FALLTHROUGH; case SEEK_SET: cursor = 0; while (1) Modified: vendor/libarchive/dist/libarchive/archive_read_disk_posix.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_disk_posix.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_disk_posix.c Wed Jan 24 11:01:12 2018 (r328323) @@ -387,7 +387,7 @@ archive_read_disk_vtable(void) } const char * -archive_read_disk_gname(struct archive *_a, int64_t gid) +archive_read_disk_gname(struct archive *_a, la_int64_t gid) { struct archive_read_disk *a = (struct archive_read_disk *)_a; if (ARCHIVE_OK != __archive_check_magic(_a, ARCHIVE_READ_DISK_MAGIC, @@ -399,7 +399,7 @@ archive_read_disk_gname(struct archive *_a, int64_t gi } const char * -archive_read_disk_uname(struct archive *_a, int64_t uid) +archive_read_disk_uname(struct archive *_a, la_int64_t uid) { struct archive_read_disk *a = (struct archive_read_disk *)_a; if (ARCHIVE_OK != __archive_check_magic(_a, ARCHIVE_READ_DISK_MAGIC, @@ -413,7 +413,7 @@ archive_read_disk_uname(struct archive *_a, int64_t ui int archive_read_disk_set_gname_lookup(struct archive *_a, void *private_data, - const char * (*lookup_gname)(void *private, int64_t gid), + const char * (*lookup_gname)(void *private, la_int64_t gid), void (*cleanup_gname)(void *private)) { struct archive_read_disk *a = (struct archive_read_disk *)_a; @@ -432,7 +432,7 @@ archive_read_disk_set_gname_lookup(struct archive *_a, int archive_read_disk_set_uname_lookup(struct archive *_a, void *private_data, - const char * (*lookup_uname)(void *private, int64_t uid), + const char * (*lookup_uname)(void *private, la_int64_t uid), void (*cleanup_uname)(void *private)) { struct archive_read_disk *a = (struct archive_read_disk *)_a; Modified: vendor/libarchive/dist/libarchive/archive_read_disk_windows.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_disk_windows.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_disk_windows.c Wed Jan 24 11:01:12 2018 (r328323) @@ -320,7 +320,7 @@ archive_read_disk_vtable(void) } const char * -archive_read_disk_gname(struct archive *_a, int64_t gid) +archive_read_disk_gname(struct archive *_a, la_int64_t gid) { struct archive_read_disk *a = (struct archive_read_disk *)_a; if (ARCHIVE_OK != __archive_check_magic(_a, ARCHIVE_READ_DISK_MAGIC, @@ -332,7 +332,7 @@ archive_read_disk_gname(struct archive *_a, int64_t gi } const char * -archive_read_disk_uname(struct archive *_a, int64_t uid) +archive_read_disk_uname(struct archive *_a, la_int64_t uid) { struct archive_read_disk *a = (struct archive_read_disk *)_a; if (ARCHIVE_OK != __archive_check_magic(_a, ARCHIVE_READ_DISK_MAGIC, @@ -346,7 +346,7 @@ archive_read_disk_uname(struct archive *_a, int64_t ui int archive_read_disk_set_gname_lookup(struct archive *_a, void *private_data, - const char * (*lookup_gname)(void *private, int64_t gid), + const char * (*lookup_gname)(void *private, la_int64_t gid), void (*cleanup_gname)(void *private)) { struct archive_read_disk *a = (struct archive_read_disk *)_a; @@ -923,6 +923,7 @@ next_entry(struct archive_read_disk *a, struct tree *t t->entry_fh = CreateFileW(tree_current_access_path(t), GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, flags, NULL); if (t->entry_fh == INVALID_HANDLE_VALUE) { + la_dosmaperr(GetLastError()); archive_set_error(&a->archive, errno, "Couldn't open %ls", tree_current_path(a->tree)); return (ARCHIVE_FAILED); Modified: vendor/libarchive/dist/libarchive/archive_read_support_format_7zip.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_support_format_7zip.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_support_format_7zip.c Wed Jan 24 11:01:12 2018 (r328323) @@ -975,18 +975,6 @@ decode_codec_id(const unsigned char *codecId, size_t i return (id); } -static void * -ppmd_alloc(void *p, size_t size) -{ - (void)p; - return malloc(size); -} -static void -ppmd_free(void *p, void *address) -{ - (void)p; - free(address); -} static Byte ppmd_read(void *p) { @@ -1006,8 +994,6 @@ ppmd_read(void *p) return (b); } -static ISzAlloc g_szalloc = { ppmd_alloc, ppmd_free }; - static int init_decompression(struct archive_read *a, struct _7zip *zip, const struct _7z_coder *coder1, const struct _7z_coder *coder2) @@ -1237,7 +1223,7 @@ init_decompression(struct archive_read *a, struct _7zi if (zip->ppmd7_valid) { __archive_ppmd7_functions.Ppmd7_Free( - &zip->ppmd7_context, &g_szalloc); + &zip->ppmd7_context); zip->ppmd7_valid = 0; } @@ -1256,7 +1242,7 @@ init_decompression(struct archive_read *a, struct _7zi } __archive_ppmd7_functions.Ppmd7_Construct(&zip->ppmd7_context); r = __archive_ppmd7_functions.Ppmd7_Alloc( - &zip->ppmd7_context, msize, &g_szalloc); + &zip->ppmd7_context, msize); if (r == 0) { archive_set_error(&a->archive, ENOMEM, "Coludn't allocate memory for PPMd"); @@ -1636,7 +1622,7 @@ free_decompression(struct archive_read *a, struct _7zi #endif if (zip->ppmd7_valid) { __archive_ppmd7_functions.Ppmd7_Free( - &zip->ppmd7_context, &g_szalloc); + &zip->ppmd7_context); zip->ppmd7_valid = 0; } return (r); @@ -2569,6 +2555,7 @@ read_Header(struct archive_read *a, struct _7z_header_ case kDummy: if (ll == 0) break; + __LA_FALLTHROUGH; default: if (header_bytes(a, ll) == NULL) return (-1); Modified: vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_support_format_mtree.c Wed Jan 24 11:01:12 2018 (r328323) @@ -1499,6 +1499,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt } if (strcmp(key, "cksum") == 0) break; + __LA_FALLTHROUGH; case 'd': if (strcmp(key, "device") == 0) { /* stat(2) st_rdev field, e.g. the major/minor IDs @@ -1512,12 +1513,14 @@ parse_keyword(struct archive_read *a, struct mtree *mt archive_entry_set_rdev(entry, dev); return r; } + __LA_FALLTHROUGH; case 'f': if (strcmp(key, "flags") == 0) { *parsed_kws |= MTREE_HAS_FFLAGS; archive_entry_copy_fflags_text(entry, val); break; } + __LA_FALLTHROUGH; case 'g': if (strcmp(key, "gid") == 0) { *parsed_kws |= MTREE_HAS_GID; @@ -1529,16 +1532,19 @@ parse_keyword(struct archive_read *a, struct mtree *mt archive_entry_copy_gname(entry, val); break; } + __LA_FALLTHROUGH; case 'i': if (strcmp(key, "inode") == 0) { archive_entry_set_ino(entry, mtree_atol(&val, 10)); break; } + __LA_FALLTHROUGH; case 'l': if (strcmp(key, "link") == 0) { archive_entry_copy_symlink(entry, val); break; } + __LA_FALLTHROUGH; case 'm': if (strcmp(key, "md5") == 0 || strcmp(key, "md5digest") == 0) break; @@ -1555,6 +1561,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt } break; } + __LA_FALLTHROUGH; case 'n': if (strcmp(key, "nlink") == 0) { *parsed_kws |= MTREE_HAS_NLINK; @@ -1562,6 +1569,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt (unsigned int)mtree_atol(&val, 10)); break; } + __LA_FALLTHROUGH; case 'r': if (strcmp(key, "resdevice") == 0) { /* stat(2) st_dev field, e.g. the device ID where the @@ -1577,6 +1585,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt if (strcmp(key, "rmd160") == 0 || strcmp(key, "rmd160digest") == 0) break; + __LA_FALLTHROUGH; case 's': if (strcmp(key, "sha1") == 0 || strcmp(key, "sha1digest") == 0) break; @@ -1593,6 +1602,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt archive_entry_set_size(entry, mtree_atol(&val, 10)); break; } + __LA_FALLTHROUGH; case 't': if (strcmp(key, "tags") == 0) { /* @@ -1635,18 +1645,21 @@ parse_keyword(struct archive_read *a, struct mtree *mt archive_entry_set_filetype(entry, AE_IFBLK); break; } + __LA_FALLTHROUGH; case 'c': if (strcmp(val, "char") == 0) { archive_entry_set_filetype(entry, AE_IFCHR); break; } + __LA_FALLTHROUGH; case 'd': if (strcmp(val, "dir") == 0) { archive_entry_set_filetype(entry, AE_IFDIR); break; } + __LA_FALLTHROUGH; case 'f': if (strcmp(val, "fifo") == 0) { archive_entry_set_filetype(entry, @@ -1658,12 +1671,14 @@ parse_keyword(struct archive_read *a, struct mtree *mt AE_IFREG); break; } + __LA_FALLTHROUGH; case 'l': if (strcmp(val, "link") == 0) { archive_entry_set_filetype(entry, AE_IFLNK); break; } + __LA_FALLTHROUGH; default: archive_set_error(&a->archive, ARCHIVE_ERRNO_FILE_FORMAT, @@ -1675,6 +1690,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt *parsed_kws |= MTREE_HAS_TYPE; break; } + __LA_FALLTHROUGH; case 'u': if (strcmp(key, "uid") == 0) { *parsed_kws |= MTREE_HAS_UID; @@ -1686,6 +1702,7 @@ parse_keyword(struct archive_read *a, struct mtree *mt archive_entry_copy_uname(entry, val); break; } + __LA_FALLTHROUGH; default: archive_set_error(&a->archive, ARCHIVE_ERRNO_FILE_FORMAT, "Unrecognized key %s=%s", key, val); Modified: vendor/libarchive/dist/libarchive/archive_read_support_format_rar.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_support_format_rar.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_support_format_rar.c Wed Jan 24 11:01:12 2018 (r328323) @@ -604,20 +604,6 @@ lzss_emit_match(struct rar *rar, int offset, int lengt rar->lzss.position += length; } -static void * -ppmd_alloc(void *p, size_t size) -{ - (void)p; - return malloc(size); -} -static void -ppmd_free(void *p, void *address) -{ - (void)p; - free(address); -} -static ISzAlloc g_szalloc = { ppmd_alloc, ppmd_free }; - static Byte ppmd_read(void *p) { @@ -1038,7 +1024,7 @@ archive_read_format_rar_read_data(struct archive_read case COMPRESS_METHOD_BEST: ret = read_data_compressed(a, buff, size, offset); if (ret != ARCHIVE_OK && ret != ARCHIVE_WARN) - __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context, &g_szalloc); + __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context); break; default: @@ -1253,7 +1239,7 @@ archive_read_format_rar_cleanup(struct archive_read *a free(rar->dbo); free(rar->unp_buffer); free(rar->lzss.window); - __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context, &g_szalloc); + __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context); free(rar); (a->format->data) = NULL; return (ARCHIVE_OK); @@ -1658,7 +1644,7 @@ read_header(struct archive_read *a, struct archive_ent rar->unp_offset = 0; rar->unp_buffer_size = UNP_BUFFER_SIZE; memset(rar->lengthtable, 0, sizeof(rar->lengthtable)); - __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context, &g_szalloc); + __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context); rar->ppmd_valid = rar->ppmd_eod = 0; /* Don't set any archive entries for non-file header types */ @@ -2122,7 +2108,7 @@ parse_codes(struct archive_read *a) /* Make sure ppmd7_contest is freed before Ppmd7_Construct * because reading a broken file cause this abnormal sequence. */ - __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context, &g_szalloc); + __archive_ppmd7_functions.Ppmd7_Free(&rar->ppmd7_context); rar->bytein.a = a; rar->bytein.Read = &ppmd_read; @@ -2137,7 +2123,7 @@ parse_codes(struct archive_read *a) } if (!__archive_ppmd7_functions.Ppmd7_Alloc(&rar->ppmd7_context, - rar->dictionary_size, &g_szalloc)) + rar->dictionary_size)) { archive_set_error(&a->archive, ENOMEM, "Out of memory"); Modified: vendor/libarchive/dist/libarchive/archive_read_support_format_tar.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_support_format_tar.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_support_format_tar.c Wed Jan 24 11:01:12 2018 (r328323) @@ -251,15 +251,15 @@ archive_read_support_format_tar(struct archive *_a) ARCHIVE_STATE_NEW, "archive_read_support_format_tar"); tar = (struct tar *)calloc(1, sizeof(*tar)); -#ifdef HAVE_COPYFILE_H - /* Set this by default on Mac OS. */ - tar->process_mac_extensions = 1; -#endif if (tar == NULL) { archive_set_error(&a->archive, ENOMEM, "Can't allocate tar data"); return (ARCHIVE_FATAL); } +#ifdef HAVE_COPYFILE_H + /* Set this by default on Mac OS. */ + tar->process_mac_extensions = 1; +#endif r = __archive_read_register_format(a, tar, "tar", archive_read_format_tar_bid, Modified: vendor/libarchive/dist/libarchive/archive_read_support_format_zip.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_read_support_format_zip.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_read_support_format_zip.c Wed Jan 24 11:01:12 2018 (r328323) @@ -886,6 +886,24 @@ zip_read_local_file_header(struct archive_read *a, str zip_entry->mode |= 0664; } + /* Windows archivers sometimes use backslash as the directory separator. + Normalize to slash. */ + if (zip_entry->system == 0 && + (wp = archive_entry_pathname_w(entry)) != NULL) { + if (wcschr(wp, L'/') == NULL && wcschr(wp, L'\\') != NULL) { + size_t i; + struct archive_wstring s; + archive_string_init(&s); + archive_wstrcpy(&s, wp); + for (i = 0; i < archive_strlen(&s); i++) { + if (s.s[i] == '\\') + s.s[i] = '/'; + } + archive_entry_copy_pathname_w(entry, s.s); + archive_wstring_free(&s); + } + } + /* Make sure that entries with a trailing '/' are marked as directories * even if the External File Attributes contains bogus values. If this * is not a directory and there is no type, assume regularfile. */ @@ -1061,6 +1079,7 @@ zip_read_local_file_header(struct archive_read *a, str zip->end_of_entry = 1; /* Set up a more descriptive format name. */ + archive_string_empty(&zip->format_name); archive_string_sprintf(&zip->format_name, "ZIP %d.%d (%s)", version / 10, version % 10, compression_name(zip->entry->compression)); Modified: vendor/libarchive/dist/libarchive/archive_util.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_util.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_util.c Wed Jan 24 11:01:12 2018 (r328323) @@ -140,7 +140,7 @@ archive_compression_name(struct archive *a) /* * Return a count of the number of compressed bytes processed. */ -int64_t +la_int64_t archive_position_compressed(struct archive *a) { return archive_filter_bytes(a, -1); @@ -149,7 +149,7 @@ archive_position_compressed(struct archive *a) /* * Return a count of the number of uncompressed bytes processed. */ -int64_t +la_int64_t archive_position_uncompressed(struct archive *a) { return archive_filter_bytes(a, 0); Modified: vendor/libarchive/dist/libarchive/archive_virtual.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_virtual.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_virtual.c Wed Jan 24 11:01:12 2018 (r328323) @@ -48,7 +48,7 @@ archive_filter_name(struct archive *a, int n) return ((a->vtable->archive_filter_name)(a, n)); } -int64_t +la_int64_t archive_filter_bytes(struct archive *a, int n) { return ((a->vtable->archive_filter_bytes)(a, n)); @@ -131,7 +131,8 @@ archive_write_data(struct archive *a, const void *buff } ssize_t -archive_write_data_block(struct archive *a, const void *buff, size_t s, int64_t o) +archive_write_data_block(struct archive *a, const void *buff, size_t s, + la_int64_t o) { if (a->vtable->archive_write_data_block == NULL) { archive_set_error(a, ARCHIVE_ERRNO_MISC, @@ -156,7 +157,7 @@ archive_read_next_header2(struct archive *a, struct ar int archive_read_data_block(struct archive *a, - const void **buff, size_t *s, int64_t *o) + const void **buff, size_t *s, la_int64_t *o) { return ((a->vtable->archive_read_data_block)(a, buff, s, o)); } Modified: vendor/libarchive/dist/libarchive/archive_write.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_write.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_write.c Wed Jan 24 11:01:12 2018 (r328323) @@ -190,7 +190,7 @@ archive_write_get_bytes_in_last_block(struct archive * * an archive to itself recursively. */ int -archive_write_set_skip_file(struct archive *_a, int64_t d, int64_t i) +archive_write_set_skip_file(struct archive *_a, la_int64_t d, la_int64_t i) { struct archive_write *a = (struct archive_write *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_MAGIC, Modified: vendor/libarchive/dist/libarchive/archive_write_disk_posix.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_write_disk_posix.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_write_disk_posix.c Wed Jan 24 11:01:12 2018 (r328323) @@ -835,7 +835,7 @@ _archive_write_disk_header(struct archive *_a, struct } int -archive_write_disk_set_skip_file(struct archive *_a, int64_t d, int64_t i) +archive_write_disk_set_skip_file(struct archive *_a, la_int64_t d, la_int64_t i) { struct archive_write_disk *a = (struct archive_write_disk *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, @@ -1786,7 +1786,7 @@ finish_metadata: int archive_write_disk_set_group_lookup(struct archive *_a, void *private_data, - int64_t (*lookup_gid)(void *private, const char *gname, int64_t gid), + la_int64_t (*lookup_gid)(void *private, const char *gname, la_int64_t gid), void (*cleanup_gid)(void *private)) { struct archive_write_disk *a = (struct archive_write_disk *)_a; @@ -1822,7 +1822,7 @@ archive_write_disk_set_user_lookup(struct archive *_a, } int64_t -archive_write_disk_gid(struct archive *_a, const char *name, int64_t id) +archive_write_disk_gid(struct archive *_a, const char *name, la_int64_t id) { struct archive_write_disk *a = (struct archive_write_disk *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, @@ -1833,7 +1833,7 @@ archive_write_disk_gid(struct archive *_a, const char } int64_t -archive_write_disk_uid(struct archive *_a, const char *name, int64_t id) +archive_write_disk_uid(struct archive *_a, const char *name, la_int64_t id) { struct archive_write_disk *a = (struct archive_write_disk *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, @@ -1981,6 +1981,10 @@ restore_entry(struct archive_write_disk *a) if ((en == EISDIR || en == EEXIST) && (a->flags & ARCHIVE_EXTRACT_NO_OVERWRITE)) { /* If we're not overwriting, we're done. */ + if (S_ISDIR(a->mode)) { + /* Don't overwrite any settings on existing directories. */ + a->todo = 0; + } archive_entry_unset_size(a->entry); return (ARCHIVE_OK); } Modified: vendor/libarchive/dist/libarchive/archive_write_disk_windows.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_write_disk_windows.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_write_disk_windows.c Wed Jan 24 11:01:12 2018 (r328323) @@ -906,7 +906,7 @@ _archive_write_disk_header(struct archive *_a, struct } int -archive_write_disk_set_skip_file(struct archive *_a, int64_t d, int64_t i) +archive_write_disk_set_skip_file(struct archive *_a, la_int64_t d, la_int64_t i) { struct archive_write_disk *a = (struct archive_write_disk *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, @@ -1156,7 +1156,7 @@ _archive_write_disk_finish_entry(struct archive *_a) int archive_write_disk_set_group_lookup(struct archive *_a, void *private_data, - int64_t (*lookup_gid)(void *private, const char *gname, int64_t gid), + la_int64_t (*lookup_gid)(void *private, const char *gname, la_int64_t gid), void (*cleanup_gid)(void *private)) { struct archive_write_disk *a = (struct archive_write_disk *)_a; @@ -1192,7 +1192,7 @@ archive_write_disk_set_user_lookup(struct archive *_a, } int64_t -archive_write_disk_gid(struct archive *_a, const char *name, int64_t id) +archive_write_disk_gid(struct archive *_a, const char *name, la_int64_t id) { struct archive_write_disk *a = (struct archive_write_disk *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, @@ -1203,7 +1203,7 @@ archive_write_disk_gid(struct archive *_a, const char } int64_t -archive_write_disk_uid(struct archive *_a, const char *name, int64_t id) +archive_write_disk_uid(struct archive *_a, const char *name, la_int64_t id) { struct archive_write_disk *a = (struct archive_write_disk *)_a; archive_check_magic(&a->archive, ARCHIVE_WRITE_DISK_MAGIC, @@ -1325,6 +1325,10 @@ restore_entry(struct archive_write_disk *a) if ((en == EISDIR || en == EEXIST) && (a->flags & ARCHIVE_EXTRACT_NO_OVERWRITE)) { /* If we're not overwriting, we're done. */ + if (S_ISDIR(a->mode)) { + /* Don't overwrite any settings on existing directories. */ + a->todo = 0; + } archive_entry_unset_size(a->entry); return (ARCHIVE_OK); } Modified: vendor/libarchive/dist/libarchive/archive_write_set_format_7zip.c ============================================================================== --- vendor/libarchive/dist/libarchive/archive_write_set_format_7zip.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/archive_write_set_format_7zip.c Wed Jan 24 11:01:12 2018 (r328323) @@ -2095,20 +2095,7 @@ compression_init_encoder_lzma2(struct archive *a, /* * _7_PPMD compressor. */ -static void * -ppmd_alloc(void *p, size_t size) -{ - (void)p; - return malloc(size); -} static void -ppmd_free(void *p, void *address) -{ - (void)p; - free(address); -} -static ISzAlloc g_szalloc = { ppmd_alloc, ppmd_free }; -static void ppmd_write(void *p, Byte b) { struct archive_write *a = ((IByteOut *)p)->a; @@ -2167,7 +2154,7 @@ compression_init_encoder_ppmd(struct archive *a, archive_le32enc(props+1, msize); __archive_ppmd7_functions.Ppmd7_Construct(&strm->ppmd7_context); r = __archive_ppmd7_functions.Ppmd7_Alloc( - &strm->ppmd7_context, msize, &g_szalloc); + &strm->ppmd7_context, msize); if (r == 0) { free(strm->buff); free(strm); @@ -2243,7 +2230,7 @@ compression_end_ppmd(struct archive *a, struct la_zstr (void)a; /* UNUSED */ strm = (struct ppmd_stream *)lastrm->real_stream; - __archive_ppmd7_functions.Ppmd7_Free(&strm->ppmd7_context, &g_szalloc); + __archive_ppmd7_functions.Ppmd7_Free(&strm->ppmd7_context); free(strm->buff); free(strm); lastrm->real_stream = NULL; Modified: vendor/libarchive/dist/libarchive/test/read_open_memory.c ============================================================================== --- vendor/libarchive/dist/libarchive/test/read_open_memory.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/test/read_open_memory.c Wed Jan 24 11:01:12 2018 (r328323) @@ -91,9 +91,11 @@ read_open_memory_internal(struct archive *a, const voi switch (level) { case 3: archive_read_set_seek_callback(a, memory_read_seek); + __LA_FALLTHROUGH; case 2: archive_read_set_open_callback(a, memory_read_open); archive_read_set_skip_callback(a, memory_read_skip); + __LA_FALLTHROUGH; case 1: mine = malloc(sizeof(*mine)); if (mine == NULL) { Modified: vendor/libarchive/dist/libarchive/test/test.h ============================================================================== --- vendor/libarchive/dist/libarchive/test/test.h Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/test/test.h Wed Jan 24 11:01:12 2018 (r328323) @@ -33,4 +33,10 @@ #define EXTRA_ERRNO(x) archive_errno((struct archive *)(x)) #define EXTRA_VERSION archive_version_details() +#if defined(__GNUC__) && (__GNUC__ >= 7) +#define __LA_FALLTHROUGH __attribute__((fallthrough)) +#else +#define __LA_FALLTHROUGH +#endif + #include "test_common.h" Modified: vendor/libarchive/dist/libarchive/test/test_acl_platform_nfs4.c ============================================================================== --- vendor/libarchive/dist/libarchive/test/test_acl_platform_nfs4.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/test/test_acl_platform_nfs4.c Wed Jan 24 11:01:12 2018 (r328323) @@ -408,7 +408,9 @@ acl_flagset_to_bitmap(acl_flagset_t opaque_fs) {ARCHIVE_ENTRY_ACL_ENTRY_INHERIT_ONLY, RICHACE_INHERIT_ONLY_ACE}, {ARCHIVE_ENTRY_ACL_ENTRY_INHERITED, RICHACE_INHERITED_ACE} #else /* FreeBSD NFSv4 ACL inheritance flags */ +#ifdef ACL_ENTRY_INHERITED {ARCHIVE_ENTRY_ACL_ENTRY_INHERITED, ACL_ENTRY_INHERITED}, +#endif {ARCHIVE_ENTRY_ACL_ENTRY_FILE_INHERIT, ACL_ENTRY_FILE_INHERIT}, {ARCHIVE_ENTRY_ACL_ENTRY_DIRECTORY_INHERIT, ACL_ENTRY_DIRECTORY_INHERIT}, {ARCHIVE_ENTRY_ACL_ENTRY_NO_PROPAGATE_INHERIT, ACL_ENTRY_NO_PROPAGATE_INHERIT}, Modified: vendor/libarchive/dist/libarchive/test/test_compat_zip.c ============================================================================== --- vendor/libarchive/dist/libarchive/test/test_compat_zip.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/test/test_compat_zip.c Wed Jan 24 11:01:12 2018 (r328323) @@ -422,3 +422,29 @@ DEFINE_TEST(test_compat_zip_7) } free(p); } + +/** + * A file with backslash path separators instead of slashes. + * PowerShell's Compress-Archive cmdlet produces such archives. + */ +DEFINE_TEST(test_compat_zip_8) +{ + const char *refname = "test_compat_zip_8.zip"; + struct archive *a; + struct archive_entry *ae; + void *p; + size_t s; + + extract_reference_file(refname); + p = slurpfile(&s, refname); + + assert((a = archive_read_new()) != NULL); + assertEqualIntA(a, ARCHIVE_OK, archive_read_support_format_zip(a)); + assertEqualIntA(a, ARCHIVE_OK, read_open_memory_minimal(a, p, s, 7)); + + assertEqualIntA(a, ARCHIVE_OK, archive_read_next_header(a, &ae)); + /* This file is in the archive as arc\test */ + assertEqualString("arc/test", archive_entry_pathname(ae)); + assertEqualIntA(a, ARCHIVE_OK, archive_read_free(a)); + free(p); +} Modified: vendor/libarchive/dist/libarchive/test/test_read_format_zip.c ============================================================================== --- vendor/libarchive/dist/libarchive/test/test_read_format_zip.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/test/test_read_format_zip.c Wed Jan 24 11:01:12 2018 (r328323) @@ -41,6 +41,7 @@ verify_basic(struct archive *a, int seek_checks) int64_t o; assertEqualIntA(a, ARCHIVE_OK, archive_read_next_header(a, &ae)); + assertEqualString("ZIP 1.0 (uncompressed)", archive_format_name(a)); assertEqualString("dir/", archive_entry_pathname(ae)); assertEqualInt(1179604249, archive_entry_mtime(ae)); assertEqualInt(0, archive_entry_size(ae)); @@ -53,6 +54,7 @@ verify_basic(struct archive *a, int seek_checks) assertEqualInt((int)s, 0); assertEqualIntA(a, ARCHIVE_OK, archive_read_next_header(a, &ae)); + assertEqualString("ZIP 2.0 (deflation)", archive_format_name(a)); assertEqualString("file1", archive_entry_pathname(ae)); assertEqualInt(1179604289, archive_entry_mtime(ae)); if (seek_checks) @@ -72,6 +74,7 @@ verify_basic(struct archive *a, int seek_checks) } assertEqualIntA(a, ARCHIVE_OK, archive_read_next_header(a, &ae)); + assertEqualString("ZIP 2.0 (deflation)", archive_format_name(a)); assertEqualString("file2", archive_entry_pathname(ae)); assertEqualInt(1179605932, archive_entry_mtime(ae)); assertEqualInt(archive_entry_is_encrypted(ae), 0); @@ -93,6 +96,7 @@ verify_basic(struct archive *a, int seek_checks) assert(archive_errno(a) != 0); } assertEqualInt(ARCHIVE_EOF, archive_read_next_header(a, &ae)); + assertEqualString("ZIP 2.0 (deflation)", archive_format_name(a)); /* Verify the number of files read. */ failure("the archive file has three files"); assertEqualInt(3, archive_file_count(a)); Modified: vendor/libarchive/dist/libarchive/test/test_write_disk_perms.c ============================================================================== --- vendor/libarchive/dist/libarchive/test/test_write_disk_perms.c Wed Jan 24 10:50:21 2018 (r328322) +++ vendor/libarchive/dist/libarchive/test/test_write_disk_perms.c Wed Jan 24 11:01:12 2018 (r328323) @@ -131,6 +131,8 @@ DEFINE_TEST(test_write_disk_perms) struct archive *a; struct archive_entry *ae; struct stat st; + uid_t original_uid; + uid_t try_to_change_uid; assertUmask(UMASK); @@ -200,6 +202,37 @@ DEFINE_TEST(test_write_disk_perms) assertEqualInt(0, stat("dir_overwrite_0744", &st)); failure("dir_overwrite_0744: st.st_mode=%o", st.st_mode); assertEqualInt(st.st_mode & 0777, 0744); + + /* For dir, the owner should get left when not overwritting. */ + assertMakeDir("dir_owner", 0744); + *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Jan 24 11:03:19 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A221ECD2A8; Wed, 24 Jan 2018 11:03:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4047F837BA; Wed, 24 Jan 2018 11:03:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1FDF31CD19; Wed, 24 Jan 2018 11:03:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OB3JOd058114; Wed, 24 Jan 2018 11:03:19 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OB3IHC058111; Wed, 24 Jan 2018 11:03:18 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201801241103.w0OB3IHC058111@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Wed, 24 Jan 2018 11:03:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328324 - in vendor/libarchive/dist: cat/test libarchive/test X-SVN-Group: vendor X-SVN-Commit-Author: mm X-SVN-Commit-Paths: in vendor/libarchive/dist: cat/test libarchive/test X-SVN-Commit-Revision: 328324 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 11:03:19 -0000 Author: mm Date: Wed Jan 24 11:03:18 2018 New Revision: 328324 URL: https://svnweb.freebsd.org/changeset/base/328324 Log: vendor/libarchive/dist: Add new files missing in r328323 Added: vendor/libarchive/dist/cat/test/test_stdin.c (contents, props changed) vendor/libarchive/dist/libarchive/test/test_compat_zip_8.zip.uu Added: vendor/libarchive/dist/cat/test/test_stdin.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/cat/test/test_stdin.c Wed Jan 24 11:03:18 2018 (r328324) @@ -0,0 +1,42 @@ +/*- + * Copyright (c) 2017 Sean Purcell + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include "test.h" + +#if !defined(_WIN32) || defined(__CYGWIN__) +#define DEV_NULL "/dev/null" +#else +#define DEV_NULL "NUL" +#endif + +DEFINE_TEST(test_stdin) +{ + int f; + + f = systemf("%s <%s >test.out 2>test.err", testprog, DEV_NULL); + assertEqualInt(0, f); + assertEmptyFile("test.out"); + assertEmptyFile("test.err"); +} + Added: vendor/libarchive/dist/libarchive/test/test_compat_zip_8.zip.uu ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libarchive/dist/libarchive/test/test_compat_zip_8.zip.uu Wed Jan 24 11:03:18 2018 (r328324) @@ -0,0 +1,6 @@ +begin 666 test_compat_zip_8.zip +M4$L#!!0````(`%A\;TOY6""D$`````X````(````87)C7'1E!BP$`4$L!`A0`%`````@`6'QO2_E8(*00````#@````@````````` +H`````````````&%R8UQT97-T4$L%!@`````!``$`-@```#8````````` +` +end From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:23:51 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E624EC3A7F; Wed, 24 Jan 2018 20:23:51 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B08FA7606D; Wed, 24 Jan 2018 20:23:50 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AB26E2278B; Wed, 24 Jan 2018 20:23:50 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKNoQL042597; Wed, 24 Jan 2018 20:23:50 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKNmj7042575; Wed, 24 Jan 2018 20:23:48 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242023.w0OKNmj7042575@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:23:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328362 - in vendor/llvm/dist-release_60: cmake/modules docs include/llvm/Analysis include/llvm/CodeGen include/llvm/MC include/llvm/Support include/llvm/Transforms/Vectorize lib/CodeGe... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/llvm/dist-release_60: cmake/modules docs include/llvm/Analysis include/llvm/CodeGen include/llvm/MC include/llvm/Support include/llvm/Transforms/Vectorize lib/CodeGen lib/CodeGen/SelectionDA... X-SVN-Commit-Revision: 328362 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:23:51 -0000 Author: dim Date: Wed Jan 24 20:23:48 2018 New Revision: 328362 URL: https://svnweb.freebsd.org/changeset/base/328362 Log: Vendor import of llvm release_60 branch r323338: https://llvm.org/svn/llvm-project/llvm/branches/release_60@323338 Added: vendor/llvm/dist-release_60/test/CodeGen/AArch64/GlobalISel/select-gv-cmodel-large.mir vendor/llvm/dist-release_60/test/CodeGen/ARM/global-merge-dllexport.ll vendor/llvm/dist-release_60/test/CodeGen/ARM/peephole-phi.mir vendor/llvm/dist-release_60/test/CodeGen/PowerPC/PR35812-neg-cmpxchg.ll vendor/llvm/dist-release_60/test/CodeGen/X86/pr35761.ll vendor/llvm/dist-release_60/test/CodeGen/X86/pr35972.ll vendor/llvm/dist-release_60/test/CodeGen/X86/pr37563.ll vendor/llvm/dist-release_60/test/ThinLTO/X86/Inputs/dicompositetype-unique2.ll vendor/llvm/dist-release_60/test/ThinLTO/X86/dicompositetype-unique2.ll vendor/llvm/dist-release_60/test/Transforms/CodeGenPrepare/X86/sink-addrmode-select.ll vendor/llvm/dist-release_60/test/Transforms/JumpThreading/ddt-crash3.ll vendor/llvm/dist-release_60/test/Transforms/JumpThreading/ddt-crash4.ll vendor/llvm/dist-release_60/test/Transforms/LoopVectorize/pr35773.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/PR35628_1.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/PR35628_2.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/PR35777.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/PR35865.ll vendor/llvm/dist-release_60/test/Transforms/StructurizeCFG/AMDGPU/ vendor/llvm/dist-release_60/test/Transforms/StructurizeCFG/AMDGPU/backedge-id-bug-xfail.ll vendor/llvm/dist-release_60/test/Transforms/StructurizeCFG/AMDGPU/backedge-id-bug.ll vendor/llvm/dist-release_60/test/Transforms/StructurizeCFG/AMDGPU/lit.local.cfg vendor/llvm/dist-release_60/test/tools/llvm-readobj/macho-needed-libs.test Modified: vendor/llvm/dist-release_60/cmake/modules/LLVMConfig.cmake.in vendor/llvm/dist-release_60/docs/ReleaseNotes.rst vendor/llvm/dist-release_60/include/llvm/Analysis/RegionInfoImpl.h vendor/llvm/dist-release_60/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h vendor/llvm/dist-release_60/include/llvm/MC/MCCodeView.h vendor/llvm/dist-release_60/include/llvm/Support/GenericDomTreeConstruction.h vendor/llvm/dist-release_60/include/llvm/Transforms/Vectorize/SLPVectorizer.h vendor/llvm/dist-release_60/lib/CodeGen/CodeGenPrepare.cpp vendor/llvm/dist-release_60/lib/CodeGen/GlobalMerge.cpp vendor/llvm/dist-release_60/lib/CodeGen/PeepholeOptimizer.cpp vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/DAGCombiner.cpp vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAG.cpp vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp vendor/llvm/dist-release_60/lib/CodeGen/TargetLoweringBase.cpp vendor/llvm/dist-release_60/lib/Linker/IRMover.cpp vendor/llvm/dist-release_60/lib/MC/MCCodeView.cpp vendor/llvm/dist-release_60/lib/Target/AArch64/AArch64InstructionSelector.cpp vendor/llvm/dist-release_60/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp vendor/llvm/dist-release_60/lib/Target/PowerPC/PPCISelLowering.cpp vendor/llvm/dist-release_60/lib/Target/PowerPC/PPCISelLowering.h vendor/llvm/dist-release_60/lib/Target/PowerPC/PPCInstrInfo.td vendor/llvm/dist-release_60/lib/Target/X86/AsmParser/X86AsmParser.cpp vendor/llvm/dist-release_60/lib/Target/X86/X86ISelLowering.cpp vendor/llvm/dist-release_60/lib/Target/X86/X86TargetTransformInfo.cpp vendor/llvm/dist-release_60/lib/Transforms/Scalar/GVNHoist.cpp vendor/llvm/dist-release_60/lib/Transforms/Scalar/StructurizeCFG.cpp vendor/llvm/dist-release_60/lib/Transforms/Vectorize/LoopVectorize.cpp vendor/llvm/dist-release_60/lib/Transforms/Vectorize/SLPVectorizer.cpp vendor/llvm/dist-release_60/test/CodeGen/AArch64/atomic-ops-lse.ll vendor/llvm/dist-release_60/test/CodeGen/AMDGPU/multilevel-break.ll vendor/llvm/dist-release_60/test/CodeGen/AMDGPU/nested-loop-conditions.ll vendor/llvm/dist-release_60/test/CodeGen/ARM/and-load-combine.ll vendor/llvm/dist-release_60/test/CodeGen/ARM/atomic-cmpxchg.ll vendor/llvm/dist-release_60/test/CodeGen/ARM/cmpxchg-O0.ll vendor/llvm/dist-release_60/test/CodeGen/ARM/global-merge-external.ll vendor/llvm/dist-release_60/test/CodeGen/PowerPC/atomics-regression.ll vendor/llvm/dist-release_60/test/CodeGen/X86/avx512-shuffles/partial_permute.ll vendor/llvm/dist-release_60/test/CodeGen/X86/darwin-bzero.ll vendor/llvm/dist-release_60/test/CodeGen/X86/inline-asm-A-constraint.ll vendor/llvm/dist-release_60/test/CodeGen/X86/var-permute-128.ll vendor/llvm/dist-release_60/test/CodeGen/X86/var-permute-256.ll vendor/llvm/dist-release_60/test/MC/COFF/cv-inline-linetable.s vendor/llvm/dist-release_60/test/MC/X86/x86-64.s vendor/llvm/dist-release_60/test/Transforms/GVNHoist/pr35222-hoist-load.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/insert-element-build-vector.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/insertvalue.ll vendor/llvm/dist-release_60/test/Transforms/SLPVectorizer/X86/value-bug.ll vendor/llvm/dist-release_60/test/Transforms/StructurizeCFG/nested-loop-order.ll vendor/llvm/dist-release_60/tools/llvm-readobj/MachODumper.cpp vendor/llvm/dist-release_60/unittests/IR/DominatorTreeBatchUpdatesTest.cpp vendor/llvm/dist-release_60/unittests/IR/DominatorTreeTest.cpp vendor/llvm/dist-release_60/utils/release/test-release.sh Modified: vendor/llvm/dist-release_60/cmake/modules/LLVMConfig.cmake.in ============================================================================== --- vendor/llvm/dist-release_60/cmake/modules/LLVMConfig.cmake.in Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/cmake/modules/LLVMConfig.cmake.in Wed Jan 24 20:23:48 2018 (r328362) @@ -37,6 +37,8 @@ set(LLVM_ENABLE_THREADS @LLVM_ENABLE_THREADS@) set(LLVM_ENABLE_ZLIB @LLVM_ENABLE_ZLIB@) +set(LLVM_LIBXML2_ENABLED @LLVM_LIBXML2_ENABLED@) + set(LLVM_ENABLE_DIA_SDK @LLVM_ENABLE_DIA_SDK@) set(LLVM_NATIVE_ARCH @LLVM_NATIVE_ARCH@) Modified: vendor/llvm/dist-release_60/docs/ReleaseNotes.rst ============================================================================== --- vendor/llvm/dist-release_60/docs/ReleaseNotes.rst Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/docs/ReleaseNotes.rst Wed Jan 24 20:23:48 2018 (r328362) @@ -54,6 +54,8 @@ Non-comprehensive list of changes in this release ``DIVariables`` to the instructions in a ``Module``. The ``CheckDebugify`` pass determines how much of the metadata is lost. +* Significantly improved quality of CodeView debug info for Windows. + * Note.. .. NOTE @@ -69,12 +71,15 @@ Non-comprehensive list of changes in this release Changes to the LLVM IR ---------------------- -Changes to the ARM Backend --------------------------- +Changes to the ARM Target +------------------------- - During this release ... +During this release the ARM target has: +* Got support for enabling SjLj exception handling on platforms where it + isn't the default. + Changes to the MIPS Target -------------------------- @@ -89,8 +94,11 @@ Changes to the PowerPC Target Changes to the X86 Target ------------------------- - During this release ... +During this release ... +* Got support for enabling SjLj exception handling on platforms where it + isn't the default. + Changes to the AMDGPU Target ----------------------------- @@ -116,8 +124,46 @@ Changes to the C API External Open Source Projects Using LLVM 6 ========================================== -* A project... +JFS - JIT Fuzzing Solver +------------------------ +`JFS `_ is an experimental constraint solver +designed to investigate using coverage guided fuzzing as an incomplete strategy +for solving boolean, BitVector, and floating-point constraints. +It is built on top of LLVM, Clang, LibFuzzer, and Z3. + +The solver works by generating a C++ program where the reachability of an +`abort()` statement is equivalent to finding a satisfying assignment to the +constraints. This program is then compiled by Clang with `SanitizerCoverage +`_ +instrumentation and then fuzzed using :doc:`LibFuzzer `. + +Zig Programming Language +------------------------ + +`Zig `_ is an open-source programming language designed +for robustness, optimality, and clarity. It is intended to replace C. It +provides high level features such as Generics, +Compile Time Function Execution, and Partial Evaluation, yet exposes low level +LLVM IR features such as Aliases. Zig uses Clang to provide automatic +import of .h symbols - even inline functions and macros. Zig uses LLD combined +with lazily building compiler-rt to provide out-of-the-box cross-compiling for +all supported targets. + +LDC - the LLVM-based D compiler +------------------------------- + +`D `_ is a language with C-like syntax and static typing. It +pragmatically combines efficiency, control, and modeling power, with safety and +programmer productivity. D supports powerful concepts like Compile-Time Function +Execution (CTFE) and Template Meta-Programming, provides an innovative approach +to concurrency and offers many classical paradigms. + +`LDC `_ uses the frontend from the reference compiler +combined with LLVM as backend to produce efficient native code. LDC targets +x86/x86_64 systems like Linux, OS X, FreeBSD and Windows and also Linux on ARM +and PowerPC (32/64 bit). Ports to other architectures like AArch64 and MIPS64 +are underway. Additional Information ====================== Modified: vendor/llvm/dist-release_60/include/llvm/Analysis/RegionInfoImpl.h ============================================================================== --- vendor/llvm/dist-release_60/include/llvm/Analysis/RegionInfoImpl.h Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/include/llvm/Analysis/RegionInfoImpl.h Wed Jan 24 20:23:48 2018 (r328362) @@ -254,23 +254,23 @@ std::string RegionBase::getNameStr() const { template void RegionBase::verifyBBInRegion(BlockT *BB) const { if (!contains(BB)) - llvm_unreachable("Broken region found: enumerated BB not in region!"); + report_fatal_error("Broken region found: enumerated BB not in region!"); BlockT *entry = getEntry(), *exit = getExit(); for (BlockT *Succ : make_range(BlockTraits::child_begin(BB), BlockTraits::child_end(BB))) { if (!contains(Succ) && exit != Succ) - llvm_unreachable("Broken region found: edges leaving the region must go " - "to the exit node!"); + report_fatal_error("Broken region found: edges leaving the region must go " + "to the exit node!"); } if (entry != BB) { for (BlockT *Pred : make_range(InvBlockTraits::child_begin(BB), InvBlockTraits::child_end(BB))) { if (!contains(Pred)) - llvm_unreachable("Broken region found: edges entering the region must " - "go to the entry node!"); + report_fatal_error("Broken region found: edges entering the region must " + "go to the entry node!"); } } } @@ -557,7 +557,7 @@ void RegionInfoBase::verifyBBMap(const RegionT *R) } else { BlockT *BB = Element->template getNodeAs(); if (getRegionFor(BB) != R) - llvm_unreachable("BB map does not match region nesting"); + report_fatal_error("BB map does not match region nesting"); } } } Modified: vendor/llvm/dist-release_60/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h ============================================================================== --- vendor/llvm/dist-release_60/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/include/llvm/CodeGen/SelectionDAGAddressAnalysis.h Wed Jan 24 20:23:48 2018 (r328362) @@ -56,7 +56,7 @@ class BaseIndexOffset { (public) int64_t &Off); /// Parses tree in Ptr for base, index, offset addresses. - static BaseIndexOffset match(SDValue Ptr, const SelectionDAG &DAG); + static BaseIndexOffset match(LSBaseSDNode *N, const SelectionDAG &DAG); }; } // end namespace llvm Modified: vendor/llvm/dist-release_60/include/llvm/MC/MCCodeView.h ============================================================================== --- vendor/llvm/dist-release_60/include/llvm/MC/MCCodeView.h Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/include/llvm/MC/MCCodeView.h Wed Jan 24 20:23:48 2018 (r328362) @@ -177,13 +177,7 @@ class CodeViewContext { (public) unsigned IACol); /// Retreive the function info if this is a valid function id, or nullptr. - MCCVFunctionInfo *getCVFunctionInfo(unsigned FuncId) { - if (FuncId >= Functions.size()) - return nullptr; - if (Functions[FuncId].isUnallocatedFunctionInfo()) - return nullptr; - return &Functions[FuncId]; - } + MCCVFunctionInfo *getCVFunctionInfo(unsigned FuncId); /// Saves the information from the currently parsed .cv_loc directive /// and sets CVLocSeen. When the next instruction is assembled an entry @@ -199,50 +193,22 @@ class CodeViewContext { (public) CurrentCVLoc.setIsStmt(IsStmt); CVLocSeen = true; } - void clearCVLocSeen() { CVLocSeen = false; } bool getCVLocSeen() { return CVLocSeen; } + void clearCVLocSeen() { CVLocSeen = false; } + const MCCVLoc &getCurrentCVLoc() { return CurrentCVLoc; } bool isValidCVFileNumber(unsigned FileNumber); /// \brief Add a line entry. - void addLineEntry(const MCCVLineEntry &LineEntry) { - size_t Offset = MCCVLines.size(); - auto I = MCCVLineStartStop.insert( - {LineEntry.getFunctionId(), {Offset, Offset + 1}}); - if (!I.second) - I.first->second.second = Offset + 1; - MCCVLines.push_back(LineEntry); - } + void addLineEntry(const MCCVLineEntry &LineEntry); - std::vector getFunctionLineEntries(unsigned FuncId) { - std::vector FilteredLines; + std::vector getFunctionLineEntries(unsigned FuncId); - auto I = MCCVLineStartStop.find(FuncId); - if (I != MCCVLineStartStop.end()) - for (size_t Idx = I->second.first, End = I->second.second; Idx != End; - ++Idx) - if (MCCVLines[Idx].getFunctionId() == FuncId) - FilteredLines.push_back(MCCVLines[Idx]); - return FilteredLines; - } + std::pair getLineExtent(unsigned FuncId); - std::pair getLineExtent(unsigned FuncId) { - auto I = MCCVLineStartStop.find(FuncId); - // Return an empty extent if there are no cv_locs for this function id. - if (I == MCCVLineStartStop.end()) - return {~0ULL, 0}; - return I->second; - } - - ArrayRef getLinesForExtent(size_t L, size_t R) { - if (R <= L) - return None; - if (L >= MCCVLines.size()) - return None; - return makeArrayRef(&MCCVLines[L], R - L); - } + ArrayRef getLinesForExtent(size_t L, size_t R); /// Emits a line table substream. void emitLineTableForFunction(MCObjectStreamer &OS, unsigned FuncId, Modified: vendor/llvm/dist-release_60/include/llvm/Support/GenericDomTreeConstruction.h ============================================================================== --- vendor/llvm/dist-release_60/include/llvm/Support/GenericDomTreeConstruction.h Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/include/llvm/Support/GenericDomTreeConstruction.h Wed Jan 24 20:23:48 2018 (r328362) @@ -628,7 +628,7 @@ struct SemiNCAInfo { DecreasingLevel> Bucket; // Queue of tree nodes sorted by level in descending order. SmallDenseSet Affected; - SmallDenseSet Visited; + SmallDenseMap Visited; SmallVector AffectedQueue; SmallVector VisitedNotAffectedQueue; }; @@ -706,7 +706,7 @@ struct SemiNCAInfo { // algorithm does not really know or use the set of roots and can make a // different (implicit) decision about which nodes within an infinite loop // becomes a root. - if (DT.isVirtualRoot(TN->getIDom())) { + if (TN && !DT.isVirtualRoot(TN->getIDom())) { DEBUG(dbgs() << "Root " << BlockNamePrinter(R) << " is not virtual root's child\n" << "The entire tree needs to be rebuilt\n"); @@ -753,14 +753,16 @@ struct SemiNCAInfo { while (!II.Bucket.empty()) { const TreeNodePtr CurrentNode = II.Bucket.top().second; + const unsigned CurrentLevel = CurrentNode->getLevel(); II.Bucket.pop(); DEBUG(dbgs() << "\tAdding to Visited and AffectedQueue: " << BlockNamePrinter(CurrentNode) << "\n"); - II.Visited.insert(CurrentNode); + + II.Visited.insert({CurrentNode, CurrentLevel}); II.AffectedQueue.push_back(CurrentNode); // Discover and collect affected successors of the current node. - VisitInsertion(DT, BUI, CurrentNode, CurrentNode->getLevel(), NCD, II); + VisitInsertion(DT, BUI, CurrentNode, CurrentLevel, NCD, II); } // Finish by updating immediate dominators and levels. @@ -772,13 +774,17 @@ struct SemiNCAInfo { const TreeNodePtr TN, const unsigned RootLevel, const TreeNodePtr NCD, InsertionInfo &II) { const unsigned NCDLevel = NCD->getLevel(); - DEBUG(dbgs() << "Visiting " << BlockNamePrinter(TN) << "\n"); + DEBUG(dbgs() << "Visiting " << BlockNamePrinter(TN) << ", RootLevel " + << RootLevel << "\n"); SmallVector Stack = {TN}; assert(TN->getBlock() && II.Visited.count(TN) && "Preconditions!"); + SmallPtrSet Processed; + do { TreeNodePtr Next = Stack.pop_back_val(); + DEBUG(dbgs() << " Next: " << BlockNamePrinter(Next) << "\n"); for (const NodePtr Succ : ChildrenGetter::Get(Next->getBlock(), BUI)) { @@ -786,19 +792,31 @@ struct SemiNCAInfo { assert(SuccTN && "Unreachable successor found at reachable insertion"); const unsigned SuccLevel = SuccTN->getLevel(); - DEBUG(dbgs() << "\tSuccessor " << BlockNamePrinter(Succ) - << ", level = " << SuccLevel << "\n"); + DEBUG(dbgs() << "\tSuccessor " << BlockNamePrinter(Succ) << ", level = " + << SuccLevel << "\n"); + // Do not process the same node multiple times. + if (Processed.count(Next) > 0) + continue; + // Succ dominated by subtree From -- not affected. // (Based on the lemma 2.5 from the second paper.) if (SuccLevel > RootLevel) { DEBUG(dbgs() << "\t\tDominated by subtree From\n"); - if (II.Visited.count(SuccTN) != 0) - continue; + if (II.Visited.count(SuccTN) != 0) { + DEBUG(dbgs() << "\t\t\talready visited at level " + << II.Visited[SuccTN] << "\n\t\t\tcurrent level " + << RootLevel << ")\n"); + // A node can be necessary to visit again if we see it again at + // a lower level than before. + if (II.Visited[SuccTN] >= RootLevel) + continue; + } + DEBUG(dbgs() << "\t\tMarking visited not affected " << BlockNamePrinter(Succ) << "\n"); - II.Visited.insert(SuccTN); + II.Visited.insert({SuccTN, RootLevel}); II.VisitedNotAffectedQueue.push_back(SuccTN); Stack.push_back(SuccTN); } else if ((SuccLevel > NCDLevel + 1) && @@ -809,6 +827,8 @@ struct SemiNCAInfo { II.Bucket.push({SuccLevel, SuccTN}); } } + + Processed.insert(Next); } while (!Stack.empty()); } @@ -920,21 +940,21 @@ struct SemiNCAInfo { const NodePtr NCDBlock = DT.findNearestCommonDominator(From, To); const TreeNodePtr NCD = DT.getNode(NCDBlock); - // To dominates From -- nothing to do. - if (ToTN == NCD) return; + // If To dominates From -- nothing to do. + if (ToTN != NCD) { + DT.DFSInfoValid = false; - DT.DFSInfoValid = false; + const TreeNodePtr ToIDom = ToTN->getIDom(); + DEBUG(dbgs() << "\tNCD " << BlockNamePrinter(NCD) << ", ToIDom " + << BlockNamePrinter(ToIDom) << "\n"); - const TreeNodePtr ToIDom = ToTN->getIDom(); - DEBUG(dbgs() << "\tNCD " << BlockNamePrinter(NCD) << ", ToIDom " - << BlockNamePrinter(ToIDom) << "\n"); - - // To remains reachable after deletion. - // (Based on the caption under Figure 4. from the second paper.) - if (FromTN != ToIDom || HasProperSupport(DT, BUI, ToTN)) - DeleteReachable(DT, BUI, FromTN, ToTN); - else - DeleteUnreachable(DT, BUI, ToTN); + // To remains reachable after deletion. + // (Based on the caption under Figure 4. from the second paper.) + if (FromTN != ToIDom || HasProperSupport(DT, BUI, ToTN)) + DeleteReachable(DT, BUI, FromTN, ToTN); + else + DeleteUnreachable(DT, BUI, ToTN); + } if (IsPostDom) UpdateRootsAfterUpdate(DT, BUI); } Modified: vendor/llvm/dist-release_60/include/llvm/Transforms/Vectorize/SLPVectorizer.h ============================================================================== --- vendor/llvm/dist-release_60/include/llvm/Transforms/Vectorize/SLPVectorizer.h Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/include/llvm/Transforms/Vectorize/SLPVectorizer.h Wed Jan 24 20:23:48 2018 (r328362) @@ -95,14 +95,9 @@ struct SLPVectorizerPass : public PassInfoMixin VL, slpvectorizer::BoUpSLP &R, - ArrayRef BuildVector = None, - bool AllowReorder = false, - bool NeedExtraction = false); + bool AllowReorder = false); /// \brief Try to vectorize a chain that may start at the operands of \p I. bool tryToVectorize(Instruction *I, slpvectorizer::BoUpSLP &R); Modified: vendor/llvm/dist-release_60/lib/CodeGen/CodeGenPrepare.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/CodeGenPrepare.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/CodeGenPrepare.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -2700,8 +2700,13 @@ class AddressingModeCombiner { (public) // we still need to collect it due to original value is different. // And later we will need all original values as anchors during // finding the common Phi node. + // We also must reject the case when base offset is different and + // scale reg is not null, we cannot handle this case due to merge of + // different offsets will be used as ScaleReg. if (DifferentField != ExtAddrMode::MultipleFields && - DifferentField != ExtAddrMode::ScaleField) { + DifferentField != ExtAddrMode::ScaleField && + (DifferentField != ExtAddrMode::BaseOffsField || + !NewAddrMode.ScaledReg)) { AddrModes.emplace_back(NewAddrMode); return true; } Modified: vendor/llvm/dist-release_60/lib/CodeGen/GlobalMerge.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/GlobalMerge.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/GlobalMerge.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -577,7 +577,8 @@ bool GlobalMerge::doInitialization(Module &M) { for (auto &GV : M.globals()) { // Merge is safe for "normal" internal or external globals only if (GV.isDeclaration() || GV.isThreadLocal() || - GV.hasSection() || GV.hasImplicitSection()) + GV.hasSection() || GV.hasImplicitSection() || + GV.hasDLLExportStorageClass()) continue; // It's not safe to merge globals that may be preempted Modified: vendor/llvm/dist-release_60/lib/CodeGen/PeepholeOptimizer.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/PeepholeOptimizer.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/PeepholeOptimizer.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -719,15 +719,14 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, u CurSrcPair = Pair; ValueTracker ValTracker(CurSrcPair.Reg, CurSrcPair.SubReg, *MRI, !DisableAdvCopyOpt, TII); - ValueTrackerResult Res; - bool ShouldRewrite = false; - do { - // Follow the chain of copies until we reach the top of the use-def chain - // or find a more suitable source. - Res = ValTracker.getNextSource(); + // Follow the chain of copies until we find a more suitable source, a phi + // or have to abort. + while (true) { + ValueTrackerResult Res = ValTracker.getNextSource(); + // Abort at the end of a chain (without finding a suitable source). if (!Res.isValid()) - break; + return false; // Insert the Def -> Use entry for the recently found source. ValueTrackerResult CurSrcRes = RewriteMap.lookup(CurSrcPair); @@ -763,26 +762,21 @@ bool PeepholeOptimizer::findNextSource(unsigned Reg, u if (TargetRegisterInfo::isPhysicalRegister(CurSrcPair.Reg)) return false; + // Keep following the chain if the value isn't any better yet. const TargetRegisterClass *SrcRC = MRI->getRegClass(CurSrcPair.Reg); - ShouldRewrite = TRI->shouldRewriteCopySrc(DefRC, SubReg, SrcRC, - CurSrcPair.SubReg); - } while (!ShouldRewrite); + if (!TRI->shouldRewriteCopySrc(DefRC, SubReg, SrcRC, CurSrcPair.SubReg)) + continue; - // Continue looking for new sources... - if (Res.isValid()) - continue; + // We currently cannot deal with subreg operands on PHI instructions + // (see insertPHI()). + if (PHICount > 0 && CurSrcPair.SubReg != 0) + continue; - // Do not continue searching for a new source if the there's at least - // one use-def which cannot be rewritten. - if (!ShouldRewrite) - return false; + // We found a suitable source, and are done with this chain. + break; + } } - if (PHICount >= RewritePHILimit) { - DEBUG(dbgs() << "findNextSource: PHI limit reached\n"); - return false; - } - // If we did not find a more suitable source, there is nothing to optimize. return CurSrcPair.Reg != Reg; } @@ -799,6 +793,9 @@ insertPHI(MachineRegisterInfo *MRI, const TargetInstrI assert(!SrcRegs.empty() && "No sources to create a PHI instruction?"); const TargetRegisterClass *NewRC = MRI->getRegClass(SrcRegs[0].Reg); + // NewRC is only correct if no subregisters are involved. findNextSource() + // should have rejected those cases already. + assert(SrcRegs[0].SubReg == 0 && "should not have subreg operand"); unsigned NewVR = MRI->createVirtualRegister(NewRC); MachineBasicBlock *MBB = OrigPHI->getParent(); MachineInstrBuilder MIB = BuildMI(*MBB, OrigPHI, OrigPHI->getDebugLoc(), Modified: vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/DAGCombiner.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/DAGCombiner.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/DAGCombiner.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -3842,9 +3842,16 @@ bool DAGCombiner::SearchForAndLoads(SDNode *N, EVT ExtVT; if (isAndLoadExtLoad(Mask, Load, Load->getValueType(0), ExtVT) && isLegalNarrowLoad(Load, ISD::ZEXTLOAD, ExtVT)) { - // Only add this load if we can make it more narrow. - if (ExtVT.bitsLT(Load->getMemoryVT())) + + // ZEXTLOAD is already small enough. + if (Load->getExtensionType() == ISD::ZEXTLOAD && + ExtVT.bitsGE(Load->getMemoryVT())) + continue; + + // Use LE to convert equal sized loads to zext. + if (ExtVT.bitsLE(Load->getMemoryVT())) Loads.insert(Load); + continue; } return false; @@ -3899,11 +3906,13 @@ bool DAGCombiner::BackwardsPropagateMask(SDNode *N, Se if (Loads.size() == 0) return false; + DEBUG(dbgs() << "Backwards propagate AND: "; N->dump()); SDValue MaskOp = N->getOperand(1); // If it exists, fixup the single node we allow in the tree that needs // masking. if (FixupNode) { + DEBUG(dbgs() << "First, need to fix up: "; FixupNode->dump()); SDValue And = DAG.getNode(ISD::AND, SDLoc(FixupNode), FixupNode->getValueType(0), SDValue(FixupNode, 0), MaskOp); @@ -3914,14 +3923,21 @@ bool DAGCombiner::BackwardsPropagateMask(SDNode *N, Se // Narrow any constants that need it. for (auto *LogicN : NodesWithConsts) { - auto *C = cast(LogicN->getOperand(1)); - SDValue And = DAG.getNode(ISD::AND, SDLoc(C), C->getValueType(0), - SDValue(C, 0), MaskOp); - DAG.UpdateNodeOperands(LogicN, LogicN->getOperand(0), And); + SDValue Op0 = LogicN->getOperand(0); + SDValue Op1 = LogicN->getOperand(1); + + if (isa(Op0)) + std::swap(Op0, Op1); + + SDValue And = DAG.getNode(ISD::AND, SDLoc(Op1), Op1.getValueType(), + Op1, MaskOp); + + DAG.UpdateNodeOperands(LogicN, Op0, And); } // Create narrow loads. for (auto *Load : Loads) { + DEBUG(dbgs() << "Propagate AND back to: "; Load->dump()); SDValue And = DAG.getNode(ISD::AND, SDLoc(Load), Load->getValueType(0), SDValue(Load, 0), MaskOp); DAG.ReplaceAllUsesOfValueWith(SDValue(Load, 0), And); @@ -5209,7 +5225,7 @@ SDValue DAGCombiner::MatchLoadCombine(SDNode *N) { return SDValue(); // Loads must share the same base address - BaseIndexOffset Ptr = BaseIndexOffset::match(L->getBasePtr(), DAG); + BaseIndexOffset Ptr = BaseIndexOffset::match(L, DAG); int64_t ByteOffsetFromBase = 0; if (!Base) Base = Ptr; @@ -12928,7 +12944,7 @@ void DAGCombiner::getStoreMergeCandidates( StoreSDNode *St, SmallVectorImpl &StoreNodes) { // This holds the base pointer, index, and the offset in bytes from the base // pointer. - BaseIndexOffset BasePtr = BaseIndexOffset::match(St->getBasePtr(), DAG); + BaseIndexOffset BasePtr = BaseIndexOffset::match(St, DAG); EVT MemVT = St->getMemoryVT(); SDValue Val = peekThroughBitcast(St->getValue()); @@ -12949,7 +12965,7 @@ void DAGCombiner::getStoreMergeCandidates( EVT LoadVT; if (IsLoadSrc) { auto *Ld = cast(Val); - LBasePtr = BaseIndexOffset::match(Ld->getBasePtr(), DAG); + LBasePtr = BaseIndexOffset::match(Ld, DAG); LoadVT = Ld->getMemoryVT(); // Load and store should be the same type. if (MemVT != LoadVT) @@ -12968,7 +12984,7 @@ void DAGCombiner::getStoreMergeCandidates( return false; // The Load's Base Ptr must also match if (LoadSDNode *OtherLd = dyn_cast(Val)) { - auto LPtr = BaseIndexOffset::match(OtherLd->getBasePtr(), DAG); + auto LPtr = BaseIndexOffset::match(OtherLd, DAG); if (LoadVT != OtherLd->getMemoryVT()) return false; if (!(LBasePtr.equalBaseIndex(LPtr, DAG))) @@ -12992,7 +13008,7 @@ void DAGCombiner::getStoreMergeCandidates( Val.getOpcode() != ISD::EXTRACT_SUBVECTOR) return false; } - Ptr = BaseIndexOffset::match(Other->getBasePtr(), DAG); + Ptr = BaseIndexOffset::match(Other, DAG); return (BasePtr.equalBaseIndex(Ptr, DAG, Offset)); }; @@ -13365,7 +13381,7 @@ bool DAGCombiner::MergeConsecutiveStores(StoreSDNode * if (Ld->getMemoryVT() != MemVT) break; - BaseIndexOffset LdPtr = BaseIndexOffset::match(Ld->getBasePtr(), DAG); + BaseIndexOffset LdPtr = BaseIndexOffset::match(Ld, DAG); // If this is not the first ptr that we check. int64_t LdOffset = 0; if (LdBasePtr.getBase().getNode()) { @@ -17432,44 +17448,46 @@ bool DAGCombiner::isAlias(LSBaseSDNode *Op0, LSBaseSDN unsigned NumBytes1 = Op1->getMemoryVT().getStoreSize(); // Check for BaseIndexOffset matching. - BaseIndexOffset BasePtr0 = BaseIndexOffset::match(Op0->getBasePtr(), DAG); - BaseIndexOffset BasePtr1 = BaseIndexOffset::match(Op1->getBasePtr(), DAG); + BaseIndexOffset BasePtr0 = BaseIndexOffset::match(Op0, DAG); + BaseIndexOffset BasePtr1 = BaseIndexOffset::match(Op1, DAG); int64_t PtrDiff; - if (BasePtr0.equalBaseIndex(BasePtr1, DAG, PtrDiff)) - return !((NumBytes0 <= PtrDiff) || (PtrDiff + NumBytes1 <= 0)); + if (BasePtr0.getBase().getNode() && BasePtr1.getBase().getNode()) { + if (BasePtr0.equalBaseIndex(BasePtr1, DAG, PtrDiff)) + return !((NumBytes0 <= PtrDiff) || (PtrDiff + NumBytes1 <= 0)); - // If both BasePtr0 and BasePtr1 are FrameIndexes, we will not be - // able to calculate their relative offset if at least one arises - // from an alloca. However, these allocas cannot overlap and we - // can infer there is no alias. - if (auto *A = dyn_cast(BasePtr0.getBase())) - if (auto *B = dyn_cast(BasePtr1.getBase())) { - MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo(); - // If the base are the same frame index but the we couldn't find a - // constant offset, (indices are different) be conservative. - if (A != B && (!MFI.isFixedObjectIndex(A->getIndex()) || - !MFI.isFixedObjectIndex(B->getIndex()))) - return false; - } + // If both BasePtr0 and BasePtr1 are FrameIndexes, we will not be + // able to calculate their relative offset if at least one arises + // from an alloca. However, these allocas cannot overlap and we + // can infer there is no alias. + if (auto *A = dyn_cast(BasePtr0.getBase())) + if (auto *B = dyn_cast(BasePtr1.getBase())) { + MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo(); + // If the base are the same frame index but the we couldn't find a + // constant offset, (indices are different) be conservative. + if (A != B && (!MFI.isFixedObjectIndex(A->getIndex()) || + !MFI.isFixedObjectIndex(B->getIndex()))) + return false; + } - bool IsFI0 = isa(BasePtr0.getBase()); - bool IsFI1 = isa(BasePtr1.getBase()); - bool IsGV0 = isa(BasePtr0.getBase()); - bool IsGV1 = isa(BasePtr1.getBase()); - bool IsCV0 = isa(BasePtr0.getBase()); - bool IsCV1 = isa(BasePtr1.getBase()); + bool IsFI0 = isa(BasePtr0.getBase()); + bool IsFI1 = isa(BasePtr1.getBase()); + bool IsGV0 = isa(BasePtr0.getBase()); + bool IsGV1 = isa(BasePtr1.getBase()); + bool IsCV0 = isa(BasePtr0.getBase()); + bool IsCV1 = isa(BasePtr1.getBase()); - // If of mismatched base types or checkable indices we can check - // they do not alias. - if ((BasePtr0.getIndex() == BasePtr1.getIndex() || (IsFI0 != IsFI1) || - (IsGV0 != IsGV1) || (IsCV0 != IsCV1)) && - (IsFI0 || IsGV0 || IsCV0) && (IsFI1 || IsGV1 || IsCV1)) - return false; + // If of mismatched base types or checkable indices we can check + // they do not alias. + if ((BasePtr0.getIndex() == BasePtr1.getIndex() || (IsFI0 != IsFI1) || + (IsGV0 != IsGV1) || (IsCV0 != IsCV1)) && + (IsFI0 || IsGV0 || IsCV0) && (IsFI1 || IsGV1 || IsCV1)) + return false; + } - // If we know required SrcValue1 and SrcValue2 have relatively large alignment - // compared to the size and offset of the access, we may be able to prove they - // do not alias. This check is conservative for now to catch cases created by - // splitting vector types. + // If we know required SrcValue1 and SrcValue2 have relatively large + // alignment compared to the size and offset of the access, we may be able + // to prove they do not alias. This check is conservative for now to catch + // cases created by splitting vector types. int64_t SrcValOffset0 = Op0->getSrcValueOffset(); int64_t SrcValOffset1 = Op1->getSrcValueOffset(); unsigned OrigAlignment0 = Op0->getOriginalAlignment(); @@ -17479,8 +17497,8 @@ bool DAGCombiner::isAlias(LSBaseSDNode *Op0, LSBaseSDN int64_t OffAlign0 = SrcValOffset0 % OrigAlignment0; int64_t OffAlign1 = SrcValOffset1 % OrigAlignment1; - // There is no overlap between these relatively aligned accesses of similar - // size. Return no alias. + // There is no overlap between these relatively aligned accesses of + // similar size. Return no alias. if ((OffAlign0 + NumBytes0) <= OffAlign1 || (OffAlign1 + NumBytes1) <= OffAlign0) return false; @@ -17643,7 +17661,7 @@ bool DAGCombiner::findBetterNeighborChains(StoreSDNode // This holds the base pointer, index, and the offset in bytes from the base // pointer. - BaseIndexOffset BasePtr = BaseIndexOffset::match(St->getBasePtr(), DAG); + BaseIndexOffset BasePtr = BaseIndexOffset::match(St, DAG); // We must have a base and an offset. if (!BasePtr.getBase().getNode()) @@ -17669,7 +17687,7 @@ bool DAGCombiner::findBetterNeighborChains(StoreSDNode break; // Find the base pointer and offset for this memory node. - BaseIndexOffset Ptr = BaseIndexOffset::match(Index->getBasePtr(), DAG); + BaseIndexOffset Ptr = BaseIndexOffset::match(Index, DAG); // Check that the base pointer is the same as the original one. if (!BasePtr.equalBaseIndex(Ptr, DAG)) Modified: vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -2965,12 +2965,12 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) { case ISD::ZERO_EXTEND: LHS = DAG.getNode(ISD::AssertZext, dl, OuterType, Res, DAG.getValueType(AtomicType)); - RHS = DAG.getNode(ISD::ZERO_EXTEND, dl, OuterType, Node->getOperand(2)); + RHS = DAG.getZeroExtendInReg(Node->getOperand(2), dl, AtomicType); ExtRes = LHS; break; case ISD::ANY_EXTEND: LHS = DAG.getZeroExtendInReg(Res, dl, AtomicType); - RHS = DAG.getNode(ISD::ZERO_EXTEND, dl, OuterType, Node->getOperand(2)); + RHS = DAG.getZeroExtendInReg(Node->getOperand(2), dl, AtomicType); break; default: llvm_unreachable("Invalid atomic op extension"); Modified: vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAG.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAG.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -7947,11 +7947,8 @@ bool SelectionDAG::areNonVolatileConsecutiveLoads(Load if (VT.getSizeInBits() / 8 != Bytes) return false; - SDValue Loc = LD->getOperand(1); - SDValue BaseLoc = Base->getOperand(1); - - auto BaseLocDecomp = BaseIndexOffset::match(BaseLoc, *this); - auto LocDecomp = BaseIndexOffset::match(Loc, *this); + auto BaseLocDecomp = BaseIndexOffset::match(Base, *this); + auto LocDecomp = BaseIndexOffset::match(LD, *this); int64_t Offset = 0; if (BaseLocDecomp.equalBaseIndex(LocDecomp, *this, Offset)) Modified: vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/SelectionDAG/SelectionDAGAddressAnalysis.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -21,6 +21,9 @@ using namespace llvm; bool BaseIndexOffset::equalBaseIndex(BaseIndexOffset &Other, const SelectionDAG &DAG, int64_t &Off) { + // Conservatively fail if we a match failed.. + if (!Base.getNode() || !Other.Base.getNode()) + return false; // Initial Offset difference. Off = Other.Offset - Offset; @@ -72,12 +75,28 @@ bool BaseIndexOffset::equalBaseIndex(BaseIndexOffset & } /// Parses tree in Ptr for base, index, offset addresses. -BaseIndexOffset BaseIndexOffset::match(SDValue Ptr, const SelectionDAG &DAG) { +BaseIndexOffset BaseIndexOffset::match(LSBaseSDNode *N, + const SelectionDAG &DAG) { + SDValue Ptr = N->getBasePtr(); + // (((B + I*M) + c)) + c ... SDValue Base = DAG.getTargetLoweringInfo().unwrapAddress(Ptr); SDValue Index = SDValue(); int64_t Offset = 0; bool IsIndexSignExt = false; + + // pre-inc/pre-dec ops are components of EA. + if (N->getAddressingMode() == ISD::PRE_INC) { + if (auto *C = dyn_cast(N->getOffset())) + Offset += C->getSExtValue(); + else // If unknown, give up now. + return BaseIndexOffset(SDValue(), SDValue(), 0, false); + } else if (N->getAddressingMode() == ISD::PRE_DEC) { + if (auto *C = dyn_cast(N->getOffset())) + Offset -= C->getSExtValue(); + else // If unknown, give up now. + return BaseIndexOffset(SDValue(), SDValue(), 0, false); + } // Consume constant adds & ors with appropriate masking. while (Base->getOpcode() == ISD::ADD || Base->getOpcode() == ISD::OR) { Modified: vendor/llvm/dist-release_60/lib/CodeGen/TargetLoweringBase.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/CodeGen/TargetLoweringBase.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/CodeGen/TargetLoweringBase.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -132,9 +132,18 @@ void TargetLoweringBase::InitLibcalls(const Triple &TT setLibcallName(RTLIB::FPEXT_F16_F32, "__extendhfsf2"); setLibcallName(RTLIB::FPROUND_F32_F16, "__truncsfhf2"); - // Darwin 10 and higher has an optimized __bzero. - if (!TT.isMacOSX() || !TT.isMacOSXVersionLT(10, 6) || TT.isArch64Bit()) { - setLibcallName(RTLIB::BZERO, TT.isAArch64() ? "bzero" : "__bzero"); + // Some darwins have an optimized __bzero/bzero function. + switch (TT.getArch()) { + case Triple::x86: + case Triple::x86_64: + if (TT.isMacOSX() && !TT.isMacOSXVersionLT(10, 6)) + setLibcallName(RTLIB::BZERO, "__bzero"); + break; + case Triple::aarch64: + setLibcallName(RTLIB::BZERO, "bzero"); + break; + default: + break; } if (darwinHasSinCos(TT)) { Modified: vendor/llvm/dist-release_60/lib/Linker/IRMover.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/Linker/IRMover.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/Linker/IRMover.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -954,7 +954,12 @@ Expected IRLinker::linkGlobalValueProto(Gl NewGV->setLinkage(GlobalValue::InternalLinkage); Constant *C = NewGV; - if (DGV) + // Only create a bitcast if necessary. In particular, with + // DebugTypeODRUniquing we may reach metadata in the destination module + // containing a GV from the source module, in which case SGV will be + // the same as DGV and NewGV, and TypeMap.get() will assert since it + // assumes it is being invoked on a type in the source module. + if (DGV && NewGV != SGV) C = ConstantExpr::getBitCast(NewGV, TypeMap.get(SGV->getType())); if (DGV && NewGV != DGV) { Modified: vendor/llvm/dist-release_60/lib/MC/MCCodeView.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/MC/MCCodeView.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/MC/MCCodeView.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -76,6 +76,14 @@ bool CodeViewContext::addFile(MCStreamer &OS, unsigned return true; } +MCCVFunctionInfo *CodeViewContext::getCVFunctionInfo(unsigned FuncId) { + if (FuncId >= Functions.size()) + return nullptr; + if (Functions[FuncId].isUnallocatedFunctionInfo()) + return nullptr; + return &Functions[FuncId]; +} + bool CodeViewContext::recordFunctionId(unsigned FuncId) { if (FuncId >= Functions.size()) Functions.resize(FuncId + 1); @@ -245,6 +253,67 @@ void CodeViewContext::emitFileChecksumOffset(MCObjectS MCSymbolRefExpr::create(Files[Idx].ChecksumTableOffset, OS.getContext()); OS.EmitValueImpl(SRE, 4); +} + +void CodeViewContext::addLineEntry(const MCCVLineEntry &LineEntry) { + size_t Offset = MCCVLines.size(); + auto I = MCCVLineStartStop.insert( + {LineEntry.getFunctionId(), {Offset, Offset + 1}}); + if (!I.second) + I.first->second.second = Offset + 1; + MCCVLines.push_back(LineEntry); +} + +std::vector +CodeViewContext::getFunctionLineEntries(unsigned FuncId) { + std::vector FilteredLines; + auto I = MCCVLineStartStop.find(FuncId); + if (I != MCCVLineStartStop.end()) { + MCCVFunctionInfo *SiteInfo = getCVFunctionInfo(FuncId); + for (size_t Idx = I->second.first, End = I->second.second; Idx != End; + ++Idx) { + unsigned LocationFuncId = MCCVLines[Idx].getFunctionId(); + if (LocationFuncId == FuncId) { + // This was a .cv_loc directly for FuncId, so record it. + FilteredLines.push_back(MCCVLines[Idx]); + } else { + // Check if the current location is inlined in this function. If it is, + // synthesize a statement .cv_loc at the original inlined call site. + auto I = SiteInfo->InlinedAtMap.find(LocationFuncId); + if (I != SiteInfo->InlinedAtMap.end()) { + MCCVFunctionInfo::LineInfo &IA = I->second; + // Only add the location if it differs from the previous location. + // Large inlined calls will have many .cv_loc entries and we only need + // one line table entry in the parent function. + if (FilteredLines.empty() || + FilteredLines.back().getFileNum() != IA.File || + FilteredLines.back().getLine() != IA.Line || + FilteredLines.back().getColumn() != IA.Col) { + FilteredLines.push_back(MCCVLineEntry( + MCCVLines[Idx].getLabel(), + MCCVLoc(FuncId, IA.File, IA.Line, IA.Col, false, false))); + } + } + } + } + } + return FilteredLines; +} + +std::pair CodeViewContext::getLineExtent(unsigned FuncId) { + auto I = MCCVLineStartStop.find(FuncId); + // Return an empty extent if there are no cv_locs for this function id. + if (I == MCCVLineStartStop.end()) + return {~0ULL, 0}; + return I->second; +} + +ArrayRef CodeViewContext::getLinesForExtent(size_t L, size_t R) { + if (R <= L) + return None; + if (L >= MCCVLines.size()) + return None; + return makeArrayRef(&MCCVLines[L], R - L); } void CodeViewContext::emitLineTableForFunction(MCObjectStreamer &OS, Modified: vendor/llvm/dist-release_60/lib/Target/AArch64/AArch64InstructionSelector.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/Target/AArch64/AArch64InstructionSelector.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/Target/AArch64/AArch64InstructionSelector.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -868,6 +868,40 @@ bool AArch64InstructionSelector::select(MachineInstr & if (OpFlags & AArch64II::MO_GOT) { I.setDesc(TII.get(AArch64::LOADgot)); I.getOperand(1).setTargetFlags(OpFlags); + } else if (TM.getCodeModel() == CodeModel::Large) { + // Materialize the global using movz/movk instructions. + unsigned MovZDstReg = MRI.createVirtualRegister(&AArch64::GPR64RegClass); + auto InsertPt = std::next(I.getIterator()); + auto MovZ = + BuildMI(MBB, InsertPt, I.getDebugLoc(), TII.get(AArch64::MOVZXi)) + .addDef(MovZDstReg); + MovZ->addOperand(MF, I.getOperand(1)); + MovZ->getOperand(1).setTargetFlags(OpFlags | AArch64II::MO_G0 | + AArch64II::MO_NC); + MovZ->addOperand(MF, MachineOperand::CreateImm(0)); + constrainSelectedInstRegOperands(*MovZ, TII, TRI, RBI); + + auto BuildMovK = [&](unsigned SrcReg, unsigned char Flags, + unsigned Offset, unsigned ForceDstReg) { + unsigned DstReg = + ForceDstReg ? ForceDstReg + : MRI.createVirtualRegister(&AArch64::GPR64RegClass); + auto MovI = BuildMI(MBB, InsertPt, MovZ->getDebugLoc(), + TII.get(AArch64::MOVKXi)) + .addDef(DstReg) + .addReg(SrcReg); + MovI->addOperand(MF, MachineOperand::CreateGA( + GV, MovZ->getOperand(1).getOffset(), Flags)); + MovI->addOperand(MF, MachineOperand::CreateImm(Offset)); + constrainSelectedInstRegOperands(*MovI, TII, TRI, RBI); + return DstReg; + }; + unsigned DstReg = BuildMovK(MovZ->getOperand(0).getReg(), + AArch64II::MO_G1 | AArch64II::MO_NC, 16, 0); + DstReg = BuildMovK(DstReg, AArch64II::MO_G2 | AArch64II::MO_NC, 32, 0); + BuildMovK(DstReg, AArch64II::MO_G3, 48, I.getOperand(0).getReg()); + I.eraseFromParent(); + return true; } else { I.setDesc(TII.get(AArch64::MOVaddr)); I.getOperand(1).setTargetFlags(OpFlags | AArch64II::MO_PAGE); Modified: vendor/llvm/dist-release_60/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -821,7 +821,6 @@ namespace llvm { MutableArrayRef NewMask, unsigned Options = None); OpRef packp(ShuffleMask SM, OpRef Va, OpRef Vb, ResultStack &Results, MutableArrayRef NewMask); - OpRef zerous(ShuffleMask SM, OpRef Va, ResultStack &Results); OpRef vmuxs(ArrayRef Bytes, OpRef Va, OpRef Vb, ResultStack &Results); OpRef vmuxp(ArrayRef Bytes, OpRef Va, OpRef Vb, @@ -1137,25 +1136,6 @@ OpRef HvxSelector::packp(ShuffleMask SM, OpRef Va, OpR } return concat(Out[0], Out[1], Results); -} - -OpRef HvxSelector::zerous(ShuffleMask SM, OpRef Va, ResultStack &Results) { - DEBUG_WITH_TYPE("isel", {dbgs() << __func__ << '\n';}); - - int VecLen = SM.Mask.size(); - SmallVector UsedBytes(VecLen); - bool HasUnused = false; - for (int I = 0; I != VecLen; ++I) { - if (SM.Mask[I] != -1) - UsedBytes[I] = 0xFF; - else - HasUnused = true; - } - if (!HasUnused) - return Va; - SDValue B = getVectorConstant(UsedBytes, SDLoc(Results.InpNode)); - Results.push(Hexagon::V6_vand, getSingleVT(MVT::i8), {Va, OpRef(B)}); - return OpRef::res(Results.top()); } OpRef HvxSelector::vmuxs(ArrayRef Bytes, OpRef Va, OpRef Vb, Modified: vendor/llvm/dist-release_60/lib/Target/PowerPC/PPCISelLowering.cpp ============================================================================== --- vendor/llvm/dist-release_60/lib/Target/PowerPC/PPCISelLowering.cpp Wed Jan 24 20:16:48 2018 (r328361) +++ vendor/llvm/dist-release_60/lib/Target/PowerPC/PPCISelLowering.cpp Wed Jan 24 20:23:48 2018 (r328362) @@ -142,6 +142,9 @@ PPCTargetLowering::PPCTargetLowering(const PPCTargetMa setOperationAction(ISD::BITREVERSE, MVT::i32, Legal); setOperationAction(ISD::BITREVERSE, MVT::i64, Legal); + // Sub-word ATOMIC_CMP_SWAP need to ensure that the input is zero-extended. + setOperationAction(ISD::ATOMIC_CMP_SWAP, MVT::i32, Custom); + // PowerPC has an i16 but no i8 (or i1) SEXTLOAD. for (MVT VT : MVT::integer_valuetypes()) { setLoadExtAction(ISD::SEXTLOAD, VT, MVT::i1, Promote); @@ -1154,6 +1157,8 @@ const char *PPCTargetLowering::getTargetNodeName(unsig *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:23:57 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2321DEC3A95; Wed, 24 Jan 2018 20:23:57 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5FF7E760FB; Wed, 24 Jan 2018 20:23:56 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 295002278C; Wed, 24 Jan 2018 20:23:55 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKNsKr042645; Wed, 24 Jan 2018 20:23:54 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKNsr1042644; Wed, 24 Jan 2018 20:23:54 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242023.w0OKNsr1042644@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:23:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328363 - vendor/llvm/llvm-release_60-r323338 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/llvm/llvm-release_60-r323338 X-SVN-Commit-Revision: 328363 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:23:57 -0000 Author: dim Date: Wed Jan 24 20:23:54 2018 New Revision: 328363 URL: https://svnweb.freebsd.org/changeset/base/328363 Log: Tag llvm release_60 branch r323338. Added: vendor/llvm/llvm-release_60-r323338/ - copied from r328362, vendor/llvm/dist-release_60/ From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:25:42 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF7FEEC3C41; Wed, 24 Jan 2018 20:25:41 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6CACB76312; Wed, 24 Jan 2018 20:25:41 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 674F822792; Wed, 24 Jan 2018 20:25:41 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKPf2p042816; Wed, 24 Jan 2018 20:25:41 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKPbbH042776; Wed, 24 Jan 2018 20:25:37 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242025.w0OKPbbH042776@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:25:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328364 - in vendor/clang/dist-release_60: docs include/clang/Basic include/clang/StaticAnalyzer/Core/BugReporter lib/AST lib/CodeGen lib/Frontend lib/Lex lib/Sema lib/StaticAnalyzer/Ch... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/clang/dist-release_60: docs include/clang/Basic include/clang/StaticAnalyzer/Core/BugReporter lib/AST lib/CodeGen lib/Frontend lib/Lex lib/Sema lib/StaticAnalyzer/Checkers test/Analysis test... X-SVN-Commit-Revision: 328364 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:25:42 -0000 Author: dim Date: Wed Jan 24 20:25:37 2018 New Revision: 328364 URL: https://svnweb.freebsd.org/changeset/base/328364 Log: Vendor import of clang release_60 branch r323338: https://llvm.org/svn/llvm-project/cfe/branches/release_60@323338 Added: vendor/clang/dist-release_60/docs/OpenMPSupport.rst vendor/clang/dist-release_60/test/CodeCompletion/Inputs/comments.h (contents, props changed) vendor/clang/dist-release_60/test/CodeCompletion/comments.cpp (contents, props changed) vendor/clang/dist-release_60/test/Lexer/null-character-in-literal.c (contents, props changed) vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/ vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/Box.h (contents, props changed) vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M1.h (contents, props changed) vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M2.h (contents, props changed) vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M3.h (contents, props changed) vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/module.modulemap vendor/clang/dist-release_60/test/Modules/odr_hash-Friend.cpp (contents, props changed) vendor/clang/dist-release_60/test/Modules/odr_hash-blocks.cpp (contents, props changed) Deleted: vendor/clang/dist-release_60/test/Sema/_Float128.c Modified: vendor/clang/dist-release_60/docs/ReleaseNotes.rst vendor/clang/dist-release_60/docs/index.rst vendor/clang/dist-release_60/include/clang/Basic/Attr.td vendor/clang/dist-release_60/include/clang/Basic/BuiltinsX86.def vendor/clang/dist-release_60/include/clang/Basic/DiagnosticGroups.td vendor/clang/dist-release_60/include/clang/Basic/TokenKinds.def vendor/clang/dist-release_60/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h vendor/clang/dist-release_60/lib/AST/DeclBase.cpp vendor/clang/dist-release_60/lib/AST/ODRHash.cpp vendor/clang/dist-release_60/lib/CodeGen/CGBuiltin.cpp vendor/clang/dist-release_60/lib/CodeGen/ObjectFilePCHContainerOperations.cpp vendor/clang/dist-release_60/lib/Frontend/InitPreprocessor.cpp vendor/clang/dist-release_60/lib/Lex/Lexer.cpp vendor/clang/dist-release_60/lib/Lex/PPCaching.cpp vendor/clang/dist-release_60/lib/Lex/PPLexerChange.cpp vendor/clang/dist-release_60/lib/Sema/Scope.cpp vendor/clang/dist-release_60/lib/Sema/SemaTemplateDeduction.cpp vendor/clang/dist-release_60/lib/Sema/SemaTemplateInstantiateDecl.cpp vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/MallocChecker.cpp vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/ValistChecker.cpp vendor/clang/dist-release_60/test/Analysis/malloc.c vendor/clang/dist-release_60/test/CodeGen/builtins-overflow.c vendor/clang/dist-release_60/test/CodeGenCXX/cxx1z-inline-variables.cpp vendor/clang/dist-release_60/test/Modules/ExtDebugInfo.cpp vendor/clang/dist-release_60/test/Modules/Inputs/DebugCXX.h vendor/clang/dist-release_60/test/Modules/ModuleDebugInfo.cpp vendor/clang/dist-release_60/test/Preprocessor/cuda-types.cu vendor/clang/dist-release_60/test/Sema/tautological-constant-compare.c vendor/clang/dist-release_60/test/SemaCXX/cxx1z-class-template-argument-deduction.cpp vendor/clang/dist-release_60/test/SemaTemplate/alignas.cpp vendor/clang/dist-release_60/test/SemaTemplate/cxx17-inline-variables.cpp vendor/clang/dist-release_60/unittests/Lex/LexerTest.cpp Added: vendor/clang/dist-release_60/docs/OpenMPSupport.rst ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/docs/OpenMPSupport.rst Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,68 @@ +.. raw:: html + + + +.. role:: none +.. role:: partial +.. role:: good + +================== +OpenMP Support +================== + +Clang fully supports OpenMP 3.1 + some elements of OpenMP 4.5. Clang supports offloading to X86_64, AArch64 and PPC64[LE] devices. +Support for Cuda devices is not ready yet. +The status of major OpenMP 4.5 features support in Clang. + +Standalone directives +===================== + +* #pragma omp [for] simd: :good:`Complete`. + +* #pragma omp declare simd: :partial:`Partial`. We support parsing/semantic + analysis + generation of special attributes for X86 target, but still + missing the LLVM pass for vectorization. + +* #pragma omp taskloop [simd]: :good:`Complete`. + +* #pragma omp target [enter|exit] data: :good:`Complete`. + +* #pragma omp target update: :good:`Complete`. + +* #pragma omp target: :partial:`Partial`. No support for the `depend` clauses. + +* #pragma omp declare target: :partial:`Partial`. No full codegen support. + +* #pragma omp teams: :good:`Complete`. + +* #pragma omp distribute [simd]: :good:`Complete`. + +* #pragma omp distribute parallel for [simd]: :good:`Complete`. + +Combined directives +=================== + +* #pragma omp parallel for simd: :good:`Complete`. + +* #pragma omp target parallel: :partial:`Partial`. No support for the `depend` clauses. + +* #pragma omp target parallel for [simd]: :partial:`Partial`. No support for the `depend` clauses. + +* #pragma omp target simd: :partial:`Partial`. No support for the `depend` clauses. + +* #pragma omp target teams: :partial:`Partial`. No support for the `depend` clauses. + +* #pragma omp teams distribute [simd]: :good:`Complete`. + +* #pragma omp target teams distribute [simd]: :partial:`Partial`. No support for the and `depend` clauses. + +* #pragma omp teams distribute parallel for [simd]: :good:`Complete`. + +* #pragma omp target teams distribute parallel for [simd]: :partial:`Partial`. No full codegen support. + +Clang does not support any constructs/updates from upcoming OpenMP 5.0 except for `reduction`-based clauses in the `task` and `target`-based directives. +In addition, the LLVM OpenMP runtime `libomp` supports the OpenMP Tools Interface (OMPT) on x86, x86_64, AArch64, and PPC64 on Linux, Windows, and mac OS. Modified: vendor/clang/dist-release_60/docs/ReleaseNotes.rst ============================================================================== --- vendor/clang/dist-release_60/docs/ReleaseNotes.rst Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/docs/ReleaseNotes.rst Wed Jan 24 20:25:37 2018 (r328364) @@ -163,6 +163,15 @@ Attribute Changes in Clang - The presence of __attribute__((availability(...))) on a declaration no longer implies default visibility for that declaration on macOS. +- Clang now supports configuration files. These are collections of driver + options, which can be applied by specifying the configuration file, either + using command line option `--config foo.cfg` or encoding it into executable + name `foo-clang`. Clang behaves as if the options from this file were inserted + before the options specified in command line. This feature is primary intended + to facilitate cross compilation. Details can be found in + `Clang Compiler User's Manual + `. + - ... Windows Support @@ -209,7 +218,7 @@ OpenCL C Language Changes in Clang OpenMP Support in Clang ---------------------------------- -- Added options `-f[no]-openmp-simd` that support code emission only foe OpenMP +- Added options `-f[no]-openmp-simd` that support code emission only for OpenMP SIMD-based directives, like `#pragma omp simd`, `#pragma omp parallel for simd` etc. The code is emitted only for simd-based part of the combined directives and clauses. @@ -221,6 +230,13 @@ OpenMP Support in Clang - Added support for `reduction`-based clauses on `task`-based directives from upcoming OpenMP 5.0. + +- The LLVM OpenMP runtime `libomp` now supports the OpenMP Tools Interface (OMPT) + on x86, x86_64, AArch64, and PPC64 on Linux, Windows, and macOS. If you observe + a measurable performance impact on one of your applications without a tool + attached, please rebuild the runtime library with `-DLIBOMP_OMPT_SUPPORT=OFF` and + file a bug at `LLVM's Bugzilla `_ or send a message to the + `OpenMP development list `_. Internal API Changes -------------------- Modified: vendor/clang/dist-release_60/docs/index.rst ============================================================================== --- vendor/clang/dist-release_60/docs/index.rst Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/docs/index.rst Wed Jan 24 20:25:37 2018 (r328364) @@ -39,6 +39,7 @@ Using Clang as a Compiler SourceBasedCodeCoverage Modules MSVCCompatibility + OpenMPSupport ThinLTO CommandGuide/index FAQ Modified: vendor/clang/dist-release_60/include/clang/Basic/Attr.td ============================================================================== --- vendor/clang/dist-release_60/include/clang/Basic/Attr.td Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/include/clang/Basic/Attr.td Wed Jan 24 20:25:37 2018 (r328364) @@ -549,6 +549,7 @@ def Aligned : InheritableAttr { Keyword<"_Alignas">]>, Accessor<"isDeclspec",[Declspec<"align">]>]; let Documentation = [Undocumented]; + let DuplicatesAllowedWhileMerging = 1; } def AlignValue : Attr { Modified: vendor/clang/dist-release_60/include/clang/Basic/BuiltinsX86.def ============================================================================== --- vendor/clang/dist-release_60/include/clang/Basic/BuiltinsX86.def Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/include/clang/Basic/BuiltinsX86.def Wed Jan 24 20:25:37 2018 (r328364) @@ -1357,15 +1357,15 @@ TARGET_BUILTIN(__builtin_ia32_vpshrdvw128_maskz, "V8sV TARGET_BUILTIN(__builtin_ia32_vpshrdvw256_maskz, "V16sV16sV16sV16sUs", "", "avx512vl,avx512vbmi2") TARGET_BUILTIN(__builtin_ia32_vpshrdvw512_maskz, "V32sV32sV32sV32sUi", "", "avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdd128_mask, "V4iV4iV4iiV4iUc", "", "avx512vl,avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdd256_mask, "V8iV8iV8iiV8iUc", "", "avx512vl,avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdd512_mask, "V16iV16iV16iiV16iUs", "", "avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdq128_mask, "V2LLiV2LLiV2LLiiV2LLiUc", "", "avx512vl,avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdq256_mask, "V4LLiV4LLiV4LLiiV4LLiUc", "", "avx512vl,avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdq512_mask, "V8LLiV8LLiV8LLiiV8LLiUc", "", "avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdw128_mask, "V8sV8sV8siV8sUc", "", "avx512vl,avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdw256_mask, "V16sV16sV16siV16sUs", "", "avx512vl,avx512vbmi2") -TARGET_BUILTIN(__builtin_ia32_vpshrdw512_mask, "V32sV32sV32siV32sUi", "", "avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdd128_mask, "V4iV4iV4iIiV4iUc", "", "avx512vl,avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdd256_mask, "V8iV8iV8iIiV8iUc", "", "avx512vl,avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdd512_mask, "V16iV16iV16iIiV16iUs", "", "avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdq128_mask, "V2LLiV2LLiV2LLiIiV2LLiUc", "", "avx512vl,avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdq256_mask, "V4LLiV4LLiV4LLiIiV4LLiUc", "", "avx512vl,avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdq512_mask, "V8LLiV8LLiV8LLiIiV8LLiUc", "", "avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdw128_mask, "V8sV8sV8sIiV8sUc", "", "avx512vl,avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdw256_mask, "V16sV16sV16sIiV16sUs", "", "avx512vl,avx512vbmi2") +TARGET_BUILTIN(__builtin_ia32_vpshrdw512_mask, "V32sV32sV32sIiV32sUi", "", "avx512vbmi2") TARGET_BUILTIN(__builtin_ia32_pmovswb512_mask, "V32cV32sV32cUi", "", "avx512bw") TARGET_BUILTIN(__builtin_ia32_pmovuswb512_mask, "V32cV32sV32cUi", "", "avx512bw") Modified: vendor/clang/dist-release_60/include/clang/Basic/DiagnosticGroups.td ============================================================================== --- vendor/clang/dist-release_60/include/clang/Basic/DiagnosticGroups.td Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/include/clang/Basic/DiagnosticGroups.td Wed Jan 24 20:25:37 2018 (r328364) @@ -444,8 +444,7 @@ def TautologicalInRangeCompare : DiagGroup<"tautologic TautologicalUnsignedEnumZeroCompare]>; def TautologicalOutOfRangeCompare : DiagGroup<"tautological-constant-out-of-range-compare">; def TautologicalConstantCompare : DiagGroup<"tautological-constant-compare", - [TautologicalInRangeCompare, - TautologicalOutOfRangeCompare]>; + [TautologicalOutOfRangeCompare]>; def TautologicalPointerCompare : DiagGroup<"tautological-pointer-compare">; def TautologicalOverlapCompare : DiagGroup<"tautological-overlap-compare">; def TautologicalUndefinedCompare : DiagGroup<"tautological-undefined-compare">; @@ -719,7 +718,6 @@ def IntToPointerCast : DiagGroup<"int-to-pointer-cast" def Move : DiagGroup<"move", [PessimizingMove, RedundantMove, SelfMove]>; def Extra : DiagGroup<"extra", [ - TautologicalInRangeCompare, MissingFieldInitializers, IgnoredQualifiers, InitializerOverrides, Modified: vendor/clang/dist-release_60/include/clang/Basic/TokenKinds.def ============================================================================== --- vendor/clang/dist-release_60/include/clang/Basic/TokenKinds.def Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/include/clang/Basic/TokenKinds.def Wed Jan 24 20:25:37 2018 (r328364) @@ -398,7 +398,6 @@ TYPE_TRAIT_2(__builtin_types_compatible_p, TypeCompati KEYWORD(__builtin_va_arg , KEYALL) KEYWORD(__extension__ , KEYALL) KEYWORD(__float128 , KEYALL) -ALIAS("_Float128", __float128 , KEYNOCXX) KEYWORD(__imag , KEYALL) KEYWORD(__int128 , KEYALL) KEYWORD(__label__ , KEYALL) Modified: vendor/clang/dist-release_60/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h ============================================================================== --- vendor/clang/dist-release_60/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h Wed Jan 24 20:25:37 2018 (r328364) @@ -32,27 +32,39 @@ class BugType { (private) const CheckName Check; const std::string Name; const std::string Category; - bool SuppressonSink; + const CheckerBase *Checker; + bool SuppressOnSink; virtual void anchor(); + public: - BugType(class CheckName check, StringRef name, StringRef cat) - : Check(check), Name(name), Category(cat), SuppressonSink(false) {} - BugType(const CheckerBase *checker, StringRef name, StringRef cat) - : Check(checker->getCheckName()), Name(name), Category(cat), - SuppressonSink(false) {} - virtual ~BugType() {} + BugType(CheckName Check, StringRef Name, StringRef Cat) + : Check(Check), Name(Name), Category(Cat), Checker(nullptr), + SuppressOnSink(false) {} + BugType(const CheckerBase *Checker, StringRef Name, StringRef Cat) + : Check(Checker->getCheckName()), Name(Name), Category(Cat), + Checker(Checker), SuppressOnSink(false) {} + virtual ~BugType() = default; - // FIXME: Should these be made strings as well? StringRef getName() const { return Name; } StringRef getCategory() const { return Category; } - StringRef getCheckName() const { return Check.getName(); } + StringRef getCheckName() const { + // FIXME: This is a workaround to ensure that the correct check name is used + // The check names are set after the constructors are run. + // In case the BugType object is initialized in the checker's ctor + // the Check field will be empty. To circumvent this problem we use + // CheckerBase whenever it is possible. + StringRef CheckName = + Checker ? Checker->getCheckName().getName() : Check.getName(); + assert(!CheckName.empty() && "Check name is not set properly."); + return CheckName; + } /// isSuppressOnSink - Returns true if bug reports associated with this bug /// type should be suppressed if the end node of the report is post-dominated /// by a sink node. - bool isSuppressOnSink() const { return SuppressonSink; } - void setSuppressOnSink(bool x) { SuppressonSink = x; } + bool isSuppressOnSink() const { return SuppressOnSink; } + void setSuppressOnSink(bool x) { SuppressOnSink = x; } virtual void FlushReports(BugReporter& BR); }; @@ -74,7 +86,7 @@ class BuiltinBug : public BugType { (public) StringRef getDescription() const { return desc; } }; -} // end GR namespace +} // end ento namespace } // end clang namespace #endif Modified: vendor/clang/dist-release_60/lib/AST/DeclBase.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/AST/DeclBase.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/AST/DeclBase.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -891,12 +891,14 @@ bool Decl::AccessDeclContextSanity() const { // 4. the context is not a record // 5. it's invalid // 6. it's a C++0x static_assert. + // 7. it's a block literal declaration if (isa(this) || isa(this) || isa(this) || !isa(getDeclContext()) || isInvalidDecl() || isa(this) || + isa(this) || // FIXME: a ParmVarDecl can have ClassTemplateSpecialization // as DeclContext (?). isa(this) || Modified: vendor/clang/dist-release_60/lib/AST/ODRHash.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/AST/ODRHash.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/AST/ODRHash.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -478,6 +478,8 @@ void ODRHash::AddFunctionDecl(const FunctionDecl *Func // TODO: Fix hashing for class methods. if (isa(Function)) return; + // And friend functions. + if (Function->getFriendObjectKind()) return; // Skip functions that are specializations or in specialization context. const DeclContext *DC = Function; Modified: vendor/clang/dist-release_60/lib/CodeGen/CGBuiltin.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/CodeGen/CGBuiltin.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/CodeGen/CGBuiltin.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -915,7 +915,11 @@ EmitCheckedMixedSignMultiply(CodeGenFunction &CGF, con Overflow = CGF.Builder.CreateOr(Overflow, TruncOverflow); } - Result = CGF.Builder.CreateTrunc(UnsignedResult, ResTy); + // Negate the product if it would be negative in infinite precision. + Result = CGF.Builder.CreateSelect( + IsNegative, CGF.Builder.CreateNeg(UnsignedResult), UnsignedResult); + + Result = CGF.Builder.CreateTrunc(Result, ResTy); } assert(Overflow && Result && "Missing overflow or result"); Modified: vendor/clang/dist-release_60/lib/CodeGen/ObjectFilePCHContainerOperations.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/CodeGen/ObjectFilePCHContainerOperations.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/CodeGen/ObjectFilePCHContainerOperations.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -229,6 +229,11 @@ class PCHContainerGenerator : public ASTConsumer { (pu Builder->getModuleDebugInfo()->completeRequiredType(RD); } + void HandleImplicitImportDecl(ImportDecl *D) override { + if (!D->getImportedOwningModule()) + Builder->getModuleDebugInfo()->EmitImportDecl(*D); + } + /// Emit a container holding the serialized AST. void HandleTranslationUnit(ASTContext &Ctx) override { assert(M && VMContext && Builder); Modified: vendor/clang/dist-release_60/lib/Frontend/InitPreprocessor.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Frontend/InitPreprocessor.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Frontend/InitPreprocessor.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -817,10 +817,6 @@ static void InitializePredefinedMacros(const TargetInf DefineFloatMacros(Builder, "FLT", &TI.getFloatFormat(), "F"); DefineFloatMacros(Builder, "DBL", &TI.getDoubleFormat(), ""); DefineFloatMacros(Builder, "LDBL", &TI.getLongDoubleFormat(), "L"); - if (TI.hasFloat128Type()) - // FIXME: Switch away from the non-standard "Q" when we can - DefineFloatMacros(Builder, "FLT128", &TI.getFloat128Format(), "Q"); - // Define a __POINTER_WIDTH__ macro for stdint.h. Builder.defineMacro("__POINTER_WIDTH__", Modified: vendor/clang/dist-release_60/lib/Lex/Lexer.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Lex/Lexer.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Lex/Lexer.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -2009,18 +2009,21 @@ bool Lexer::LexAngledStringLiteral(Token &Result, cons const char *AfterLessPos = CurPtr; char C = getAndAdvanceChar(CurPtr, Result); while (C != '>') { - // Skip escaped characters. - if (C == '\\' && CurPtr < BufferEnd) { - // Skip the escaped character. - getAndAdvanceChar(CurPtr, Result); - } else if (C == '\n' || C == '\r' || // Newline. - (C == 0 && (CurPtr-1 == BufferEnd || // End of file. - isCodeCompletionPoint(CurPtr-1)))) { + // Skip escaped characters. Escaped newlines will already be processed by + // getAndAdvanceChar. + if (C == '\\') + C = getAndAdvanceChar(CurPtr, Result); + + if (C == '\n' || C == '\r' || // Newline. + (C == 0 && (CurPtr-1 == BufferEnd || // End of file. + isCodeCompletionPoint(CurPtr-1)))) { // If the filename is unterminated, then it must just be a lone < // character. Return this as such. FormTokenWithChars(Result, AfterLessPos, tok::less); return true; - } else if (C == 0) { + } + + if (C == 0) { NulCharacter = CurPtr-1; } C = getAndAdvanceChar(CurPtr, Result); Modified: vendor/clang/dist-release_60/lib/Lex/PPCaching.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Lex/PPCaching.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Lex/PPCaching.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -105,8 +105,10 @@ void Preprocessor::CachingLex(Token &Result) { } void Preprocessor::EnterCachingLexMode() { - if (InCachingLexMode()) + if (InCachingLexMode()) { + assert(CurLexerKind == CLK_CachingLexer && "Unexpected lexer kind"); return; + } PushIncludeMacroStack(); CurLexerKind = CLK_CachingLexer; Modified: vendor/clang/dist-release_60/lib/Lex/PPLexerChange.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Lex/PPLexerChange.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Lex/PPLexerChange.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -444,6 +444,7 @@ bool Preprocessor::HandleEndOfFile(Token &Result, bool } CurPPLexer = nullptr; + recomputeCurLexerKind(); return true; } Modified: vendor/clang/dist-release_60/lib/Sema/Scope.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Sema/Scope.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Sema/Scope.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -143,72 +143,43 @@ void Scope::dumpImpl(raw_ostream &OS) const { if (HasFlags) OS << "Flags: "; - while (Flags) { - if (Flags & FnScope) { - OS << "FnScope"; - Flags &= ~FnScope; - } else if (Flags & BreakScope) { - OS << "BreakScope"; - Flags &= ~BreakScope; - } else if (Flags & ContinueScope) { - OS << "ContinueScope"; - Flags &= ~ContinueScope; - } else if (Flags & DeclScope) { - OS << "DeclScope"; - Flags &= ~DeclScope; - } else if (Flags & ControlScope) { - OS << "ControlScope"; - Flags &= ~ControlScope; - } else if (Flags & ClassScope) { - OS << "ClassScope"; - Flags &= ~ClassScope; - } else if (Flags & BlockScope) { - OS << "BlockScope"; - Flags &= ~BlockScope; - } else if (Flags & TemplateParamScope) { - OS << "TemplateParamScope"; - Flags &= ~TemplateParamScope; - } else if (Flags & FunctionPrototypeScope) { - OS << "FunctionPrototypeScope"; - Flags &= ~FunctionPrototypeScope; - } else if (Flags & FunctionDeclarationScope) { - OS << "FunctionDeclarationScope"; - Flags &= ~FunctionDeclarationScope; - } else if (Flags & AtCatchScope) { - OS << "AtCatchScope"; - Flags &= ~AtCatchScope; - } else if (Flags & ObjCMethodScope) { - OS << "ObjCMethodScope"; - Flags &= ~ObjCMethodScope; - } else if (Flags & SwitchScope) { - OS << "SwitchScope"; - Flags &= ~SwitchScope; - } else if (Flags & TryScope) { - OS << "TryScope"; - Flags &= ~TryScope; - } else if (Flags & FnTryCatchScope) { - OS << "FnTryCatchScope"; - Flags &= ~FnTryCatchScope; - } else if (Flags & SEHTryScope) { - OS << "SEHTryScope"; - Flags &= ~SEHTryScope; - } else if (Flags & SEHExceptScope) { - OS << "SEHExceptScope"; - Flags &= ~SEHExceptScope; - } else if (Flags & OpenMPDirectiveScope) { - OS << "OpenMPDirectiveScope"; - Flags &= ~OpenMPDirectiveScope; - } else if (Flags & OpenMPLoopDirectiveScope) { - OS << "OpenMPLoopDirectiveScope"; - Flags &= ~OpenMPLoopDirectiveScope; - } else if (Flags & OpenMPSimdDirectiveScope) { - OS << "OpenMPSimdDirectiveScope"; - Flags &= ~OpenMPSimdDirectiveScope; - } + std::pair FlagInfo[] = { + {FnScope, "FnScope"}, + {BreakScope, "BreakScope"}, + {ContinueScope, "ContinueScope"}, + {DeclScope, "DeclScope"}, + {ControlScope, "ControlScope"}, + {ClassScope, "ClassScope"}, + {BlockScope, "BlockScope"}, + {TemplateParamScope, "TemplateParamScope"}, + {FunctionPrototypeScope, "FunctionPrototypeScope"}, + {FunctionDeclarationScope, "FunctionDeclarationScope"}, + {AtCatchScope, "AtCatchScope"}, + {ObjCMethodScope, "ObjCMethodScope"}, + {SwitchScope, "SwitchScope"}, + {TryScope, "TryScope"}, + {FnTryCatchScope, "FnTryCatchScope"}, + {OpenMPDirectiveScope, "OpenMPDirectiveScope"}, + {OpenMPLoopDirectiveScope, "OpenMPLoopDirectiveScope"}, + {OpenMPSimdDirectiveScope, "OpenMPSimdDirectiveScope"}, + {EnumScope, "EnumScope"}, + {SEHTryScope, "SEHTryScope"}, + {SEHExceptScope, "SEHExceptScope"}, + {SEHFilterScope, "SEHFilterScope"}, + {CompoundStmtScope, "CompoundStmtScope"}, + {ClassInheritanceScope, "ClassInheritanceScope"}}; - if (Flags) - OS << " | "; + for (auto Info : FlagInfo) { + if (Flags & Info.first) { + OS << Info.second; + Flags &= ~Info.first; + if (Flags) + OS << " | "; + } } + + assert(Flags == 0 && "Unknown scope flags"); + if (HasFlags) OS << '\n'; Modified: vendor/clang/dist-release_60/lib/Sema/SemaTemplateDeduction.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Sema/SemaTemplateDeduction.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Sema/SemaTemplateDeduction.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -502,6 +502,10 @@ DeduceTemplateArguments(Sema &S, SmallVectorImpl &Deduced) { assert(Arg.isCanonical() && "Argument type must be canonical"); + // Treat an injected-class-name as its underlying template-id. + if (auto *Injected = dyn_cast(Arg)) + Arg = Injected->getInjectedSpecializationType(); + // Check whether the template argument is a dependent template-id. if (const TemplateSpecializationType *SpecArg = dyn_cast(Arg)) { Modified: vendor/clang/dist-release_60/lib/Sema/SemaTemplateInstantiateDecl.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/Sema/SemaTemplateInstantiateDecl.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/Sema/SemaTemplateInstantiateDecl.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -4160,7 +4160,8 @@ void Sema::BuildVariableInstantiation( // it right away if the type contains 'auto'. if ((!isa(NewVar) && !InstantiatingVarTemplate && - !(OldVar->isInline() && OldVar->isThisDeclarationADefinition())) || + !(OldVar->isInline() && OldVar->isThisDeclarationADefinition() && + !NewVar->isThisDeclarationADefinition())) || NewVar->getType()->isUndeducedType()) InstantiateVariableInitializer(NewVar, OldVar, TemplateArgs); Modified: vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/MallocChecker.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/MallocChecker.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/MallocChecker.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -2900,8 +2900,13 @@ void ento::registerNewDeleteLeaksChecker(CheckerManage mgr.getCurrentCheckName(); // We currently treat NewDeleteLeaks checker as a subchecker of NewDelete // checker. - if (!checker->ChecksEnabled[MallocChecker::CK_NewDeleteChecker]) + if (!checker->ChecksEnabled[MallocChecker::CK_NewDeleteChecker]) { checker->ChecksEnabled[MallocChecker::CK_NewDeleteChecker] = true; + // FIXME: This does not set the correct name, but without this workaround + // no name will be set at all. + checker->CheckNames[MallocChecker::CK_NewDeleteChecker] = + mgr.getCurrentCheckName(); + } } #define REGISTER_CHECKER(name) \ Modified: vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/ValistChecker.cpp ============================================================================== --- vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/ValistChecker.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/lib/StaticAnalyzer/Checkers/ValistChecker.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -64,7 +64,7 @@ class ValistChecker : public CheckersetSuppressOnSink(true); } @@ -375,7 +379,7 @@ void ValistChecker::checkVAListEndCall(const CallEvent std::shared_ptr ValistChecker::ValistBugVisitor::VisitNode( const ExplodedNode *N, const ExplodedNode *PrevN, BugReporterContext &BRC, - BugReport &BR) { + BugReport &) { ProgramStateRef State = N->getState(); ProgramStateRef StatePrev = PrevN->getState(); Modified: vendor/clang/dist-release_60/test/Analysis/malloc.c ============================================================================== --- vendor/clang/dist-release_60/test/Analysis/malloc.c Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/Analysis/malloc.c Wed Jan 24 20:25:37 2018 (r328364) @@ -1720,13 +1720,6 @@ void *smallocWarn(size_t size) { } } -char *dupstrWarn(const char *s) { - const int len = strlen(s); - char *p = (char*) smallocWarn(len + 1); - strcpy(p, s); // expected-warning{{String copy function overflows destination buffer}} - return p; -} - int *radar15580979() { int *data = (int *)malloc(32); int *p = data ?: (int*)malloc(32); // no warning Added: vendor/clang/dist-release_60/test/CodeCompletion/Inputs/comments.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/CodeCompletion/Inputs/comments.h Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,4 @@ +// PR32732 +struct B { + // <- code completion +}; Added: vendor/clang/dist-release_60/test/CodeCompletion/comments.cpp ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/CodeCompletion/comments.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,13 @@ +// Note: the run lines follow their respective tests, since line/column +// matter in this test. + +#include "comments.h" + +struct A { + // <- code completion + /* <- code completion */ +}; + +// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%s:7:6 %s +// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%s:8:6 %s +// RUN: %clang_cc1 -I %S/Inputs -fsyntax-only -code-completion-at=%S/Inputs/comments.h:3:6 %s Modified: vendor/clang/dist-release_60/test/CodeGen/builtins-overflow.c ============================================================================== --- vendor/clang/dist-release_60/test/CodeGen/builtins-overflow.c Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/CodeGen/builtins-overflow.c Wed Jan 24 20:25:37 2018 (r328364) @@ -373,7 +373,9 @@ int test_mixed_sign_mull_overflow_unsigned(int x, unsi // CHECK-NEXT: [[NotNull:%.*]] = icmp ne i32 [[UnsignedResult]], 0 // CHECK-NEXT: [[Underflow:%.*]] = and i1 [[IsNeg]], [[NotNull]] // CHECK-NEXT: [[OFlow:%.*]] = or i1 [[UnsignedOFlow]], [[Underflow]] -// CHECK-NEXT: store i32 [[UnsignedResult]], i32* %{{.*}}, align 4 +// CHECK-NEXT: [[NegatedResult:%.*]] = sub i32 0, [[UnsignedResult]] +// CHECK-NEXT: [[Result:%.*]] = select i1 [[IsNeg]], i32 [[NegatedResult]], i32 [[UnsignedResult]] +// CHECK-NEXT: store i32 [[Result]], i32* %{{.*}}, align 4 // CHECK: br i1 [[OFlow]] unsigned result; @@ -432,7 +434,9 @@ long long test_mixed_sign_mulll_overflow_trunc_unsigne // CHECK-NEXT: [[OVERFLOW_PRE_TRUNC:%.*]] = or i1 {{.*}}, [[UNDERFLOW]] // CHECK-NEXT: [[TRUNC_OVERFLOW:%.*]] = icmp ugt i64 [[UNSIGNED_RESULT]], 4294967295 // CHECK-NEXT: [[OVERFLOW:%.*]] = or i1 [[OVERFLOW_PRE_TRUNC]], [[TRUNC_OVERFLOW]] -// CHECK-NEXT: trunc i64 [[UNSIGNED_RESULT]] to i32 +// CHECK-NEXT: [[NEGATED:%.*]] = sub i64 0, [[UNSIGNED_RESULT]] +// CHECK-NEXT: [[RESULT:%.*]] = select i1 {{.*}}, i64 [[NEGATED]], i64 [[UNSIGNED_RESULT]] +// CHECK-NEXT: trunc i64 [[RESULT]] to i32 // CHECK-NEXT: store unsigned result; if (__builtin_mul_overflow(y, x, &result)) Modified: vendor/clang/dist-release_60/test/CodeGenCXX/cxx1z-inline-variables.cpp ============================================================================== --- vendor/clang/dist-release_60/test/CodeGenCXX/cxx1z-inline-variables.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/CodeGenCXX/cxx1z-inline-variables.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -58,14 +58,22 @@ template struct X { static int a; static inline int b; static int c; + static const int d; + static int e; }; // CHECK: @_ZN1XIiE1aE = linkonce_odr global i32 10 // CHECK: @_ZN1XIiE1bE = global i32 20 // CHECK-NOT: @_ZN1XIiE1cE +// CHECK: @_ZN1XIiE1dE = linkonce_odr constant i32 40 +// CHECK: @_ZN1XIiE1eE = linkonce_odr global i32 50 template<> inline int X::a = 10; int &use3 = X::a; template<> int X::b = 20; template<> inline int X::c = 30; +template constexpr int X::d = 40; +template inline int X::e = 50; +const int *use_x_int_d = &X::d; +const int *use_x_int_e = &X::e; template struct Y; template<> struct Y { Added: vendor/clang/dist-release_60/test/Lexer/null-character-in-literal.c ============================================================================== Binary file. No diff available. Modified: vendor/clang/dist-release_60/test/Modules/ExtDebugInfo.cpp ============================================================================== --- vendor/clang/dist-release_60/test/Modules/ExtDebugInfo.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/Modules/ExtDebugInfo.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -187,7 +187,7 @@ void foo() { // CHECK: !DIGlobalVariable(name: "anon_enum", {{.*}}, type: ![[ANON_ENUM:[0-9]+]] // CHECK: !DICompositeType(tag: DW_TAG_enumeration_type, scope: ![[NS]], -// CHECK-SAME: line: 16 +// CHECK-SAME: line: 19 // CHECK: !DIGlobalVariable(name: "GlobalUnion", // CHECK-SAME: type: ![[GLOBAL_UNION:[0-9]+]] Modified: vendor/clang/dist-release_60/test/Modules/Inputs/DebugCXX.h ============================================================================== --- vendor/clang/dist-release_60/test/Modules/Inputs/DebugCXX.h Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/Modules/Inputs/DebugCXX.h Wed Jan 24 20:25:37 2018 (r328364) @@ -1,4 +1,7 @@ /* -*- C++ -*- */ + +#include "dummy.h" + namespace DebugCXX { // Records. struct Struct { Added: vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/Box.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/Box.h Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,14 @@ +template +struct iterator { + void Compare(const iterator &x) { } + friend void Check(iterator) {} +}; + +template struct Box { + iterator I; + + void test() { + Check(I); + I.Compare(I); + } +}; Added: vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M1.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M1.h Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,6 @@ +#include "Box.h" + +void Peek() { + Box<> Gift; + Gift.test(); +} Added: vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M2.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M2.h Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,5 @@ +#include "Box.h" +void x() { + Box<> Unused; + //Unused.test(); +} Added: vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M3.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/M3.h Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,7 @@ +#include "Box.h" +#include "M2.h" + +void Party() { + Box<> Present; + Present.test(); +} Added: vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/module.modulemap ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/Inputs/odr_hash-Friend/module.modulemap Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,15 @@ +module Box { + header "Box.h" +} + +module Module1 { + header "M1.h" +} + +module Module2 { + header "M2.h" +} + +module Module3 { + header "M3.h" +} Modified: vendor/clang/dist-release_60/test/Modules/ModuleDebugInfo.cpp ============================================================================== --- vendor/clang/dist-release_60/test/Modules/ModuleDebugInfo.cpp Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/Modules/ModuleDebugInfo.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -5,12 +5,13 @@ // Modules: // RUN: rm -rf %t -// RUN: %clang_cc1 -triple %itanium_abi_triple -x objective-c++ -std=c++11 -debug-info-kind=limited -fmodules -fmodule-format=obj -fimplicit-module-maps -DMODULES -fmodules-cache-path=%t %s -I %S/Inputs -I %t -emit-llvm -o %t.ll -mllvm -debug-only=pchcontainer &>%t-mod.ll +// RUN: %clang_cc1 -triple %itanium_abi_triple -x objective-c++ -std=c++11 -debugger-tuning=lldb -debug-info-kind=limited -fmodules -fmodule-format=obj -fimplicit-module-maps -DMODULES -fmodules-cache-path=%t %s -I %S/Inputs -I %t -emit-llvm -o %t.ll -mllvm -debug-only=pchcontainer &>%t-mod.ll // RUN: cat %t-mod.ll | FileCheck %s // RUN: cat %t-mod.ll | FileCheck --check-prefix=CHECK-NEG %s +// RUN: cat %t-mod.ll | FileCheck --check-prefix=CHECK-MOD %s // PCH: -// RUN: %clang_cc1 -triple %itanium_abi_triple -x c++ -std=c++11 -emit-pch -fmodule-format=obj -I %S/Inputs -o %t.pch %S/Inputs/DebugCXX.h -mllvm -debug-only=pchcontainer &>%t-pch.ll +// RUN: %clang_cc1 -triple %itanium_abi_triple -x c++ -std=c++11 -debugger-tuning=lldb -emit-pch -fmodule-format=obj -I %S/Inputs -o %t.pch %S/Inputs/DebugCXX.h -mllvm -debug-only=pchcontainer &>%t-pch.ll // RUN: cat %t-pch.ll | FileCheck %s // RUN: cat %t-pch.ll | FileCheck --check-prefix=CHECK-NEG %s @@ -18,6 +19,9 @@ @import DebugCXX; #endif +// CHECK-MOD: distinct !DICompileUnit(language: DW_LANG_{{.*}}C_plus_plus, +// CHECK-MOD: distinct !DICompileUnit(language: DW_LANG_{{.*}}C_plus_plus, + // CHECK: distinct !DICompileUnit(language: DW_LANG_{{.*}}C_plus_plus, // CHECK-SAME: isOptimized: false, // CHECK-NOT: splitDebugFilename: @@ -27,6 +31,8 @@ // CHECK-SAME: identifier: "_ZTSN8DebugCXX4EnumE") // CHECK: !DINamespace(name: "DebugCXX" +// CHECK-MOD: ![[DEBUGCXX:.*]] = !DIModule(scope: null, name: "DebugCXX + // CHECK: !DICompositeType(tag: DW_TAG_enumeration_type, // CHECK-NOT: name: // CHECK-SAME: ) @@ -149,5 +155,12 @@ // CHECK: ![[SPECIALIZEDBASE]] = !DICompositeType(tag: DW_TAG_class_type, // CHECK-SAME: name: "WithSpecializedBase", // CHECK-SAME: flags: DIFlagFwdDecl, + +// CHECK-MOD: !DIImportedEntity(tag: DW_TAG_imported_declaration, scope: ![[DEBUGCXX]], +// CHECK-MOD-SAME: entity: ![[DUMMY:[0-9]+]], +// CHECK-MOD-SAME: line: 3) +// CHECK-MOD: ![[DUMMY]] = !DIModule(scope: null, name: "dummy", +// CHECK-MOD: distinct !DICompileUnit(language: DW_LANG_ObjC_plus_plus, +// CHECK-MOD-SAME: splitDebugFilename: "{{.*}}dummy{{.*}}.pcm", // CHECK-NEG-NOT: !DICompositeType(tag: DW_TAG_structure_type, name: "PureForwardDecl" Added: vendor/clang/dist-release_60/test/Modules/odr_hash-Friend.cpp ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/odr_hash-Friend.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,22 @@ +// RUN: rm -rf %t + +// RUN: %clang_cc1 -fmodules -fmodules-cache-path=%t/modules.cache \ +// RUN: -I %S/Inputs/odr_hash-Friend \ +// RUN: -emit-obj -o /dev/null \ +// RUN: -fmodules \ +// RUN: -fimplicit-module-maps \ +// RUN: -fmodules-cache-path=%t/modules.cache \ +// RUN: -std=c++11 -x c++ %s -verify + +// PR35939: MicrosoftMangle.cpp triggers an assertion failure on this test. +// UNSUPPORTED: system-windows + +// expected-no-diagnostics + +#include "Box.h" +#include "M1.h" +#include "M3.h" + +void Run() { + Box<> Present; +} Added: vendor/clang/dist-release_60/test/Modules/odr_hash-blocks.cpp ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/clang/dist-release_60/test/Modules/odr_hash-blocks.cpp Wed Jan 24 20:25:37 2018 (r328364) @@ -0,0 +1,119 @@ +// Clear and create directories +// RUN: rm -rf %t +// RUN: mkdir %t +// RUN: mkdir %t/cache +// RUN: mkdir %t/Inputs + +// Build first header file +// RUN: echo "#define FIRST" >> %t/Inputs/first.h +// RUN: cat %s >> %t/Inputs/first.h + +// Build second header file +// RUN: echo "#define SECOND" >> %t/Inputs/second.h +// RUN: cat %s >> %t/Inputs/second.h + +// Test that each header can compile +// RUN: %clang_cc1 -fsyntax-only -x c++ -std=c++11 -fblocks %t/Inputs/first.h +// RUN: %clang_cc1 -fsyntax-only -x c++ -std=c++11 -fblocks %t/Inputs/second.h + +// Build module map file +// RUN: echo "module FirstModule {" >> %t/Inputs/module.map +// RUN: echo " header \"first.h\"" >> %t/Inputs/module.map +// RUN: echo "}" >> %t/Inputs/module.map +// RUN: echo "module SecondModule {" >> %t/Inputs/module.map +// RUN: echo " header \"second.h\"" >> %t/Inputs/module.map +// RUN: echo "}" >> %t/Inputs/module.map + +// Run test +// RUN: %clang_cc1 -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-cache-path=%t/cache -x c++ -I%t/Inputs \ +// RUN: -verify %s -std=c++11 -fblocks + +#if !defined(FIRST) && !defined(SECOND) +#include "first.h" +#include "second.h" +#endif + +// Used for testing +#if defined(FIRST) +#define ACCESS public: +#elif defined(SECOND) +#define ACCESS private: +#endif + +// TODO: S1, S2, and S3 should generate errors. +namespace Blocks { +#if defined(FIRST) +struct S1 { + void (^block)(int x) = ^(int x) { }; +}; +#elif defined(SECOND) +struct S1 { + void (^block)(int x) = ^(int y) { }; +}; +#else +S1 s1; +#endif + +#if defined(FIRST) +struct S2 { + int (^block)(int x) = ^(int x) { return x + 1; }; +}; +#elif defined(SECOND) +struct S2 { + int (^block)(int x) = ^(int x) { return x; }; +}; +#else +S2 s2; +#endif + +#if defined(FIRST) +struct S3 { + void run(int (^block)(int x)); +}; +#elif defined(SECOND) +struct S3 { + void run(int (^block)(int x, int y)); +}; +#else +S3 s3; +#endif + +#define DECLS \ + int (^block)(int x) = ^(int x) { return x + x; }; \ + void run(int (^block)(int x, int y)); + +#if defined(FIRST) || defined(SECOND) +struct Valid1 { + DECLS +}; +#else +Valid1 v1; +#endif + +#if defined(FIRST) || defined(SECOND) +struct Invalid1 { + DECLS + ACCESS +}; +#else +Invalid1 i1; +// expected-error@second.h:* {{'Blocks::Invalid1' has different definitions in different modules; first difference is definition in module 'SecondModule' found private access specifier}} +// expected-note@first.h:* {{but in 'FirstModule' found public access specifier}} +#endif + +#undef DECLS +} + +// Keep macros contained to one file. +#ifdef FIRST +#undef FIRST +#endif + +#ifdef SECOND +#undef SECOND +#endif + +#ifdef ACCESS +#undef ACCESS +#endif Modified: vendor/clang/dist-release_60/test/Preprocessor/cuda-types.cu ============================================================================== --- vendor/clang/dist-release_60/test/Preprocessor/cuda-types.cu Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/Preprocessor/cuda-types.cu Wed Jan 24 20:25:37 2018 (r328364) @@ -9,40 +9,40 @@ // RUN: %clang --cuda-host-only -nocudainc -target i386-unknown-linux-gnu -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/i386-host-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/i386-host-defines-filtered // RUN: %clang --cuda-device-only -nocudainc -nocudalib -target i386-unknown-linux-gnu -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/i386-device-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/i386-device-defines-filtered // RUN: diff %t/i386-host-defines-filtered %t/i386-device-defines-filtered // RUN: %clang --cuda-host-only -nocudainc -target x86_64-unknown-linux-gnu -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/x86_64-host-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/x86_64-host-defines-filtered // RUN: %clang --cuda-device-only -nocudainc -nocudalib -target x86_64-unknown-linux-gnu -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/x86_64-device-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/x86_64-device-defines-filtered // RUN: diff %t/x86_64-host-defines-filtered %t/x86_64-device-defines-filtered // RUN: %clang --cuda-host-only -nocudainc -target powerpc64-unknown-linux-gnu -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/powerpc64-host-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/powerpc64-host-defines-filtered // RUN: %clang --cuda-device-only -nocudainc -nocudalib -target powerpc64-unknown-linux-gnu -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/powerpc64-device-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/powerpc64-device-defines-filtered // RUN: diff %t/powerpc64-host-defines-filtered %t/powerpc64-device-defines-filtered // RUN: %clang --cuda-host-only -nocudainc -target i386-windows-msvc -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/i386-msvc-host-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/i386-msvc-host-defines-filtered // RUN: %clang --cuda-device-only -nocudainc -nocudalib -target i386-windows-msvc -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/i386-msvc-device-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/i386-msvc-device-defines-filtered // RUN: diff %t/i386-msvc-host-defines-filtered %t/i386-msvc-device-defines-filtered // RUN: %clang --cuda-host-only -nocudainc -target x86_64-windows-msvc -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/x86_64-msvc-host-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/x86_64-msvc-host-defines-filtered // RUN: %clang --cuda-device-only -nocudainc -nocudalib -target x86_64-windows-msvc -x cuda -E -dM -o - /dev/null \ // RUN: | grep 'define __[^ ]*\(TYPE\|MAX\|SIZEOF|WIDTH\)\|define __GCC_ATOMIC' \ -// RUN: | grep -v '__FLT128\|__LDBL\|_LONG_DOUBLE' > %t/x86_64-msvc-device-defines-filtered +// RUN: | grep -v '__LDBL\|_LONG_DOUBLE' > %t/x86_64-msvc-device-defines-filtered // RUN: diff %t/x86_64-msvc-host-defines-filtered %t/x86_64-msvc-device-defines-filtered Modified: vendor/clang/dist-release_60/test/Sema/tautological-constant-compare.c ============================================================================== --- vendor/clang/dist-release_60/test/Sema/tautological-constant-compare.c Wed Jan 24 20:23:54 2018 (r328363) +++ vendor/clang/dist-release_60/test/Sema/tautological-constant-compare.c Wed Jan 24 20:25:37 2018 (r328364) @@ -2,8 +2,8 @@ // RUN: %clang_cc1 -triple x86_64-linux-gnu -fsyntax-only -Wtautological-constant-in-range-compare -DTEST -verify -x c++ %s // RUN: %clang_cc1 -triple x86_64-linux-gnu -fsyntax-only -Wtautological-type-limit-compare -DTEST -verify %s *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:25:49 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 819F0EC3C67; Wed, 24 Jan 2018 20:25:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 088AF763EA; Wed, 24 Jan 2018 20:25:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DA60722794; Wed, 24 Jan 2018 20:25:48 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKPmHT042911; Wed, 24 Jan 2018 20:25:48 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKPml1042910; Wed, 24 Jan 2018 20:25:48 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242025.w0OKPml1042910@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:25:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328366 - vendor/compiler-rt/dist-release_60/lib/builtins X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/dist-release_60/lib/builtins X-SVN-Commit-Revision: 328366 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:25:49 -0000 Author: dim Date: Wed Jan 24 20:25:48 2018 New Revision: 328366 URL: https://svnweb.freebsd.org/changeset/base/328366 Log: Vendor import of compiler-rt release_60 branch r323338: https://llvm.org/svn/llvm-project/compiler-rt/branches/release_60@323338 Modified: vendor/compiler-rt/dist-release_60/lib/builtins/clear_cache.c Modified: vendor/compiler-rt/dist-release_60/lib/builtins/clear_cache.c ============================================================================== --- vendor/compiler-rt/dist-release_60/lib/builtins/clear_cache.c Wed Jan 24 20:25:46 2018 (r328365) +++ vendor/compiler-rt/dist-release_60/lib/builtins/clear_cache.c Wed Jan 24 20:25:48 2018 (r328366) @@ -33,6 +33,11 @@ uintptr_t GetCurrentProcess(void); #include #endif +#if defined(__OpenBSD__) && defined(__mips__) + #include + #include +#endif + #if defined(__linux__) && defined(__mips__) #include #include @@ -142,6 +147,8 @@ void __clear_cache(void *start, void *end) { #else syscall(__NR_cacheflush, start, (end_int - start_int), BCACHE); #endif +#elif defined(__mips__) && defined(__OpenBSD__) + cacheflush(start, (uintptr_t)end - (uintptr_t)start, BCACHE); #elif defined(__aarch64__) && !defined(__APPLE__) uint64_t xstart = (uint64_t)(uintptr_t) start; uint64_t xend = (uint64_t)(uintptr_t) end; @@ -156,12 +163,14 @@ void __clear_cache(void *start, void *end) { * uintptr_t in case this runs in an IPL32 environment. */ const size_t dcache_line_size = 4 << ((ctr_el0 >> 16) & 15); - for (addr = xstart; addr < xend; addr += dcache_line_size) + for (addr = xstart & ~(dcache_line_size - 1); addr < xend; + addr += dcache_line_size) __asm __volatile("dc cvau, %0" :: "r"(addr)); __asm __volatile("dsb ish"); const size_t icache_line_size = 4 << ((ctr_el0 >> 0) & 15); - for (addr = xstart; addr < xend; addr += icache_line_size) + for (addr = xstart & ~(icache_line_size - 1); addr < xend; + addr += icache_line_size) __asm __volatile("ic ivau, %0" :: "r"(addr)); __asm __volatile("isb sy"); #elif defined (__powerpc64__) From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:25:48 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC71DEC3C60; Wed, 24 Jan 2018 20:25:48 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF75F763BE; Wed, 24 Jan 2018 20:25:47 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 80A6322793; Wed, 24 Jan 2018 20:25:46 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKPkZk042864; Wed, 24 Jan 2018 20:25:46 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKPkIJ042863; Wed, 24 Jan 2018 20:25:46 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242025.w0OKPkIJ042863@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:25:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328365 - vendor/clang/clang-release_60-r323338 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/clang/clang-release_60-r323338 X-SVN-Commit-Revision: 328365 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:25:49 -0000 Author: dim Date: Wed Jan 24 20:25:46 2018 New Revision: 328365 URL: https://svnweb.freebsd.org/changeset/base/328365 Log: Tag clang release_60 branch r323338. Added: vendor/clang/clang-release_60-r323338/ - copied from r328364, vendor/clang/dist-release_60/ From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:26:00 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E5B8EC3CBB; Wed, 24 Jan 2018 20:26:00 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9CA2765B5; Wed, 24 Jan 2018 20:25:59 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9206722797; Wed, 24 Jan 2018 20:25:59 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKPxS8043057; Wed, 24 Jan 2018 20:25:59 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKPx3Z043056; Wed, 24 Jan 2018 20:25:59 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242025.w0OKPx3Z043056@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:25:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328369 - vendor/libc++/libc++-release_60-r323338 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libc++/libc++-release_60-r323338 X-SVN-Commit-Revision: 328369 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:26:00 -0000 Author: dim Date: Wed Jan 24 20:25:59 2018 New Revision: 328369 URL: https://svnweb.freebsd.org/changeset/base/328369 Log: Tag libc++ release_60 branch r323338. Added: vendor/libc++/libc++-release_60-r323338/ - copied from r328368, vendor/libc++/dist-release_60/ From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:26:13 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5E49EC3D54; Wed, 24 Jan 2018 20:26:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5DA376770; Wed, 24 Jan 2018 20:26:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7ECF922799; Wed, 24 Jan 2018 20:26:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKQ8A0043173; Wed, 24 Jan 2018 20:26:08 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKQ8DF043172; Wed, 24 Jan 2018 20:26:08 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242026.w0OKQ8DF043172@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:26:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328371 - vendor/lld/lld-release_60-r323338 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lld/lld-release_60-r323338 X-SVN-Commit-Revision: 328371 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:26:13 -0000 Author: dim Date: Wed Jan 24 20:26:08 2018 New Revision: 328371 URL: https://svnweb.freebsd.org/changeset/base/328371 Log: Tag lld release_60 branch r323338. Added: vendor/lld/lld-release_60-r323338/ - copied from r328370, vendor/lld/dist-release_60/ From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:25:54 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B5FEEC3C8B; Wed, 24 Jan 2018 20:25:54 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F607764B0; Wed, 24 Jan 2018 20:25:52 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E92C422795; Wed, 24 Jan 2018 20:25:51 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKPpWR042960; Wed, 24 Jan 2018 20:25:51 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKPppe042959; Wed, 24 Jan 2018 20:25:51 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242025.w0OKPppe042959@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:25:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328367 - vendor/compiler-rt/compiler-rt-release_60-r323338 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/compiler-rt-release_60-r323338 X-SVN-Commit-Revision: 328367 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:25:54 -0000 Author: dim Date: Wed Jan 24 20:25:51 2018 New Revision: 328367 URL: https://svnweb.freebsd.org/changeset/base/328367 Log: Tag compiler-rt release_60 branch r323338. Added: vendor/compiler-rt/compiler-rt-release_60-r323338/ - copied from r328366, vendor/compiler-rt/dist-release_60/ From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:25:58 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41ED1EC3CB7; Wed, 24 Jan 2018 20:25:58 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65E0C76551; Wed, 24 Jan 2018 20:25:56 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8834C22796; Wed, 24 Jan 2018 20:25:56 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKPuW1043008; Wed, 24 Jan 2018 20:25:56 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKPuoI043006; Wed, 24 Jan 2018 20:25:56 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242025.w0OKPuoI043006@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:25:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328368 - in vendor/libc++/dist-release_60: include test/std/utilities/meta/meta.unary/meta.unary.prop X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/libc++/dist-release_60: include test/std/utilities/meta/meta.unary/meta.unary.prop X-SVN-Commit-Revision: 328368 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:25:58 -0000 Author: dim Date: Wed Jan 24 20:25:56 2018 New Revision: 328368 URL: https://svnweb.freebsd.org/changeset/base/328368 Log: Vendor import of libc++ release_60 branch r323338: https://llvm.org/svn/llvm-project/libcxx/branches/release_60@323338 Modified: vendor/libc++/dist-release_60/include/type_traits vendor/libc++/dist-release_60/test/std/utilities/meta/meta.unary/meta.unary.prop/is_constructible.pass.cpp Modified: vendor/libc++/dist-release_60/include/type_traits ============================================================================== --- vendor/libc++/dist-release_60/include/type_traits Wed Jan 24 20:25:51 2018 (r328367) +++ vendor/libc++/dist-release_60/include/type_traits Wed Jan 24 20:25:56 2018 (r328368) @@ -3172,6 +3172,14 @@ template false_type __is_constructible2_test(__any, _A0&, _A1&); +template +decltype((_Tp(_VSTD::declval<_A0>(), _VSTD::declval<_A1>(), _VSTD::declval<_A2>()), true_type())) +__is_constructible3_test(_Tp&, _A0&, _A1&, _A2&); + +template +false_type +__is_constructible3_test(__any, _A0&, _A1&, _A2&); + template struct __is_constructible0_imp // false, _Tp is not a scalar : public common_type @@ -3196,6 +3204,14 @@ struct __is_constructible2_imp // false, _Tp is not a >::type {}; +template +struct __is_constructible3_imp // false, _Tp is not a scalar + : public common_type + < + decltype(__is_constructible3_test(declval<_Tp&>(), declval<_A0>(), declval<_A1>(), declval<_A2>())) + >::type + {}; + // handle scalars and reference types // Scalars are default constructible, references are not @@ -3215,6 +3231,11 @@ struct __is_constructible2_imp : public false_type {}; +template +struct __is_constructible3_imp + : public false_type + {}; + // Treat scalars and reference types separately template @@ -3235,6 +3256,12 @@ struct __is_constructible2_void_check _Tp, _A0, _A1> {}; +template +struct __is_constructible3_void_check + : public __is_constructible3_imp::value || is_reference<_Tp>::value, + _Tp, _A0, _A1, _A2> + {}; + // If any of T or Args is void, is_constructible should be false template @@ -3252,17 +3279,24 @@ struct __is_constructible2_void_check +struct __is_constructible3_void_check + : public false_type + {}; + // is_constructible entry point template + class _A1 = __is_construct::__nat, + class _A2 = __is_construct::__nat> struct _LIBCPP_TEMPLATE_VIS is_constructible - : public __is_constructible2_void_check::value + : public __is_constructible3_void_check::value || is_abstract<_Tp>::value || is_function<_Tp>::value || is_void<_A0>::value - || is_void<_A1>::value, - _Tp, _A0, _A1> + || is_void<_A1>::value + || is_void<_A2>::value, + _Tp, _A0, _A1, _A2> {}; template @@ -3282,6 +3316,16 @@ struct _LIBCPP_TEMPLATE_VIS is_constructible<_Tp, _A0, _Tp, _A0> {}; +template +struct _LIBCPP_TEMPLATE_VIS is_constructible<_Tp, _A0, _A1, __is_construct::__nat> + : public __is_constructible2_void_check::value + || is_abstract<_Tp>::value + || is_function<_Tp>::value + || is_void<_A0>::value + || is_void<_A1>::value, + _Tp, _A0, _A1> + {}; + // Array types are default constructible if their element type // is default constructible @@ -3300,6 +3344,11 @@ struct __is_constructible2_imp +struct __is_constructible3_imp + : public false_type + {}; + // Incomplete array types are not constructible template @@ -3314,6 +3363,11 @@ struct __is_constructible1_imp template struct __is_constructible2_imp + : public false_type + {}; + +template +struct __is_constructible3_imp : public false_type {}; Modified: vendor/libc++/dist-release_60/test/std/utilities/meta/meta.unary/meta.unary.prop/is_constructible.pass.cpp ============================================================================== --- vendor/libc++/dist-release_60/test/std/utilities/meta/meta.unary/meta.unary.prop/is_constructible.pass.cpp Wed Jan 24 20:25:51 2018 (r328367) +++ vendor/libc++/dist-release_60/test/std/utilities/meta/meta.unary/meta.unary.prop/is_constructible.pass.cpp Wed Jan 24 20:25:56 2018 (r328368) @@ -30,6 +30,7 @@ struct A { explicit A(int); A(int, double); + A(int, long, double); #if TEST_STD_VER >= 11 private: #endif @@ -106,6 +107,16 @@ void test_is_constructible() #endif } +template +void test_is_constructible() +{ + static_assert(( std::is_constructible::value), ""); + LIBCPP11_STATIC_ASSERT((std::__libcpp_is_constructible::type::value), ""); +#if TEST_STD_VER > 14 + static_assert(( std::is_constructible_v), ""); +#endif +} + template void test_is_not_constructible() { @@ -146,6 +157,7 @@ int main() test_is_constructible (); test_is_constructible (); test_is_constructible (); + test_is_constructible (); test_is_constructible (); test_is_not_constructible (); From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:26:10 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6788AEC3D36; Wed, 24 Jan 2018 20:26:10 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8820C766AB; Wed, 24 Jan 2018 20:26:05 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7367422798; Wed, 24 Jan 2018 20:26:05 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKQ5nF043125; Wed, 24 Jan 2018 20:26:05 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKQ3G1043107; Wed, 24 Jan 2018 20:26:03 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242026.w0OKQ3G1043107@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:26:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328370 - in vendor/lld/dist-release_60: COFF ELF test/ELF test/ELF/Inputs test/ELF/linkerscript X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/lld/dist-release_60: COFF ELF test/ELF test/ELF/Inputs test/ELF/linkerscript X-SVN-Commit-Revision: 328370 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:26:10 -0000 Author: dim Date: Wed Jan 24 20:26:03 2018 New Revision: 328370 URL: https://svnweb.freebsd.org/changeset/base/328370 Log: Vendor import of lld release_60 branch r323338: https://llvm.org/svn/llvm-project/lld/branches/release_60@323338 Added: vendor/lld/dist-release_60/test/ELF/Inputs/as-needed-lazy.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/Inputs/compress-debug.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/as-needed-lazy.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/compress-debug-sections-reloc.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/linkerscript/at-self-reference.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/linkerscript/at2.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/linkerscript/compress-debug-sections-custom.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/linkerscript/parse-section-in-addr.s (contents, props changed) vendor/lld/dist-release_60/test/ELF/sysv-hash-no-rosegment.s (contents, props changed) Modified: vendor/lld/dist-release_60/COFF/Driver.cpp vendor/lld/dist-release_60/ELF/LinkerScript.cpp vendor/lld/dist-release_60/ELF/OutputSections.cpp vendor/lld/dist-release_60/ELF/OutputSections.h vendor/lld/dist-release_60/ELF/ScriptParser.cpp vendor/lld/dist-release_60/ELF/SymbolTable.cpp vendor/lld/dist-release_60/ELF/SyntheticSections.cpp vendor/lld/dist-release_60/ELF/Writer.cpp Modified: vendor/lld/dist-release_60/COFF/Driver.cpp ============================================================================== --- vendor/lld/dist-release_60/COFF/Driver.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/COFF/Driver.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -57,6 +57,7 @@ bool link(ArrayRef Args, bool CanExitEar errorHandler().ErrorLimitExceededMsg = "too many errors emitted, stopping now" " (use /ERRORLIMIT:0 to see all errors)"; + errorHandler().ExitEarly = CanExitEarly; Config = make(); Config->Argv = {Args.begin(), Args.end()}; Config->CanExitEarly = CanExitEarly; Modified: vendor/lld/dist-release_60/ELF/LinkerScript.cpp ============================================================================== --- vendor/lld/dist-release_60/ELF/LinkerScript.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/LinkerScript.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -608,13 +608,6 @@ void LinkerScript::switchTo(OutputSection *Sec) { Ctx->OutSec = Sec; Ctx->OutSec->Addr = advance(0, Ctx->OutSec->Alignment); - - // If neither AT nor AT> is specified for an allocatable section, the linker - // will set the LMA such that the difference between VMA and LMA for the - // section is the same as the preceding output section in the same region - // https://sourceware.org/binutils/docs-2.20/ld/Output-Section-LMA.html - if (Ctx->LMAOffset) - Ctx->OutSec->LMAOffset = Ctx->LMAOffset(); } // This function searches for a memory region to place the given output @@ -662,17 +655,28 @@ void LinkerScript::assignOffsets(OutputSection *Sec) { if (Ctx->MemRegion) Dot = Ctx->MemRegionOffset[Ctx->MemRegion]; + switchTo(Sec); + if (Sec->LMAExpr) { uint64_t D = Dot; Ctx->LMAOffset = [=] { return Sec->LMAExpr().getValue() - D; }; } - switchTo(Sec); + if (!Sec->LMARegionName.empty()) { + if (MemoryRegion *MR = MemoryRegions.lookup(Sec->LMARegionName)) { + uint64_t Offset = MR->Origin - Dot; + Ctx->LMAOffset = [=] { return Offset; }; + } else { + error("memory region '" + Sec->LMARegionName + "' not declared"); + } + } - // We do not support custom layout for compressed debug sectons. - // At this point we already know their size and have compressed content. - if (Ctx->OutSec->Flags & SHF_COMPRESSED) - return; + // If neither AT nor AT> is specified for an allocatable section, the linker + // will set the LMA such that the difference between VMA and LMA for the + // section is the same as the preceding output section in the same region + // https://sourceware.org/binutils/docs-2.20/ld/Output-Section-LMA.html + if (Ctx->LMAOffset) + Ctx->OutSec->LMAOffset = Ctx->LMAOffset(); // The Size previously denoted how many InputSections had been added to this // section, and was used for sorting SHF_LINK_ORDER sections. Reset it to Modified: vendor/lld/dist-release_60/ELF/OutputSections.cpp ============================================================================== --- vendor/lld/dist-release_60/ELF/OutputSections.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/OutputSections.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -183,15 +183,6 @@ template void OutputSection::maybeCompres !Name.startswith(".debug_")) return; - // Calculate the section offsets and size pre-compression. - Size = 0; - for (BaseCommand *Cmd : SectionCommands) - if (auto *ISD = dyn_cast(Cmd)) - for (InputSection *IS : ISD->Sections) { - IS->OutSecOff = alignTo(Size, IS->Alignment); - this->Size = IS->OutSecOff + IS->getSize(); - } - // Create a section header. ZDebugHeader.resize(sizeof(Elf_Chdr)); auto *Hdr = reinterpret_cast(ZDebugHeader.data()); Modified: vendor/lld/dist-release_60/ELF/OutputSections.h ============================================================================== --- vendor/lld/dist-release_60/ELF/OutputSections.h Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/OutputSections.h Wed Jan 24 20:26:03 2018 (r328370) @@ -99,6 +99,7 @@ class OutputSection final : public BaseCommand, public ConstraintKind Constraint = ConstraintKind::NoConstraint; std::string Location; std::string MemoryRegionName; + std::string LMARegionName; bool Noload = false; template void finalize(); Modified: vendor/lld/dist-release_60/ELF/ScriptParser.cpp ============================================================================== --- vendor/lld/dist-release_60/ELF/ScriptParser.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/ScriptParser.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -709,6 +709,14 @@ OutputSection *ScriptParser::readOutputSectionDescript if (consume(">")) Cmd->MemoryRegionName = next(); + if (consume("AT")) { + expect(">"); + Cmd->LMARegionName = next(); + } + + if (Cmd->LMAExpr && !Cmd->LMARegionName.empty()) + error("section can't have both LMA and a load region"); + Cmd->Phdrs = readOutputSectionPhdrs(); if (consume("=")) @@ -922,7 +930,10 @@ ByteCommand *ScriptParser::readByteCommand(StringRef T StringRef ScriptParser::readParenLiteral() { expect("("); + bool Orig = InExpr; + InExpr = false; StringRef Tok = next(); + InExpr = Orig; expect(")"); return Tok; } Modified: vendor/lld/dist-release_60/ELF/SymbolTable.cpp ============================================================================== --- vendor/lld/dist-release_60/ELF/SymbolTable.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/SymbolTable.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -491,12 +491,13 @@ void SymbolTable::addShared(StringRef Name, SharedFile if (WasInserted || ((S->isUndefined() || S->isLazy()) && S->getVisibility() == STV_DEFAULT)) { uint8_t Binding = S->Binding; + bool WasUndefined = S->isUndefined(); replaceSymbol(S, File, Name, Sym.getBinding(), Sym.st_other, Sym.getType(), Sym.st_value, Sym.st_size, Alignment, VerdefIndex); if (!WasInserted) { S->Binding = Binding; - if (!S->isWeak() && !Config->GcSections) + if (!S->isWeak() && !Config->GcSections && WasUndefined) File.IsNeeded = true; } } Modified: vendor/lld/dist-release_60/ELF/SyntheticSections.cpp ============================================================================== --- vendor/lld/dist-release_60/ELF/SyntheticSections.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/SyntheticSections.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -1823,6 +1823,9 @@ void HashTableSection::finalizeContents() { } void HashTableSection::writeTo(uint8_t *Buf) { + // See comment in GnuHashTableSection::writeTo. + memset(Buf, 0, Size); + unsigned NumSymbols = InX::DynSymTab->getNumSymbols(); uint32_t *P = reinterpret_cast(Buf); @@ -2435,10 +2438,8 @@ void MergeNoTailSection::finalizeContents() { parallelForEachN(0, Concurrency, [&](size_t ThreadId) { for (MergeInputSection *Sec : Sections) { for (size_t I = 0, E = Sec->Pieces.size(); I != E; ++I) { - if (!Sec->Pieces[I].Live) - continue; size_t ShardId = getShardId(Sec->Pieces[I].Hash); - if ((ShardId & (Concurrency - 1)) == ThreadId) + if ((ShardId & (Concurrency - 1)) == ThreadId && Sec->Pieces[I].Live) Sec->Pieces[I].OutputOff = Shards[ShardId].add(Sec->getData(I)); } } Modified: vendor/lld/dist-release_60/ELF/Writer.cpp ============================================================================== --- vendor/lld/dist-release_60/ELF/Writer.cpp Wed Jan 24 20:25:59 2018 (r328369) +++ vendor/lld/dist-release_60/ELF/Writer.cpp Wed Jan 24 20:26:03 2018 (r328370) @@ -427,13 +427,14 @@ template void Writer::run() { if (errorCount()) return; + Script->assignAddresses(); + // If -compressed-debug-sections is specified, we need to compress // .debug_* sections. Do it right now because it changes the size of // output sections. - parallelForEach(OutputSections, - [](OutputSection *Sec) { Sec->maybeCompress(); }); + for (OutputSection *Sec : OutputSections) + Sec->maybeCompress(); - Script->assignAddresses(); Script->allocateHeaders(Phdrs); // Remove empty PT_LOAD to avoid causing the dynamic linker to try to mmap a Added: vendor/lld/dist-release_60/test/ELF/Inputs/as-needed-lazy.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/Inputs/as-needed-lazy.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,3 @@ +.global foo +foo: + nop Added: vendor/lld/dist-release_60/test/ELF/Inputs/compress-debug.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/Inputs/compress-debug.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,5 @@ +.text +.fill 0x44 + +.section .debug_info,"",@progbits +.fill 0x43 Added: vendor/lld/dist-release_60/test/ELF/as-needed-lazy.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/as-needed-lazy.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,14 @@ +# REQUIRES: x86 +# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t1.o +# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %p/Inputs/as-needed-lazy.s -o %t2.o +# RUN: ld.lld %t2.o -o %t2.so -shared +# RUN: rm -f %t2.a +# RUN: llvm-ar rc %t2.a %t2.o +# RUN: ld.lld %t1.o %t2.a --as-needed %t2.so -o %t +# RUN: llvm-readobj -d %t | FileCheck %s + +# CHECK-NOT: NEEDED + +.global _start +_start: + nop Added: vendor/lld/dist-release_60/test/ELF/compress-debug-sections-reloc.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/compress-debug-sections-reloc.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,26 @@ +# REQUIRES: x86, zlib + +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/compress-debug.s -o %t2.o +# RUN: ld.lld %t2.o %t.o -o %t1 --compress-debug-sections=zlib -Ttext=0 +# RUN: llvm-dwarfdump %t1 -debug-str | FileCheck %s +# These two checks correspond to the patched values of a_sym and a_debug_sym. +# D = 0x44 - address of .text input section for this file (the start address of +# .text is 0 as requested on the command line, and the size of the +# preceding .text in the other input file is 0x44). +# C = 0x43 - offset of .debug_info section for this file (the size of +# the preceding .debug_info from the other input file is 0x43). +# CHECK: 0x00000000: "D" +# CHECK: 0x00000004: "C" + +.text +a_sym: +nop + +.section .debug_str,"",@progbits +.long a_sym +.long a_debug_sym + +.section .debug_info,"",@progbits +a_debug_sym: +.long 0x88776655 Added: vendor/lld/dist-release_60/test/ELF/linkerscript/at-self-reference.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/linkerscript/at-self-reference.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,63 @@ +# REQUIRES: x86 +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t +# RUN: echo "SECTIONS { \ +# RUN: . = 0x1000; \ +# RUN: .aaa : AT(ADDR(.aaa)) { *(.aaa) } \ +# RUN: .bbb : AT(ADDR(.bbb)) { *(.bbb) } \ +# RUN: }" > %t.script +# RUN: ld.lld %t --script %t.script -o %t2 +# RUN: llvm-readobj -program-headers %t2 | FileCheck %s + +# CHECK: ProgramHeaders [ +# CHECK-NEXT: ProgramHeader { +# CHECK-NEXT: Type: PT_LOAD (0x1) +# CHECK-NEXT: Offset: 0x1000 +# CHECK-NEXT: VirtualAddress: 0x1000 +# CHECK-NEXT: PhysicalAddress: 0x1000 +# CHECK-NEXT: FileSize: 3 +# CHECK-NEXT: MemSize: 3 +# CHECK-NEXT: Flags [ (0x5) +# CHECK-NEXT: PF_R (0x4) +# CHECK-NEXT: PF_X (0x1) +# CHECK-NEXT: ] +# CHECK-NEXT: Alignment: 4096 +# CHECK-NEXT: } +# CHECK-NEXT: ProgramHeader { +# CHECK-NEXT: Type: PT_LOAD (0x1) +# CHECK-NEXT: Offset: 0x1008 +# CHECK-NEXT: VirtualAddress: 0x1008 +# CHECK-NEXT: PhysicalAddress: 0x1008 +# CHECK-NEXT: FileSize: 9 +# CHECK-NEXT: MemSize: 9 +# CHECK-NEXT: Flags [ (0x5) +# CHECK-NEXT: PF_R (0x4) +# CHECK-NEXT: PF_X (0x1) +# CHECK-NEXT: ] +# CHECK-NEXT: Alignment: 4096 +# CHECK-NEXT: } +# CHECK-NEXT: ProgramHeader { +# CHECK-NEXT: Type: PT_GNU_STACK (0x6474E551) +# CHECK-NEXT: Offset: 0x0 +# CHECK-NEXT: VirtualAddress: 0x0 +# CHECK-NEXT: PhysicalAddress: 0x0 +# CHECK-NEXT: FileSize: 0 +# CHECK-NEXT: MemSize: 0 +# CHECK-NEXT: Flags [ (0x6) +# CHECK-NEXT: PF_R (0x4) +# CHECK-NEXT: PF_W (0x2) +# CHECK-NEXT: ] +# CHECK-NEXT: Alignment: 0 +# CHECK-NEXT: } +# CHECK-NEXT:] + +.global _start +_start: + nop + + +.section .aaa, "a" +.asciz "aa" + +.section .bbb, "a" +.align 8 +.quad 0 Added: vendor/lld/dist-release_60/test/ELF/linkerscript/at2.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/linkerscript/at2.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,81 @@ +# REQUIRES: x86 +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t +# RUN: echo "MEMORY { \ +# RUN: AX (ax) : ORIGIN = 0x2000, LENGTH = 0x100 \ +# RUN: AW (aw) : ORIGIN = 0x3000, LENGTH = 0x100 \ +# RUN: FLASH (ax) : ORIGIN = 0x6000, LENGTH = 0x100 \ +# RUN: RAM (aw) : ORIGIN = 0x7000, LENGTH = 0x100 } \ +# RUN: SECTIONS { \ +# RUN: .foo1 : { *(.foo1) } > AX AT>FLASH \ +# RUN: .foo2 : { *(.foo2) } > AX \ +# RUN: .bar1 : { *(.bar1) } > AW AT> RAM \ +# RUN: .bar2 : { *(.bar2) } > AW AT > RAM \ +# RUN: .bar3 : { *(.bar3) } > AW AT >RAM \ +# RUN: }" > %t.script +# RUN: ld.lld %t --script %t.script -o %t2 +# RUN: llvm-readobj -program-headers %t2 | FileCheck %s +# RUN: llvm-objdump -section-headers %t2 | FileCheck %s --check-prefix=SECTIONS + +# CHECK: ProgramHeaders [ +# CHECK-NEXT: ProgramHeader { +# CHECK-NEXT: Type: PT_LOAD +# CHECK-NEXT: Offset: 0x1000 +# CHECK-NEXT: VirtualAddress: 0x2000 +# CHECK-NEXT: PhysicalAddress: 0x6000 +# CHECK-NEXT: FileSize: 16 +# CHECK-NEXT: MemSize: 16 +# CHECK-NEXT: Flags [ +# CHECK-NEXT: PF_R +# CHECK-NEXT: PF_X +# CHECK-NEXT: ] +# CHECK-NEXT: Alignment: +# CHECK-NEXT: } +# CHECK-NEXT: ProgramHeader { +# CHECK-NEXT: Type: PT_LOAD +# CHECK-NEXT: Offset: 0x2000 +# CHECK-NEXT: VirtualAddress: 0x3000 +# CHECK-NEXT: PhysicalAddress: 0x7000 +# CHECK-NEXT: FileSize: 24 +# CHECK-NEXT: MemSize: 24 +# CHECK-NEXT: Flags [ +# CHECK-NEXT: PF_R +# CHECK-NEXT: PF_W +# CHECK-NEXT: ] +# CHECK-NEXT: Alignment: 4096 +# CHECK-NEXT: } + +# SECTIONS: Sections: +# SECTIONS-NEXT: Idx Name Size Address +# SECTIONS-NEXT: 0 00000000 0000000000000000 +# SECTIONS-NEXT: 1 .foo1 00000008 0000000000002000 +# SECTIONS-NEXT: 2 .foo2 00000008 0000000000002008 +# SECTIONS-NEXT: 3 .text 00000000 0000000000002010 +# SECTIONS-NEXT: 4 .bar1 00000008 0000000000003000 +# SECTIONS-NEXT: 5 .bar2 00000008 0000000000003008 +# SECTIONS-NEXT: 6 .bar3 00000008 0000000000003010 + +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t +# RUN: echo "MEMORY { \ +# RUN: FLASH (ax) : ORIGIN = 0x2000, LENGTH = 0x100 \ +# RUN: RAM (aw) : ORIGIN = 0x5000, LENGTH = 0x100 } \ +# RUN: SECTIONS { \ +# RUN: .foo1 : AT(0x500) { *(.foo1) } > FLASH AT>FLASH \ +# RUN: }" > %t2.script +# RUN: not ld.lld %t --script %t2.script -o %t2 2>&1 | \ +# RUN: FileCheck %s --check-prefix=ERR +# ERR: error: section can't have both LMA and a load region + +.section .foo1, "ax" +.quad 0 + +.section .foo2, "ax" +.quad 0 + +.section .bar1, "aw" +.quad 0 + +.section .bar2, "aw" +.quad 0 + +.section .bar3, "aw" +.quad 0 Added: vendor/lld/dist-release_60/test/ELF/linkerscript/compress-debug-sections-custom.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/linkerscript/compress-debug-sections-custom.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,35 @@ +# REQUIRES: x86, zlib + +# RUN: echo "SECTIONS { \ +# RUN: .text : { . += 0x10; *(.text) } \ +# RUN: .debug_str : { . += 0x10; *(.debug_str) } \ +# RUN: .debug_info : { . += 0x10; *(.debug_info) } \ +# RUN: }" > %t.script + +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o +# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/../Inputs/compress-debug.s -o %t2.o +# RUN: ld.lld %t2.o %t.o -o %t1 --compress-debug-sections=zlib -T %t.script +# RUN: llvm-dwarfdump %t1 -debug-str | FileCheck %s +# These two checks correspond to the patched values of a_sym and a_debug_sym. +# T = 0x54 - address of .text input section for this file (the start address of +# .text is 0 by default, the size of the preceding .text in the other input +# file is 0x44, and the linker script adds an additional 0x10). +# S = 0x53 - offset of .debug_info section for this file (the size of +# the preceding .debug_info from the other input file is 0x43, and the +# linker script adds an additional 0x10). +# Also note that the .debug_str offsets are also offset by 0x10, as directed by +# the linker script. +# CHECK: 0x00000010: "T" +# CHECK: 0x00000014: "S" + +.text +a_sym: +nop + +.section .debug_str,"",@progbits +.long a_sym +.long a_debug_sym + +.section .debug_info,"",@progbits +a_debug_sym: +.long 0x88776655 Added: vendor/lld/dist-release_60/test/ELF/linkerscript/parse-section-in-addr.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/linkerscript/parse-section-in-addr.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,10 @@ +# REQUIRES: x86 +# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o + +# RUN: echo "SECTIONS { \ +# RUN: .foo-bar : AT(ADDR(.foo-bar)) { *(.text) } \ +# RUN: }" > %t.script +# RUN: ld.lld -o %t.so --script %t.script %t.o -shared +# RUN: llvm-readelf -S %t.so | FileCheck %s + +# CHECK: .foo-bar Added: vendor/lld/dist-release_60/test/ELF/sysv-hash-no-rosegment.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist-release_60/test/ELF/sysv-hash-no-rosegment.s Wed Jan 24 20:26:03 2018 (r328370) @@ -0,0 +1,13 @@ +# REQUIRES: x86 +# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o +# RUN: ld.lld -shared --no-rosegment -o %t %t.o +# RUN: llvm-readobj -hash-table %t | FileCheck %s + +# CHECK: HashTable { +# CHECK-NEXT: Num Buckets: 2 +# CHECK-NEXT: Num Chains: 2 +# CHECK-NEXT: Buckets: [1, 0] +# CHECK-NEXT: Chains: [0, 0] +# CHECK-NEXT: } + +callq undef@PLT From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:26:17 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 538A0EC3D94; Wed, 24 Jan 2018 20:26:17 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F8E576851; Wed, 24 Jan 2018 20:26:14 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 36A922279A; Wed, 24 Jan 2018 20:26:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKQDhm043223; Wed, 24 Jan 2018 20:26:13 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKQCmo043220; Wed, 24 Jan 2018 20:26:12 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242026.w0OKQCmo043220@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:26:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328372 - vendor/lldb/dist-release_60/lit X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lldb/dist-release_60/lit X-SVN-Commit-Revision: 328372 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:26:17 -0000 Author: dim Date: Wed Jan 24 20:26:12 2018 New Revision: 328372 URL: https://svnweb.freebsd.org/changeset/base/328372 Log: Vendor import of lldb release_60 branch r323338: https://llvm.org/svn/llvm-project/lldb/branches/release_60@323338 Modified: vendor/lldb/dist-release_60/lit/CMakeLists.txt vendor/lldb/dist-release_60/lit/lit.cfg vendor/lldb/dist-release_60/lit/lit.site.cfg.in Modified: vendor/lldb/dist-release_60/lit/CMakeLists.txt ============================================================================== --- vendor/lldb/dist-release_60/lit/CMakeLists.txt Wed Jan 24 20:26:08 2018 (r328371) +++ vendor/lldb/dist-release_60/lit/CMakeLists.txt Wed Jan 24 20:26:12 2018 (r328372) @@ -11,6 +11,10 @@ else() set(ENABLE_SHARED 0) endif(BUILD_SHARED_LIBS) +# the value is not canonicalized within LLVM +llvm_canonicalize_cmake_booleans( + LLVM_ENABLE_ZLIB) + configure_lit_site_cfg( ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg) Modified: vendor/lldb/dist-release_60/lit/lit.cfg ============================================================================== --- vendor/lldb/dist-release_60/lit/lit.cfg Wed Jan 24 20:26:08 2018 (r328371) +++ vendor/lldb/dist-release_60/lit/lit.cfg Wed Jan 24 20:26:12 2018 (r328372) @@ -91,11 +91,11 @@ for pattern in [r"\bFileCheck\b", pattern) tool_pipe = tool_match.group(2) tool_name = tool_match.group(4) - tool_path = lit.util.which(tool_name, config.llvm_tools_dir) + tool_path = lit.util.which(tool_name, config.environment['PATH']) if not tool_path: # Warn, but still provide a substitution. lit_config.note( - 'Did not find ' + tool_name + ' in ' + config.llvm_tools_dir) + 'Did not find ' + tool_name + ' in ' + config.environment['PATH']) config.substitutions.append((pattern, tool_pipe + tool_path)) # Shell execution Modified: vendor/lldb/dist-release_60/lit/lit.site.cfg.in ============================================================================== --- vendor/lldb/dist-release_60/lit/lit.site.cfg.in Wed Jan 24 20:26:08 2018 (r328371) +++ vendor/lldb/dist-release_60/lit/lit.site.cfg.in Wed Jan 24 20:26:12 2018 (r328372) @@ -12,7 +12,7 @@ config.target_triple = "@TARGET_TRIPLE@" config.python_executable = "@PYTHON_EXECUTABLE@" config.cc = "@LLDB_TEST_C_COMPILER@" config.cxx = "@LLDB_TEST_CXX_COMPILER@" -config.have_zlib = @HAVE_LIBZ@ +config.have_zlib = @LLVM_ENABLE_ZLIB@ # Support substitution of the tools and libs dirs with user parameters. This is # used when we can't determine the tool dir at configuration time. From owner-svn-src-vendor@freebsd.org Wed Jan 24 20:26:17 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D46C5EC3D9C; Wed, 24 Jan 2018 20:26:17 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82BE376882; Wed, 24 Jan 2018 20:26:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5B34B2279B; Wed, 24 Jan 2018 20:26:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0OKQGjK043271; Wed, 24 Jan 2018 20:26:16 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0OKQG9U043270; Wed, 24 Jan 2018 20:26:16 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201801242026.w0OKQG9U043270@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Wed, 24 Jan 2018 20:26:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328373 - vendor/lldb/lldb-release_60-r323338 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lldb/lldb-release_60-r323338 X-SVN-Commit-Revision: 328373 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 20:26:18 -0000 Author: dim Date: Wed Jan 24 20:26:15 2018 New Revision: 328373 URL: https://svnweb.freebsd.org/changeset/base/328373 Log: Tag lldb release_60 branch r323338. Added: vendor/lldb/lldb-release_60-r323338/ - copied from r328372, vendor/lldb/dist-release_60/ From owner-svn-src-vendor@freebsd.org Sat Jan 27 21:22:11 2018 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 711B9EC315C; Sat, 27 Jan 2018 21:22:11 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CFA47A4F2; Sat, 27 Jan 2018 21:22:11 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 17CD210135; Sat, 27 Jan 2018 21:22:11 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w0RLMAJZ032976; Sat, 27 Jan 2018 21:22:10 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w0RLMAec032972; Sat, 27 Jan 2018 21:22:10 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201801272122.w0RLMAec032972@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sat, 27 Jan 2018 21:22:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r328490 - vendor-sys/libfdt/dist X-SVN-Group: vendor-sys X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: vendor-sys/libfdt/dist X-SVN-Commit-Revision: 328490 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jan 2018 21:22:11 -0000 Author: kevans Date: Sat Jan 27 21:22:10 2018 New Revision: 328490 URL: https://svnweb.freebsd.org/changeset/base/328490 Log: Update libfdt to github:f1879e1 Pulls in a patch, now upstreamed, from nwhitehorn to add limited read-only support for older (v2 and v3) FDT blobs. Modified: vendor-sys/libfdt/dist/fdt.c vendor-sys/libfdt/dist/fdt_overlay.c vendor-sys/libfdt/dist/fdt_ro.c vendor-sys/libfdt/dist/libfdt.h Modified: vendor-sys/libfdt/dist/fdt.c ============================================================================== --- vendor-sys/libfdt/dist/fdt.c Sat Jan 27 20:13:36 2018 (r328489) +++ vendor-sys/libfdt/dist/fdt.c Sat Jan 27 21:22:10 2018 (r328490) @@ -123,6 +123,9 @@ uint32_t fdt_next_tag(const void *fdt, int startoffset /* skip-name offset, length and value */ offset += sizeof(struct fdt_property) - FDT_TAGSIZE + fdt32_to_cpu(*lenp); + if (fdt_version(fdt) < 0x10 && fdt32_to_cpu(*lenp) >= 8 && + ((offset - fdt32_to_cpu(*lenp)) % 8) != 0) + offset += 4; break; case FDT_END: Modified: vendor-sys/libfdt/dist/fdt_overlay.c ============================================================================== --- vendor-sys/libfdt/dist/fdt_overlay.c Sat Jan 27 20:13:36 2018 (r328489) +++ vendor-sys/libfdt/dist/fdt_overlay.c Sat Jan 27 21:22:10 2018 (r328490) @@ -1,3 +1,54 @@ +/* + * libfdt - Flat Device Tree manipulation + * Copyright (C) 2016 Free Electrons + * Copyright (C) 2016 NextThing Co. + * + * libfdt is dual licensed: you can use it either under the terms of + * the GPL, or the BSD license, at your option. + * + * a) This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of the + * License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this library; if not, write to the Free + * Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, + * MA 02110-1301 USA + * + * Alternatively, + * + * b) Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND + * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR + * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, + * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ #include "libfdt_env.h" #include Modified: vendor-sys/libfdt/dist/fdt_ro.c ============================================================================== --- vendor-sys/libfdt/dist/fdt_ro.c Sat Jan 27 20:13:36 2018 (r328489) +++ vendor-sys/libfdt/dist/fdt_ro.c Sat Jan 27 21:22:10 2018 (r328490) @@ -58,9 +58,10 @@ static int fdt_nodename_eq_(const void *fdt, int offset, const char *s, int len) { - const char *p = fdt_offset_ptr(fdt, offset + FDT_TAGSIZE, len+1); + int olen; + const char *p = fdt_get_name(fdt, offset, &olen); - if (!p) + if (!p || olen < len) /* short match */ return 0; @@ -233,16 +234,34 @@ int fdt_path_offset(const void *fdt, const char *path) const char *fdt_get_name(const void *fdt, int nodeoffset, int *len) { const struct fdt_node_header *nh = fdt_offset_ptr_(fdt, nodeoffset); + const char *nameptr; int err; if (((err = fdt_check_header(fdt)) != 0) || ((err = fdt_check_node_offset_(fdt, nodeoffset)) < 0)) goto fail; + nameptr = nh->name; + + if (fdt_version(fdt) < 0x10) { + /* + * For old FDT versions, match the naming conventions of V16: + * give only the leaf name (after all /). The actual tree + * contents are loosely checked. + */ + const char *leaf; + leaf = strrchr(nameptr, '/'); + if (leaf == NULL) { + err = -FDT_ERR_BADSTRUCTURE; + goto fail; + } + nameptr = leaf+1; + } + if (len) - *len = strlen(nh->name); + *len = strlen(nameptr); - return nh->name; + return nameptr; fail: if (len) @@ -268,9 +287,9 @@ int fdt_next_property_offset(const void *fdt, int offs return nextprop_(fdt, offset); } -const struct fdt_property *fdt_get_property_by_offset(const void *fdt, - int offset, - int *lenp) +static const struct fdt_property *fdt_get_property_by_offset_(const void *fdt, + int offset, + int *lenp) { int err; const struct fdt_property *prop; @@ -289,23 +308,44 @@ const struct fdt_property *fdt_get_property_by_offset( return prop; } -const struct fdt_property *fdt_get_property_namelen(const void *fdt, - int offset, - const char *name, - int namelen, int *lenp) +const struct fdt_property *fdt_get_property_by_offset(const void *fdt, + int offset, + int *lenp) { + /* Prior to version 16, properties may need realignment + * and this API does not work. fdt_getprop_*() will, however. */ + + if (fdt_version(fdt) < 0x10) { + if (lenp) + *lenp = -FDT_ERR_BADVERSION; + return NULL; + } + + return fdt_get_property_by_offset_(fdt, offset, lenp); +} + +static const struct fdt_property *fdt_get_property_namelen_(const void *fdt, + int offset, + const char *name, + int namelen, + int *lenp, + int *poffset) +{ for (offset = fdt_first_property_offset(fdt, offset); (offset >= 0); (offset = fdt_next_property_offset(fdt, offset))) { const struct fdt_property *prop; - if (!(prop = fdt_get_property_by_offset(fdt, offset, lenp))) { + if (!(prop = fdt_get_property_by_offset_(fdt, offset, lenp))) { offset = -FDT_ERR_INTERNAL; break; } if (fdt_string_eq_(fdt, fdt32_to_cpu(prop->nameoff), - name, namelen)) + name, namelen)) { + if (poffset) + *poffset = offset; return prop; + } } if (lenp) @@ -313,6 +353,25 @@ const struct fdt_property *fdt_get_property_namelen(co return NULL; } + +const struct fdt_property *fdt_get_property_namelen(const void *fdt, + int offset, + const char *name, + int namelen, int *lenp) +{ + /* Prior to version 16, properties may need realignment + * and this API does not work. fdt_getprop_*() will, however. */ + if (fdt_version(fdt) < 0x10) { + if (lenp) + *lenp = -FDT_ERR_BADVERSION; + return NULL; + } + + return fdt_get_property_namelen_(fdt, offset, name, namelen, lenp, + NULL); +} + + const struct fdt_property *fdt_get_property(const void *fdt, int nodeoffset, const char *name, int *lenp) @@ -324,12 +383,18 @@ const struct fdt_property *fdt_get_property(const void const void *fdt_getprop_namelen(const void *fdt, int nodeoffset, const char *name, int namelen, int *lenp) { + int poffset; const struct fdt_property *prop; - prop = fdt_get_property_namelen(fdt, nodeoffset, name, namelen, lenp); + prop = fdt_get_property_namelen_(fdt, nodeoffset, name, namelen, lenp, + &poffset); if (!prop) return NULL; + /* Handle realignment */ + if (fdt_version(fdt) < 0x10 && (poffset + sizeof(*prop)) % 8 && + fdt32_to_cpu(prop->len) >= 8) + return prop->data + 4; return prop->data; } @@ -338,11 +403,16 @@ const void *fdt_getprop_by_offset(const void *fdt, int { const struct fdt_property *prop; - prop = fdt_get_property_by_offset(fdt, offset, lenp); + prop = fdt_get_property_by_offset_(fdt, offset, lenp); if (!prop) return NULL; if (namep) *namep = fdt_string(fdt, fdt32_to_cpu(prop->nameoff)); + + /* Handle realignment */ + if (fdt_version(fdt) < 0x10 && (offset + sizeof(*prop)) % 8 && + fdt32_to_cpu(prop->len) >= 8) + return prop->data + 4; return prop->data; } Modified: vendor-sys/libfdt/dist/libfdt.h ============================================================================== --- vendor-sys/libfdt/dist/libfdt.h Sat Jan 27 20:13:36 2018 (r328489) +++ vendor-sys/libfdt/dist/libfdt.h Sat Jan 27 21:22:10 2018 (r328490) @@ -54,7 +54,7 @@ #include #include -#define FDT_FIRST_SUPPORTED_VERSION 0x10 +#define FDT_FIRST_SUPPORTED_VERSION 0x02 #define FDT_LAST_SUPPORTED_VERSION 0x11 /* Error codes: informative error codes */ @@ -526,6 +526,9 @@ int fdt_next_property_offset(const void *fdt, int offs * fdt_property structure within the device tree blob at the given * offset. If lenp is non-NULL, the length of the property value is * also returned, in the integer pointed to by lenp. + * + * Note that this code only works on device tree versions >= 16. fdt_getprop() + * works on all versions. * * returns: * pointer to the structure representing the property