From owner-svn-src-vendor@freebsd.org Mon Oct 12 08:54:51 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E78379D26B0; Mon, 12 Oct 2015 08:54:51 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 649561AB2; Mon, 12 Oct 2015 08:54:51 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9C8so5U016378; Mon, 12 Oct 2015 08:54:50 GMT (envelope-from peter@FreeBSD.org) Received: (from peter@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9C8soWg016373; Mon, 12 Oct 2015 08:54:50 GMT (envelope-from peter@FreeBSD.org) Message-Id: <201510120854.t9C8soWg016373@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: peter set sender to peter@FreeBSD.org using -f From: Peter Wemm Date: Mon, 12 Oct 2015 08:54:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289177 - in vendor/subversion/dist: . doc/programmer doc/user subversion subversion/include subversion/include/private subversion/libsvn_auth_gnome_keyring subversion/libsvn_auth_kwall... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 08:54:52 -0000 Author: peter Date: Mon Oct 12 08:54:49 2015 New Revision: 289177 URL: https://svnweb.freebsd.org/changeset/base/289177 Log: Vendor import of subversion-1.9.2 Added: vendor/subversion/dist/.ycm_extra_conf.py (contents, props changed) vendor/subversion/dist/doc/programmer/gtest-guide.txt (contents, props changed) vendor/subversion/dist/subversion/include/private/svn_client_mtcc.h (contents, props changed) vendor/subversion/dist/subversion/include/private/svn_fs_fs_private.h (contents, props changed) vendor/subversion/dist/subversion/include/private/svn_object_pool.h (contents, props changed) vendor/subversion/dist/subversion/include/private/svn_packed_data.h (contents, props changed) vendor/subversion/dist/subversion/include/private/svn_sorts_private.h (contents, props changed) vendor/subversion/dist/subversion/include/svn_x509.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_auth_gnome_keyring/libsvn_auth_gnome_keyring.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_auth_kwallet/libsvn_auth_kwallet.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_client/libsvn_client.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_client/mtcc.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_delta/libsvn_delta.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_diff/binary_diff.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_diff/libsvn_diff.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs/deprecated.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs/libsvn_fs.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_base/libsvn_fs_base.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/cached_data.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/dump-index.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/hotcopy.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/hotcopy.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/index.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/index.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/libsvn_fs_fs.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/load-index.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/low_level.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/low_level.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/pack.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/pack.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/recovery.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/recovery.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/rev_file.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/rev_file.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/revprops.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/revprops.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/stats.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/structure-indexes vendor/subversion/dist/subversion/libsvn_fs_fs/transaction.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/transaction.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/util.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/util.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/verify.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_fs/verify.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_util/libsvn_fs_util.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/ vendor/subversion/dist/subversion/libsvn_fs_x/TODO vendor/subversion/dist/subversion/libsvn_fs_x/cached_data.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/cached_data.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/caching.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/changes.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/changes.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/dag.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/dag.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/fs.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/fs.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/fs_id.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/fs_id.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/fs_x.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/fs_x.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/hotcopy.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/hotcopy.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/id.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/id.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/index.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/index.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/libsvn_fs_x.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/lock.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/lock.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/low_level.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/low_level.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/noderevs.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/noderevs.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/pack.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/pack.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/recovery.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/recovery.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache-db.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache-db.sql vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/rep-cache.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/reps.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/reps.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/rev_file.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/rev_file.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/revprops.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/revprops.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/string_table.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/string_table.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/structure vendor/subversion/dist/subversion/libsvn_fs_x/temp_serializer.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/temp_serializer.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/transaction.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/transaction.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/tree.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/tree.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/util.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/util.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/verify.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_fs_x/verify.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra/libsvn_ra.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_local/libsvn_ra_local.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/eagain_bucket.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/get_file.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/get_lock.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/libsvn_ra_serf.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/lock.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/multistatus.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_serf/stat.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_ra_svn/libsvn_ra_svn.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_repos/authz_pool.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_repos/config_pool.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_repos/libsvn_repos.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/bit_array.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/checksum.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/compress.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/config_keys.inc (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/errorcode.inc (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/fnv1a.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/fnv1a.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/libsvn_subr.pc.in (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/object_pool.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/packed_data.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/prefix_string.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/root_pools.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/utf8proc/ vendor/subversion/dist/subversion/libsvn_subr/utf8proc.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/utf8proc/LICENSE vendor/subversion/dist/subversion/libsvn_subr/utf8proc/README vendor/subversion/dist/subversion/libsvn_subr/utf8proc/utf8proc.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/utf8proc/utf8proc.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/utf8proc/utf8proc_data.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/x509.h (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/x509info.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_subr/x509parse.c (contents, props changed) vendor/subversion/dist/subversion/libsvn_wc/libsvn_wc.pc.in (contents, props changed) vendor/subversion/dist/subversion/svn/auth-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svn/cl-log.h (contents, props changed) vendor/subversion/dist/subversion/svn/similarity.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/ vendor/subversion/dist/subversion/svnbench/cl.h (contents, props changed) vendor/subversion/dist/subversion/svnbench/help-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/notify.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/null-blame-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/null-export-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/null-info-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/null-list-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/null-log-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/svnbench.c (contents, props changed) vendor/subversion/dist/subversion/svnbench/util.c (contents, props changed) vendor/subversion/dist/subversion/svnfsfs/ vendor/subversion/dist/subversion/svnfsfs/dump-index-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnfsfs/load-index-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnfsfs/stats-cmd.c (contents, props changed) vendor/subversion/dist/subversion/svnfsfs/svnfsfs.c (contents, props changed) vendor/subversion/dist/subversion/svnfsfs/svnfsfs.h (contents, props changed) vendor/subversion/dist/subversion/svnserve/logger.c (contents, props changed) vendor/subversion/dist/subversion/svnserve/logger.h (contents, props changed) Deleted: vendor/subversion/dist/subversion/include/private/svn_named_atomic.h vendor/subversion/dist/subversion/include/private/svn_pseudo_md5.h vendor/subversion/dist/subversion/libsvn_fs_fs/key-gen.c vendor/subversion/dist/subversion/libsvn_fs_fs/key-gen.h vendor/subversion/dist/subversion/libsvn_ra_serf/locks.c vendor/subversion/dist/subversion/libsvn_subr/md5.h vendor/subversion/dist/subversion/libsvn_subr/named_atomic.c vendor/subversion/dist/subversion/libsvn_subr/pseudo_md5.c vendor/subversion/dist/subversion/libsvn_subr/sha1.c vendor/subversion/dist/subversion/libsvn_subr/sha1.h vendor/subversion/dist/subversion/svn/client_errors.h Modified: vendor/subversion/dist/CHANGES vendor/subversion/dist/COMMITTERS vendor/subversion/dist/INSTALL vendor/subversion/dist/LICENSE vendor/subversion/dist/Makefile.in vendor/subversion/dist/NOTICE vendor/subversion/dist/autogen.sh vendor/subversion/dist/build-outputs.mk vendor/subversion/dist/build.conf vendor/subversion/dist/configure vendor/subversion/dist/configure.ac vendor/subversion/dist/doc/user/svn-best-practices.html vendor/subversion/dist/gen-make.py vendor/subversion/dist/get-deps.sh vendor/subversion/dist/subversion/include/mod_dav_svn.h vendor/subversion/dist/subversion/include/private/svn_atomic.h vendor/subversion/dist/subversion/include/private/svn_auth_private.h vendor/subversion/dist/subversion/include/private/svn_cache.h vendor/subversion/dist/subversion/include/private/svn_client_private.h vendor/subversion/dist/subversion/include/private/svn_cmdline_private.h vendor/subversion/dist/subversion/include/private/svn_delta_private.h vendor/subversion/dist/subversion/include/private/svn_dep_compat.h vendor/subversion/dist/subversion/include/private/svn_diff_private.h vendor/subversion/dist/subversion/include/private/svn_diff_tree.h vendor/subversion/dist/subversion/include/private/svn_editor.h vendor/subversion/dist/subversion/include/private/svn_error_private.h vendor/subversion/dist/subversion/include/private/svn_fs_private.h vendor/subversion/dist/subversion/include/private/svn_fs_util.h vendor/subversion/dist/subversion/include/private/svn_io_private.h vendor/subversion/dist/subversion/include/private/svn_log.h vendor/subversion/dist/subversion/include/private/svn_magic.h vendor/subversion/dist/subversion/include/private/svn_mergeinfo_private.h vendor/subversion/dist/subversion/include/private/svn_mutex.h vendor/subversion/dist/subversion/include/private/svn_opt_private.h vendor/subversion/dist/subversion/include/private/svn_ra_private.h vendor/subversion/dist/subversion/include/private/svn_ra_svn_private.h vendor/subversion/dist/subversion/include/private/svn_repos_private.h vendor/subversion/dist/subversion/include/private/svn_sqlite.h vendor/subversion/dist/subversion/include/private/svn_string_private.h vendor/subversion/dist/subversion/include/private/svn_subr_private.h vendor/subversion/dist/subversion/include/private/svn_temp_serializer.h vendor/subversion/dist/subversion/include/private/svn_utf_private.h vendor/subversion/dist/subversion/include/private/svn_wc_private.h vendor/subversion/dist/subversion/include/svn_auth.h vendor/subversion/dist/subversion/include/svn_cache_config.h vendor/subversion/dist/subversion/include/svn_checksum.h vendor/subversion/dist/subversion/include/svn_client.h vendor/subversion/dist/subversion/include/svn_cmdline.h vendor/subversion/dist/subversion/include/svn_compat.h vendor/subversion/dist/subversion/include/svn_config.h vendor/subversion/dist/subversion/include/svn_delta.h vendor/subversion/dist/subversion/include/svn_diff.h vendor/subversion/dist/subversion/include/svn_dirent_uri.h vendor/subversion/dist/subversion/include/svn_error.h vendor/subversion/dist/subversion/include/svn_error_codes.h vendor/subversion/dist/subversion/include/svn_fs.h vendor/subversion/dist/subversion/include/svn_hash.h vendor/subversion/dist/subversion/include/svn_io.h vendor/subversion/dist/subversion/include/svn_iter.h vendor/subversion/dist/subversion/include/svn_mergeinfo.h vendor/subversion/dist/subversion/include/svn_opt.h vendor/subversion/dist/subversion/include/svn_path.h vendor/subversion/dist/subversion/include/svn_props.h vendor/subversion/dist/subversion/include/svn_ra.h vendor/subversion/dist/subversion/include/svn_ra_svn.h vendor/subversion/dist/subversion/include/svn_repos.h vendor/subversion/dist/subversion/include/svn_sorts.h vendor/subversion/dist/subversion/include/svn_string.h vendor/subversion/dist/subversion/include/svn_types.h vendor/subversion/dist/subversion/include/svn_version.h vendor/subversion/dist/subversion/include/svn_wc.h vendor/subversion/dist/subversion/include/svn_xml.h vendor/subversion/dist/subversion/libsvn_auth_kwallet/kwallet.cpp vendor/subversion/dist/subversion/libsvn_client/add.c vendor/subversion/dist/subversion/libsvn_client/blame.c vendor/subversion/dist/subversion/libsvn_client/cat.c vendor/subversion/dist/subversion/libsvn_client/checkout.c vendor/subversion/dist/subversion/libsvn_client/cleanup.c vendor/subversion/dist/subversion/libsvn_client/client.h vendor/subversion/dist/subversion/libsvn_client/cmdline.c vendor/subversion/dist/subversion/libsvn_client/commit.c vendor/subversion/dist/subversion/libsvn_client/commit_util.c vendor/subversion/dist/subversion/libsvn_client/compat_providers.c vendor/subversion/dist/subversion/libsvn_client/copy.c vendor/subversion/dist/subversion/libsvn_client/copy_foreign.c vendor/subversion/dist/subversion/libsvn_client/ctx.c vendor/subversion/dist/subversion/libsvn_client/delete.c vendor/subversion/dist/subversion/libsvn_client/deprecated.c vendor/subversion/dist/subversion/libsvn_client/diff.c vendor/subversion/dist/subversion/libsvn_client/diff_local.c vendor/subversion/dist/subversion/libsvn_client/diff_summarize.c vendor/subversion/dist/subversion/libsvn_client/export.c vendor/subversion/dist/subversion/libsvn_client/externals.c vendor/subversion/dist/subversion/libsvn_client/import.c vendor/subversion/dist/subversion/libsvn_client/info.c vendor/subversion/dist/subversion/libsvn_client/iprops.c vendor/subversion/dist/subversion/libsvn_client/list.c vendor/subversion/dist/subversion/libsvn_client/locking_commands.c vendor/subversion/dist/subversion/libsvn_client/log.c vendor/subversion/dist/subversion/libsvn_client/merge.c vendor/subversion/dist/subversion/libsvn_client/mergeinfo.c vendor/subversion/dist/subversion/libsvn_client/mergeinfo.h vendor/subversion/dist/subversion/libsvn_client/patch.c vendor/subversion/dist/subversion/libsvn_client/prop_commands.c vendor/subversion/dist/subversion/libsvn_client/ra.c vendor/subversion/dist/subversion/libsvn_client/relocate.c vendor/subversion/dist/subversion/libsvn_client/repos_diff.c vendor/subversion/dist/subversion/libsvn_client/resolved.c vendor/subversion/dist/subversion/libsvn_client/revert.c vendor/subversion/dist/subversion/libsvn_client/revisions.c vendor/subversion/dist/subversion/libsvn_client/status.c vendor/subversion/dist/subversion/libsvn_client/switch.c vendor/subversion/dist/subversion/libsvn_client/update.c vendor/subversion/dist/subversion/libsvn_client/upgrade.c vendor/subversion/dist/subversion/libsvn_client/util.c vendor/subversion/dist/subversion/libsvn_delta/compat.c vendor/subversion/dist/subversion/libsvn_delta/compose_delta.c vendor/subversion/dist/subversion/libsvn_delta/debug_editor.c vendor/subversion/dist/subversion/libsvn_delta/debug_editor.h vendor/subversion/dist/subversion/libsvn_delta/editor.c vendor/subversion/dist/subversion/libsvn_delta/path_driver.c vendor/subversion/dist/subversion/libsvn_delta/svndiff.c vendor/subversion/dist/subversion/libsvn_delta/text_delta.c vendor/subversion/dist/subversion/libsvn_delta/xdelta.c vendor/subversion/dist/subversion/libsvn_diff/deprecated.c vendor/subversion/dist/subversion/libsvn_diff/diff4.c vendor/subversion/dist/subversion/libsvn_diff/diff_file.c vendor/subversion/dist/subversion/libsvn_diff/diff_memory.c vendor/subversion/dist/subversion/libsvn_diff/lcs.c vendor/subversion/dist/subversion/libsvn_diff/parse-diff.c vendor/subversion/dist/subversion/libsvn_diff/util.c vendor/subversion/dist/subversion/libsvn_fs/access.c vendor/subversion/dist/subversion/libsvn_fs/editor.c vendor/subversion/dist/subversion/libsvn_fs/fs-loader.c vendor/subversion/dist/subversion/libsvn_fs/fs-loader.h vendor/subversion/dist/subversion/libsvn_fs_base/bdb/changes-table.c vendor/subversion/dist/subversion/libsvn_fs_base/bdb/locks-table.c vendor/subversion/dist/subversion/libsvn_fs_base/bdb/strings-table.c vendor/subversion/dist/subversion/libsvn_fs_base/dag.c vendor/subversion/dist/subversion/libsvn_fs_base/dag.h vendor/subversion/dist/subversion/libsvn_fs_base/fs.c vendor/subversion/dist/subversion/libsvn_fs_base/fs.h vendor/subversion/dist/subversion/libsvn_fs_base/id.c vendor/subversion/dist/subversion/libsvn_fs_base/id.h vendor/subversion/dist/subversion/libsvn_fs_base/key-gen.c vendor/subversion/dist/subversion/libsvn_fs_base/key-gen.h vendor/subversion/dist/subversion/libsvn_fs_base/lock.c vendor/subversion/dist/subversion/libsvn_fs_base/lock.h vendor/subversion/dist/subversion/libsvn_fs_base/reps-strings.c vendor/subversion/dist/subversion/libsvn_fs_base/revs-txns.c vendor/subversion/dist/subversion/libsvn_fs_base/tree.c vendor/subversion/dist/subversion/libsvn_fs_fs/caching.c vendor/subversion/dist/subversion/libsvn_fs_fs/dag.c vendor/subversion/dist/subversion/libsvn_fs_fs/dag.h vendor/subversion/dist/subversion/libsvn_fs_fs/fs.c vendor/subversion/dist/subversion/libsvn_fs_fs/fs.h vendor/subversion/dist/subversion/libsvn_fs_fs/fs_fs.c vendor/subversion/dist/subversion/libsvn_fs_fs/fs_fs.h vendor/subversion/dist/subversion/libsvn_fs_fs/id.c vendor/subversion/dist/subversion/libsvn_fs_fs/id.h vendor/subversion/dist/subversion/libsvn_fs_fs/lock.c vendor/subversion/dist/subversion/libsvn_fs_fs/lock.h vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache-db.h vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache-db.sql vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache.c vendor/subversion/dist/subversion/libsvn_fs_fs/rep-cache.h vendor/subversion/dist/subversion/libsvn_fs_fs/structure vendor/subversion/dist/subversion/libsvn_fs_fs/temp_serializer.c vendor/subversion/dist/subversion/libsvn_fs_fs/temp_serializer.h vendor/subversion/dist/subversion/libsvn_fs_fs/tree.c vendor/subversion/dist/subversion/libsvn_fs_fs/tree.h vendor/subversion/dist/subversion/libsvn_fs_util/fs-util.c vendor/subversion/dist/subversion/libsvn_ra/compat.c vendor/subversion/dist/subversion/libsvn_ra/ra_loader.c vendor/subversion/dist/subversion/libsvn_ra/ra_loader.h vendor/subversion/dist/subversion/libsvn_ra/wrapper_template.h vendor/subversion/dist/subversion/libsvn_ra_local/ra_local.h vendor/subversion/dist/subversion/libsvn_ra_local/ra_plugin.c vendor/subversion/dist/subversion/libsvn_ra_local/split_url.c vendor/subversion/dist/subversion/libsvn_ra_serf/README vendor/subversion/dist/subversion/libsvn_ra_serf/blame.c vendor/subversion/dist/subversion/libsvn_ra_serf/blncache.c vendor/subversion/dist/subversion/libsvn_ra_serf/blncache.h vendor/subversion/dist/subversion/libsvn_ra_serf/commit.c vendor/subversion/dist/subversion/libsvn_ra_serf/get_deleted_rev.c vendor/subversion/dist/subversion/libsvn_ra_serf/getdate.c vendor/subversion/dist/subversion/libsvn_ra_serf/getlocations.c vendor/subversion/dist/subversion/libsvn_ra_serf/getlocationsegments.c vendor/subversion/dist/subversion/libsvn_ra_serf/getlocks.c vendor/subversion/dist/subversion/libsvn_ra_serf/inherited_props.c vendor/subversion/dist/subversion/libsvn_ra_serf/log.c vendor/subversion/dist/subversion/libsvn_ra_serf/merge.c vendor/subversion/dist/subversion/libsvn_ra_serf/mergeinfo.c vendor/subversion/dist/subversion/libsvn_ra_serf/options.c vendor/subversion/dist/subversion/libsvn_ra_serf/property.c vendor/subversion/dist/subversion/libsvn_ra_serf/ra_serf.h vendor/subversion/dist/subversion/libsvn_ra_serf/replay.c vendor/subversion/dist/subversion/libsvn_ra_serf/serf.c vendor/subversion/dist/subversion/libsvn_ra_serf/update.c vendor/subversion/dist/subversion/libsvn_ra_serf/util.c vendor/subversion/dist/subversion/libsvn_ra_serf/util_error.c vendor/subversion/dist/subversion/libsvn_ra_serf/xml.c vendor/subversion/dist/subversion/libsvn_ra_svn/client.c vendor/subversion/dist/subversion/libsvn_ra_svn/cram.c vendor/subversion/dist/subversion/libsvn_ra_svn/cyrus_auth.c vendor/subversion/dist/subversion/libsvn_ra_svn/deprecated.c vendor/subversion/dist/subversion/libsvn_ra_svn/editorp.c vendor/subversion/dist/subversion/libsvn_ra_svn/internal_auth.c vendor/subversion/dist/subversion/libsvn_ra_svn/marshal.c vendor/subversion/dist/subversion/libsvn_ra_svn/protocol vendor/subversion/dist/subversion/libsvn_ra_svn/ra_svn.h vendor/subversion/dist/subversion/libsvn_ra_svn/streams.c vendor/subversion/dist/subversion/libsvn_repos/authz.c vendor/subversion/dist/subversion/libsvn_repos/commit.c vendor/subversion/dist/subversion/libsvn_repos/delta.c vendor/subversion/dist/subversion/libsvn_repos/deprecated.c vendor/subversion/dist/subversion/libsvn_repos/dump.c vendor/subversion/dist/subversion/libsvn_repos/fs-wrap.c vendor/subversion/dist/subversion/libsvn_repos/hooks.c vendor/subversion/dist/subversion/libsvn_repos/load-fs-vtable.c vendor/subversion/dist/subversion/libsvn_repos/load.c vendor/subversion/dist/subversion/libsvn_repos/log.c vendor/subversion/dist/subversion/libsvn_repos/replay.c vendor/subversion/dist/subversion/libsvn_repos/reporter.c vendor/subversion/dist/subversion/libsvn_repos/repos.c vendor/subversion/dist/subversion/libsvn_repos/repos.h vendor/subversion/dist/subversion/libsvn_repos/rev_hunt.c vendor/subversion/dist/subversion/libsvn_subr/adler32.c vendor/subversion/dist/subversion/libsvn_subr/auth.c vendor/subversion/dist/subversion/libsvn_subr/auth.h vendor/subversion/dist/subversion/libsvn_subr/cache-inprocess.c vendor/subversion/dist/subversion/libsvn_subr/cache-membuffer.c vendor/subversion/dist/subversion/libsvn_subr/cache-memcache.c vendor/subversion/dist/subversion/libsvn_subr/cache.c vendor/subversion/dist/subversion/libsvn_subr/cache.h vendor/subversion/dist/subversion/libsvn_subr/cache_config.c vendor/subversion/dist/subversion/libsvn_subr/checksum.c vendor/subversion/dist/subversion/libsvn_subr/cmdline.c vendor/subversion/dist/subversion/libsvn_subr/compat.c vendor/subversion/dist/subversion/libsvn_subr/config.c vendor/subversion/dist/subversion/libsvn_subr/config_auth.c vendor/subversion/dist/subversion/libsvn_subr/config_file.c vendor/subversion/dist/subversion/libsvn_subr/config_impl.h vendor/subversion/dist/subversion/libsvn_subr/config_win.c vendor/subversion/dist/subversion/libsvn_subr/ctype.c vendor/subversion/dist/subversion/libsvn_subr/debug.c vendor/subversion/dist/subversion/libsvn_subr/deprecated.c vendor/subversion/dist/subversion/libsvn_subr/dirent_uri.c vendor/subversion/dist/subversion/libsvn_subr/dso.c vendor/subversion/dist/subversion/libsvn_subr/eol.c vendor/subversion/dist/subversion/libsvn_subr/error.c vendor/subversion/dist/subversion/libsvn_subr/gpg_agent.c vendor/subversion/dist/subversion/libsvn_subr/hash.c vendor/subversion/dist/subversion/libsvn_subr/internal_statements.h vendor/subversion/dist/subversion/libsvn_subr/io.c vendor/subversion/dist/subversion/libsvn_subr/iter.c vendor/subversion/dist/subversion/libsvn_subr/log.c vendor/subversion/dist/subversion/libsvn_subr/macos_keychain.c vendor/subversion/dist/subversion/libsvn_subr/magic.c vendor/subversion/dist/subversion/libsvn_subr/md5.c vendor/subversion/dist/subversion/libsvn_subr/mergeinfo.c vendor/subversion/dist/subversion/libsvn_subr/mutex.c vendor/subversion/dist/subversion/libsvn_subr/nls.c vendor/subversion/dist/subversion/libsvn_subr/opt.c vendor/subversion/dist/subversion/libsvn_subr/path.c vendor/subversion/dist/subversion/libsvn_subr/pool.c vendor/subversion/dist/subversion/libsvn_subr/prompt.c vendor/subversion/dist/subversion/libsvn_subr/simple_providers.c vendor/subversion/dist/subversion/libsvn_subr/sorts.c vendor/subversion/dist/subversion/libsvn_subr/spillbuf.c vendor/subversion/dist/subversion/libsvn_subr/sqlite.c vendor/subversion/dist/subversion/libsvn_subr/sqlite3wrapper.c vendor/subversion/dist/subversion/libsvn_subr/ssl_client_cert_providers.c vendor/subversion/dist/subversion/libsvn_subr/ssl_client_cert_pw_providers.c vendor/subversion/dist/subversion/libsvn_subr/ssl_server_trust_providers.c vendor/subversion/dist/subversion/libsvn_subr/stream.c vendor/subversion/dist/subversion/libsvn_subr/string.c vendor/subversion/dist/subversion/libsvn_subr/subst.c vendor/subversion/dist/subversion/libsvn_subr/sysinfo.c vendor/subversion/dist/subversion/libsvn_subr/sysinfo.h vendor/subversion/dist/subversion/libsvn_subr/temp_serializer.c vendor/subversion/dist/subversion/libsvn_subr/time.c vendor/subversion/dist/subversion/libsvn_subr/types.c vendor/subversion/dist/subversion/libsvn_subr/username_providers.c vendor/subversion/dist/subversion/libsvn_subr/utf.c vendor/subversion/dist/subversion/libsvn_subr/utf_validate.c vendor/subversion/dist/subversion/libsvn_subr/utf_width.c vendor/subversion/dist/subversion/libsvn_subr/version.c vendor/subversion/dist/subversion/libsvn_subr/win32_crashrpt.c vendor/subversion/dist/subversion/libsvn_subr/win32_crypto.c vendor/subversion/dist/subversion/libsvn_subr/win32_xlate.c vendor/subversion/dist/subversion/libsvn_subr/win32_xlate.h vendor/subversion/dist/subversion/libsvn_subr/xml.c vendor/subversion/dist/subversion/libsvn_wc/adm_crawler.c vendor/subversion/dist/subversion/libsvn_wc/adm_files.c vendor/subversion/dist/subversion/libsvn_wc/adm_files.h vendor/subversion/dist/subversion/libsvn_wc/adm_ops.c vendor/subversion/dist/subversion/libsvn_wc/cleanup.c vendor/subversion/dist/subversion/libsvn_wc/conflicts.c vendor/subversion/dist/subversion/libsvn_wc/conflicts.h vendor/subversion/dist/subversion/libsvn_wc/copy.c vendor/subversion/dist/subversion/libsvn_wc/crop.c vendor/subversion/dist/subversion/libsvn_wc/delete.c vendor/subversion/dist/subversion/libsvn_wc/deprecated.c vendor/subversion/dist/subversion/libsvn_wc/diff.h vendor/subversion/dist/subversion/libsvn_wc/diff_editor.c vendor/subversion/dist/subversion/libsvn_wc/diff_local.c vendor/subversion/dist/subversion/libsvn_wc/entries.c vendor/subversion/dist/subversion/libsvn_wc/externals.c vendor/subversion/dist/subversion/libsvn_wc/info.c vendor/subversion/dist/subversion/libsvn_wc/lock.c vendor/subversion/dist/subversion/libsvn_wc/merge.c vendor/subversion/dist/subversion/libsvn_wc/node.c vendor/subversion/dist/subversion/libsvn_wc/old-and-busted.c vendor/subversion/dist/subversion/libsvn_wc/props.c vendor/subversion/dist/subversion/libsvn_wc/props.h vendor/subversion/dist/subversion/libsvn_wc/questions.c vendor/subversion/dist/subversion/libsvn_wc/relocate.c vendor/subversion/dist/subversion/libsvn_wc/revert.c vendor/subversion/dist/subversion/libsvn_wc/revision_status.c vendor/subversion/dist/subversion/libsvn_wc/status.c vendor/subversion/dist/subversion/libsvn_wc/token-map.h vendor/subversion/dist/subversion/libsvn_wc/translate.c vendor/subversion/dist/subversion/libsvn_wc/tree_conflicts.c vendor/subversion/dist/subversion/libsvn_wc/update_editor.c vendor/subversion/dist/subversion/libsvn_wc/upgrade.c vendor/subversion/dist/subversion/libsvn_wc/util.c vendor/subversion/dist/subversion/libsvn_wc/wc-checks.h vendor/subversion/dist/subversion/libsvn_wc/wc-checks.sql vendor/subversion/dist/subversion/libsvn_wc/wc-metadata.h vendor/subversion/dist/subversion/libsvn_wc/wc-metadata.sql vendor/subversion/dist/subversion/libsvn_wc/wc-queries.h vendor/subversion/dist/subversion/libsvn_wc/wc-queries.sql vendor/subversion/dist/subversion/libsvn_wc/wc.h vendor/subversion/dist/subversion/libsvn_wc/wc_db.c vendor/subversion/dist/subversion/libsvn_wc/wc_db.h vendor/subversion/dist/subversion/libsvn_wc/wc_db_pristine.c vendor/subversion/dist/subversion/libsvn_wc/wc_db_private.h vendor/subversion/dist/subversion/libsvn_wc/wc_db_update_move.c vendor/subversion/dist/subversion/libsvn_wc/wc_db_util.c vendor/subversion/dist/subversion/libsvn_wc/wc_db_wcroot.c vendor/subversion/dist/subversion/libsvn_wc/workqueue.c vendor/subversion/dist/subversion/libsvn_wc/workqueue.h vendor/subversion/dist/subversion/svn/add-cmd.c vendor/subversion/dist/subversion/svn/blame-cmd.c vendor/subversion/dist/subversion/svn/cat-cmd.c vendor/subversion/dist/subversion/svn/changelist-cmd.c vendor/subversion/dist/subversion/svn/checkout-cmd.c vendor/subversion/dist/subversion/svn/cl-conflicts.c vendor/subversion/dist/subversion/svn/cl-conflicts.h vendor/subversion/dist/subversion/svn/cl.h vendor/subversion/dist/subversion/svn/cleanup-cmd.c vendor/subversion/dist/subversion/svn/commit-cmd.c vendor/subversion/dist/subversion/svn/conflict-callbacks.c vendor/subversion/dist/subversion/svn/copy-cmd.c vendor/subversion/dist/subversion/svn/diff-cmd.c vendor/subversion/dist/subversion/svn/export-cmd.c vendor/subversion/dist/subversion/svn/file-merge.c vendor/subversion/dist/subversion/svn/help-cmd.c vendor/subversion/dist/subversion/svn/info-cmd.c vendor/subversion/dist/subversion/svn/list-cmd.c vendor/subversion/dist/subversion/svn/log-cmd.c vendor/subversion/dist/subversion/svn/merge-cmd.c vendor/subversion/dist/subversion/svn/mergeinfo-cmd.c vendor/subversion/dist/subversion/svn/notify.c vendor/subversion/dist/subversion/svn/propget-cmd.c vendor/subversion/dist/subversion/svn/proplist-cmd.c vendor/subversion/dist/subversion/svn/props.c vendor/subversion/dist/subversion/svn/resolve-cmd.c vendor/subversion/dist/subversion/svn/revert-cmd.c vendor/subversion/dist/subversion/svn/status-cmd.c vendor/subversion/dist/subversion/svn/status.c vendor/subversion/dist/subversion/svn/svn.c vendor/subversion/dist/subversion/svn/util.c vendor/subversion/dist/subversion/svn_private_config.h.in vendor/subversion/dist/subversion/svn_private_config.hw vendor/subversion/dist/subversion/svnadmin/svnadmin.c vendor/subversion/dist/subversion/svndumpfilter/svndumpfilter.c vendor/subversion/dist/subversion/svnlook/svnlook.c vendor/subversion/dist/subversion/svnmucc/svnmucc.c vendor/subversion/dist/subversion/svnrdump/dump_editor.c vendor/subversion/dist/subversion/svnrdump/load_editor.c vendor/subversion/dist/subversion/svnrdump/svnrdump.c vendor/subversion/dist/subversion/svnrdump/svnrdump.h vendor/subversion/dist/subversion/svnrdump/util.c vendor/subversion/dist/subversion/svnserve/cyrus_auth.c vendor/subversion/dist/subversion/svnserve/serve.c vendor/subversion/dist/subversion/svnserve/server.h vendor/subversion/dist/subversion/svnserve/svnserve.c vendor/subversion/dist/subversion/svnsync/svnsync.c vendor/subversion/dist/subversion/svnsync/sync.c vendor/subversion/dist/subversion/svnversion/svnversion.c vendor/subversion/dist/win-tests.py Added: vendor/subversion/dist/.ycm_extra_conf.py ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/subversion/dist/.ycm_extra_conf.py Mon Oct 12 08:54:49 2015 (r289177) @@ -0,0 +1,88 @@ +# Configuration file for YouCompleteMe vim plugin to allow the plugin +# to determine the compile flags. This file is based on: +# https://github.com/Valloric/YouCompleteMe/blob/master/cpp/ycm/.ycm_extra_conf.py +# +# This is free and unencumbered software released into the public domain. +# +# Anyone is free to copy, modify, publish, use, compile, sell, or +# distribute this software, either in source code form or as a compiled +# binary, for any purpose, commercial or non-commercial, and by any +# means. +# +# In jurisdictions that recognize copyright laws, the author or authors +# of this software dedicate any and all copyright interest in the +# software to the public domain. We make this dedication for the benefit +# of the public at large and to the detriment of our heirs and +# successors. We intend this dedication to be an overt act of +# relinquishment in perpetuity of all present and future rights to this +# software under copyright law. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR +# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, +# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +# OTHER DEALINGS IN THE SOFTWARE. +# +# For more information, please refer to + +import os +import ycm_core +from clang_helpers import PrepareClangFlags + +compilation_database_folder = os.path.dirname(os.path.realpath(__file__)) + +if compilation_database_folder: + database = ycm_core.CompilationDatabase( compilation_database_folder ) +else: + database = None + +def MakeRelativePathsInFlagsAbsolute( flags, working_directory ): + if not working_directory: + return flags + new_flags = [] + make_next_absolute = False + path_flags = [ '-isystem', '-I', '-iquote', '--sysroot=' ] + for flag in flags: + new_flag = flag + + if make_next_absolute: + make_next_absolute = False + if not flag.startswith( '/' ): + new_flag = os.path.join( working_directory, flag ) + + for path_flag in path_flags: + if flag == path_flag: + make_next_absolute = True + break + + if flag.startswith( path_flag ): + path = flag[ len( path_flag ): ] + new_flag = path_flag + os.path.join( working_directory, path ) + break + + if new_flag: + new_flags.append( new_flag ) + return new_flags + + +def FlagsForFile( filename ): + if database: + # Bear in mind that compilation_info.compiler_flags_ does NOT return a + # python list, but a "list-like" StringVec object + compilation_info = database.GetCompilationInfoForFile( filename ) + final_flags = PrepareClangFlags( + MakeRelativePathsInFlagsAbsolute( + compilation_info.compiler_flags_, + compilation_info.compiler_working_dir_ ), + filename ) + do_cache = True + else: + final_flags = [ ] + do_cache = False + + return { + 'flags': final_flags, + 'do_cache': do_cache + } Modified: vendor/subversion/dist/CHANGES ============================================================================== --- vendor/subversion/dist/CHANGES Mon Oct 12 08:47:46 2015 (r289176) +++ vendor/subversion/dist/CHANGES Mon Oct 12 08:54:49 2015 (r289177) @@ -1,3 +1,799 @@ +Version 1.9.2 +(30 Sep 2015, from /branches/1.9.x) +http://svn.apache.org/repos/asf/subversion/tags/1.9.2 + + User-visible changes: + - Client-side bugfixes: + * svn: fix crash when saving credentials in kwallet (r1700740, r1700951) + * checkout/update: fix "access denied" error on Windows (r1701064 et al) + * update: fix crash when updating a conflicted tree (r1702198, r1702200) + * commit: fix possible crash (r1702231) + * ra_serf: do not crash on unexpected 'X-SVN-VR-Base' headers (r1702288) + * merge: fix crash when merging to a local add (r1702299 et al) + * svnmucc: fix error during propset+put for existing file (r1702467 et al) + * update: fix crash without .svn/tmp folder (r1701838, r1702203) + * checkout: remove unnecessary I/O operation (r1701638) + * merge: fix possible crash (r1701997) + * update: fix crash with some of the incoming deletes (r1702247) + * upgrade: fix crash for pre-1.3 wc with externals (r1702218 et al) + * revert: fix crash when reverting the root of a move (r1702237 et al) + * svn: do not crash upon specific database corruptions (r1702974, r1702991) + * svn: show utf8proc version in svn --version --verbose (r1702533, r1702891) + + - Server-side bugfixes: + * fix reporting for empty representations in svnfsfs stats (r1698312 et al) + + Developer-visible changes: + - General: + * fix svnfsfs_tests.py in fsfs-v4 and fsfs-v6 modes (r1700215 et al) + + - API changes: + * disable unsupported operations for standard streams (r1701633 et al) + + +Version 1.9.1 +(02 Sep 2015, from /branches/1.9.x) +http://svn.apache.org/repos/asf/subversion/tags/1.9.1 + + User-visible changes: + - Client-side bugfixes: + * Fix crash with GPG-agent with non-canonical $HOME (r1691928, issue #4584) + * Fix checkout errors with svn+ssh:// on Windows (r1696222, r1696225) + * svn: expose expat and zlib versions in svn --version --verbose (r1696387, r1697664) + * svn: improve help text for 'svn info --show-item' (r1698106) + + - Server-side bugfixes: + * svnserve: fixed minor typo in help text (r1694023) + * Enable caching with memcached on Windows (1674626, r1674785) + * Fix an error leak in FSFS verification (r1693886) + * Fix incomplete membuffer cache initialization (r1695022) + * svnfsfs: fix some bugs and inconsistencies in load-index (r1697381 et al.) + + - Client-side and server-side bugfixes: + * Fix alignment fault in ra_svn on 32 bit SPARC machines (r1697914) + + - Bindings bugfixes: + * Fix memory corruption in copy source SWIG bindings (r1694929) + + Developer-visible changes: + * Better configure-time detection of httpd version and authz fix (r1687304 et al.) + * Correct a parameter name in svn_repos_get_fs_build_parser5 (r1694194) + * Resolve circular library reference in libsvn_fs_x (r1696695) + * Fix Unix build on systems without GPG agent (r1694481, r1697824) + +Version 1.9.0 +(5 Aug 2015, from /branches/1.9.x) +http://svn.apache.org/repos/asf/subversion/tags/1.9.0 + + User-visible changes: + - General: + * make all commands provide brief description in help output (r1522518) + * flush stdout before exiting to avoid information being lost (r1543868) + + - Major new features: + * fsfs: new format 7 with more efficient on-disk layout (r1547045 et al) + * blame: support showing prospective as well as previous changes + * info: support printing of individual values with --show-item (r1662620) + * svn auth: new subcommand to manage cached credentials and certs + * svnserve: cache config and authz to lower resource usage and be able to + serve large numbers of connections with a limited number of threads + * membuffer: quadruple the maximum cacheable directory size (r1545948 et al) + * new filesystem fsx (faster, smaller); experimental - see release notes + + - Minor new features and improvements: + * new 'diff-ignore-content-type' runtime configuration option + * new option for 'svnadmin verify': --check-normalization + * new option for 'svnadmin verify': --keep-going + * svnadmin info: new subcommand to print info about a repository + * print summary of conflicts before/after interactive conflict resolution + * import: reduce number of connections to the server needed (r1482962) + * membuffer: rework cache eviction heuristics (r1476664 et at) + * membuffer: improved cache usage statistics (r1489883) + * mergeinfo: new '--log' option (r1492168) + * svnadmin upgrade: progress and cancellation support (r1495545, r1495566) + * cleanup: add '--remove-unversioned' and '--remove-ignored' (issue #3549) + * cleanup: add '--include-externals' option (issue #2325) + * cleanup: add '--quiet' option (r1498661) + * svnadmin load: speedup by setting revprops in one call (r1504079) + * svnadmin load: set svn:date revprop in the initial commit (r1504951) + * reimplement UTF-8 fuzzy conversion using utf8proc (r1511676) + * svnadmin verify: speed up for repos with large directories (r1520419) + * svn merge: interactive conflict resolver tries external tools (r1524145) + * minor speed up in string to time conversion (r1533387) + * windows: speed up console output (r1533994) + * update: optimize wc db usage when obtaining children (r1537065 et al) + * decreased overhead for case-sensitive configuration access (r1538068) + * avoid re-opening repo for in-repo authz if already open (r1538120) + * svnserve: output errors in a more standard way (r1544250) + * faster parsing of config file comments (r1544716) + * avoid trying to open the hooks-env file when it doesn't exist (r1544721) + * svnserve: provide the same logging detail in "run once" mode as provided + in the log file (r1544731) + * svnserve: reduce connection latency (r1544732) + * wc: reduce the number of locks and transactions required (r1545127 et al) + * cat: add '--ignore-keywords' option (r1547517) + * merge and mergeinfo: use fewer RA sessions (r1552265) + * fsfs: limit delta chains from crossing too many shards (r1554942) + * fsfs: option to configure compression level in deltas (r1559748) + * fsfs: enable dir and prop deltas by default for formats that support + it (r1555286) + * fsfs: avoid out of date errors from files in a directory changing when + you change a property on the directory (r1558224) + * fsfs: temporary transaction properties are preserved when commit is + interrupted (r1549907 et al) + * fsfs: speed up transaction creation (r1544719) + * fsfs: avoid trying to open lock digest files that don't exist (r1544721) + * fsfs: reduce internal overhead by using sorted array instead of hash for + directory representation (r1554711) + * fsfs: skip decoding txdelta windows that are already cached (r1555284) + * fsfs: avoid constructing fulltext when delta we need is stored (r1555297) + * fsfs: improvements to format 6 reading/writing (r1517479) + * fsfs: reduce overhead of parsing noderev structures (r1544717) + * fsfs: speed up node relation comparison (r1554807) + * fsfs: speed up critical open_path() call (r1483301 et al) + * fsfs: speed up node verification (r1520421 et al) + * fsfs: speed up serialization into cache buffer format (r1505056) + * fsfs: avoid caching intermediate fulltexts (r1565100) + * fsfs: reduce lock contention on txn-list-lock (r1569549) + * svnadmin: don't display warnings as errors (r1556297) + * ra_serf: avoid caching properties in the update editor (r1557538) + * ra_serf: decrease latency of requests to get directory contents by + pipelining requests, speeds up 'svn ls --include externals' and + some cases of multi-url diff, including merge (r1557548) + * ra_serf: spool small update reports in memory to avoid making temp files + for small requests (r1557599) + * ra_serf: allow the reuse of serf connections after an error (r1553341) + * ra_serf: improve many error messages (r1551910 et al) + * ra_serf: pipeline lock/unlock requests (r1551918, r1551993, r1552008) + * ra_serf: pipeline requests for inherited properties against old servers + that don't support the faster REPORT (r1552455, r1552475) + * ra_serf: allow reuse of sessions after a canceled request (r1557686) + * ra_serf: reduce memory usage when retrieving revision props (r1557689) + * mod_dav_svn: make out of date error message consistent with generic repos + logic (r1558247) + * allow SQLite to optimize functions that are deterministic (r1559352) + * speed up delta calculations on non-deltifyable sections (r1559767) + * ra_serf: improve memory usage in commit processing (r1520028) + * report progress as cumulative across all sessions (issue #3260) + * ra_serf: don't send DAV headers with GET requests (r1517472) + * mod_dav_svn: SVNCacheTextDeltas defaults to on (r1517479) + * fs: improve scalability of fs_open and similar functions (r1523450 et al) + * svnserve: improve performance and scalability (r1523465 et al) + * svnadmin verify: output progress messages to stdout (r1471095) + * svnadmin crashtest: make output less misleading (r1486046) + * mod_dav_svn: discover copy to src earlier in some cases (r1537440) + * speed up mergeinfo parsing (r1483292, r1483309 et al) + * optimize hash lookups used in mergeinfo and caching (r1483434 et al) + * log: optimize string handling in repos layer (r1483570, r1483572) + * ask disk hardware to sync instead of only syncing to hardware buffers + when OS supports it (r1484439, r1484445) + * optimize diff code to avoid unneeded comparisons (r1485488) + * optimize check if relpaths are canonical (r1485496, r1489828) + * ra_svn: reduce protocol implementation overhead (r1485499, r1485504 et al) + * optimize file translation without keyword substitution (r1486058) + * optimize config file parsing by using unbuffered I/O (r1486897) + * patch: apply ignore settings when deciding to delete dirs (r1490378) + * use a cheaper RA operation for common client calculation (r1496468 et al) + * ra_svn: avoid unnecessary work when doing a blame (r1503046) + * optimize reading files into memory if size is known (r1505068) + * copy: do not error on properties when doing a foreign copy (r1563361) + * membuffer: reduce memory usage by using shorter lived pools (r1564217) + * svnadmin load: add '--ignore-dates' option (r1564789) + * update: reduce sessions used with '--parents' option (r1565920) + * diff: report properties on deleted nodes (r1569320, r1570055) + * diff: switch to diff processor APIs instead of old style callbacks, step + towards resolving long standing bugs and feature requests (r1569551 et al) + * diff: use the proper revision in headers for addition and deletion + of files (r1570053) + * svnadmin lslocks: add cancellation (r1571992) + * svn --version: list available authentication credential caches (r1572106) + * fs: improved detection of changes between two nodes, this should reduce + the number of out of date errors clients see (r1572336) + * allow the use of libmagic to be configured via subversion configuration + file or SVN_CONFIG_OPTION_ENABLE_MAGIC_FILE env variable (r1572916) + * new '--pin-externals' option for svn copy (issue #1258) + * the '--strict' option was renamed '--no-newline' (r1662224) + * merge/update: switch to three-way text conflict markers + (r1591951, r1595522) + * patch: handle renames in git formatted patches (r1594636) + * svnfsfs: new expert tool (r1594860) + * mod_dav_svn: allow server admin to GET the FSFS global cache stats + (r1595160) + * diff: support git-like binary file diffs with '--git' (r1599552) + * diff: support arbitrary context size for internal diff tool with + '-U' option (r1603847, 1603871) + * commit: print progress notification between client finishing + transmitting text deltas and asking server to commit (r1604179) + * fsfs: optimize log commands for repos on Windows by not + using a locale specific function to parse ids (r1605123) + * fsfs: reduce memory usage of reading/writing changed paths caches + (r1605188 et al) + * mod_dav_svn: log post-commit errors to httpd error log as well + as returning them to client (r1606004) + * make server config groups work with svn:// URLs (issue #4512) + * svnadmin hotcopy: report progress when copying revisions and packed + shards for FSFS repositories (r1613339) + * info: show revisions for missing tree conflicts (r1617932) + * fsfs: avoid shared data clashes due to multiple distinct repositories + having identical UUIDs (r1618138 et al) + * status: ignore thumbs.db files by default (r1620955) + * fsfs: harden parsers against data corruption (r1622931, r1622937, + r1622942) + * diff: respect svn:keywords and svn:eol-style when doing arbitrary + diffs (r1623820) + * ra_serf: reduce size of XML generated for reports (r1627333) + * replace generic --trust-server-cert with more specific options to + override specific certificate failures (r1630117 et al) + * commit: improve speed of commits with many files (r1630312 et al) + * svnadmin setrevprop: add '--transaction' option (r1631435) + * svnadmin delrevprop: new subcommand (1592723) + * svnadmin verify: validate the index data against checksums (r1631598) + * svnadmin verify: new option '--metadata-only' (r1593753) + * cp: improve performance of local copies (r1632284, et al) + * fsfs: speed up operations that use revision properties (r1634875, + r1634879) + * checkout/update: use fewer RA sessions (r1635753 et al) + * log: do less work when '--with-no-revprops' is used (r1642231) + * patch: improve command to be more capable as compared to GNU patch + (issue #4533) + * limit server caches to avoid running out of memory if server admin + configured caches larger than supported by the platform (r1645572) + * mod_authz_svn: log implicit read access failures with INFO log level; + Explicit read access failures are still logged with ERROR log level + (r1653032) + * ra_serf: reduce memory usage by removing some extra intermediary state + (r1654681, r1654690) + * revert: improve performance on large working copies without changes + (r1657083) + * ra_svn: improve efficiency of editor processing (r1658194 et al) + * merge: provide different conflict reasons for local additions and + unversioned obstructions (r1659381) + * windows: improve checkout performance on windows by using sqlite truncate + journal mode rather than delete (r1659426) + * status: now accept '-r' argument (r1671164, 1672578, 1673228) + * ls: improve performance of '-v' on tag directories (r1673153) + * resolve: improve conflict prompts for binary files (r1667228 et al) + * fsfs: improve error messages for parsing errors (r1678147, r1678149) + * warn when the '--config-option' FILE:SECTION:OPTION combination may + be invalid. (r1674301 et al) + * ensure full key matching in membuffer cache (r1678950 et al) + * mod_dav_svn: expose cache statistics via HTTP (r1595160) + + - Client-side bugfixes: + * export: fix problem exporting symlinks on windows (r1476093) + * fix non-ascii character handling of command line options (r1476461) + * log: strip EOL marker only after converting to UTF-8 (r1476608) + * ra_serf: avoid dropping errors when making a lock request (r1489526) + * ra_serf: fix an error leak in update logic (r1499686) + * windows: fix issues with wcs in symlinked folders (r1501251) + * fix regression that broke parallel nested checkouts (issue #4390) + * svnmucc: ignore leading r in revision number arguments to -r (r1502636) + * mkdir: use absolute paths to avoid Windows path length limits (r1516816) + * avoid encoding support code when C runtime doesn't support it (r1530582) + * ra_svn: add check against dirents with path separators (r1533812) + * properly delete temporary files when atomic write fails (r1537466) + * wc: fix corner cases in move-update tree conflicts (r1538639 et al) + * windows: fix crash report indentation for x64 reports (r1543589) + * mergeinfo: allow to work on a moved target (issue #4301) + * windows: avoid delay when no homedir is available (r1546814) + * status: report externals in a deterministic way (r1550200) + * copy: avoid an unneeded extra RA session for wc to repo case (r1551564) + * ra_serf: show pre-revprop-change failure on revprop delete (issue #3086) + * svnsync: avoid extra request at end of every revision sync (r1553370) + * ra_serf: fix replace + propset of locked file failures (issue #3674) + * ra_serf: prevent overwriting directory during copy (issue #3314) + * commit: limit number of temporary files open at same time (issue #4172) + * ra_serf: verify incoming integers are really integers (r1557709 et al) + * log: -g --xml doesn't differentiate between forward and reverse merges + (issue #4463) + * windows: improve update and checkout speed (issue #4450) + * log: reduce performance penalties of using -g (r1559912) + * blame: reduce performance penalties of using -g (r1560112) + * ra_local: set svn:txn-user-agent for consistency (r1498608) + * ra_svn: use the stream API properly for communications (r1562072) + * update: provide error when none of targets are wcs (r1565388) + * wc: reduce dependence on unspecified SQLite behavior (r1567080 et al) + * diff: fix diffing directory without permissions to read parent (r1569265, + r1569290) + * diff: improve resolving peg revisions (r1570205 et al) + * diff: fix bug in calculating header paths (r1570584) + * ra_serf: add SSL certificate issuers common name to output (r1573728) + * updates keywords in files not modified during 'svn switch' (issue #1975) + * svnmucc: Normalize line endings with '-F' and '-m' options + (r1592148, r1592150) + * fix problems with read-only authentication caches (issue #4504) + * commit: don't bump just locked files (r1603617) + * log: reduce memory with '-v' (r1604569 et al) + * diff: fix diffing locally deleted nodes under copied directories + (r1605832) + * diff: fix missing node error when diffing a replaced node (r1605866) + * propget: forbid using 'show-inherited-properties' with 'strict' (r1611698) + * windows: avoid delay when user profile isn't writable (r1617926) + * merge: display the correct node kinds for tree conflicts (r1618024, + r1619418, r1619717) + * diff: show the correct revisions in the header (r1619452) + * diff: fix missing header for files with only property changes (r1619476) + * update: raise a tree conflict rather than an obstruction when an + incoming server-excluded node conflicts with a locally added node + (r1619495) + * update: improve tree conflict reason and action descriptions (r1619777) + * ra_serf: fix stalls during checkout/update over http/https (r1621596) + * svnmucc: don't crash when '--version' is used with other arguments + (r1625496) + * checkout: report svn:externals failures via the exit code as other + commands already do (r1628398) + * svn & svnlook: use the right error code when fputs() fails (r1630369) + * export: reject peg specifiers on local destination path (r1635085) + * don't reject command-line arguments in the form of ".@abc", where + "abc" is a peg specifier that may be empty (r1635118) + * fix directory externals not following history (issue #4529) + * remove 'df' and 'm' options from the interactive conflict resolver for + binary files (r1645578) + * mergeinfo parsing: allow source path to be empty (issue #4537) + * mkdir: when using '--parents' don't add entire contents recursively if + target already exists (r1649951) + * resolve errors with move of a nested delete (r1651980, r1651997) + * update: prevent breaking a working copy when a directory is replaced + with an external to a foreign repository (issue #4550) + * update: prevent an invalid wc state when applying a move (r1652184 et al) + * resolve: fix a segfault when breaking a move inside a delete (issue #4491) + * ra_serf: don't handle a commit that didn't produce a new revision as + a successful commit (r1653532) + * export: fix the lack of notifications when starting to handle externals + (issue #4527) + * update: fix a case where we reported an error rather than a tree conflict + (r1655017) + * info: Use local platform style paths in all cases (r1659283) + * handle lack of a configuration file properly (r1660369) + * update: resolve issues with tree conflicts caused by an incoming + delete removing a mixed revision tree (r1660742) + * don't hold onto locks of deleted paths in the client on commit (r1661363) + * info: fix url calculation for a few statuses (r1661476) + * update: when using '--set-depth' avoid removing local changes (r1661585) + * update: fix tree conflict detection on unversioned nodes that exist where + there used to be a deleted node (r1661664) + * status: display tree conflicts even if the node with the tree conflict is + shadowed by a file (r1662331) + * pre-1.6 wc compatibility: fix with obstructed working copies (r1662412) + * resolve: allow directly resolving tree conflicts (r1658435) + * copy: when copying from a wc to a url show all the changes (r1655729) + * info: provide results in a stable order (r1662030) + * revert: allow depth limited reverts of nodes that only have not-present + and/or excluded child nodes (r1662091) + * wc: fix calculating repo path after commits of nodes that shadow a + switched (not-present) node (r1663991, r1666258, r1674032) + * update: resolve assertion on bad update report involving incomplete + status (r1663671, r1666832) + * update: allow a real file to replace a file external (r1664035) + * merge: raise a tree conflict on root of obstructing dir (r1666690) + * cp: fix 'svn cp ^/A/D/H@1 ^/A' to properly create A (r1674455, r1674456) + * status: fix incorrect output with file externals (issue #4580) + * merge: fix part of issue #4582 (r1686175, r1687029, r1688258) + + - Server-side bugfixes: + * svnserve: don't ignore socket initialization errors (r1544253) + * svnserve: don't hide fatal errors in inetd and tunnel modes (r1544256) + * fsfs: log repo path in local style for cache init failure (r1494314) + * fsfs: fix potential transaction corruption (r1519624) + * svnserve: fix logging in multi-threaded servers (r1523502) + * fsfs: don't report out of date errors due to FS corruption (r1527084) + * svnadmin verify: detect inconsistencies that prevent loading (r1536854) + * mod_dav_svn: use 404 status for errors caused by invalid URIs (r1542063) + * mod_dav_svn: use 404 when the repository doesn't exist (r1544259) + * mod_dav_svn: use 'dav_svn:' prefix for filename instead of 'svn:' + (r1544711) + * mod_dav_svn: XML escape lock tokens (r1547427) + * hotcopy: don't create config files when copying pre-1.5 repos (r1547454) + * hotcopy: preserve the rep-cache.db permissions when hotcopying (r1547877) + * mod_dav_svn: fix SVNCacheTextDeltas and SVNAdvertisV2Protocol directive + merging (r1548124) + * mod_dav_svn: always produce an error text even when text specifies + the default message to make diagnosing issues easier (r1553441) + * mod_dav_svn: fix some pool lifetime issues with error messages (r1553868) + * mod_dav_svn: avoid setting option headers multiple times (r1557103) + * fsfs: prevent some commits that could cause future corruption (r1560673) + * cache: fix premature eviction due to 64-bit underflows (r1567996 et al) + * svnserve: fix potential integer overflow in Cyrus SASL support (r1570434) + * bdb: fix potential integer overflow and underflow (r1570701) + * bdb: prevent silent propogation of some corruption (r1570778) + * svnadmin hotcopy: do not corrupt db/current contents when copying old + FSFS repos (r1603485) + * svnadmin hotcopy: don't produce broken copies when a concurrent pack + happens (r1605633) + * log: reduce memory consumption of '-v' (r1605195) + * mod_dav_svn: fix performance issue on Windows with named atomics + (r1611379) + * log: fix a segfault in handling changed paths (r1615364) + * mod_dav_svn: properly forward post-lock/post-unlock failures (r1622235) + * diff: fix handling of depth empty in added directories (r1622024) + * fix a segfault with corrupted changed path lists that try to add root + nodes (r1622944) + * svnadmin verify: report errors nicely rather than possibly aborting + (r1622955) + * svnlook propget: report transaction name rather than revision number in + errors if run with '-t' option (r1623317) + * svnadmin verify: prevent stack overflow conditions in corrupted + repositories (r1623398) + * fsfs upgrade: fix an issue that could block an upgraded format 1 or 2 + repository from being committed to (r1624011) + * fs backends: fix memory lifetime issue (r1632646) + * bdb: output correct checksum in error message about corruption (r1640707) + * fsfs: fix a pool lifetime error related to transaction local caches + (r1643139, 1643233) + * fsfs: avoid "random DAG walks" during checkout over ra_serf (r1645567) + * fsfs/bdb: reduce memory use during tree deletions (r1647820, r1655022) + * mod_authz_svn: fix unbounded memory use when SVNPathAuthz short_circuit + is used (r1647887) + * fsfs: reduce memory use when walking back in history (r1648230) + * dump: remove incorrect kind header on replaced nodes (issue #4553) + * dump: remove duplicate headers for replace-with-copy (issue #4552) + * mod_dav_svn: don't send XML-unsafe characters in svn:author + (issue #4415) + * dump: don't write broken dump files in some ambiguously encoded fsfs + repositories (issue #4554) + * mod_dav_svn: provide a more intuitive error message to the client when + trying to create a directory that already exists (issue #2295) + * fsfs: fix uninitialized memory use in lock/unlock code (r1657525) + * fix a segfault executing a pre-commit hook with legacy locks (r1657893) + * mod_dav_svn: do not ignore skel parsing errors (r1658168) + * fsfs: fix multiple reporting of the same lock (r1658482) + * fsfs: fix pool lifetime issue in lock handling (r1659314) + * mod_dav_svn: properly log assertions and malfuctions (r1660480) + * svnadmin load/dump: preserve deletion of svn:date from r0 (issue #4563) + * svnrdump: don't provide HEAD+1 as base revision when loading deletes + (r1664684) + * mod_dav_svn: improve error message sent with a 405 status code + (r1665195, r1666096, r1666379) + * make detection of invalid base revision on commit behave the same on + all RA layers (r1664664, r1664672, r1664674) + * mod_dav_svn: emit the first few log items as soon as they are available + (r1666965, r1667120) + * mod_dav_svn: prevent a tree walk on copy sources (issue #4351) + * fsfs: fix 'EOF found' error when reading repo (issue #4577) + * svnadmin freeze: unlock rep-cache.db as part of unfreezing + (r1679169, r1679287) + * fsfs: improve stability in the presence of power or network + disk failures during 'svnadmin pack' (r1683378) + * detect invalid svndiff data earlier (r1684077) + + - Client-side and server-side bugfixes: + * use less memory when retrieving extension from filename (r1548480) + * use more optimal code path with old txdelta v1 data (r1485480) + * windows: allow opening SQLite databases on long paths (r1564338) + * fix an out-of-bounds read in the delta combiner (r1569415) + * fix a number of cases of undefined behavior when passing invalid + arguments to memcpy() (r1632530, et al) + * windows: avoid a 12 second delay due to a retry loop in some error + conditions when opening a file (r1657583) + + - Other tool improvements and bugfixes: + * windows: add build-svn-deps-win.pl tool to build dependencies (r1467715) + * svnpubsub: add support for revprop changes (r1486463) + * svnpubsub: do not pass svn commands through shell (r148466, r1486467) + * svnpubsub: hooks exit with an error if they fail (r1486500, r1486597) + * svnpubsub: hooks use "--" to signal end of arguments (r1486513) + * mailer: properly encode mail headers per RFC2047 (r1487532) + * svnwcsub: add a pre-update hook, which can deny update (r1494542 et al) + * fsfs-stats: count file nodes that are added without history (r1519283) + * fsfs-stats: replaced by 'svnfsfs stats' (r1594860) + * fsfs-access-map: count empty reads and unnecessary seeks (r1523441) + * fsfs-access-map: generate scaled/rectangular heatmaps (r1505065) + * fsfs-access-map: deal with strace format issues (r1505065) + * fsfs-access-map: update to know about index files (r1505065) + * svnbench: renamed from svn-bench (r1659226) + * svnbench: add null-info command (r1532196) + * svnlook.py: made usable as a library by adding getter methods (r1541558) + * svnbench, svnraisetreeconflict, svnauthz, svn-rep-sharing-stats: More + consistent error reporting, following pattern of core command-line + programs (r1544194) + * which-error.py: allow which-error.py to be run from symlink (r1547977) + * bash_completion: add svnlook filesize command and options to svn cat, + info, and mergeinfo commands (r1569021) + * new '--conflict-style' option to standalone diff3 tool (r1591750) + * update standalone diff3 tool so it can be used with --diff-cmd + directly (r1591871, r1591876) + * bash_completion: support for auth command (r1596841) + * diff: support '-U' option (r1618618) + * svn-rep-sharing-stats: replaced by 'svnfsfs stats' (r1618861) + * add svn-vendor.py as an alternative to svn_load_dirs.pl that can + auto-detect renames and copies (r1623660) + * svnpredumpfilter.py: fix a scalability problem that made run time + increase greatly on large repositories (r1625674 et al) + * svnpredumpfilter.py: detect copies of copies and handle properly + (r1626182) + * bash_completion: add svnadmin delrevprop (r1631473) + * showchange.pl: removed, obsoleted by 'svn log --diff' (r1631686) + * bash_completion: add new trust options (r1660373) + * bash_completion: add '--pin-externals' (r1662250) + * bash_completion: stop offering deprecated options (r1662291) + * bash_completion: add '--show-item' and '--no-newline' (r1662622) + * svnbench: add null-blame command (r1673785, r1673803, r1674015) + * svnbench: install with default 'make install' (r1685085) + + Developer-visible changes: + - General: + * require Python 2.7+ for development and testing (r1691712, r1691713, r1692448) + * include symbolic names for error codes in maintainer mode (r1467643) + * include symbolic names for warning codes in maintainer mode (r1469855) + * support YouCompleteMe vim plugin (r1476374 et al) + * cache: add pretend_empty mode for maintainers (r1461689, et al) + * allow tests to be run with exclusive wc locks (r1496475) + * fix check for Microsoft C/C++ compiler (r1503148) + * improve windows build to use pre-built dependencies + * optimize ramdrive usage in the windows test runner (r1504511) + * SVN_CMDLINE_DISABLE_CRASH_HANDLER env var added for Windows (r1506507) + * gen-make.py: support Visual Studio 2013 by --vsnet-version=2013 (r1533907) + * windows: don't require APR source when building (r1534037, et al) + * don't compile SQLite amalgamation twice (r1536364) + * make C tests use same temp path as Python tests (r1536700) + * davautocheck: access log logs the r->filename now (r1538048) + * added test to detect duplicate SQL queries (r1538962) + * windows test runner: copy additional apr dlls (r1539300) + * support for debug logging that should be available in serf 1.4.0 (r1541561) + * OS X: fix build when Gnome is installed (r1541698) + * properly detect if expat is libexpat.lib or xml.lib on windows (r1541981) + * doc-clean target works properly with separate build dir (r1542303) + * don't break entire test run if cleaning temp directory failed but return + an error anyway (r1542817) + * windows tests: find java.exe via the sdk if not found via PATH (r1543876) + * fix davautocheck with httpd 2.3.x+ and worker/prefork MPMs (r1544302-3) + * add parallel mode for C tests (r1545334 et al) + * use --bin path for svnrdump when running Python tests (r1548706) + * remove unused --with-openssl option from configure (r1548714) + * tests: stop rewriting shared authz file for every sandbox (r1552064 et al) + * fix unnecessary aborts in maintainer mode on sqlite errors (r1536325) + * new --sqlite-logging option for C tests (r1560409) + * allow building with APR-2 (r1560586) + * remove support for ASM Zlib from Windows build system (r1560864) + * gen-make.py: new -D option (r1567046) + * improve how we fetch the version of Windows (r1568798, 1568806) + * move common code in python tests to library files to avoid side effects of + having tests depending on each other (r1570654) + * disable compiling on platforms where int is shorter than 32-bits and add + the SVN_ALLOW_SHORT_INTS compile definition to override (r1570882) + * disable building *.sbr files with Visual C++ 2010 or later (r1571020) + * require serf 1.3.4 (r1572261) + * require APR and APR-Util 1.3.x (r1572261) + * add mod_dontdothat to davautocheck.sh (r1572312) + * require Apache httpd 2.2.x (r1572339) + * support pkg-config for serf when using a prefix (r1572762) + * don't depend on stdbool.h and inttypes.h for utf8proc (r1573069 et al) + * avoid non-portable find extensions so the build works on Solaris (r1573780) + * Allow PARALLEL value to specify the number of threads to run tests with, + PARALLEL=1 means to use the default (r1573320) + * pkg-config support for libsvn_* libraries (issue #4435) + * binaries built on Mac OS X 10.6+ will not run on older versions (r1651549) + * fix random failure of Makefile-based parallel builds on Unix + when the amalgamated SQLite is being used (r1658357) + * C tests only run with ra_local to avoid duplicate testing (r1609477) + * allow skipping the C tests for any RA method (r1609489) + * new tools for benchmarking on Windows (r1610264) + * svnbench: no longer part of tools but main subversion (r1618860) + * improve detection for libtool (r1627276) + * check the runtime version of the fs_util library in the fs backends + (r1651567) + * properly initialize the src_kind value to avoid problems for other API + users during a copy (r1655723) + * try to improve reliability of applications that do not call + svn_dso_initialize2() right after apr_initialize() (r1659604) + * add svn-wc-db-tester tool for testing working copies (r1660874) + * fix bugs and performance issues using svn_wc_walk_entries3() (r1661110) + * support using Python 3 for building (r1661247) + * don't add -lstdc++ on FreeBSD 10 (r1662329) + * add build support for Visual Studio 2015 (r1663183, r1663184, r1689721 et al.) + * fix test failures when running from a directory whose name contains + characters that must be escaped when used in a URL. (r1664997) + * fix breakage of the serf ra session with svn_ra_get_dir2() and + svn_ra_get_log2(). (r1665213, r1665259, r1665609) + * resolve a race condition in some test suite cleanup code (r1683303) + * fix some tests on non-US default locale on Windows (r1684034) + * document the meaning of "XFAIL" for users building from source (r1683071) + + - API changes: + * new RA callbacks for managing ra_svn tunnels: + svn_ra_callbacks2_t::check_tunnel_func, + svn_ra_callbacks2_t::open_tunnel_func and + svn_ra_callbacks2_t::close_tunnel_func + * new API for retrieving info about a file system: svn_fs_info() + * new API svn_io_file_flush() that wraps apr_file_flush() + * new API svn_io_write_atomic(), that writes, sync and renames a file + * new API svn_compat_log_revprops_out_string() + * deprecate unused datatype svn_ra_get_latest_revnum_func_t + * new API svn_client_cleanup2() (r1496954 et al) + * new API svn_stringbuf_create_wrap() (r1502248) + * new API svn_io_file_aligned_seek() (r1502539) + * ra_svn: fix svn_ra_get_log*() to apply limit when server can't (r1503043) + * svn_client_commit6: notify which path prevents a mv commit (r1503662) + * new APIs svn_io_file_create_empty() and svn_io_file_create_bytes() + (r1505006) + * new API svn_ver_check_list2() (r1502267) + * new API svn_stringbuf_from_stream() (r1532193) + * svn_auth_get_platform_specific_client_providers() now includes ssl + providers (r1534153) + * change dav_svn_split_uri() repos_path argument (r1537812) + * new API svn_repos_fs_type() returns filesystem type (r1538585) + * svn_client__get_inheritable_props() clears wcroot_iprops when revision is + not valid (r1538602) + * SVN_VA_NULL: New macro which is null-pointer constant (r1536307) + * SVN_NEEDS_SENTINEL_NULL: New macro to mark functions that require final + NULL sentinel value, so compilers can warn when missing (r1543394) + * platform specific svn_auth_get_* functions are deprecated, use the + svn_auth_get_platform_specific_provider() function instead (r1543992) + * SVN_INT_ERR macro is deprecated; use svn_handle_error2() or + svn_cmdline_handle_exit_error() instead (r1544142) + * new API svn_client_cat3() allow disabling keyword expansion and retrieving + props (r1544182) + * svn_auth_first_credentials(): provides an error rather than crashing if + auth_baton is NULL (r1544320) + * svn_auth_set_parameter(): Do nothing if auth_baton is NULL (r1544320) + * new API svn_relpath_prefix which returns a relpath with a maximum number + of path components (r1545123, r1673282) + * svn_fs_initialize() is now threadsafe (r1546409) + * svn_checksum_kind_t has two new FNV-1a checksums (r1546699) + * svn_fs_lock: control characters are not allowed in lock tokens (r1547445) + * new API svn_wc_cleanup4() makes some functionality optional and adds + notifications (r1548075 et al) + * new API svn_client_info4() supports walking externals (r1550206 et al) + * new flag for svn_fs_begin_txn2() SVN_FS_TXN_CLIENT_DATE to allow client + to set the final svn:date (r1550228) + * new APIs for easier RA commits svn_client_mtcc_* (r1550758 et al) + * new API svn_ra_session_dup() (r1552324 et al) + * svn_ra_stat() now handles compatibility with 1.0-1.1 svnserve (r1552441) + * new error code SVN_ERR_COMPOSED_ERROR added to allow detection of + composed errors by API consumers (r1553266) + * new error code SVN_ERR_RA_DAV_PRECONDITION_FAILED (r1553668) + * new error code SVN_ERR_RA_CANNOT_CREATE_SESSION (r1554027) + * new API svn_fs_node_relation() (r1554800) + * SVN_EXPERIMENTAL to mark functions that might change in future (r1526012) + * fix bug in svn_client_get_merging_summary() with some params (r1532396) + * new error code SVN_ERR_XML_UNEXPECTED_ELEMENT (r1498938) + * extend stream API to also support incomplete reads (r1561688 et al) + * new API svn_wc_add_from_disk3() (r1563361) + * new API svn_client_revert3() to allow clearing changelists and to have + a metadata_only flag (r1568635, r1657026) + * svn_rangelist_inheritable2(): don't change inheritabilty of remaining + ranges (r1569731) + * svn_rangelist_inheritable2() and svn_mergeinfo_inheritable2(): fix a + pool lifetime issue (r1569764) + * new APIs to support cancelation during unified diff output and + allow the context size to be specified (r1570149 et al) + * APIs related to retrieving logs are now documented to be unlimited when + a negative value is passed for the limit (r1570330, 1570335) + * new APIs: svn_fs_props_different() and svn_fs_contents_different() + (r1572363, r1573111) + * expose SVN_CONFIG_AUTHN_* macros in public API (r1572640) + * add SVN_CONFIG_OPTION_SQLITE_TIMEOUT to allow SQLite busy timeout + to be configured by clients (r1592093) + * add new notification between transmitting deltas and finalizing + commit as svn_wc_notify_commit_finalizing (r1603388) + * svn_client_cat3() API no longer returns entry or WC props (r1603501) + * properly handle canonical paths in svn_io_start_cmd3() (r1604761) + * add SVN_FS_CONFIG_FSFS_BLOCK_READ to control FSFS format 7 block + read feature (r1604933) + * new API svn_cstring_skip_prefix() (r1612823) + * new API svn_diff_mem_string_output_merge3() (r1618599) + * extend svn_wc_info_t with conflicts2 (r1618643) + * new API svn_diff_mem_string_output_unified3() (r1618839) + * new API svn_diff_file_output_merge3() (r1618857) + * svn_fs_props_changed() and svn_fs_contents_changed() no longer return + false positives (r1618880) + * new API svn_wc_conflict_description2_dup() (r1618883) + * extend svn_wc_conflict_description2_t with prop_reject_abspath field + (r1619096) + * extend svn_wc_conflict_description2_t with property values as + svn_string_t's (r1619122) + * fix svn_client_import5() to use absolute paths (r1623974) + * fix a few cases of invalid filling of svn_client_commit_item3_t (r1623981) + * add SVN_FS_CONFIG_FSFS_SHARD_SIZE to allow creation of repositories with + a custom shard size (r1624512) + * new API svn_cmdline_create_auth_baton2 (r1630117) + * make svn_string_*() and svn_stringbuf_*() functions handle C strings + of NULL with 0 length (r1632530, et al) + * fix svn_stream_compressed() for streams without partial read support + (r1639626) + * do not segfault on svn_stream_read_full() or svn_stream_skip() are called + for a no-op stream created via svn_stream_create() (r1639637) + * add SVN_FS_CONFIG_FSFS_LOG_ADDRESSING to control logical addressing + feature of fsfs format 7 (r1640915) + * rename the 'parent_directory' parameter of + svn_wc_parse_externals_description3() to 'defining_directory' and improve + the documentation (r1642690) + * new API svn_repos_get_fs_build_parser5() (r1647563) + * SVN_VERSION_BODY changed so that it is embedded into libraries, ultimately + allowing SVN_VER_TAG constant to be modified by patches (r1651565) + * rename repos_url to repos_url_deocded argument to + svn_repos_get_commit_editor5() to clairfy usage (r1653609) + * make svn_io_set_file_read_only() and svn_io_remove_dir2() ignore + ENOTDIR when ignore_enoent argument is set so the behavior is consistent + across platforms (r1653680) + * make svn_ra_open4() return the documented error when a repository + should be opened in a different location (r1655648) + * no longer return an error when using svn_ra_get_lock() and the path does + not exist (r1656875) + * svn_string_dup() will accept a NULL pointer and return a NULL (r1657254) + * svn_ra_get_file_revs2() now handles SVN_INVALID_REVNUM as HEAD (r1660463) + * new API svn_error_quick_wrapf() (r1662668) + * new API svn_fs_node_has_props() (r1673170, r1673172, r1673692, r1673746) + * new API svn_repos_verify_fs3() (r1492651 ... r1687769) + + - Bindings: + * javahl: add support for the RA layer (r1494650 et al) + * javahl: add ignore-keywords knob to ISVNCLient.doExport (r1494936) + * javahl: improve performance of rev_range_vector_to_apr_array (r1496243) + * javahl: provide whole stack of errors from native exceptions (r1496353) + * javahl: new framework for manipulating in-memory config data (r1497746) + * javahl: add methods to set config change handler callback (r1497753) + * javahl: add inheritance info to JavaHL's RevisionRange type (r1499308) + * swig-rb: fixes to building on Windows (1505406 et al) + * libsvn_swig_*libraries now work as proper shared librares (r1506520 et al) + * swig-py: 'import svn.core.*' imports libsvn_subr public symbols (r1507860) + * javahl: expose the svn_rangelist operations (r1509025) + * javahl: new utility class SVNUtil (r1512354) + * javahl: serialize init of native library (r1519803) + * javahl: fix value truncation checks (r1519913) + * javahl: add ISVNClient.info which exposes svn_client_info4 (r1603481) + * javahl: include name of exception class in error messages (r1532117) + * javahl: simplify JNI environment handling (r1533804) + * javahl: Revsion.UNSPECIFIED added as alias to Revision.START (r1533928) + * javahl: do not require JUnit to build javahl; only to test (r1535603) + * javahl: update Action enum with new entries added in 1.9 (r1536319) + * javahl: expose node property validation utility (r1538133) + * javahl: new style JNI wrapper (r1539114 et al) + * javahl: new utility API to parse/unparse svn:externals (r1539130 et al) + * javahl: allow tests to be run from a RAM disk (r1539215 et al) + * javahl: expose and use the url member of svn_wc_notify_t (r1539601) + * javahl: expose a utility API to resolve relative externals (r1540921) + * javahl: new utility API for file content translation (r1542401) + * javahl: throw NullPointerException when sources is empty (r1543328) + * swig-pl: fix compilation against perl 5.18 on Windows (r1543980) + * javahl: implement streamed file contents translation (r1543985) + * javahl: enable warnings with javac (r1544163, r1544169) + * javahl: cleanup warnings (r1544578 et al) + * javahl: add common utilities for new-style map iterations (r1545925) + * javahl: add an API to get runtime version of native libs (r1545945) + * javahl: use default cache size instead of no cache (r1547248) + * swig-py: add typemap for hunks field of svn_patch_t (r1548379) + * javahl: don't ignore difference between NULL and empty changelist in + ISVNClient.getChangelists (r1553254) + * swig-pl: fix pool issues with setting dates as revisions (r1553822) + * swig-pl: make svn_auth_set_parameter() usable (r1553823) + * javahl: expose inheritiable property names (r1560338) + * javahl: fix bug in parsing single revision merginfo data (r1563140) + * javahl: create JVM crashlogs from svn malfunctions (r1563927, r1564252) + * javahl: avoid problems when building without NLS support (r1566578 et al) + * javahl: expose --alow-mixed-revisions merge option (r1567602) + * javahl: support incomplete reads from streams (r1569631) + * javahl: allow revert API to take a set of paths (r1571461) + * javahl: improve authentication callback (r1597758 et al) + * javahl: load (most) classes on demand (r1602822) + * javahl: expose confg file keys in JavaHL (r1604448) + * javahl: update ISVNClient.info, ISVnClient.cleanup and add + ISvnClient.vacuum (r1604449) + * javahl: update ISVNRepos.hotcopy (r1618894) + * swig-py: Add close to the core.Stream class (r1619077) + * javahl: add example clients that use the authn API (r1640533) + * swig-py: implement dump stream parser (r1642813) + * swig-pl: remove some unneded cleanup code that triggered a cleanup + failure on windows (r1643072) + * swig-pl: make cancel_func, cancel_baton parameter pairs work (r1648852) + * javahl: expose whitespace diff parameters to blame method (issue #4475) + * javahl: update ConflictDescriptor (r1655842) + * javahl: update FileRevision to provide textDelta (r1656911) + * swig-pl: install into prefix (r1658459) + * javahl: expose the metadataOnly option to copy support (r1661451) + * swig-py: add support for svn_fs_lock_many(), svn_fs_unlock_many(), + svn_repos_fs_lock_many() and svn_repos_fs_unlock_many() (r1662867) + * swig-py: fix support for svn_ra_lock() and svn_ra_unlock() (r1662891) + * javahl: allow java callbacks to throw errors back to the svn code + (r1664938, r1664939, r1664940, r1664978, r1664984) + * swig-pl: fix some stack memory problems (r1668618, 1671388) + * swig: warn on using SWIG 3.x as we have compatibility problems + with newer versions of SWIG (1675149) + * javahl: requires Java 1.6 (r1677003) + * javahl: on OS X use /usr/libexec/java_home to find the JDK (r1675774) + * javahl: allow compiling with a C++11 compiler (r1684412) + + Version 1.8.14 (5 Aug 2015, from /branches/1.8.x) http://svn.apache.org/repos/asf/subversion/tags/1.8.14 @@ -13,6 +809,9 @@ http://svn.apache.org/repos/asf/subversi of user and revision after 'svn up' (r1680242) - Server-side bugfixes: + * mod_authz_svn: do not leak information in mixed anonymous/authenticated + httpd (dav) configurations (CVE-2015-3184) + * do not leak paths that were hidden by path-based authz (CVE-2015-3187) * mod_dav_svn: do not ignore skel parsing errors (r1658168) * detect invalid svndiff data earlier (r1684077) * prevent possible repository corruption on power/disk failures (r1680819) @@ -863,10 +1662,17 @@ http://svn.apache.org/repos/asf/subversi * add missing API functions to JavaHL bindings (issue #4326) * fix some reference counting bugs in swig-py bindings (r1464899, r1466524) +Version 1.7.22 +(12 Aug 2015, from /branches/1.7.x) +http://svn.apache.org/repos/asf/subversion/tags/1.7.22 + + Developer-visible changes: + - General: + * fix the regression test suite which was broken in 1.7.21 (r1694012) Version 1.7.21 (5 Aug 2015, from /branches/1.7.x) -http://svn.apache.org/repos/asf/subversion/tags/1.8.21 +http://svn.apache.org/repos/asf/subversion/tags/1.7.21 User-visible changes: - Client-side bugfixes: @@ -875,6 +1681,9 @@ http://svn.apache.org/repos/asf/subversi non-deltas dumpfile (r1652182 et al.) - Server-side bugfixes: + * mod_authz_svn: do not leak information in mixed anonymous/authenticated + httpd (dav) configurations (CVE-2015-3184) + * do not leak paths that were hidden by path-based authz (CVE-2015-3187) * fix 'svnadmin recover' for pre-1.4 FSFS repositories (r1561419) Developer-visible changes: Modified: vendor/subversion/dist/COMMITTERS ============================================================================== --- vendor/subversion/dist/COMMITTERS Mon Oct 12 08:47:46 2015 (r289176) +++ vendor/subversion/dist/COMMITTERS Mon Oct 12 08:54:49 2015 (r289177) @@ -1,4 +1,4 @@ -The following people have commit access to the Subversion sources. +The following people have commit access to the Subversion sources. Note that this is not a full list of Subversion's authors, however -- for that, you'd need to look over the log messages to see all the patch contributors. @@ -58,6 +58,7 @@ Blanket commit access: stefan2 Stefan Fuhrmann jcorvel Johan Corveleyn trent Trent Nelson + kotkov Evgeny Kotkov [[END ACTIVE FULL COMMITTERS. LEAVE THIS LINE HERE; SCRIPTS LOOK FOR IT.]] @@ -96,6 +97,9 @@ Commit access for specific areas: jrvernooij Jelmer Vernooij (Python bindings) sage Sage LaTorra (Ctypes-Python b.) vmpn Vladimir Berezniker (JavaHL bindings) + rschupp Roderich Schupp (Swig bindings) + stilor Alexey Neyman (Python bindings, + svn-vendor.py) Packages: @@ -157,8 +161,10 @@ Commit access for specific areas: artagnon Ramkumar Ramachandra (svnrdump, svntest) arwin Arwin Arni (svn-bisect) joes Joe Schaefer (svnpubsub) - prabhugs Prabhu Gnana Sundar (verify-keep-going) - + humbedooh Daniel Gruno (svnpubsub) + prabhugs Prabhu Gnana Sundar (verify-keep-going) + schabi Markus Schaber (testsuite) + gbg Gabriela Gibson (gtest) Translation of message files: @@ -186,6 +192,7 @@ giorgio_valoti Giorgio Valoti (po: fr) marcelg Marcel Gosselin (po: fr) + mattiase Mattias Engdegård (po: sv) Experimental branches: Modified: vendor/subversion/dist/INSTALL ============================================================================== --- vendor/subversion/dist/INSTALL Mon Oct 12 08:47:46 2015 (r289176) +++ vendor/subversion/dist/INSTALL Mon Oct 12 08:54:49 2015 (r289177) @@ -3,7 +3,7 @@ A Quick Guide ====================================== -$LastChangedDate: 2013-09-27 06:57:44 +0000 (Fri, 27 Sep 2013) $ +$LastChangedDate: 2015-07-26 23:03:10 +0000 (Sun, 26 Jul 2015) $ Contents: @@ -15,7 +15,7 @@ Contents: D. Documentation II. INSTALLATION - A. Building from a Tarball or RPM + A. Building from a Tarball B. Building the Latest Source under Unix C. Building under Unix in Different Directories D. Installing from a Zip or Installer File under Windows @@ -123,7 +123,8 @@ I. INTRODUCTION *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Mon Oct 12 08:55:55 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0A569D2796; Mon, 12 Oct 2015 08:55:55 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E78E1C81; Mon, 12 Oct 2015 08:55:55 +0000 (UTC) (envelope-from peter@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9C8tsTo016552; Mon, 12 Oct 2015 08:55:54 GMT (envelope-from peter@FreeBSD.org) Received: (from peter@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9C8tsJp016551; Mon, 12 Oct 2015 08:55:54 GMT (envelope-from peter@FreeBSD.org) Message-Id: <201510120855.t9C8tsJp016551@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: peter set sender to peter@FreeBSD.org using -f From: Peter Wemm Date: Mon, 12 Oct 2015 08:55:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289178 - vendor/subversion/subversion-1.9.2 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 08:55:55 -0000 Author: peter Date: Mon Oct 12 08:55:54 2015 New Revision: 289178 URL: https://svnweb.freebsd.org/changeset/base/289178 Log: Tag subversion-1.9.2 import. Added: vendor/subversion/subversion-1.9.2/ - copied from r289177, vendor/subversion/dist/ From owner-svn-src-vendor@freebsd.org Mon Oct 12 14:23:01 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F32DA11842; Mon, 12 Oct 2015 14:23:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 640F71B1F; Mon, 12 Oct 2015 14:23:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9CEN09M019800; Mon, 12 Oct 2015 14:23:00 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9CEN0Pq019799; Mon, 12 Oct 2015 14:23:00 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510121423.t9CEN0Pq019799@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 12 Oct 2015 14:23:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289185 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 14:23:01 -0000 Author: mav Date: Mon Oct 12 14:23:00 2015 New Revision: 289185 URL: https://svnweb.freebsd.org/changeset/base/289185 Log: 6250 zvol_dump_init() can hold txg open Reviewed by: Matthew Ahrens Reviewed by: Prakash Surya Reviewed by: Albert Lee Reviewed by: Xin Li Approved by: Garrett D'Amore Author: George Wilson illumos/illumos-gate@b10bba72460aeaa53119c76ff5e647fd5585bece Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Mon Oct 12 13:20:17 2015 (r289184) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Mon Oct 12 14:23:00 2015 (r289185) @@ -1893,13 +1893,15 @@ zvol_dump_init(zvol_state_t *zv, boolean vdev_t *vd = spa->spa_root_vdev; nvlist_t *nv = NULL; uint64_t version = spa_version(spa); - enum zio_checksum checksum; + uint64_t checksum, compress, refresrv, vbs, dedup; ASSERT(MUTEX_HELD(&zfsdev_state_lock)); ASSERT(vd->vdev_ops == &vdev_root_ops); error = dmu_free_long_range(zv->zv_objset, ZVOL_OBJ, 0, DMU_OBJECT_END); + if (error != 0) + return (error); /* wait for dmu_free_long_range to actually free the blocks */ txg_wait_synced(dmu_objset_pool(zv->zv_objset), 0); @@ -1923,24 +1925,42 @@ zvol_dump_init(zvol_state_t *zv, boolean 2, ZFS_SPACE_CHECK_RESERVED); } + if (!resize) { + error = dsl_prop_get_integer(zv->zv_name, + zfs_prop_to_name(ZFS_PROP_COMPRESSION), &compress, NULL); + if (error == 0) { + error = dsl_prop_get_integer(zv->zv_name, + zfs_prop_to_name(ZFS_PROP_CHECKSUM), &checksum, + NULL); + } + if (error == 0) { + error = dsl_prop_get_integer(zv->zv_name, + zfs_prop_to_name(ZFS_PROP_REFRESERVATION), + &refresrv, NULL); + } + if (error == 0) { + error = dsl_prop_get_integer(zv->zv_name, + zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), &vbs, + NULL); + } + if (version >= SPA_VERSION_DEDUP && error == 0) { + error = dsl_prop_get_integer(zv->zv_name, + zfs_prop_to_name(ZFS_PROP_DEDUP), &dedup, NULL); + } + } + if (error != 0) + return (error); + tx = dmu_tx_create(os); dmu_tx_hold_zap(tx, ZVOL_ZAP_OBJ, TRUE, NULL); dmu_tx_hold_bonus(tx, ZVOL_OBJ); error = dmu_tx_assign(tx, TXG_WAIT); - if (error) { + if (error != 0) { dmu_tx_abort(tx); return (error); } /* - * If MULTI_VDEV_CRASH_DUMP is active, use the NOPARITY checksum - * function. Otherwise, use the old default -- OFF. - */ - checksum = spa_feature_is_active(spa, - SPA_FEATURE_MULTI_VDEV_CRASH_DUMP) ? ZIO_CHECKSUM_NOPARITY : - ZIO_CHECKSUM_OFF; - - /* * If we are resizing the dump device then we only need to * update the refreservation to match the newly updated * zvolsize. Otherwise, we save off the original state of the @@ -1951,37 +1971,30 @@ zvol_dump_init(zvol_state_t *zv, boolean zfs_prop_to_name(ZFS_PROP_REFRESERVATION), 8, 1, &zv->zv_volsize, tx); } else { - uint64_t checksum, compress, refresrv, vbs, dedup; - - error = dsl_prop_get_integer(zv->zv_name, - zfs_prop_to_name(ZFS_PROP_COMPRESSION), &compress, NULL); - error = error ? error : dsl_prop_get_integer(zv->zv_name, - zfs_prop_to_name(ZFS_PROP_CHECKSUM), &checksum, NULL); - error = error ? error : dsl_prop_get_integer(zv->zv_name, - zfs_prop_to_name(ZFS_PROP_REFRESERVATION), &refresrv, NULL); - error = error ? error : dsl_prop_get_integer(zv->zv_name, - zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), &vbs, NULL); - if (version >= SPA_VERSION_DEDUP) { - error = error ? error : - dsl_prop_get_integer(zv->zv_name, - zfs_prop_to_name(ZFS_PROP_DEDUP), &dedup, NULL); - } - - error = error ? error : zap_update(os, ZVOL_ZAP_OBJ, + error = zap_update(os, ZVOL_ZAP_OBJ, zfs_prop_to_name(ZFS_PROP_COMPRESSION), 8, 1, &compress, tx); - error = error ? error : zap_update(os, ZVOL_ZAP_OBJ, - zfs_prop_to_name(ZFS_PROP_CHECKSUM), 8, 1, &checksum, tx); - error = error ? error : zap_update(os, ZVOL_ZAP_OBJ, - zfs_prop_to_name(ZFS_PROP_REFRESERVATION), 8, 1, - &refresrv, tx); - error = error ? error : zap_update(os, ZVOL_ZAP_OBJ, - zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), 8, 1, - &vbs, tx); - error = error ? error : dmu_object_set_blocksize( - os, ZVOL_OBJ, SPA_OLD_MAXBLOCKSIZE, 0, tx); - if (version >= SPA_VERSION_DEDUP) { - error = error ? error : zap_update(os, ZVOL_ZAP_OBJ, + if (error == 0) { + error = zap_update(os, ZVOL_ZAP_OBJ, + zfs_prop_to_name(ZFS_PROP_CHECKSUM), 8, 1, + &checksum, tx); + } + if (error == 0) { + error = zap_update(os, ZVOL_ZAP_OBJ, + zfs_prop_to_name(ZFS_PROP_REFRESERVATION), 8, 1, + &refresrv, tx); + } + if (error == 0) { + error = zap_update(os, ZVOL_ZAP_OBJ, + zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), 8, 1, + &vbs, tx); + } + if (error == 0) { + error = dmu_object_set_blocksize( + os, ZVOL_OBJ, SPA_OLD_MAXBLOCKSIZE, 0, tx); + } + if (version >= SPA_VERSION_DEDUP && error == 0) { + error = zap_update(os, ZVOL_ZAP_OBJ, zfs_prop_to_name(ZFS_PROP_DEDUP), 8, 1, &dedup, tx); } @@ -1994,7 +2007,15 @@ zvol_dump_init(zvol_state_t *zv, boolean * We only need update the zvol's property if we are initializing * the dump area for the first time. */ - if (!resize) { + if (error == 0 && !resize) { + /* + * If MULTI_VDEV_CRASH_DUMP is active, use the NOPARITY checksum + * function. Otherwise, use the old default -- OFF. + */ + checksum = spa_feature_is_active(spa, + SPA_FEATURE_MULTI_VDEV_CRASH_DUMP) ? ZIO_CHECKSUM_NOPARITY : + ZIO_CHECKSUM_OFF; + VERIFY(nvlist_alloc(&nv, NV_UNIQUE_NAME, KM_SLEEP) == 0); VERIFY(nvlist_add_uint64(nv, zfs_prop_to_name(ZFS_PROP_REFRESERVATION), 0) == 0); @@ -2013,13 +2034,11 @@ zvol_dump_init(zvol_state_t *zv, boolean error = zfs_set_prop_nvlist(zv->zv_name, ZPROP_SRC_LOCAL, nv, NULL); nvlist_free(nv); - - if (error) - return (error); } /* Allocate the space for the dump */ - error = zvol_prealloc(zv); + if (error == 0) + error = zvol_prealloc(zv); return (error); } From owner-svn-src-vendor@freebsd.org Mon Oct 12 14:24:13 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43DB8A1192E; Mon, 12 Oct 2015 14:24:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E5ED61F26; Mon, 12 Oct 2015 14:24:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9CEOBFZ020072; Mon, 12 Oct 2015 14:24:11 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9CEOBYv020071; Mon, 12 Oct 2015 14:24:11 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510121424.t9CEOBYv020071@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 12 Oct 2015 14:24:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289187 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 14:24:13 -0000 Author: mav Date: Mon Oct 12 14:24:11 2015 New Revision: 289187 URL: https://svnweb.freebsd.org/changeset/base/289187 Log: 6251 add tunable to disable free_bpobj processing Reviewed by: Matthew Ahrens Reviewed by: Prakash Surya Reviewed by: Simon Klinkert Reviewed by: Richard Elling Reviewed by: Albert Lee Reviewed by: Xin Li Approved by: Garrett D'Amore Author: George Wilson illumos/illumos-gate@139510fb6efa97dbe5f5479594b308d940cab8d1 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Mon Oct 12 14:23:10 2015 (r289186) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Mon Oct 12 14:24:11 2015 (r289187) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2014 by Delphix. All rights reserved. + * Copyright (c) 2011, 2015 by Delphix. All rights reserved. */ #include @@ -78,6 +78,11 @@ uint64_t zfs_free_max_blocks = UINT64_MA extern int zfs_txg_timeout; +/* + * Enable/disable the processing of the free_bpobj object. + */ +boolean_t zfs_free_bpobj_enabled = B_TRUE; + /* the order has to match pool_scan_type */ static scan_cb_t *scan_funcs[POOL_SCAN_FUNCS] = { NULL, @@ -1426,7 +1431,8 @@ dsl_scan_sync(dsl_pool_t *dp, dmu_tx_t * * have to worry about traversing it. It is also faster to free the * blocks than to scrub them. */ - if (spa_version(dp->dp_spa) >= SPA_VERSION_DEADLISTS) { + if (zfs_free_bpobj_enabled && + spa_version(dp->dp_spa) >= SPA_VERSION_DEADLISTS) { scn->scn_is_bptree = B_FALSE; scn->scn_zio_root = zio_root(dp->dp_spa, NULL, NULL, ZIO_FLAG_MUSTSUCCEED); From owner-svn-src-vendor@freebsd.org Mon Oct 12 14:27:28 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B49B3A11A66; Mon, 12 Oct 2015 14:27:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 671AB687; Mon, 12 Oct 2015 14:27:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9CERRoG020403; Mon, 12 Oct 2015 14:27:27 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9CERRqi020401; Mon, 12 Oct 2015 14:27:27 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510121427.t9CERRqi020401@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 12 Oct 2015 14:27:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289188 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2015 14:27:28 -0000 Author: mav Date: Mon Oct 12 14:27:27 2015 New Revision: 289188 URL: https://svnweb.freebsd.org/changeset/base/289188 Log: 6281 prefetching should apply to 1MB reads Reviewed by: Matthew Ahrens Reviewed by: Paul Dagnelie Reviewed by: Alexander Motin Reviewed by: Dan McDonald Reviewed by: Justin Gibbs Reviewed by: Xin Li Approved by: Gordon Ross Author: George Wilson illumos/illumos-gate@632802744ef6d17e06d6980a95f631615c3b060f Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Mon Oct 12 14:24:11 2015 (r289187) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Mon Oct 12 14:27:27 2015 (r289188) @@ -442,7 +442,7 @@ dmu_buf_hold_array_by_dnode(dnode_t *dn, } if ((flags & DMU_READ_NO_PREFETCH) == 0 && read && - length < zfetch_array_rd_sz) { + length <= zfetch_array_rd_sz) { dmu_zfetch(&dn->dn_zfetch, blkid, nblks); } rw_exit(&dn->dn_struct_rwlock); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c Mon Oct 12 14:24:11 2015 (r289187) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_zfetch.c Mon Oct 12 14:27:27 2015 (r289188) @@ -24,7 +24,7 @@ */ /* - * Copyright (c) 2013, 2014 by Delphix. All rights reserved. + * Copyright (c) 2013, 2015 by Delphix. All rights reserved. */ #include @@ -49,7 +49,7 @@ uint32_t zfetch_max_streams = 8; uint32_t zfetch_min_sec_reap = 2; /* max bytes to prefetch per stream (default 8MB) */ uint32_t zfetch_max_distance = 8 * 1024 * 1024; -/* number of bytes in a array_read at which we stop prefetching (1MB) */ +/* max number of bytes in an array_read in which we allow prefetching (1MB) */ uint64_t zfetch_array_rd_sz = 1024 * 1024; typedef struct zfetch_stats { From owner-svn-src-vendor@freebsd.org Wed Oct 14 04:30:19 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD499A121CB; Wed, 14 Oct 2015 04:30:19 +0000 (UTC) (envelope-from rpaulo@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 711C51B2C; Wed, 14 Oct 2015 04:30:19 +0000 (UTC) (envelope-from rpaulo@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9E4UIZX000744; Wed, 14 Oct 2015 04:30:18 GMT (envelope-from rpaulo@FreeBSD.org) Received: (from rpaulo@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9E4UINX000738; Wed, 14 Oct 2015 04:30:18 GMT (envelope-from rpaulo@FreeBSD.org) Message-Id: <201510140430.t9E4UINX000738@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rpaulo set sender to rpaulo@FreeBSD.org using -f From: Rui Paulo Date: Wed, 14 Oct 2015 04:30:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289284 - in vendor/wpa/dist: hostapd hs20/client patches src src/ap src/common src/crypto src/drivers src/eap_common src/eap_peer src/eap_server src/eapol_auth src/eapol_supp src/fst s... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 04:30:19 -0000 Author: rpaulo Date: Wed Oct 14 04:30:17 2015 New Revision: 289284 URL: https://svnweb.freebsd.org/changeset/base/289284 Log: Import wpa_supplicant/hostapd 2.5. Major changes: bunch of CVEs fixed, tab completion for wpa_cli and misc bug fixes. Added: vendor/wpa/dist/patches/openssl-0.9.8zf-tls-extensions.patch vendor/wpa/dist/src/crypto/sha384-prf.c (contents, props changed) vendor/wpa/dist/src/fst/ vendor/wpa/dist/src/fst/Makefile (contents, props changed) vendor/wpa/dist/src/fst/fst.c (contents, props changed) vendor/wpa/dist/src/fst/fst.h (contents, props changed) vendor/wpa/dist/src/fst/fst_ctrl_aux.c (contents, props changed) vendor/wpa/dist/src/fst/fst_ctrl_aux.h (contents, props changed) vendor/wpa/dist/src/fst/fst_ctrl_defs.h (contents, props changed) vendor/wpa/dist/src/fst/fst_ctrl_iface.c (contents, props changed) vendor/wpa/dist/src/fst/fst_ctrl_iface.h (contents, props changed) vendor/wpa/dist/src/fst/fst_defs.h (contents, props changed) vendor/wpa/dist/src/fst/fst_group.c (contents, props changed) vendor/wpa/dist/src/fst/fst_group.h (contents, props changed) vendor/wpa/dist/src/fst/fst_iface.c (contents, props changed) vendor/wpa/dist/src/fst/fst_iface.h (contents, props changed) vendor/wpa/dist/src/fst/fst_internal.h (contents, props changed) vendor/wpa/dist/src/fst/fst_session.c (contents, props changed) vendor/wpa/dist/src/fst/fst_session.h (contents, props changed) vendor/wpa/dist/wpa_supplicant/eapol_test.py (contents, props changed) vendor/wpa/dist/wpa_supplicant/p2p_supplicant_sd.c (contents, props changed) Deleted: vendor/wpa/dist/src/crypto/crypto_cryptoapi.c vendor/wpa/dist/src/crypto/tls_schannel.c Modified: vendor/wpa/dist/hostapd/ChangeLog vendor/wpa/dist/hostapd/Makefile vendor/wpa/dist/hostapd/config_file.c vendor/wpa/dist/hostapd/config_file.h vendor/wpa/dist/hostapd/ctrl_iface.c vendor/wpa/dist/hostapd/defconfig vendor/wpa/dist/hostapd/hlr_auc_gw.c vendor/wpa/dist/hostapd/hlr_auc_gw.milenage_db vendor/wpa/dist/hostapd/hostapd.conf vendor/wpa/dist/hostapd/hostapd_cli.c vendor/wpa/dist/hostapd/main.c vendor/wpa/dist/hs20/client/Makefile vendor/wpa/dist/hs20/client/osu_client.c vendor/wpa/dist/hs20/client/spp_client.c vendor/wpa/dist/src/Makefile vendor/wpa/dist/src/ap/Makefile vendor/wpa/dist/src/ap/accounting.c vendor/wpa/dist/src/ap/acs.c vendor/wpa/dist/src/ap/ap_config.c vendor/wpa/dist/src/ap/ap_config.h vendor/wpa/dist/src/ap/ap_drv_ops.c vendor/wpa/dist/src/ap/ap_drv_ops.h vendor/wpa/dist/src/ap/ap_list.c vendor/wpa/dist/src/ap/ap_list.h vendor/wpa/dist/src/ap/authsrv.c vendor/wpa/dist/src/ap/beacon.c vendor/wpa/dist/src/ap/beacon.h vendor/wpa/dist/src/ap/ctrl_iface_ap.c vendor/wpa/dist/src/ap/dfs.c vendor/wpa/dist/src/ap/drv_callbacks.c vendor/wpa/dist/src/ap/eap_user_db.c vendor/wpa/dist/src/ap/hostapd.c vendor/wpa/dist/src/ap/hostapd.h vendor/wpa/dist/src/ap/hw_features.c vendor/wpa/dist/src/ap/hw_features.h vendor/wpa/dist/src/ap/ieee802_11.c vendor/wpa/dist/src/ap/ieee802_11.h vendor/wpa/dist/src/ap/ieee802_11_auth.c vendor/wpa/dist/src/ap/ieee802_11_auth.h vendor/wpa/dist/src/ap/ieee802_11_ht.c vendor/wpa/dist/src/ap/ieee802_11_vht.c vendor/wpa/dist/src/ap/ieee802_1x.c vendor/wpa/dist/src/ap/ieee802_1x.h vendor/wpa/dist/src/ap/ndisc_snoop.c vendor/wpa/dist/src/ap/sta_info.c vendor/wpa/dist/src/ap/sta_info.h vendor/wpa/dist/src/ap/utils.c vendor/wpa/dist/src/ap/vlan_init.c vendor/wpa/dist/src/ap/vlan_init.h vendor/wpa/dist/src/ap/vlan_util.c vendor/wpa/dist/src/ap/wmm.c vendor/wpa/dist/src/ap/wpa_auth.c vendor/wpa/dist/src/ap/wpa_auth.h vendor/wpa/dist/src/ap/wpa_auth_ft.c vendor/wpa/dist/src/ap/wpa_auth_glue.c vendor/wpa/dist/src/ap/wpa_auth_i.h vendor/wpa/dist/src/ap/wpa_auth_ie.c vendor/wpa/dist/src/ap/wps_hostapd.c vendor/wpa/dist/src/ap/x_snoop.c vendor/wpa/dist/src/common/Makefile vendor/wpa/dist/src/common/common_module_tests.c vendor/wpa/dist/src/common/defs.h vendor/wpa/dist/src/common/hw_features_common.c vendor/wpa/dist/src/common/hw_features_common.h vendor/wpa/dist/src/common/ieee802_11_common.c vendor/wpa/dist/src/common/ieee802_11_common.h vendor/wpa/dist/src/common/ieee802_11_defs.h vendor/wpa/dist/src/common/privsep_commands.h vendor/wpa/dist/src/common/qca-vendor.h vendor/wpa/dist/src/common/sae.c vendor/wpa/dist/src/common/sae.h vendor/wpa/dist/src/common/version.h vendor/wpa/dist/src/common/wpa_common.c vendor/wpa/dist/src/common/wpa_common.h vendor/wpa/dist/src/common/wpa_ctrl.c vendor/wpa/dist/src/common/wpa_ctrl.h vendor/wpa/dist/src/crypto/crypto.h vendor/wpa/dist/src/crypto/crypto_module_tests.c vendor/wpa/dist/src/crypto/crypto_openssl.c vendor/wpa/dist/src/crypto/dh_groups.c vendor/wpa/dist/src/crypto/fips_prf_openssl.c vendor/wpa/dist/src/crypto/ms_funcs.c vendor/wpa/dist/src/crypto/ms_funcs.h vendor/wpa/dist/src/crypto/random.c vendor/wpa/dist/src/crypto/sha1-tlsprf.c vendor/wpa/dist/src/crypto/sha1-tprf.c vendor/wpa/dist/src/crypto/sha256-kdf.c vendor/wpa/dist/src/crypto/sha384.h vendor/wpa/dist/src/crypto/tls.h vendor/wpa/dist/src/crypto/tls_gnutls.c vendor/wpa/dist/src/crypto/tls_internal.c vendor/wpa/dist/src/crypto/tls_none.c vendor/wpa/dist/src/crypto/tls_openssl.c vendor/wpa/dist/src/drivers/driver.h vendor/wpa/dist/src/drivers/driver_atheros.c vendor/wpa/dist/src/drivers/driver_bsd.c vendor/wpa/dist/src/drivers/driver_hostap.c vendor/wpa/dist/src/drivers/driver_hostap.h vendor/wpa/dist/src/drivers/driver_ndis.c vendor/wpa/dist/src/drivers/driver_nl80211.c vendor/wpa/dist/src/drivers/driver_nl80211.h vendor/wpa/dist/src/drivers/driver_nl80211_android.c vendor/wpa/dist/src/drivers/driver_nl80211_capa.c vendor/wpa/dist/src/drivers/driver_nl80211_event.c vendor/wpa/dist/src/drivers/driver_nl80211_scan.c vendor/wpa/dist/src/drivers/driver_privsep.c vendor/wpa/dist/src/drivers/driver_wext.c vendor/wpa/dist/src/drivers/drivers.c vendor/wpa/dist/src/drivers/drivers.mak vendor/wpa/dist/src/drivers/linux_ioctl.c vendor/wpa/dist/src/drivers/linux_ioctl.h vendor/wpa/dist/src/drivers/nl80211_copy.h vendor/wpa/dist/src/eap_common/Makefile vendor/wpa/dist/src/eap_common/eap_common.c vendor/wpa/dist/src/eap_common/eap_fast_common.c vendor/wpa/dist/src/eap_common/eap_pwd_common.c vendor/wpa/dist/src/eap_common/eap_pwd_common.h vendor/wpa/dist/src/eap_common/eap_sake_common.c vendor/wpa/dist/src/eap_common/ikev2_common.c vendor/wpa/dist/src/eap_peer/Makefile vendor/wpa/dist/src/eap_peer/eap.c vendor/wpa/dist/src/eap_peer/eap.h vendor/wpa/dist/src/eap_peer/eap_aka.c vendor/wpa/dist/src/eap_peer/eap_eke.c vendor/wpa/dist/src/eap_peer/eap_fast.c vendor/wpa/dist/src/eap_peer/eap_gpsk.c vendor/wpa/dist/src/eap_peer/eap_i.h vendor/wpa/dist/src/eap_peer/eap_mschapv2.c vendor/wpa/dist/src/eap_peer/eap_pax.c vendor/wpa/dist/src/eap_peer/eap_peap.c vendor/wpa/dist/src/eap_peer/eap_pwd.c vendor/wpa/dist/src/eap_peer/eap_sake.c vendor/wpa/dist/src/eap_peer/eap_sim.c vendor/wpa/dist/src/eap_peer/eap_tls.c vendor/wpa/dist/src/eap_peer/eap_tls_common.c vendor/wpa/dist/src/eap_peer/eap_tls_common.h vendor/wpa/dist/src/eap_peer/eap_ttls.c vendor/wpa/dist/src/eap_peer/eap_wsc.c vendor/wpa/dist/src/eap_server/Makefile vendor/wpa/dist/src/eap_server/eap.h vendor/wpa/dist/src/eap_server/eap_i.h vendor/wpa/dist/src/eap_server/eap_server.c vendor/wpa/dist/src/eap_server/eap_server_eke.c vendor/wpa/dist/src/eap_server/eap_server_fast.c vendor/wpa/dist/src/eap_server/eap_server_mschapv2.c vendor/wpa/dist/src/eap_server/eap_server_peap.c vendor/wpa/dist/src/eap_server/eap_server_pwd.c vendor/wpa/dist/src/eap_server/eap_server_tls.c vendor/wpa/dist/src/eap_server/eap_server_tls_common.c vendor/wpa/dist/src/eap_server/eap_server_ttls.c vendor/wpa/dist/src/eap_server/eap_tls_common.h vendor/wpa/dist/src/eapol_auth/Makefile vendor/wpa/dist/src/eapol_auth/eapol_auth_sm.c vendor/wpa/dist/src/eapol_auth/eapol_auth_sm.h vendor/wpa/dist/src/eapol_supp/Makefile vendor/wpa/dist/src/eapol_supp/eapol_supp_sm.c vendor/wpa/dist/src/l2_packet/Makefile vendor/wpa/dist/src/p2p/Makefile vendor/wpa/dist/src/p2p/p2p.c vendor/wpa/dist/src/p2p/p2p.h vendor/wpa/dist/src/p2p/p2p_build.c vendor/wpa/dist/src/p2p/p2p_dev_disc.c vendor/wpa/dist/src/p2p/p2p_go_neg.c vendor/wpa/dist/src/p2p/p2p_group.c vendor/wpa/dist/src/p2p/p2p_i.h vendor/wpa/dist/src/p2p/p2p_invitation.c vendor/wpa/dist/src/p2p/p2p_parse.c vendor/wpa/dist/src/p2p/p2p_pd.c vendor/wpa/dist/src/p2p/p2p_utils.c vendor/wpa/dist/src/radius/Makefile vendor/wpa/dist/src/radius/radius.c vendor/wpa/dist/src/radius/radius_das.c vendor/wpa/dist/src/radius/radius_server.c vendor/wpa/dist/src/radius/radius_server.h vendor/wpa/dist/src/rsn_supp/Makefile vendor/wpa/dist/src/rsn_supp/tdls.c vendor/wpa/dist/src/rsn_supp/wpa.c vendor/wpa/dist/src/rsn_supp/wpa_ft.c vendor/wpa/dist/src/rsn_supp/wpa_ie.c vendor/wpa/dist/src/rsn_supp/wpa_ie.h vendor/wpa/dist/src/tls/libtommath.c vendor/wpa/dist/src/tls/tlsv1_client.c vendor/wpa/dist/src/tls/tlsv1_client.h vendor/wpa/dist/src/tls/tlsv1_server.c vendor/wpa/dist/src/tls/tlsv1_server.h vendor/wpa/dist/src/tls/x509v3.c vendor/wpa/dist/src/utils/browser-wpadebug.c vendor/wpa/dist/src/utils/common.c vendor/wpa/dist/src/utils/common.h vendor/wpa/dist/src/utils/eloop.c vendor/wpa/dist/src/utils/http_curl.c vendor/wpa/dist/src/utils/includes.h vendor/wpa/dist/src/utils/os.h vendor/wpa/dist/src/utils/os_internal.c vendor/wpa/dist/src/utils/os_none.c vendor/wpa/dist/src/utils/os_unix.c vendor/wpa/dist/src/utils/os_win32.c vendor/wpa/dist/src/utils/radiotap.c vendor/wpa/dist/src/utils/utils_module_tests.c vendor/wpa/dist/src/utils/wpa_debug.c vendor/wpa/dist/src/utils/wpa_debug.h vendor/wpa/dist/src/utils/wpabuf.c vendor/wpa/dist/src/wps/Makefile vendor/wpa/dist/src/wps/http_client.c vendor/wpa/dist/src/wps/http_server.c vendor/wpa/dist/src/wps/httpread.c vendor/wpa/dist/src/wps/ndef.c vendor/wpa/dist/src/wps/wps.c vendor/wpa/dist/src/wps/wps.h vendor/wpa/dist/src/wps/wps_attr_parse.c vendor/wpa/dist/src/wps/wps_attr_parse.h vendor/wpa/dist/src/wps/wps_common.c vendor/wpa/dist/src/wps/wps_defs.h vendor/wpa/dist/src/wps/wps_enrollee.c vendor/wpa/dist/src/wps/wps_er.c vendor/wpa/dist/src/wps/wps_er_ssdp.c vendor/wpa/dist/src/wps/wps_module_tests.c vendor/wpa/dist/src/wps/wps_registrar.c vendor/wpa/dist/src/wps/wps_upnp.c vendor/wpa/dist/src/wps/wps_upnp_ap.c vendor/wpa/dist/src/wps/wps_upnp_event.c vendor/wpa/dist/src/wps/wps_upnp_ssdp.c vendor/wpa/dist/src/wps/wps_upnp_web.c vendor/wpa/dist/src/wps/wps_validate.c vendor/wpa/dist/wpa_supplicant/ChangeLog vendor/wpa/dist/wpa_supplicant/Makefile vendor/wpa/dist/wpa_supplicant/ap.c vendor/wpa/dist/wpa_supplicant/ap.h vendor/wpa/dist/wpa_supplicant/bss.c vendor/wpa/dist/wpa_supplicant/bss.h vendor/wpa/dist/wpa_supplicant/config.c vendor/wpa/dist/wpa_supplicant/config.h vendor/wpa/dist/wpa_supplicant/config_file.c vendor/wpa/dist/wpa_supplicant/config_ssid.h vendor/wpa/dist/wpa_supplicant/ctrl_iface.c vendor/wpa/dist/wpa_supplicant/ctrl_iface_named_pipe.c vendor/wpa/dist/wpa_supplicant/ctrl_iface_udp.c vendor/wpa/dist/wpa_supplicant/ctrl_iface_unix.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_new.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_new.h vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_handlers.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_handlers.h vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_handlers_p2p.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_handlers_p2p.h vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_handlers_wps.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_helpers.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_new_introspect.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_old.c vendor/wpa/dist/wpa_supplicant/dbus/dbus_old_handlers.c vendor/wpa/dist/wpa_supplicant/defconfig vendor/wpa/dist/wpa_supplicant/doc/docbook/eapol_test.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_background.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_cli.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_gui.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_gui.sgml vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_passphrase.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_priv.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_supplicant.8 vendor/wpa/dist/wpa_supplicant/doc/docbook/wpa_supplicant.conf.5 vendor/wpa/dist/wpa_supplicant/driver_i.h vendor/wpa/dist/wpa_supplicant/eapol_test.c vendor/wpa/dist/wpa_supplicant/events.c vendor/wpa/dist/wpa_supplicant/hs20_supplicant.c vendor/wpa/dist/wpa_supplicant/ibss_rsn.c vendor/wpa/dist/wpa_supplicant/interworking.c vendor/wpa/dist/wpa_supplicant/main.c vendor/wpa/dist/wpa_supplicant/mesh.c vendor/wpa/dist/wpa_supplicant/mesh_mpm.c vendor/wpa/dist/wpa_supplicant/mesh_rsn.c vendor/wpa/dist/wpa_supplicant/notify.c vendor/wpa/dist/wpa_supplicant/notify.h vendor/wpa/dist/wpa_supplicant/p2p_supplicant.c vendor/wpa/dist/wpa_supplicant/p2p_supplicant.h vendor/wpa/dist/wpa_supplicant/preauth_test.c vendor/wpa/dist/wpa_supplicant/scan.c vendor/wpa/dist/wpa_supplicant/sme.c vendor/wpa/dist/wpa_supplicant/wpa_cli.c vendor/wpa/dist/wpa_supplicant/wpa_priv.c vendor/wpa/dist/wpa_supplicant/wpa_supplicant.c vendor/wpa/dist/wpa_supplicant/wpa_supplicant.conf vendor/wpa/dist/wpa_supplicant/wpa_supplicant_i.h vendor/wpa/dist/wpa_supplicant/wpas_glue.c vendor/wpa/dist/wpa_supplicant/wpas_glue.h vendor/wpa/dist/wpa_supplicant/wps_supplicant.c vendor/wpa/dist/wpa_supplicant/wps_supplicant.h Modified: vendor/wpa/dist/hostapd/ChangeLog ============================================================================== --- vendor/wpa/dist/hostapd/ChangeLog Wed Oct 14 02:43:04 2015 (r289283) +++ vendor/wpa/dist/hostapd/ChangeLog Wed Oct 14 04:30:17 2015 (r289284) @@ -1,5 +1,41 @@ ChangeLog for hostapd +2015-09-27 - v2.5 + * fixed WPS UPnP vulnerability with HTTP chunked transfer encoding + [http://w1.fi/security/2015-2/] (CVE-2015-4141) + * fixed WMM Action frame parser + [http://w1.fi/security/2015-3/] (CVE-2015-4142) + * fixed EAP-pwd server missing payload length validation + [http://w1.fi/security/2015-4/] + (CVE-2015-4143, CVE-2015-4144, CVE-2015-4145) + * fixed validation of WPS and P2P NFC NDEF record payload length + [http://w1.fi/security/2015-5/] + * nl80211: + - fixed vendor command handling to check OUI properly + * fixed hlr_auc_gw build with OpenSSL + * hlr_auc_gw: allow Milenage RES length to be reduced + * disable HT for a station that does not support WMM/QoS + * added support for hashed password (NtHash) in EAP-pwd server + * fixed and extended dynamic VLAN cases + * added EAP-EKE server support for deriving Session-Id + * set Acct-Session-Id to a random value to make it more likely to be + unique even if the device does not have a proper clock + * added more 2.4 GHz channels for 20/40 MHz HT co-ex scan + * modified SAE routines to be more robust and PWE generation to be + stronger against timing attacks + * added support for Brainpool Elliptic Curves with SAE + * increases maximum value accepted for cwmin/cwmax + * added support for CCMP-256 and GCMP-256 as group ciphers with FT + * added Fast Session Transfer (FST) module + * removed optional fields from RSNE when using FT with PMF + (workaround for interoperability issues with iOS 8.4) + * added EAP server support for TLS session resumption + * fixed key derivation for Suite B 192-bit AKM (this breaks + compatibility with the earlier version) + * added mechanism to track unconnected stations and do minimal band + steering + * number of small fixes + 2015-03-15 - v2.4 * allow OpenSSL cipher configuration to be set for internal EAP server (openssl_ciphers parameter) Modified: vendor/wpa/dist/hostapd/Makefile ============================================================================== --- vendor/wpa/dist/hostapd/Makefile Wed Oct 14 02:43:04 2015 (r289283) +++ vendor/wpa/dist/hostapd/Makefile Wed Oct 14 04:30:17 2015 (r289284) @@ -107,7 +107,16 @@ LIBS_h += -lrt LIBS_n += -lrt endif +ifdef CONFIG_ELOOP_POLL +CFLAGS += -DCONFIG_ELOOP_POLL +endif + +ifdef CONFIG_ELOOP_EPOLL +CFLAGS += -DCONFIG_ELOOP_EPOLL +endif + OBJS += ../src/utils/common.o +OBJS_c += ../src/utils/common.o OBJS += ../src/utils/wpa_debug.o OBJS_c += ../src/utils/wpa_debug.o OBJS += ../src/utils/wpabuf.o @@ -227,6 +236,7 @@ CFLAGS += -DCONFIG_SAE OBJS += ../src/common/sae.o NEED_ECC=y NEED_DH_GROUPS=y +NEED_AP_MLME=y endif ifdef CONFIG_WNM @@ -531,8 +541,14 @@ HOBJS += ../src/crypto/crypto_openssl.o ifdef NEED_FIPS186_2_PRF OBJS += ../src/crypto/fips_prf_openssl.o endif +NEED_SHA256=y +NEED_TLS_PRF_SHA256=y LIBS += -lcrypto LIBS_h += -lcrypto +ifdef CONFIG_TLS_ADD_DL +LIBS += -ldl +LIBS_h += -ldl +endif endif ifeq ($(CONFIG_TLS), gnutls) @@ -553,17 +569,6 @@ CONFIG_INTERNAL_RC4=y CONFIG_INTERNAL_DH_GROUP5=y endif -ifeq ($(CONFIG_TLS), schannel) -ifdef TLS_FUNCS -OBJS += ../src/crypto/tls_schannel.o -endif -OBJS += ../src/crypto/crypto_cryptoapi.o -OBJS_p += ../src/crypto/crypto_cryptoapi.o -CONFIG_INTERNAL_SHA256=y -CONFIG_INTERNAL_RC4=y -CONFIG_INTERNAL_DH_GROUP5=y -endif - ifeq ($(CONFIG_TLS), internal) ifndef CONFIG_CRYPTO CONFIG_CRYPTO=internal @@ -694,8 +699,10 @@ endif endif ifdef NEED_AES_CBC NEED_AES_DEC=y +ifneq ($(CONFIG_TLS), openssl) AESOBJS += ../src/crypto/aes-cbc.o endif +endif ifdef NEED_AES_DEC ifdef CONFIG_INTERNAL_AES AESOBJS += ../src/crypto/aes-internal-dec.o @@ -754,11 +761,17 @@ OBJS += ../src/crypto/des-internal.o endif endif +ifdef CONFIG_NO_RC4 +CFLAGS += -DCONFIG_NO_RC4 +endif + ifdef NEED_RC4 ifdef CONFIG_INTERNAL_RC4 +ifndef CONFIG_NO_RC4 OBJS += ../src/crypto/rc4.o endif endif +endif ifdef NEED_SHA256 CFLAGS += -DCONFIG_SHA256 @@ -778,6 +791,7 @@ endif endif ifdef NEED_SHA384 CFLAGS += -DCONFIG_SHA384 +OBJS += ../src/crypto/sha384-prf.o endif ifdef NEED_DH_GROUPS @@ -803,8 +817,10 @@ OBJS += ../src/crypto/random.o HOBJS += ../src/crypto/random.o HOBJS += ../src/utils/eloop.o HOBJS += $(SHA1OBJS) +ifneq ($(CONFIG_TLS), openssl) HOBJS += ../src/crypto/md5.o endif +endif ifdef CONFIG_RADIUS_SERVER CFLAGS += -DRADIUS_SERVER @@ -903,6 +919,21 @@ LIBS += -lsqlite3 LIBS_h += -lsqlite3 endif +ifdef CONFIG_FST +CFLAGS += -DCONFIG_FST +OBJS += ../src/fst/fst.o +OBJS += ../src/fst/fst_group.o +OBJS += ../src/fst/fst_iface.o +OBJS += ../src/fst/fst_session.o +OBJS += ../src/fst/fst_ctrl_aux.o +ifdef CONFIG_FST_TEST +CFLAGS += -DCONFIG_FST_TEST +endif +ifndef CONFIG_NO_CTRL_IFACE +OBJS += ../src/fst/fst_ctrl_iface.o +endif +endif + ALL=hostapd hostapd_cli all: verify_config $(ALL) @@ -965,9 +996,11 @@ NOBJS = nt_password_hash.o ../src/crypto NOBJS += ../src/utils/common.o ifdef NEED_RC4 ifdef CONFIG_INTERNAL_RC4 +ifndef CONFIG_NO_RC4 NOBJS += ../src/crypto/rc4.o endif endif +endif ifdef CONFIG_INTERNAL_MD5 NOBJS += ../src/crypto/md5-internal.o endif Modified: vendor/wpa/dist/hostapd/config_file.c ============================================================================== --- vendor/wpa/dist/hostapd/config_file.c Wed Oct 14 02:43:04 2015 (r289283) +++ vendor/wpa/dist/hostapd/config_file.c Wed Oct 14 04:30:17 2015 (r289284) @@ -222,9 +222,15 @@ static int hostapd_config_read_eap_user( return 0; if (os_strncmp(fname, "sqlite:", 7) == 0) { +#ifdef CONFIG_SQLITE os_free(conf->eap_user_sqlite); conf->eap_user_sqlite = os_strdup(fname + 7); return 0; +#else /* CONFIG_SQLITE */ + wpa_printf(MSG_ERROR, + "EAP user file in SQLite DB, but CONFIG_SQLITE was not enabled in the build."); + return -1; +#endif /* CONFIG_SQLITE */ } f = fopen(fname, "r"); @@ -775,6 +781,24 @@ static int hostapd_config_read_wep(struc } +static int hostapd_parse_chanlist(struct hostapd_config *conf, char *val) +{ + char *pos; + + /* for backwards compatibility, translate ' ' in conf str to ',' */ + pos = val; + while (pos) { + pos = os_strchr(pos, ' '); + if (pos) + *pos++ = ','; + } + if (freq_range_list_parse(&conf->acs_ch_list, val)) + return -1; + + return 0; +} + + static int hostapd_parse_intlist(int **int_list, char *val) { int *list; @@ -875,7 +899,9 @@ static int hostapd_config_read_int10(con static int valid_cw(int cw) { return (cw == 1 || cw == 3 || cw == 7 || cw == 15 || cw == 31 || - cw == 63 || cw == 127 || cw == 255 || cw == 511 || cw == 1023); + cw == 63 || cw == 127 || cw == 255 || cw == 511 || cw == 1023 || + cw == 2047 || cw == 4095 || cw == 8191 || cw == 16383 || + cw == 32767); } @@ -886,11 +912,11 @@ enum { IEEE80211_TX_QUEUE_DATA3 = 3 /* used for EDCA AC_BK data */ }; -static int hostapd_config_tx_queue(struct hostapd_config *conf, char *name, - char *val) +static int hostapd_config_tx_queue(struct hostapd_config *conf, + const char *name, const char *val) { int num; - char *pos; + const char *pos; struct hostapd_tx_queue_params *queue; /* skip 'tx_queue_' prefix */ @@ -1134,13 +1160,23 @@ static int hostapd_config_vht_capab(stru if (os_strstr(capab, "[BF-ANTENNA-2]") && (conf->vht_capab & VHT_CAP_SU_BEAMFORMEE_CAPABLE)) conf->vht_capab |= (1 << VHT_CAP_BEAMFORMEE_STS_OFFSET); + if (os_strstr(capab, "[BF-ANTENNA-3]") && + (conf->vht_capab & VHT_CAP_SU_BEAMFORMEE_CAPABLE)) + conf->vht_capab |= (2 << VHT_CAP_BEAMFORMEE_STS_OFFSET); + if (os_strstr(capab, "[BF-ANTENNA-4]") && + (conf->vht_capab & VHT_CAP_SU_BEAMFORMEE_CAPABLE)) + conf->vht_capab |= (3 << VHT_CAP_BEAMFORMEE_STS_OFFSET); if (os_strstr(capab, "[SOUNDING-DIMENSION-2]") && (conf->vht_capab & VHT_CAP_SU_BEAMFORMER_CAPABLE)) conf->vht_capab |= (1 << VHT_CAP_SOUNDING_DIMENSION_OFFSET); + if (os_strstr(capab, "[SOUNDING-DIMENSION-3]") && + (conf->vht_capab & VHT_CAP_SU_BEAMFORMER_CAPABLE)) + conf->vht_capab |= (2 << VHT_CAP_SOUNDING_DIMENSION_OFFSET); + if (os_strstr(capab, "[SOUNDING-DIMENSION-4]") && + (conf->vht_capab & VHT_CAP_SU_BEAMFORMER_CAPABLE)) + conf->vht_capab |= (3 << VHT_CAP_SOUNDING_DIMENSION_OFFSET); if (os_strstr(capab, "[MU-BEAMFORMER]")) conf->vht_capab |= VHT_CAP_MU_BEAMFORMER_CAPABLE; - if (os_strstr(capab, "[MU-BEAMFORMEE]")) - conf->vht_capab |= VHT_CAP_MU_BEAMFORMEE_CAPABLE; if (os_strstr(capab, "[VHT-TXOP-PS]")) conf->vht_capab |= VHT_CAP_VHT_TXOP_PS; if (os_strstr(capab, "[HTC-VHT]")) @@ -1699,7 +1735,7 @@ static int hs20_parse_osu_ssid(struct ho char *str; str = wpa_config_parse_string(pos, &slen); - if (str == NULL || slen < 1 || slen > HOSTAPD_MAX_SSID_LEN) { + if (str == NULL || slen < 1 || slen > SSID_MAX_LEN) { wpa_printf(MSG_ERROR, "Line %d: Invalid SSID '%s'", line, pos); os_free(str); return -1; @@ -1900,7 +1936,7 @@ fail: static int hostapd_config_fill(struct hostapd_config *conf, struct hostapd_bss_config *bss, - char *buf, char *pos, int line) + const char *buf, char *pos, int line) { if (os_strcmp(buf, "interface") == 0) { os_strlcpy(conf->bss[0]->iface, pos, @@ -1946,7 +1982,7 @@ static int hostapd_config_fill(struct ho line); } else if (os_strcmp(buf, "ssid") == 0) { bss->ssid.ssid_len = os_strlen(pos); - if (bss->ssid.ssid_len > HOSTAPD_MAX_SSID_LEN || + if (bss->ssid.ssid_len > SSID_MAX_LEN || bss->ssid.ssid_len < 1) { wpa_printf(MSG_ERROR, "Line %d: invalid SSID '%s'", line, pos); @@ -1957,7 +1993,7 @@ static int hostapd_config_fill(struct ho } else if (os_strcmp(buf, "ssid2") == 0) { size_t slen; char *str = wpa_config_parse_string(pos, &slen); - if (str == NULL || slen < 1 || slen > HOSTAPD_MAX_SSID_LEN) { + if (str == NULL || slen < 1 || slen > SSID_MAX_LEN) { wpa_printf(MSG_ERROR, "Line %d: invalid SSID '%s'", line, pos); os_free(str); @@ -2043,6 +2079,8 @@ static int hostapd_config_fill(struct ho bss->private_key_passwd = os_strdup(pos); } else if (os_strcmp(buf, "check_crl") == 0) { bss->check_crl = atoi(pos); + } else if (os_strcmp(buf, "tls_session_lifetime") == 0) { + bss->tls_session_lifetime = atoi(pos); } else if (os_strcmp(buf, "ocsp_stapling_response") == 0) { os_free(bss->ocsp_stapling_response); bss->ocsp_stapling_response = os_strdup(pos); @@ -2515,13 +2553,17 @@ static int hostapd_config_fill(struct ho conf->hw_mode = HOSTAPD_MODE_IEEE80211G; else if (os_strcmp(pos, "ad") == 0) conf->hw_mode = HOSTAPD_MODE_IEEE80211AD; + else if (os_strcmp(pos, "any") == 0) + conf->hw_mode = HOSTAPD_MODE_IEEE80211ANY; else { wpa_printf(MSG_ERROR, "Line %d: unknown hw_mode '%s'", line, pos); return 1; } } else if (os_strcmp(buf, "wps_rf_bands") == 0) { - if (os_strcmp(pos, "a") == 0) + if (os_strcmp(pos, "ad") == 0) + bss->wps_rf_bands = WPS_RF_60GHZ; + else if (os_strcmp(pos, "a") == 0) bss->wps_rf_bands = WPS_RF_50GHZ; else if (os_strcmp(pos, "g") == 0 || os_strcmp(pos, "b") == 0) @@ -2542,12 +2584,15 @@ static int hostapd_config_fill(struct ho line); return 1; #else /* CONFIG_ACS */ + conf->acs = 1; conf->channel = 0; #endif /* CONFIG_ACS */ - } else + } else { conf->channel = atoi(pos); + conf->acs = conf->channel == 0; + } } else if (os_strcmp(buf, "chanlist") == 0) { - if (hostapd_parse_intlist(&conf->chanlist, pos)) { + if (hostapd_parse_chanlist(conf, pos)) { wpa_printf(MSG_ERROR, "Line %d: invalid channel list", line); return 1; @@ -2810,7 +2855,7 @@ static int hostapd_config_fill(struct ho os_free(bss->wps_pin_requests); bss->wps_pin_requests = os_strdup(pos); } else if (os_strcmp(buf, "device_name") == 0) { - if (os_strlen(pos) > 32) { + if (os_strlen(pos) > WPS_DEV_NAME_MAX_LEN) { wpa_printf(MSG_ERROR, "Line %d: Too long " "device_name", line); return 1; @@ -3111,6 +3156,8 @@ static int hostapd_config_fill(struct ho bss->disable_dgaf = atoi(pos); } else if (os_strcmp(buf, "proxy_arp") == 0) { bss->proxy_arp = atoi(pos); + } else if (os_strcmp(buf, "na_mcast_to_ucast") == 0) { + bss->na_mcast_to_ucast = atoi(pos); } else if (os_strcmp(buf, "osen") == 0) { bss->osen = atoi(pos); } else if (os_strcmp(buf, "anqp_domain_id") == 0) { @@ -3223,6 +3270,24 @@ static int hostapd_config_fill(struct ho bss->bss_load_test_set = 1; } else if (os_strcmp(buf, "radio_measurements") == 0) { bss->radio_measurements = atoi(pos); + } else if (os_strcmp(buf, "own_ie_override") == 0) { + struct wpabuf *tmp; + size_t len = os_strlen(pos) / 2; + + tmp = wpabuf_alloc(len); + if (!tmp) + return 1; + + if (hexstr2bin(pos, wpabuf_put(tmp, len), len)) { + wpabuf_free(tmp); + wpa_printf(MSG_ERROR, + "Line %d: Invalid own_ie_override '%s'", + line, pos); + return 1; + } + + wpabuf_free(bss->own_ie_override); + bss->own_ie_override = tmp; #endif /* CONFIG_TESTING_OPTIONS */ } else if (os_strcmp(buf, "vendor_elements") == 0) { struct wpabuf *elems; @@ -3276,6 +3341,74 @@ static int hostapd_config_fill(struct ho } else if (os_strcmp(buf, "wowlan_triggers") == 0) { os_free(bss->wowlan_triggers); bss->wowlan_triggers = os_strdup(pos); +#ifdef CONFIG_FST + } else if (os_strcmp(buf, "fst_group_id") == 0) { + size_t len = os_strlen(pos); + + if (!len || len >= sizeof(conf->fst_cfg.group_id)) { + wpa_printf(MSG_ERROR, + "Line %d: Invalid fst_group_id value '%s'", + line, pos); + return 1; + } + + if (conf->fst_cfg.group_id[0]) { + wpa_printf(MSG_ERROR, + "Line %d: Duplicate fst_group value '%s'", + line, pos); + return 1; + } + + os_strlcpy(conf->fst_cfg.group_id, pos, + sizeof(conf->fst_cfg.group_id)); + } else if (os_strcmp(buf, "fst_priority") == 0) { + char *endp; + long int val; + + if (!*pos) { + wpa_printf(MSG_ERROR, + "Line %d: fst_priority value not supplied (expected 1..%u)", + line, FST_MAX_PRIO_VALUE); + return -1; + } + + val = strtol(pos, &endp, 0); + if (*endp || val < 1 || val > FST_MAX_PRIO_VALUE) { + wpa_printf(MSG_ERROR, + "Line %d: Invalid fst_priority %ld (%s) (expected 1..%u)", + line, val, pos, FST_MAX_PRIO_VALUE); + return 1; + } + conf->fst_cfg.priority = (u8) val; + } else if (os_strcmp(buf, "fst_llt") == 0) { + char *endp; + long int val; + + if (!*pos) { + wpa_printf(MSG_ERROR, + "Line %d: fst_llt value not supplied (expected 1..%u)", + line, FST_MAX_LLT_MS); + return -1; + } + val = strtol(pos, &endp, 0); + if (*endp || val < 1 || val > FST_MAX_LLT_MS) { + wpa_printf(MSG_ERROR, + "Line %d: Invalid fst_llt %ld (%s) (expected 1..%u)", + line, val, pos, FST_MAX_LLT_MS); + return 1; + } + conf->fst_cfg.llt = (u32) val; +#endif /* CONFIG_FST */ + } else if (os_strcmp(buf, "track_sta_max_num") == 0) { + conf->track_sta_max_num = atoi(pos); + } else if (os_strcmp(buf, "track_sta_max_age") == 0) { + conf->track_sta_max_age = atoi(pos); + } else if (os_strcmp(buf, "no_probe_resp_if_seen_on") == 0) { + os_free(bss->no_probe_resp_if_seen_on); + bss->no_probe_resp_if_seen_on = os_strdup(pos); + } else if (os_strcmp(buf, "no_auth_if_seen_on") == 0) { + os_free(bss->no_auth_if_seen_on); + bss->no_auth_if_seen_on = os_strdup(pos); } else { wpa_printf(MSG_ERROR, "Line %d: unknown configuration item '%s'", @@ -3378,7 +3511,8 @@ struct hostapd_config * hostapd_config_r int hostapd_set_iface(struct hostapd_config *conf, - struct hostapd_bss_config *bss, char *field, char *value) + struct hostapd_bss_config *bss, const char *field, + char *value) { int errors; size_t i; Modified: vendor/wpa/dist/hostapd/config_file.h ============================================================================== --- vendor/wpa/dist/hostapd/config_file.h Wed Oct 14 02:43:04 2015 (r289283) +++ vendor/wpa/dist/hostapd/config_file.h Wed Oct 14 04:30:17 2015 (r289284) @@ -11,7 +11,7 @@ struct hostapd_config * hostapd_config_read(const char *fname); int hostapd_set_iface(struct hostapd_config *conf, - struct hostapd_bss_config *bss, char *field, + struct hostapd_bss_config *bss, const char *field, char *value); #endif /* CONFIG_FILE_H */ Modified: vendor/wpa/dist/hostapd/ctrl_iface.c ============================================================================== --- vendor/wpa/dist/hostapd/ctrl_iface.c Wed Oct 14 02:43:04 2015 (r289283) +++ vendor/wpa/dist/hostapd/ctrl_iface.c Wed Oct 14 04:30:17 2015 (r289284) @@ -25,6 +25,7 @@ #include "common/ieee802_11_defs.h" #include "crypto/tls.h" #include "drivers/driver.h" +#include "eapol_auth/eapol_auth_sm.h" #include "radius/radius_client.h" #include "radius/radius_server.h" #include "l2_packet/l2_packet.h" @@ -43,10 +44,13 @@ #include "ap/beacon.h" #include "wps/wps_defs.h" #include "wps/wps.h" +#include "fst/fst_ctrl_iface.h" #include "config_file.h" #include "ctrl_iface.h" +#define HOSTAPD_CLI_DUP_VALUE_MAX_LEN 256 + struct wpa_ctrl_dst { struct wpa_ctrl_dst *next; struct sockaddr_un addr; @@ -57,6 +61,7 @@ struct wpa_ctrl_dst { static void hostapd_ctrl_iface_send(struct hostapd_data *hapd, int level, + enum wpa_msg_type type, const char *buf, size_t len); @@ -1055,6 +1060,97 @@ static int hostapd_ctrl_iface_bss_tm_req #endif /* CONFIG_WNM */ +static int hostapd_ctrl_iface_get_key_mgmt(struct hostapd_data *hapd, + char *buf, size_t buflen) +{ + int ret = 0; + char *pos, *end; + + pos = buf; + end = buf + buflen; + + WPA_ASSERT(hapd->conf->wpa_key_mgmt); + + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_PSK) { + ret = os_snprintf(pos, end - pos, "WPA-PSK "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_IEEE8021X) { + ret = os_snprintf(pos, end - pos, "WPA-EAP "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } +#ifdef CONFIG_IEEE80211R + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_FT_PSK) { + ret = os_snprintf(pos, end - pos, "FT-PSK "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_FT_IEEE8021X) { + ret = os_snprintf(pos, end - pos, "FT-EAP "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } +#ifdef CONFIG_SAE + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_FT_SAE) { + ret = os_snprintf(pos, end - pos, "FT-SAE "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } +#endif /* CONFIG_SAE */ +#endif /* CONFIG_IEEE80211R */ +#ifdef CONFIG_IEEE80211W + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_PSK_SHA256) { + ret = os_snprintf(pos, end - pos, "WPA-PSK-SHA256 "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_IEEE8021X_SHA256) { + ret = os_snprintf(pos, end - pos, "WPA-EAP-SHA256 "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } +#endif /* CONFIG_IEEE80211W */ +#ifdef CONFIG_SAE + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_SAE) { + ret = os_snprintf(pos, end - pos, "SAE "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } +#endif /* CONFIG_SAE */ + if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_IEEE8021X_SUITE_B) { + ret = os_snprintf(pos, end - pos, "WPA-EAP-SUITE-B "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } + if (hapd->conf->wpa_key_mgmt & + WPA_KEY_MGMT_IEEE8021X_SUITE_B_192) { + ret = os_snprintf(pos, end - pos, + "WPA-EAP-SUITE-B-192 "); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } + + if (pos > buf && *(pos - 1) == ' ') { + *(pos - 1) = '\0'; + pos--; + } + + return pos - buf; +} + + static int hostapd_ctrl_iface_get_config(struct hostapd_data *hapd, char *buf, size_t buflen) { @@ -1104,82 +1200,20 @@ static int hostapd_ctrl_iface_get_config } #endif /* CONFIG_WPS */ + if (hapd->conf->wpa) { + ret = os_snprintf(pos, end - pos, "wpa=%d\n", hapd->conf->wpa); + if (os_snprintf_error(end - pos, ret)) + return pos - buf; + pos += ret; + } + if (hapd->conf->wpa && hapd->conf->wpa_key_mgmt) { ret = os_snprintf(pos, end - pos, "key_mgmt="); if (os_snprintf_error(end - pos, ret)) return pos - buf; pos += ret; - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_PSK) { - ret = os_snprintf(pos, end - pos, "WPA-PSK "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_IEEE8021X) { - ret = os_snprintf(pos, end - pos, "WPA-EAP "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } -#ifdef CONFIG_IEEE80211R - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_FT_PSK) { - ret = os_snprintf(pos, end - pos, "FT-PSK "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_FT_IEEE8021X) { - ret = os_snprintf(pos, end - pos, "FT-EAP "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } -#ifdef CONFIG_SAE - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_FT_SAE) { - ret = os_snprintf(pos, end - pos, "FT-SAE "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } -#endif /* CONFIG_SAE */ -#endif /* CONFIG_IEEE80211R */ -#ifdef CONFIG_IEEE80211W - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_PSK_SHA256) { - ret = os_snprintf(pos, end - pos, "WPA-PSK-SHA256 "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_IEEE8021X_SHA256) { - ret = os_snprintf(pos, end - pos, "WPA-EAP-SHA256 "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } -#endif /* CONFIG_IEEE80211W */ -#ifdef CONFIG_SAE - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_SAE) { - ret = os_snprintf(pos, end - pos, "SAE "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } -#endif /* CONFIG_SAE */ - if (hapd->conf->wpa_key_mgmt & WPA_KEY_MGMT_IEEE8021X_SUITE_B) { - ret = os_snprintf(pos, end - pos, "WPA-EAP-SUITE-B "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } - if (hapd->conf->wpa_key_mgmt & - WPA_KEY_MGMT_IEEE8021X_SUITE_B_192) { - ret = os_snprintf(pos, end - pos, - "WPA-EAP-SUITE-B-192 "); - if (os_snprintf_error(end - pos, ret)) - return pos - buf; - pos += ret; - } + pos += hostapd_ctrl_iface_get_key_mgmt(hapd, pos, end - pos); ret = os_snprintf(pos, end - pos, "\n"); if (os_snprintf_error(end - pos, ret)) @@ -1528,7 +1562,7 @@ void hostapd_data_test_rx(void *ctx, con { struct hostapd_data *hapd = ctx; const struct ether_header *eth; - const struct iphdr *ip; + struct iphdr ip; const u8 *pos; unsigned int i; @@ -1536,14 +1570,14 @@ void hostapd_data_test_rx(void *ctx, con return; eth = (const struct ether_header *) buf; - ip = (const struct iphdr *) (eth + 1); - pos = (const u8 *) (ip + 1); + os_memcpy(&ip, eth + 1, sizeof(ip)); + pos = &buf[sizeof(*eth) + sizeof(ip)]; - if (ip->ihl != 5 || ip->version != 4 || - ntohs(ip->tot_len) != HWSIM_IP_LEN) + if (ip.ihl != 5 || ip.version != 4 || + ntohs(ip.tot_len) != HWSIM_IP_LEN) return; - for (i = 0; i < HWSIM_IP_LEN - sizeof(*ip); i++) { + for (i = 0; i < HWSIM_IP_LEN - sizeof(ip); i++) { if (*pos != (u8) i) return; pos++; @@ -1599,7 +1633,7 @@ static int hostapd_ctrl_iface_data_test_ int used; long int val; u8 tos; - u8 buf[HWSIM_PACKETLEN]; + u8 buf[2 + HWSIM_PACKETLEN]; struct ether_header *eth; struct iphdr *ip; u8 *dpos; @@ -1627,7 +1661,7 @@ static int hostapd_ctrl_iface_data_test_ return -1; tos = val; - eth = (struct ether_header *) buf; + eth = (struct ether_header *) &buf[2]; os_memcpy(eth->ether_dhost, dst, ETH_ALEN); os_memcpy(eth->ether_shost, src, ETH_ALEN); eth->ether_type = htons(ETHERTYPE_IP); @@ -1639,14 +1673,14 @@ static int hostapd_ctrl_iface_data_test_ ip->tos = tos; ip->tot_len = htons(HWSIM_IP_LEN); ip->protocol = 1; - ip->saddr = htonl(192 << 24 | 168 << 16 | 1 << 8 | 1); - ip->daddr = htonl(192 << 24 | 168 << 16 | 1 << 8 | 2); + ip->saddr = htonl(192U << 24 | 168 << 16 | 1 << 8 | 1); + ip->daddr = htonl(192U << 24 | 168 << 16 | 1 << 8 | 2); ip->check = ipv4_hdr_checksum(ip, sizeof(*ip)); dpos = (u8 *) (ip + 1); for (i = 0; i < HWSIM_IP_LEN - sizeof(*ip); i++) *dpos++ = i; - if (l2_packet_send(hapd->l2_test, dst, ETHERTYPE_IP, buf, + if (l2_packet_send(hapd->l2_test, dst, ETHERTYPE_IP, &buf[2], HWSIM_PACKETLEN) < 0) return -1; @@ -1746,6 +1780,45 @@ static int hostapd_ctrl_get_alloc_fail(s #endif /* WPA_TRACE_BFD */ } + +static int hostapd_ctrl_test_fail(struct hostapd_data *hapd, char *cmd) +{ +#ifdef WPA_TRACE_BFD + extern char wpa_trace_test_fail_func[256]; + extern unsigned int wpa_trace_test_fail_after; + char *pos; + + wpa_trace_test_fail_after = atoi(cmd); + pos = os_strchr(cmd, ':'); + if (pos) { + pos++; + os_strlcpy(wpa_trace_test_fail_func, pos, + sizeof(wpa_trace_test_fail_func)); + } else { + wpa_trace_test_fail_after = 0; + } + + return 0; +#else /* WPA_TRACE_BFD */ + return -1; +#endif /* WPA_TRACE_BFD */ +} + + +static int hostapd_ctrl_get_fail(struct hostapd_data *hapd, + char *buf, size_t buflen) +{ +#ifdef WPA_TRACE_BFD + extern char wpa_trace_test_fail_func[256]; + extern unsigned int wpa_trace_test_fail_after; + + return os_snprintf(buf, buflen, "%u:%s", wpa_trace_test_fail_after, + wpa_trace_test_fail_func); +#else /* WPA_TRACE_BFD */ + return -1; +#endif /* WPA_TRACE_BFD */ +} + #endif /* CONFIG_TESTING_OPTIONS */ @@ -1847,41 +1920,134 @@ static int hostapd_ctrl_iface_vendor(str } -static void hostapd_ctrl_iface_receive(int sock, void *eloop_ctx, - void *sock_ctx) +static int hostapd_ctrl_iface_eapol_reauth(struct hostapd_data *hapd, + const char *cmd) { - struct hostapd_data *hapd = eloop_ctx; - char buf[4096]; - int res; - struct sockaddr_un from; - socklen_t fromlen = sizeof(from); - char *reply; - const int reply_size = 4096; - int reply_len; - int level = MSG_DEBUG; + u8 addr[ETH_ALEN]; + struct sta_info *sta; - res = recvfrom(sock, buf, sizeof(buf) - 1, 0, - (struct sockaddr *) &from, &fromlen); - if (res < 0) { - wpa_printf(MSG_ERROR, "recvfrom(ctrl_iface): %s", - strerror(errno)); - return; + if (hwaddr_aton(cmd, addr)) + return -1; + + sta = ap_get_sta(hapd, addr); + if (!sta || !sta->eapol_sm) + return -1; + + eapol_auth_reauthenticate(sta->eapol_sm); + return 0; +} + + +static int hostapd_ctrl_iface_eapol_set(struct hostapd_data *hapd, char *cmd) +{ + u8 addr[ETH_ALEN]; + struct sta_info *sta; + char *pos = cmd, *param; + + if (hwaddr_aton(pos, addr) || pos[17] != ' ') + return -1; + pos += 18; + param = pos; + pos = os_strchr(pos, ' '); + if (!pos) + return -1; + *pos++ = '\0'; + + sta = ap_get_sta(hapd, addr); + if (!sta || !sta->eapol_sm) + return -1; + + return eapol_auth_set_conf(sta->eapol_sm, param, pos); +} + + +static int hostapd_ctrl_iface_log_level(struct hostapd_data *hapd, char *cmd, + char *buf, size_t buflen) +{ + char *pos, *end, *stamp; + int ret; + + /* cmd: "LOG_LEVEL []" */ + if (*cmd == '\0') { + pos = buf; + end = buf + buflen; + ret = os_snprintf(pos, end - pos, "Current level: %s\n" + "Timestamp: %d\n", + debug_level_str(wpa_debug_level), + wpa_debug_timestamp); + if (os_snprintf_error(end - pos, ret)) + ret = 0; + + return ret; } - buf[res] = '\0'; - if (os_strcmp(buf, "PING") == 0) - level = MSG_EXCESSIVE; - wpa_hexdump_ascii(level, "RX ctrl_iface", (u8 *) buf, res); - reply = os_malloc(reply_size); - if (reply == NULL) { - if (sendto(sock, "FAIL\n", 5, 0, (struct sockaddr *) &from, - fromlen) < 0) { - wpa_printf(MSG_DEBUG, "CTRL: sendto failed: %s", - strerror(errno)); + while (*cmd == ' ') + cmd++; + + stamp = os_strchr(cmd, ' '); + if (stamp) { + *stamp++ = '\0'; + while (*stamp == ' ') { + stamp++; } - return; } + if (os_strlen(cmd)) { + int level = str_to_debug_level(cmd); + if (level < 0) + return -1; + wpa_debug_level = level; + } + + if (stamp && os_strlen(stamp)) + wpa_debug_timestamp = atoi(stamp); + + os_memcpy(buf, "OK\n", 3); + return 3; +} + + +#ifdef NEED_AP_MLME +static int hostapd_ctrl_iface_track_sta_list(struct hostapd_data *hapd, + char *buf, size_t buflen) +{ + struct hostapd_iface *iface = hapd->iface; + char *pos, *end; + struct hostapd_sta_info *info; + struct os_reltime now; + + sta_track_expire(iface, 0); + + pos = buf; + end = buf + buflen; + + os_get_reltime(&now); + dl_list_for_each_reverse(info, &iface->sta_seen, + struct hostapd_sta_info, list) { + struct os_reltime age; + int ret; + + os_reltime_sub(&now, &info->last_seen, &age); + ret = os_snprintf(pos, end - pos, MACSTR " %u\n", + MAC2STR(info->addr), (unsigned int) age.sec); + if (os_snprintf_error(end - pos, ret)) + break; + pos += ret; + } + + return pos - buf; +} +#endif /* NEED_AP_MLME */ + + +static int hostapd_ctrl_iface_receive_process(struct hostapd_data *hapd, + char *buf, char *reply, + int reply_size, + struct sockaddr_un *from, + socklen_t fromlen) +{ + int reply_len, res; + os_memcpy(reply, "OK\n", 3); reply_len = 3; @@ -1938,13 +2104,13 @@ static void hostapd_ctrl_iface_receive(i reply_len = hostapd_ctrl_iface_sta_next(hapd, buf + 9, reply, reply_size); *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Oct 14 04:32:57 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22737A12379; Wed, 14 Oct 2015 04:32:57 +0000 (UTC) (envelope-from rpaulo@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2EF31EC5; Wed, 14 Oct 2015 04:32:56 +0000 (UTC) (envelope-from rpaulo@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9E4WtEw003455; Wed, 14 Oct 2015 04:32:55 GMT (envelope-from rpaulo@FreeBSD.org) Received: (from rpaulo@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9E4WtJA003454; Wed, 14 Oct 2015 04:32:55 GMT (envelope-from rpaulo@FreeBSD.org) Message-Id: <201510140432.t9E4WtJA003454@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rpaulo set sender to rpaulo@FreeBSD.org using -f From: Rui Paulo Date: Wed, 14 Oct 2015 04:32:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289285 - vendor/wpa/2.5 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 04:32:57 -0000 Author: rpaulo Date: Wed Oct 14 04:32:55 2015 New Revision: 289285 URL: https://svnweb.freebsd.org/changeset/base/289285 Log: Tag wpa_supplicant/hostapd 2.5. Added: - copied from r289284, vendor/wpa/dist/ Directory Properties: vendor/wpa/2.5/ (props changed) From owner-svn-src-vendor@freebsd.org Wed Oct 14 07:28:37 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEC54A1231B; Wed, 14 Oct 2015 07:28:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 902FC11A1; Wed, 14 Oct 2015 07:28:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9E7Safq053108; Wed, 14 Oct 2015 07:28:36 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9E7SaTF053107; Wed, 14 Oct 2015 07:28:36 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510140728.t9E7SaTF053107@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 07:28:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289294 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 07:28:37 -0000 Author: mav Date: Wed Oct 14 07:28:36 2015 New Revision: 289294 URL: https://svnweb.freebsd.org/changeset/base/289294 Log: 5219 l2arc_write_buffers() may write beyond target_sz Reviewed by: Matthew Ahrens Reviewed by: Saso Kiselkov Reviewed by: George Wilson Reviewed by: Steven Hartland Reviewed by: Justin Gibbs Approved by: Matthew Ahrens Author: Andriy Gapon illumos/illumos-gate@d7d9a6d919f92d74ea0510a53f8441396048e800 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Oct 14 06:57:28 2015 (r289293) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Oct 14 07:28:36 2015 (r289294) @@ -5902,7 +5902,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_de boolean_t *headroom_boost) { arc_buf_hdr_t *hdr, *hdr_prev, *head; - uint64_t write_asize, write_psize, write_sz, headroom, + uint64_t write_asize, write_sz, headroom, buf_compress_minsz; void *buf_data; boolean_t full; @@ -5917,7 +5917,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_de *headroom_boost = B_FALSE; pio = NULL; - write_sz = write_asize = write_psize = 0; + write_sz = write_asize = 0; full = B_FALSE; head = kmem_cache_alloc(hdr_l2only_cache, KM_PUSHPAGE); head->b_flags |= ARC_FLAG_L2_WRITE_HEAD; @@ -5954,6 +5954,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_de for (; hdr; hdr = hdr_prev) { kmutex_t *hash_lock; uint64_t buf_sz; + uint64_t buf_a_sz; if (arc_warm == B_FALSE) hdr_prev = multilist_sublist_next(mls, hdr); @@ -5982,7 +5983,15 @@ l2arc_write_buffers(spa_t *spa, l2arc_de continue; } - if ((write_sz + hdr->b_size) > target_sz) { + /* + * Assume that the buffer is not going to be compressed + * and could take more space on disk because of a larger + * disk block size. + */ + buf_sz = hdr->b_size; + buf_a_sz = vdev_psize_to_asize(dev->l2ad_vdev, buf_sz); + + if ((write_asize + buf_a_sz) > target_sz) { full = B_TRUE; mutex_exit(hash_lock); break; @@ -6046,7 +6055,6 @@ l2arc_write_buffers(spa_t *spa, l2arc_de */ hdr->b_l2hdr.b_daddr = L2ARC_ADDR_UNSET; - buf_sz = hdr->b_size; hdr->b_flags |= ARC_FLAG_HAS_L2HDR; mutex_enter(&dev->l2ad_mtx); @@ -6063,6 +6071,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_de mutex_exit(hash_lock); write_sz += buf_sz; + write_asize += buf_a_sz; } multilist_sublist_unlock(mls); @@ -6082,6 +6091,19 @@ l2arc_write_buffers(spa_t *spa, l2arc_de mutex_enter(&dev->l2ad_mtx); /* + * Note that elsewhere in this file arcstat_l2_asize + * and the used space on l2ad_vdev are updated using b_asize, + * which is not necessarily rounded up to the device block size. + * Too keep accounting consistent we do the same here as well: + * stats_size accumulates the sum of b_asize of the written buffers, + * while write_asize accumulates the sum of b_asize rounded up + * to the device block size. + * The latter sum is used only to validate the corectness of the code. + */ + uint64_t stats_size = 0; + write_asize = 0; + + /* * Now start writing the buffers. We're starting at the write head * and work backwards, retracing the course of the buffer selector * loop above. @@ -6134,7 +6156,7 @@ l2arc_write_buffers(spa_t *spa, l2arc_de /* Compression may have squashed the buffer to zero length. */ if (buf_sz != 0) { - uint64_t buf_p_sz; + uint64_t buf_a_sz; wzio = zio_write_phys(pio, dev->l2ad_vdev, dev->l2ad_hand, buf_sz, buf_data, ZIO_CHECKSUM_OFF, @@ -6145,14 +6167,14 @@ l2arc_write_buffers(spa_t *spa, l2arc_de zio_t *, wzio); (void) zio_nowait(wzio); - write_asize += buf_sz; + stats_size += buf_sz; /* * Keep the clock hand suitably device-aligned. */ - buf_p_sz = vdev_psize_to_asize(dev->l2ad_vdev, buf_sz); - write_psize += buf_p_sz; - dev->l2ad_hand += buf_p_sz; + buf_a_sz = vdev_psize_to_asize(dev->l2ad_vdev, buf_sz); + write_asize += buf_a_sz; + dev->l2ad_hand += buf_a_sz; } } @@ -6162,8 +6184,8 @@ l2arc_write_buffers(spa_t *spa, l2arc_de ARCSTAT_BUMP(arcstat_l2_writes_sent); ARCSTAT_INCR(arcstat_l2_write_bytes, write_asize); ARCSTAT_INCR(arcstat_l2_size, write_sz); - ARCSTAT_INCR(arcstat_l2_asize, write_asize); - vdev_space_update(dev->l2ad_vdev, write_asize, 0, 0); + ARCSTAT_INCR(arcstat_l2_asize, stats_size); + vdev_space_update(dev->l2ad_vdev, stats_size, 0, 0); /* * Bump device hand to the device start if it is approaching the end. From owner-svn-src-vendor@freebsd.org Wed Oct 14 07:44:21 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B485A12DD8; Wed, 14 Oct 2015 07:44:21 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE2431D20; Wed, 14 Oct 2015 07:44:20 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9E7iJ8L058834; Wed, 14 Oct 2015 07:44:19 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9E7iJfw058833; Wed, 14 Oct 2015 07:44:19 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510140744.t9E7iJfw058833@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 07:44:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289296 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 07:44:21 -0000 Author: mav Date: Wed Oct 14 07:44:19 2015 New Revision: 289296 URL: https://svnweb.freebsd.org/changeset/base/289296 Log: 6288 dmu_buf_will_dirty could be faster Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Justin Gibbs Reviewed by: Richard Elling Approved by: Robert Mustacchi Author: Matthew Ahrens illumos/illumos-gate@0f2e7d03b8f588387cb8dd8dd500cbe5ff4484e0 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed Oct 14 07:37:02 2015 (r289295) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed Oct 14 07:44:19 2015 (r289296) @@ -1115,6 +1115,32 @@ dbuf_release_bp(dmu_buf_impl_t *db) (void) arc_release(db->db_buf, db); } +/* + * We already have a dirty record for this TXG, and we are being + * dirtied again. + */ +static void +dbuf_redirty(dbuf_dirty_record_t *dr) +{ + dmu_buf_impl_t *db = dr->dr_dbuf; + + ASSERT(MUTEX_HELD(&db->db_mtx)); + + if (db->db_level == 0 && db->db_blkid != DMU_BONUS_BLKID) { + /* + * If this buffer has already been written out, + * we now need to reset its state. + */ + dbuf_unoverride(dr); + if (db->db.db_object != DMU_META_DNODE_OBJECT && + db->db_state != DB_NOFILL) { + /* Already released on initial dirty, so just thaw. */ + ASSERT(arc_released(db->db_buf)); + arc_buf_thaw(db->db_buf); + } + } +} + dbuf_dirty_record_t * dbuf_dirty(dmu_buf_impl_t *db, dmu_tx_t *tx) { @@ -1187,16 +1213,7 @@ dbuf_dirty(dmu_buf_impl_t *db, dmu_tx_t if (dr && dr->dr_txg == tx->tx_txg) { DB_DNODE_EXIT(db); - if (db->db_level == 0 && db->db_blkid != DMU_BONUS_BLKID) { - /* - * If this buffer has already been written out, - * we now need to reset its state. - */ - dbuf_unoverride(dr); - if (db->db.db_object != DMU_META_DNODE_OBJECT && - db->db_state != DB_NOFILL) - arc_buf_thaw(db->db_buf); - } + dbuf_redirty(dr); mutex_exit(&db->db_mtx); return (dr); } @@ -1500,6 +1517,30 @@ dmu_buf_will_dirty(dmu_buf_t *db_fake, d ASSERT(tx->tx_txg != 0); ASSERT(!refcount_is_zero(&db->db_holds)); + /* + * Quick check for dirtyness. For already dirty blocks, this + * reduces runtime of this function by >90%, and overall performance + * by 50% for some workloads (e.g. file deletion with indirect blocks + * cached). + */ + mutex_enter(&db->db_mtx); + dbuf_dirty_record_t *dr; + for (dr = db->db_last_dirty; + dr != NULL && dr->dr_txg >= tx->tx_txg; dr = dr->dr_next) { + /* + * It's possible that it is already dirty but not cached, + * because there are some calls to dbuf_dirty() that don't + * go through dmu_buf_will_dirty(). + */ + if (dr->dr_txg == tx->tx_txg && db->db_state == DB_CACHED) { + /* This dbuf is already dirty and cached. */ + dbuf_redirty(dr); + mutex_exit(&db->db_mtx); + return; + } + } + mutex_exit(&db->db_mtx); + DB_DNODE_ENTER(db); if (RW_WRITE_HELD(&DB_DNODE(db)->dn_struct_rwlock)) rf |= DB_RF_HAVESTRUCT; From owner-svn-src-vendor@freebsd.org Wed Oct 14 07:48:41 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39AD0A1322F; Wed, 14 Oct 2015 07:48:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 002812FB; Wed, 14 Oct 2015 07:48:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9E7mdlC059078; Wed, 14 Oct 2015 07:48:39 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9E7mdOU059077; Wed, 14 Oct 2015 07:48:39 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510140748.t9E7mdOU059077@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 07:48:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289298 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 07:48:41 -0000 Author: mav Date: Wed Oct 14 07:48:39 2015 New Revision: 289298 URL: https://svnweb.freebsd.org/changeset/base/289298 Log: 6286 ZFS internal error when set large block on bootfs Reviewed by: Paul Dagnelie Reviewed by: George Wilson Reviewed by: Andriy Gapon Approved by: Robert Mustacchi Author: Matthew Ahrens illumos/illumos-gate@6de9bb5603e65b16816b7ab29e39bac820e2da2b Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Wed Oct 14 07:45:44 2015 (r289297) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Wed Oct 14 07:48:39 2015 (r289298) @@ -24,7 +24,7 @@ * Portions Copyright 2011 Martin Matuska * Copyright 2015 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2014, Joyent, Inc. All rights reserved. - * Copyright (c) 2011, 2014 by Delphix. All rights reserved. + * Copyright (c) 2011, 2015 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. */ @@ -3835,7 +3835,7 @@ zfs_check_settable(const char *dsname, n */ if (zfs_is_bootfs(dsname) && intval > SPA_OLD_MAXBLOCKSIZE) { - return (SET_ERROR(EDOM)); + return (SET_ERROR(ERANGE)); } /* @@ -3844,7 +3844,7 @@ zfs_check_settable(const char *dsname, n */ if (intval > zfs_max_recordsize || intval > SPA_MAXBLOCKSIZE) - return (SET_ERROR(EDOM)); + return (SET_ERROR(ERANGE)); if ((err = spa_open(dsname, &spa, FTAG)) != 0) return (err); From owner-svn-src-vendor@freebsd.org Wed Oct 14 10:21:22 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8078FA1228D; Wed, 14 Oct 2015 10:21:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49AF015F6; Wed, 14 Oct 2015 10:21:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EALLLk005602; Wed, 14 Oct 2015 10:21:21 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EALLCh005601; Wed, 14 Oct 2015 10:21:21 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141021.t9EALLCh005601@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 10:21:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289304 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 10:21:22 -0000 Author: mav Date: Wed Oct 14 10:21:21 2015 New Revision: 289304 URL: https://svnweb.freebsd.org/changeset/base/289304 Log: 6293 ztest failure: error == 28 (0xc == 0x1c) in ztest_tx_assign() Reviewed by: George Wilson Reviewed by: Prakash Surya Reviewed by: Richard Elling Approved by: Richard Lowe Author: Matthew Ahrens illumos/illumos-gate@8fe00bfb8790ad51653f67b01d5ac14256cbb404 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Oct 14 10:01:08 2015 (r289303) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Oct 14 10:21:21 2015 (r289304) @@ -5021,6 +5021,16 @@ arc_init(void) arc_c_max = MAX(allmem * 3 / 4, arc_c_max); /* + * In userland, there's only the memory pressure that we artificially + * create (see arc_available_memory()). Don't let arc_c get too + * small, because it can cause transactions to be larger than + * arc_c, causing arc_tempreserve_space() to fail. + */ +#ifndef _KERNEL + arc_c_min = arc_c_max / 2; +#endif + + /* * Allow the tunables to override our calculations if they are * reasonable (ie. over 64MB) */ From owner-svn-src-vendor@freebsd.org Wed Oct 14 10:30:26 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 705D7A12535; Wed, 14 Oct 2015 10:30:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F7D01E20; Wed, 14 Oct 2015 10:30:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EAUPWH006007; Wed, 14 Oct 2015 10:30:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EAUPPa006006; Wed, 14 Oct 2015 10:30:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141030.t9EAUPPa006006@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 10:30:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289306 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 10:30:26 -0000 Author: mav Date: Wed Oct 14 10:30:24 2015 New Revision: 289306 URL: https://svnweb.freebsd.org/changeset/base/289306 Log: 6295 metaslab_condense's dbgmsg should include vdev id Reviewed by: George Wilson Reviewed by: Matthew Ahrens Reviewed by: Andriy Gapon Reviewed by: Xin Li Reviewed by: Justin Gibbs Approved by: Richard Lowe Author: Joe Stein illumos/illumos-gate@daec38ecb4fb5e73e4ca9e99be84f6b8c50c02fa Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Wed Oct 14 10:28:29 2015 (r289305) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Wed Oct 14 10:30:24 2015 (r289306) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2014 by Delphix. All rights reserved. + * Copyright (c) 2011, 2015 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. */ @@ -1705,10 +1705,11 @@ metaslab_condense(metaslab_t *msp, uint6 ASSERT(msp->ms_loaded); - spa_dbgmsg(spa, "condensing: txg %llu, msp[%llu] %p, " - "smp size %llu, segments %lu, forcing condense=%s", txg, - msp->ms_id, msp, space_map_length(msp->ms_sm), - avl_numnodes(&msp->ms_tree->rt_root), + spa_dbgmsg(spa, "condensing: txg %llu, msp[%llu] %p, vdev id %llu, " + "spa %s, smp size %llu, segments %lu, forcing condense=%s", txg, + msp->ms_id, msp, msp->ms_group->mg_vd->vdev_id, + msp->ms_group->mg_vd->vdev_spa->spa_name, + space_map_length(msp->ms_sm), avl_numnodes(&msp->ms_tree->rt_root), msp->ms_condense_wanted ? "TRUE" : "FALSE"); msp->ms_condense_wanted = B_FALSE; From owner-svn-src-vendor@freebsd.org Wed Oct 14 10:36:45 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55128A127F6; Wed, 14 Oct 2015 10:36:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 15F66134D; Wed, 14 Oct 2015 10:36:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EAaiJU008848; Wed, 14 Oct 2015 10:36:44 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EAahSw008843; Wed, 14 Oct 2015 10:36:43 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141036.t9EAahSw008843@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 10:36:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289308 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 10:36:45 -0000 Author: mav Date: Wed Oct 14 10:36:43 2015 New Revision: 289308 URL: https://svnweb.freebsd.org/changeset/base/289308 Log: 6267 dn_bonus evicted too early Reviewed by: Richard Yao Reviewed by: Xin LI Reviewed by: Matthew Ahrens Approved by: Richard Lowe Author: Justin T. Gibbs illumos/illumos-gate@d2058105c61ec61df3a2dd3f839fed8c3fe7bfd6 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed Oct 14 10:31:50 2015 (r289307) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed Oct 14 10:36:43 2015 (r289308) @@ -272,7 +272,7 @@ dbuf_verify_user(dmu_buf_impl_t *db, dbv */ ASSERT3U(holds, >=, db->db_dirtycnt); } else { - if (db->db_immediate_evict == TRUE) + if (db->db_user_immediate_evict == TRUE) ASSERT3U(holds, >=, db->db_dirtycnt); else ASSERT3U(holds, >, 0); @@ -1875,8 +1875,9 @@ dbuf_create(dnode_t *dn, uint8_t level, db->db_blkptr = blkptr; db->db_user = NULL; - db->db_immediate_evict = 0; - db->db_freed_in_flight = 0; + db->db_user_immediate_evict = FALSE; + db->db_freed_in_flight = FALSE; + db->db_pending_evict = FALSE; if (blkid == DMU_BONUS_BLKID) { ASSERT3P(parent, ==, dn->dn_dbuf); @@ -2432,12 +2433,13 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, arc_buf_freeze(db->db_buf); if (holds == db->db_dirtycnt && - db->db_level == 0 && db->db_immediate_evict) + db->db_level == 0 && db->db_user_immediate_evict) dbuf_evict_user(db); if (holds == 0) { if (db->db_blkid == DMU_BONUS_BLKID) { dnode_t *dn; + boolean_t evict_dbuf = db->db_pending_evict; /* * If the dnode moves here, we cannot cross this @@ -2452,7 +2454,7 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, * Decrementing the dbuf count means that the bonus * buffer's dnode hold is no longer discounted in * dnode_move(). The dnode cannot move until after - * the dnode_rele_and_unlock() below. + * the dnode_rele() below. */ DB_DNODE_EXIT(db); @@ -2462,35 +2464,10 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, */ mutex_exit(&db->db_mtx); - /* - * If the dnode has been freed, evict the bonus - * buffer immediately. The data in the bonus - * buffer is no longer relevant and this prevents - * a stale bonus buffer from being associated - * with this dnode_t should the dnode_t be reused - * prior to being destroyed. - */ - mutex_enter(&dn->dn_mtx); - if (dn->dn_type == DMU_OT_NONE || - dn->dn_free_txg != 0) { - /* - * Drop dn_mtx. It is a leaf lock and - * cannot be held when dnode_evict_bonus() - * acquires other locks in order to - * perform the eviction. - * - * Freed dnodes cannot be reused until the - * last hold is released. Since this bonus - * buffer has a hold, the dnode will remain - * in the free state, even without dn_mtx - * held, until the dnode_rele_and_unlock() - * below. - */ - mutex_exit(&dn->dn_mtx); + if (evict_dbuf) dnode_evict_bonus(dn); - mutex_enter(&dn->dn_mtx); - } - dnode_rele_and_unlock(dn, db); + + dnode_rele(dn, db); } else if (db->db_buf == NULL) { /* * This is a special case: we never associated this @@ -2537,7 +2514,7 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, } else { dbuf_clear(db); } - } else if (db->db_objset->os_evicting || + } else if (db->db_pending_evict || arc_buf_eviction_needed(db->db_buf)) { dbuf_clear(db); } else { @@ -2585,7 +2562,7 @@ dmu_buf_set_user_ie(dmu_buf_t *db_fake, { dmu_buf_impl_t *db = (dmu_buf_impl_t *)db_fake; - db->db_immediate_evict = TRUE; + db->db_user_immediate_evict = TRUE; return (dmu_buf_set_user(db_fake, user)); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Wed Oct 14 10:31:50 2015 (r289307) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Wed Oct 14 10:36:43 2015 (r289308) @@ -686,7 +686,6 @@ dmu_objset_evict(objset_t *os) if (os->os_sa) sa_tear_down(os); - os->os_evicting = B_TRUE; dmu_objset_evict_dbufs(os); mutex_enter(&os->os_lock); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Wed Oct 14 10:31:50 2015 (r289307) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Wed Oct 14 10:36:43 2015 (r289308) @@ -424,6 +424,7 @@ dnode_evict_dbufs(dnode_t *dn) db_next = AVL_NEXT(&dn->dn_dbufs, &db_marker); avl_remove(&dn->dn_dbufs, &db_marker); } else { + db->db_pending_evict = TRUE; mutex_exit(&db->db_mtx); db_next = AVL_NEXT(&dn->dn_dbufs, db); } @@ -437,10 +438,14 @@ void dnode_evict_bonus(dnode_t *dn) { rw_enter(&dn->dn_struct_rwlock, RW_WRITER); - if (dn->dn_bonus && refcount_is_zero(&dn->dn_bonus->db_holds)) { - mutex_enter(&dn->dn_bonus->db_mtx); - dbuf_evict(dn->dn_bonus); - dn->dn_bonus = NULL; + if (dn->dn_bonus != NULL) { + if (refcount_is_zero(&dn->dn_bonus->db_holds)) { + mutex_enter(&dn->dn_bonus->db_mtx); + dbuf_evict(dn->dn_bonus); + dn->dn_bonus = NULL; + } else { + dn->dn_bonus->db_pending_evict = TRUE; + } } rw_exit(&dn->dn_struct_rwlock); } @@ -492,7 +497,6 @@ dnode_sync_free(dnode_t *dn, dmu_tx_t *t dnode_undirty_dbufs(&dn->dn_dirty_records[txgoff]); dnode_evict_dbufs(dn); - ASSERT(avl_is_empty(&dn->dn_dbufs)); /* * XXX - It would be nice to assert this, but we may still Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Wed Oct 14 10:31:50 2015 (r289307) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Wed Oct 14 10:36:43 2015 (r289308) @@ -230,9 +230,25 @@ typedef struct dmu_buf_impl { /* User callback information. */ dmu_buf_user_t *db_user; - uint8_t db_immediate_evict; + /* + * Evict user data as soon as the dirty and reference + * counts are equal. + */ + uint8_t db_user_immediate_evict; + + /* + * This block was freed while a read or write was + * active. + */ uint8_t db_freed_in_flight; + /* + * dnode_evict_dbufs() or dnode_evict_bonus() tried to + * evict this dbuf, but couldn't due to outstanding + * references. Evict once the refcount drops to 0. + */ + uint8_t db_pending_evict; + uint8_t db_dirtycnt; } dmu_buf_impl_t; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Wed Oct 14 10:31:50 2015 (r289307) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Wed Oct 14 10:36:43 2015 (r289308) @@ -93,7 +93,6 @@ struct objset { uint8_t os_copies; enum zio_checksum os_dedup_checksum; boolean_t os_dedup_verify; - boolean_t os_evicting; zfs_logbias_op_t os_logbias; zfs_cache_type_t os_primary_cache; zfs_cache_type_t os_secondary_cache; From owner-svn-src-vendor@freebsd.org Wed Oct 14 11:12:50 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06C4FA1271F; Wed, 14 Oct 2015 11:12:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F002F82; Wed, 14 Oct 2015 11:12:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EBCmaT022225; Wed, 14 Oct 2015 11:12:48 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EBClT3022215; Wed, 14 Oct 2015 11:12:47 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141112.t9EBClT3022215@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 11:12:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 11:12:50 -0000 Author: mav Date: Wed Oct 14 11:12:47 2015 New Revision: 289310 URL: https://svnweb.freebsd.org/changeset/base/289310 Log: 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R Reviewed by: George Wilson Reviewed by: Prakash Surya Reviewed by: Saso Kiselkov Reviewed by: Richard Lowe Approved by: Garrett D'Amore Author: Matthew Ahrens illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f Added: vendor-sys/illumos/dist/uts/common/crypto/ vendor-sys/illumos/dist/uts/common/crypto/io/ vendor-sys/illumos/dist/uts/common/crypto/io/edonr_mod.c (contents, props changed) vendor-sys/illumos/dist/uts/common/crypto/io/skein_mod.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/edonr_zfs.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/skein_zfs.c (contents, props changed) vendor-sys/illumos/dist/uts/common/sys/edonr.h (contents, props changed) vendor-sys/illumos/dist/uts/common/sys/skein.h (contents, props changed) Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/common/zfs/zfs_fletcher.c vendor-sys/illumos/dist/common/zfs/zfs_fletcher.h vendor-sys/illumos/dist/common/zfs/zfs_prop.c vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/ddt.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sha256.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_checksum.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio_checksum.c vendor-sys/illumos/dist/uts/common/sys/debug.h Changes in other areas also in this revision: Added: vendor/illumos/dist/common/crypto/ vendor/illumos/dist/common/crypto/edonr/ vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) vendor/illumos/dist/common/crypto/skein/ vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man5/zpool-features.5 Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.c Wed Oct 14 10:38:05 2015 (r289309) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.c Wed Oct 14 11:12:47 2015 (r289310) @@ -231,4 +231,17 @@ zpool_feature_init(void) "org.open-zfs:large_blocks", "large_blocks", "Support for blocks larger than 128KB.", ZFEATURE_FLAG_PER_DATASET, large_blocks_deps); + + zfeature_register(SPA_FEATURE_SHA512, + "org.illumos:sha512", "sha512", + "SHA-512/256 hash algorithm.", + ZFEATURE_FLAG_PER_DATASET, NULL); + zfeature_register(SPA_FEATURE_SKEIN, + "org.illumos:skein", "skein", + "Skein hash algorithm.", + ZFEATURE_FLAG_PER_DATASET, NULL); + zfeature_register(SPA_FEATURE_EDONR, + "org.illumos:edonr", "edonr", + "Edon-R hash algorithm.", + ZFEATURE_FLAG_PER_DATASET, NULL); } Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.h ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.h Wed Oct 14 10:38:05 2015 (r289309) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.h Wed Oct 14 11:12:47 2015 (r289310) @@ -52,6 +52,9 @@ typedef enum spa_feature { SPA_FEATURE_BOOKMARKS, SPA_FEATURE_FS_SS_LIMIT, SPA_FEATURE_LARGE_BLOCKS, + SPA_FEATURE_SHA512, + SPA_FEATURE_SKEIN, + SPA_FEATURE_EDONR, SPA_FEATURES } spa_feature_t; Modified: vendor-sys/illumos/dist/common/zfs/zfs_fletcher.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_fletcher.c Wed Oct 14 10:38:05 2015 (r289309) +++ vendor-sys/illumos/dist/common/zfs/zfs_fletcher.c Wed Oct 14 11:12:47 2015 (r289310) @@ -22,6 +22,9 @@ * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. */ +/* + * Copyright 2013 Saso Kiselkov. All rights reserved. + */ /* * Fletcher Checksums @@ -131,8 +134,10 @@ #include #include +/*ARGSUSED*/ void -fletcher_2_native(const void *buf, uint64_t size, zio_cksum_t *zcp) +fletcher_2_native(const void *buf, uint64_t size, + const void *ctx_template, zio_cksum_t *zcp) { const uint64_t *ip = buf; const uint64_t *ipend = ip + (size / sizeof (uint64_t)); @@ -148,8 +153,10 @@ fletcher_2_native(const void *buf, uint6 ZIO_SET_CHECKSUM(zcp, a0, a1, b0, b1); } +/*ARGSUSED*/ void -fletcher_2_byteswap(const void *buf, uint64_t size, zio_cksum_t *zcp) +fletcher_2_byteswap(const void *buf, uint64_t size, + const void *ctx_template, zio_cksum_t *zcp) { const uint64_t *ip = buf; const uint64_t *ipend = ip + (size / sizeof (uint64_t)); @@ -165,8 +172,10 @@ fletcher_2_byteswap(const void *buf, uin ZIO_SET_CHECKSUM(zcp, a0, a1, b0, b1); } +/*ARGSUSED*/ void -fletcher_4_native(const void *buf, uint64_t size, zio_cksum_t *zcp) +fletcher_4_native(const void *buf, uint64_t size, + const void *ctx_template, zio_cksum_t *zcp) { const uint32_t *ip = buf; const uint32_t *ipend = ip + (size / sizeof (uint32_t)); @@ -182,8 +191,10 @@ fletcher_4_native(const void *buf, uint6 ZIO_SET_CHECKSUM(zcp, a, b, c, d); } +/*ARGSUSED*/ void -fletcher_4_byteswap(const void *buf, uint64_t size, zio_cksum_t *zcp) +fletcher_4_byteswap(const void *buf, uint64_t size, + const void *ctx_template, zio_cksum_t *zcp) { const uint32_t *ip = buf; const uint32_t *ipend = ip + (size / sizeof (uint32_t)); Modified: vendor-sys/illumos/dist/common/zfs/zfs_fletcher.h ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_fletcher.h Wed Oct 14 10:38:05 2015 (r289309) +++ vendor-sys/illumos/dist/common/zfs/zfs_fletcher.h Wed Oct 14 11:12:47 2015 (r289310) @@ -22,6 +22,9 @@ * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. */ +/* + * Copyright 2013 Saso Kiselkov. All rights reserved. + */ #ifndef _ZFS_FLETCHER_H #define _ZFS_FLETCHER_H @@ -37,14 +40,12 @@ extern "C" { * fletcher checksum functions */ -void fletcher_2_native(const void *, uint64_t, zio_cksum_t *); -void fletcher_2_byteswap(const void *, uint64_t, zio_cksum_t *); -void fletcher_4_native(const void *, uint64_t, zio_cksum_t *); -void fletcher_4_byteswap(const void *, uint64_t, zio_cksum_t *); -void fletcher_4_incremental_native(const void *, uint64_t, - zio_cksum_t *); -void fletcher_4_incremental_byteswap(const void *, uint64_t, - zio_cksum_t *); +void fletcher_2_native(const void *, uint64_t, const void *, zio_cksum_t *); +void fletcher_2_byteswap(const void *, uint64_t, const void *, zio_cksum_t *); +void fletcher_4_native(const void *, uint64_t, const void *, zio_cksum_t *); +void fletcher_4_byteswap(const void *, uint64_t, const void *, zio_cksum_t *); +void fletcher_4_incremental_native(const void *, uint64_t, zio_cksum_t *); +void fletcher_4_incremental_byteswap(const void *, uint64_t, zio_cksum_t *); #ifdef __cplusplus } Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_prop.c Wed Oct 14 10:38:05 2015 (r289309) +++ vendor-sys/illumos/dist/common/zfs/zfs_prop.c Wed Oct 14 11:12:47 2015 (r289310) @@ -71,6 +71,9 @@ zfs_prop_init(void) { "fletcher4", ZIO_CHECKSUM_FLETCHER_4 }, { "sha256", ZIO_CHECKSUM_SHA256 }, { "noparity", ZIO_CHECKSUM_NOPARITY }, + { "sha512", ZIO_CHECKSUM_SHA512 }, + { "skein", ZIO_CHECKSUM_SKEIN }, + { "edonr", ZIO_CHECKSUM_EDONR }, { NULL } }; @@ -81,6 +84,14 @@ zfs_prop_init(void) { "sha256", ZIO_CHECKSUM_SHA256 }, { "sha256,verify", ZIO_CHECKSUM_SHA256 | ZIO_CHECKSUM_VERIFY }, + { "sha512", ZIO_CHECKSUM_SHA512 }, + { "sha512,verify", + ZIO_CHECKSUM_SHA512 | ZIO_CHECKSUM_VERIFY }, + { "skein", ZIO_CHECKSUM_SKEIN }, + { "skein,verify", + ZIO_CHECKSUM_SKEIN | ZIO_CHECKSUM_VERIFY }, + { "edonr,verify", + ZIO_CHECKSUM_EDONR | ZIO_CHECKSUM_VERIFY }, { NULL } }; @@ -217,12 +228,12 @@ zfs_prop_init(void) zprop_register_index(ZFS_PROP_CHECKSUM, "checksum", ZIO_CHECKSUM_DEFAULT, PROP_INHERIT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - "on | off | fletcher2 | fletcher4 | sha256", "CHECKSUM", - checksum_table); + "on | off | fletcher2 | fletcher4 | sha256 | sha512 | " + "skein | edonr", "CHECKSUM", checksum_table); zprop_register_index(ZFS_PROP_DEDUP, "dedup", ZIO_CHECKSUM_OFF, PROP_INHERIT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, - "on | off | verify | sha256[,verify]", "DEDUP", - dedup_table); + "on | off | verify | sha256[,verify], sha512[,verify], " + "skein[,verify], edonr,verify", "DEDUP", dedup_table); zprop_register_index(ZFS_PROP_COMPRESSION, "compression", ZIO_COMPRESS_DEFAULT, PROP_INHERIT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Oct 14 10:38:05 2015 (r289309) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Oct 14 11:12:47 2015 (r289310) @@ -509,6 +509,10 @@ SHA1_OBJS += sha1.o sha1_mod.o SHA2_OBJS += sha2.o sha2_mod.o +SKEIN_OBJS += skein.o skein_block.o skein_iv.o skein_mod.o + +EDONR_OBJS += edonr.o edonr_mod.o + IPGPC_OBJS += classifierddi.o classifier.o filters.o trie.o table.o \ ba_table.o @@ -1370,6 +1374,8 @@ ZFS_COMMON_OBJS += \ rrwlock.o \ sa.o \ sha256.o \ + edonr_zfs.o \ + skein_zfs.o \ spa.o \ spa_config.o \ spa_errlog.o \ Added: vendor-sys/illumos/dist/uts/common/crypto/io/edonr_mod.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/crypto/io/edonr_mod.c Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1,63 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2013 Saso Kiselkov. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include + +/* + * Unlike sha2 or skein, we won't expose edonr via the Kernel Cryptographic + * Framework (KCF), because Edon-R is *NOT* suitable for general-purpose + * cryptographic use. Users of Edon-R must interface directly to this module. + */ + +static struct modlmisc modlmisc = { + &mod_miscops, + "Edon-R Message-Digest Algorithm" +}; + +static struct modlinkage modlinkage = { + MODREV_1, &modlmisc, NULL +}; + +int +_init(void) +{ + int error; + + if ((error = mod_install(&modlinkage)) != 0) + return (error); + + return (0); +} + +int +_info(struct modinfo *modinfop) +{ + return (mod_info(&modlinkage, modinfop)); +} Added: vendor-sys/illumos/dist/uts/common/crypto/io/skein_mod.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/crypto/io/skein_mod.c Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1,830 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2013 Saso Kiselkov. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#define SKEIN_MODULE_IMPL +#include + +/* + * Like the sha2 module, we create the skein module with two modlinkages: + * - modlmisc to allow direct calls to Skein_* API functions. + * - modlcrypto to integrate well into the Kernel Crypto Framework (KCF). + */ +static struct modlmisc modlmisc = { + &mod_miscops, + "Skein Message-Digest Algorithm" +}; + +static struct modlcrypto modlcrypto = { + &mod_cryptoops, + "Skein Kernel SW Provider" +}; + +static struct modlinkage modlinkage = { + MODREV_1, &modlmisc, &modlcrypto, NULL +}; + +static crypto_mech_info_t skein_mech_info_tab[] = { + {CKM_SKEIN_256, SKEIN_256_MECH_INFO_TYPE, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, + 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + {CKM_SKEIN_256_MAC, SKEIN_256_MAC_MECH_INFO_TYPE, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, 1, INT_MAX, + CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + {CKM_SKEIN_512, SKEIN_512_MECH_INFO_TYPE, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, + 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + {CKM_SKEIN_512_MAC, SKEIN_512_MAC_MECH_INFO_TYPE, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, 1, INT_MAX, + CRYPTO_KEYSIZE_UNIT_IN_BYTES}, + {CKM_SKEIN1024, SKEIN1024_MECH_INFO_TYPE, + CRYPTO_FG_DIGEST | CRYPTO_FG_DIGEST_ATOMIC, + 0, 0, CRYPTO_KEYSIZE_UNIT_IN_BITS}, + {CKM_SKEIN1024_MAC, SKEIN1024_MAC_MECH_INFO_TYPE, + CRYPTO_FG_MAC | CRYPTO_FG_MAC_ATOMIC, 1, INT_MAX, + CRYPTO_KEYSIZE_UNIT_IN_BYTES} +}; + +static void skein_provider_status(crypto_provider_handle_t, uint_t *); + +static crypto_control_ops_t skein_control_ops = { + skein_provider_status +}; + +static int skein_digest_init(crypto_ctx_t *, crypto_mechanism_t *, + crypto_req_handle_t); +static int skein_digest(crypto_ctx_t *, crypto_data_t *, crypto_data_t *, + crypto_req_handle_t); +static int skein_update(crypto_ctx_t *, crypto_data_t *, crypto_req_handle_t); +static int skein_final(crypto_ctx_t *, crypto_data_t *, crypto_req_handle_t); +static int skein_digest_atomic(crypto_provider_handle_t, crypto_session_id_t, + crypto_mechanism_t *, crypto_data_t *, crypto_data_t *, + crypto_req_handle_t); + +static crypto_digest_ops_t skein_digest_ops = { + skein_digest_init, + skein_digest, + skein_update, + NULL, + skein_final, + skein_digest_atomic +}; + +static int skein_mac_init(crypto_ctx_t *, crypto_mechanism_t *, crypto_key_t *, + crypto_spi_ctx_template_t, crypto_req_handle_t); +static int skein_mac_atomic(crypto_provider_handle_t, crypto_session_id_t, + crypto_mechanism_t *, crypto_key_t *, crypto_data_t *, crypto_data_t *, + crypto_spi_ctx_template_t, crypto_req_handle_t); + +static crypto_mac_ops_t skein_mac_ops = { + skein_mac_init, + NULL, + skein_update, /* using regular digest update is OK here */ + skein_final, /* using regular digest final is OK here */ + skein_mac_atomic, + NULL +}; + +static int skein_create_ctx_template(crypto_provider_handle_t, + crypto_mechanism_t *, crypto_key_t *, crypto_spi_ctx_template_t *, + size_t *, crypto_req_handle_t); +static int skein_free_context(crypto_ctx_t *); + +static crypto_ctx_ops_t skein_ctx_ops = { + skein_create_ctx_template, + skein_free_context +}; + +static crypto_ops_t skein_crypto_ops = { + &skein_control_ops, + &skein_digest_ops, + NULL, + &skein_mac_ops, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + NULL, + &skein_ctx_ops, + NULL, + NULL, + NULL +}; + +static crypto_provider_info_t skein_prov_info = { + CRYPTO_SPI_VERSION_4, + "Skein Software Provider", + CRYPTO_SW_PROVIDER, + {&modlinkage}, + NULL, + &skein_crypto_ops, + sizeof (skein_mech_info_tab) / sizeof (crypto_mech_info_t), + skein_mech_info_tab +}; + +static crypto_kcf_provider_handle_t skein_prov_handle = NULL; + +typedef struct skein_ctx { + skein_mech_type_t sc_mech_type; + size_t sc_digest_bitlen; + /*LINTED(E_ANONYMOUS_UNION_DECL)*/ + union { + Skein_256_Ctxt_t sc_256; + Skein_512_Ctxt_t sc_512; + Skein1024_Ctxt_t sc_1024; + }; +} skein_ctx_t; +#define SKEIN_CTX(_ctx_) ((skein_ctx_t *)((_ctx_)->cc_provider_private)) +#define SKEIN_CTX_LVALUE(_ctx_) (_ctx_)->cc_provider_private +#define SKEIN_OP(_skein_ctx, _op, ...) \ + do { \ + skein_ctx_t *sc = (_skein_ctx); \ + switch (sc->sc_mech_type) { \ + case SKEIN_256_MECH_INFO_TYPE: \ + case SKEIN_256_MAC_MECH_INFO_TYPE: \ + (void) Skein_256_ ## _op(&sc->sc_256, __VA_ARGS__);\ + break; \ + case SKEIN_512_MECH_INFO_TYPE: \ + case SKEIN_512_MAC_MECH_INFO_TYPE: \ + (void) Skein_512_ ## _op(&sc->sc_512, __VA_ARGS__);\ + break; \ + case SKEIN1024_MECH_INFO_TYPE: \ + case SKEIN1024_MAC_MECH_INFO_TYPE: \ + (void) Skein1024_ ## _op(&sc->sc_1024, __VA_ARGS__);\ + break; \ + } \ + _NOTE(CONSTCOND) \ + } while (0) + +static int +skein_get_digest_bitlen(const crypto_mechanism_t *mechanism, size_t *result) +{ + if (mechanism->cm_param != NULL) { + /*LINTED(E_BAD_PTR_CAST_ALIGN)*/ + skein_param_t *param = (skein_param_t *)mechanism->cm_param; + + if (mechanism->cm_param_len != sizeof (*param) || + param->sp_digest_bitlen == 0) { + return (CRYPTO_MECHANISM_PARAM_INVALID); + } + *result = param->sp_digest_bitlen; + } else { + switch (mechanism->cm_type) { + case SKEIN_256_MECH_INFO_TYPE: + *result = 256; + break; + case SKEIN_512_MECH_INFO_TYPE: + *result = 512; + break; + case SKEIN1024_MECH_INFO_TYPE: + *result = 1024; + break; + default: + return (CRYPTO_MECHANISM_INVALID); + } + } + return (CRYPTO_SUCCESS); +} + +int +_init(void) +{ + int error; + + if ((error = mod_install(&modlinkage)) != 0) + return (error); + + /* + * Try to register with KCF - failure shouldn't unload us, since we + * still may want to continue providing misc/skein functionality. + */ + (void) crypto_register_provider(&skein_prov_info, &skein_prov_handle); + + return (0); +} + +int +_info(struct modinfo *modinfop) +{ + return (mod_info(&modlinkage, modinfop)); +} + +/* + * KCF software provider control entry points. + */ +/* ARGSUSED */ +static void +skein_provider_status(crypto_provider_handle_t provider, uint_t *status) +{ + *status = CRYPTO_PROVIDER_READY; +} + +/* + * General Skein hashing helper functions. + */ + +/* + * Performs an Update on a context with uio input data. + */ +static int +skein_digest_update_uio(skein_ctx_t *ctx, const crypto_data_t *data) +{ + off_t offset = data->cd_offset; + size_t length = data->cd_length; + uint_t vec_idx; + size_t cur_len; + const uio_t *uio = data->cd_uio; + + /* we support only kernel buffer */ + if (uio->uio_segflg != UIO_SYSSPACE) + return (CRYPTO_ARGUMENTS_BAD); + + /* + * Jump to the first iovec containing data to be + * digested. + */ + for (vec_idx = 0; vec_idx < uio->uio_iovcnt && + offset >= uio->uio_iov[vec_idx].iov_len; + offset -= uio->uio_iov[vec_idx++].iov_len) + ; + if (vec_idx == uio->uio_iovcnt) { + /* + * The caller specified an offset that is larger than the + * total size of the buffers it provided. + */ + return (CRYPTO_DATA_LEN_RANGE); + } + + /* + * Now do the digesting on the iovecs. + */ + while (vec_idx < uio->uio_iovcnt && length > 0) { + cur_len = MIN(uio->uio_iov[vec_idx].iov_len - offset, length); + SKEIN_OP(ctx, Update, (uint8_t *)uio->uio_iov[vec_idx].iov_base + + offset, cur_len); + length -= cur_len; + vec_idx++; + offset = 0; + } + + if (vec_idx == uio->uio_iovcnt && length > 0) { + /* + * The end of the specified iovec's was reached but + * the length requested could not be processed, i.e. + * The caller requested to digest more data than it provided. + */ + return (CRYPTO_DATA_LEN_RANGE); + } + + return (CRYPTO_SUCCESS); +} + +/* + * Performs a Final on a context and writes to a uio digest output. + */ +static int +skein_digest_final_uio(skein_ctx_t *ctx, crypto_data_t *digest, + crypto_req_handle_t req) +{ + off_t offset = digest->cd_offset; + uint_t vec_idx; + uio_t *uio = digest->cd_uio; + + /* we support only kernel buffer */ + if (uio->uio_segflg != UIO_SYSSPACE) + return (CRYPTO_ARGUMENTS_BAD); + + /* + * Jump to the first iovec containing ptr to the digest to be returned. + */ + for (vec_idx = 0; offset >= uio->uio_iov[vec_idx].iov_len && + vec_idx < uio->uio_iovcnt; + offset -= uio->uio_iov[vec_idx++].iov_len) + ; + if (vec_idx == uio->uio_iovcnt) { + /* + * The caller specified an offset that is larger than the + * total size of the buffers it provided. + */ + return (CRYPTO_DATA_LEN_RANGE); + } + if (offset + CRYPTO_BITS2BYTES(ctx->sc_digest_bitlen) <= + uio->uio_iov[vec_idx].iov_len) { + /* The computed digest will fit in the current iovec. */ + SKEIN_OP(ctx, Final, + (uchar_t *)uio->uio_iov[vec_idx].iov_base + offset); + } else { + uint8_t *digest_tmp; + off_t scratch_offset = 0; + size_t length = CRYPTO_BITS2BYTES(ctx->sc_digest_bitlen); + size_t cur_len; + + digest_tmp = kmem_alloc(CRYPTO_BITS2BYTES( + ctx->sc_digest_bitlen), crypto_kmflag(req)); + if (digest_tmp == NULL) + return (CRYPTO_HOST_MEMORY); + SKEIN_OP(ctx, Final, digest_tmp); + while (vec_idx < uio->uio_iovcnt && length > 0) { + cur_len = MIN(uio->uio_iov[vec_idx].iov_len - offset, + length); + bcopy(digest_tmp + scratch_offset, + uio->uio_iov[vec_idx].iov_base + offset, cur_len); + + length -= cur_len; + vec_idx++; + scratch_offset += cur_len; + offset = 0; + } + kmem_free(digest_tmp, CRYPTO_BITS2BYTES(ctx->sc_digest_bitlen)); + + if (vec_idx == uio->uio_iovcnt && length > 0) { + /* + * The end of the specified iovec's was reached but + * the length requested could not be processed, i.e. + * The caller requested to digest more data than it + * provided. + */ + return (CRYPTO_DATA_LEN_RANGE); + } + } + + return (CRYPTO_SUCCESS); +} + +/* + * Performs an Update on a context with mblk input data. + */ +static int +skein_digest_update_mblk(skein_ctx_t *ctx, crypto_data_t *data) +{ + off_t offset = data->cd_offset; + size_t length = data->cd_length; + mblk_t *mp; + size_t cur_len; + + /* Jump to the first mblk_t containing data to be digested. */ + for (mp = data->cd_mp; mp != NULL && offset >= MBLKL(mp); + offset -= MBLKL(mp), mp = mp->b_cont) + ; + if (mp == NULL) { + /* + * The caller specified an offset that is larger than the + * total size of the buffers it provided. + */ + return (CRYPTO_DATA_LEN_RANGE); + } + + /* Now do the digesting on the mblk chain. */ + while (mp != NULL && length > 0) { + cur_len = MIN(MBLKL(mp) - offset, length); + SKEIN_OP(ctx, Update, mp->b_rptr + offset, cur_len); + length -= cur_len; + offset = 0; + mp = mp->b_cont; + } + + if (mp == NULL && length > 0) { + /* + * The end of the mblk was reached but the length requested + * could not be processed, i.e. The caller requested + * to digest more data than it provided. + */ + return (CRYPTO_DATA_LEN_RANGE); + } + + return (CRYPTO_SUCCESS); +} + +/* + * Performs a Final on a context and writes to an mblk digest output. + */ +static int +skein_digest_final_mblk(skein_ctx_t *ctx, crypto_data_t *digest, + crypto_req_handle_t req) +{ + off_t offset = digest->cd_offset; + mblk_t *mp; + + /* Jump to the first mblk_t that will be used to store the digest. */ + for (mp = digest->cd_mp; mp != NULL && offset >= MBLKL(mp); + offset -= MBLKL(mp), mp = mp->b_cont) + ; + if (mp == NULL) { + /* caller specified offset is too large */ + return (CRYPTO_DATA_LEN_RANGE); + } + + if (offset + CRYPTO_BITS2BYTES(ctx->sc_digest_bitlen) <= MBLKL(mp)) { + /* The digest will fit in the current mblk. */ + SKEIN_OP(ctx, Final, mp->b_rptr + offset); + } else { + /* Split the digest up between the individual buffers. */ + uint8_t *digest_tmp; + off_t scratch_offset = 0; + size_t length = CRYPTO_BITS2BYTES(ctx->sc_digest_bitlen); + size_t cur_len; + + digest_tmp = kmem_alloc(CRYPTO_BITS2BYTES( + ctx->sc_digest_bitlen), crypto_kmflag(req)); + if (digest_tmp == NULL) + return (CRYPTO_HOST_MEMORY); + SKEIN_OP(ctx, Final, digest_tmp); + while (mp != NULL && length > 0) { + cur_len = MIN(MBLKL(mp) - offset, length); + bcopy(digest_tmp + scratch_offset, + mp->b_rptr + offset, cur_len); + length -= cur_len; + mp = mp->b_cont; + scratch_offset += cur_len; + offset = 0; + } + kmem_free(digest_tmp, CRYPTO_BITS2BYTES(ctx->sc_digest_bitlen)); + if (mp == NULL && length > 0) { + /* digest too long to fit in the mblk buffers */ + return (CRYPTO_DATA_LEN_RANGE); + } + } + + return (CRYPTO_SUCCESS); +} + +/* + * KCF software provider digest entry points. + */ + +/* + * Initializes a skein digest context to the configuration in `mechanism'. + * The mechanism cm_type must be one of SKEIN_*_MECH_INFO_TYPE. The cm_param + * field may contain a skein_param_t structure indicating the length of the + * digest the algorithm should produce. Otherwise the default output lengths + * are applied (32 bytes for Skein-256, 64 bytes for Skein-512 and 128 bytes + * for Skein-1024). + */ +static int +skein_digest_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, + crypto_req_handle_t req) +{ + int error = CRYPTO_SUCCESS; + + if (!VALID_SKEIN_DIGEST_MECH(mechanism->cm_type)) + return (CRYPTO_MECHANISM_INVALID); + + SKEIN_CTX_LVALUE(ctx) = kmem_alloc(sizeof (*SKEIN_CTX(ctx)), + crypto_kmflag(req)); + if (SKEIN_CTX(ctx) == NULL) + return (CRYPTO_HOST_MEMORY); + + SKEIN_CTX(ctx)->sc_mech_type = mechanism->cm_type; + error = skein_get_digest_bitlen(mechanism, + &SKEIN_CTX(ctx)->sc_digest_bitlen); + if (error != CRYPTO_SUCCESS) + goto errout; + SKEIN_OP(SKEIN_CTX(ctx), Init, SKEIN_CTX(ctx)->sc_digest_bitlen); + + return (CRYPTO_SUCCESS); +errout: + bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + SKEIN_CTX_LVALUE(ctx) = NULL; + return (error); +} + +/* + * Executes a skein_update and skein_digest on a pre-initialized crypto + * context in a single step. See the documentation to these functions to + * see what to pass here. + */ +static int +skein_digest(crypto_ctx_t *ctx, crypto_data_t *data, crypto_data_t *digest, + crypto_req_handle_t req) +{ + int error = CRYPTO_SUCCESS; + + ASSERT(SKEIN_CTX(ctx) != NULL); + + if (digest->cd_length < + CRYPTO_BITS2BYTES(SKEIN_CTX(ctx)->sc_digest_bitlen)) { + digest->cd_length = + CRYPTO_BITS2BYTES(SKEIN_CTX(ctx)->sc_digest_bitlen); + return (CRYPTO_BUFFER_TOO_SMALL); + } + + error = skein_update(ctx, data, req); + if (error != CRYPTO_SUCCESS) { + bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + SKEIN_CTX_LVALUE(ctx) = NULL; + digest->cd_length = 0; + return (error); + } + error = skein_final(ctx, digest, req); + + return (error); +} + +/* + * Performs a skein Update with the input message in `data' (successive calls + * can push more data). This is used both for digest and MAC operation. + * Supported input data formats are raw, uio and mblk. + */ +/*ARGSUSED*/ +static int +skein_update(crypto_ctx_t *ctx, crypto_data_t *data, crypto_req_handle_t req) +{ + int error = CRYPTO_SUCCESS; + + ASSERT(SKEIN_CTX(ctx) != NULL); + + switch (data->cd_format) { + case CRYPTO_DATA_RAW: + SKEIN_OP(SKEIN_CTX(ctx), Update, + (uint8_t *)data->cd_raw.iov_base + data->cd_offset, + data->cd_length); + break; + case CRYPTO_DATA_UIO: + error = skein_digest_update_uio(SKEIN_CTX(ctx), data); + break; + case CRYPTO_DATA_MBLK: + error = skein_digest_update_mblk(SKEIN_CTX(ctx), data); + break; + default: + error = CRYPTO_ARGUMENTS_BAD; + } + + return (error); +} + +/* + * Performs a skein Final, writing the output to `digest'. This is used both + * for digest and MAC operation. + * Supported output digest formats are raw, uio and mblk. + */ +/*ARGSUSED*/ +static int +skein_final(crypto_ctx_t *ctx, crypto_data_t *digest, crypto_req_handle_t req) +{ + int error = CRYPTO_SUCCESS; + + ASSERT(SKEIN_CTX(ctx) != NULL); + + if (digest->cd_length < + CRYPTO_BITS2BYTES(SKEIN_CTX(ctx)->sc_digest_bitlen)) { + digest->cd_length = + CRYPTO_BITS2BYTES(SKEIN_CTX(ctx)->sc_digest_bitlen); + return (CRYPTO_BUFFER_TOO_SMALL); + } + + switch (digest->cd_format) { + case CRYPTO_DATA_RAW: + SKEIN_OP(SKEIN_CTX(ctx), Final, + (uint8_t *)digest->cd_raw.iov_base + digest->cd_offset); + break; + case CRYPTO_DATA_UIO: + error = skein_digest_final_uio(SKEIN_CTX(ctx), digest, req); + break; + case CRYPTO_DATA_MBLK: + error = skein_digest_final_mblk(SKEIN_CTX(ctx), digest, req); + break; + default: + error = CRYPTO_ARGUMENTS_BAD; + } + + if (error == CRYPTO_SUCCESS) + digest->cd_length = + CRYPTO_BITS2BYTES(SKEIN_CTX(ctx)->sc_digest_bitlen); + else + digest->cd_length = 0; + + bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + kmem_free(SKEIN_CTX(ctx), sizeof (*(SKEIN_CTX(ctx)))); + SKEIN_CTX_LVALUE(ctx) = NULL; + + return (error); +} + +/* + * Performs a full skein digest computation in a single call, configuring the + * algorithm according to `mechanism', reading the input to be digested from + * `data' and writing the output to `digest'. + * Supported input/output formats are raw, uio and mblk. + */ +/*ARGSUSED*/ +static int +skein_digest_atomic(crypto_provider_handle_t provider, + crypto_session_id_t session_id, crypto_mechanism_t *mechanism, + crypto_data_t *data, crypto_data_t *digest, crypto_req_handle_t req) +{ + int error; + skein_ctx_t skein_ctx; + crypto_ctx_t ctx; + SKEIN_CTX_LVALUE(&ctx) = &skein_ctx; + + /* Init */ + if (!VALID_SKEIN_DIGEST_MECH(mechanism->cm_type)) + return (CRYPTO_MECHANISM_INVALID); + skein_ctx.sc_mech_type = mechanism->cm_type; + error = skein_get_digest_bitlen(mechanism, &skein_ctx.sc_digest_bitlen); + if (error != CRYPTO_SUCCESS) + goto out; + SKEIN_OP(&skein_ctx, Init, skein_ctx.sc_digest_bitlen); + + if ((error = skein_update(&ctx, data, digest)) != CRYPTO_SUCCESS) + goto out; + if ((error = skein_final(&ctx, data, digest)) != CRYPTO_SUCCESS) + goto out; + +out: + if (error == CRYPTO_SUCCESS) + digest->cd_length = + CRYPTO_BITS2BYTES(skein_ctx.sc_digest_bitlen); + else + digest->cd_length = 0; + bzero(&skein_ctx, sizeof (skein_ctx)); + + return (error); +} + +/* + * Helper function that builds a Skein MAC context from the provided + * mechanism and key. + */ +static int +skein_mac_ctx_build(skein_ctx_t *ctx, crypto_mechanism_t *mechanism, + crypto_key_t *key) +{ + int error; + + if (!VALID_SKEIN_MAC_MECH(mechanism->cm_type)) + return (CRYPTO_MECHANISM_INVALID); + if (key->ck_format != CRYPTO_KEY_RAW) + return (CRYPTO_ARGUMENTS_BAD); + ctx->sc_mech_type = mechanism->cm_type; + error = skein_get_digest_bitlen(mechanism, &ctx->sc_digest_bitlen); + if (error != CRYPTO_SUCCESS) + return (error); + SKEIN_OP(ctx, InitExt, ctx->sc_digest_bitlen, 0, key->ck_data, + CRYPTO_BITS2BYTES(key->ck_length)); + + return (CRYPTO_SUCCESS); +} + +/* + * KCF software provide mac entry points. + */ +/* + * Initializes a skein MAC context. You may pass a ctx_template, in which + * case the template will be reused to make initialization more efficient. + * Otherwise a new context will be constructed. The mechanism cm_type must + * be one of SKEIN_*_MAC_MECH_INFO_TYPE. Same as in skein_digest_init, you + * may pass a skein_param_t in cm_param to configure the length of the + * digest. The key must be in raw format. + */ +static int +skein_mac_init(crypto_ctx_t *ctx, crypto_mechanism_t *mechanism, + crypto_key_t *key, crypto_spi_ctx_template_t ctx_template, + crypto_req_handle_t req) +{ + int error; + + SKEIN_CTX_LVALUE(ctx) = kmem_alloc(sizeof (*SKEIN_CTX(ctx)), + crypto_kmflag(req)); + if (SKEIN_CTX(ctx) == NULL) + return (CRYPTO_HOST_MEMORY); + + if (ctx_template != NULL) { + bcopy(ctx_template, SKEIN_CTX(ctx), + sizeof (*SKEIN_CTX(ctx))); + } else { + error = skein_mac_ctx_build(SKEIN_CTX(ctx), mechanism, key); + if (error != CRYPTO_SUCCESS) + goto errout; + } + + return (CRYPTO_SUCCESS); +errout: + bzero(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + kmem_free(SKEIN_CTX(ctx), sizeof (*SKEIN_CTX(ctx))); + return (error); +} + *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Oct 14 11:12:50 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D263A12722; Wed, 14 Oct 2015 11:12:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D340F83; Wed, 14 Oct 2015 11:12:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EBCnK4022236; Wed, 14 Oct 2015 11:12:49 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EBCmFZ022230; Wed, 14 Oct 2015 11:12:48 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141112.t9EBCmFZ022230@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 11:12:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 11:12:50 -0000 Author: mav Date: Wed Oct 14 11:12:47 2015 New Revision: 289310 URL: https://svnweb.freebsd.org/changeset/base/289310 Log: 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R Reviewed by: George Wilson Reviewed by: Prakash Surya Reviewed by: Saso Kiselkov Reviewed by: Richard Lowe Approved by: Garrett D'Amore Author: Matthew Ahrens illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f Added: vendor/illumos/dist/common/crypto/ vendor/illumos/dist/common/crypto/edonr/ vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) vendor/illumos/dist/common/crypto/skein/ vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man5/zpool-features.5 Changes in other areas also in this revision: Added: vendor-sys/illumos/dist/uts/common/crypto/ vendor-sys/illumos/dist/uts/common/crypto/io/ vendor-sys/illumos/dist/uts/common/crypto/io/edonr_mod.c (contents, props changed) vendor-sys/illumos/dist/uts/common/crypto/io/skein_mod.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/edonr_zfs.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/skein_zfs.c (contents, props changed) vendor-sys/illumos/dist/uts/common/sys/edonr.h (contents, props changed) vendor-sys/illumos/dist/uts/common/sys/skein.h (contents, props changed) Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/common/zfs/zfs_fletcher.c vendor-sys/illumos/dist/common/zfs/zfs_fletcher.h vendor-sys/illumos/dist/common/zfs/zfs_prop.c vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/ddt.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sha256.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_checksum.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio_checksum.c vendor-sys/illumos/dist/uts/common/sys/debug.h Added: vendor/illumos/dist/common/crypto/edonr/edonr.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/illumos/dist/common/crypto/edonr/edonr.c Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1,729 @@ +/* + * IDI,NTNU + * + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * Copyright (C) 2009, 2010, Jorn Amundsen + * Tweaked Edon-R implementation for SUPERCOP, based on NIST API. + * + * $Id: edonr.c 517 2013-02-17 20:34:39Z joern $ + */ +/* + * Portions copyright (c) 2013, Saso Kiselkov, All rights reserved + */ + +/* determine where we can get bcopy/bzero declarations */ +#ifdef _KERNEL +#include +#else +#include +#endif +#include +#include + +/* big endian support, provides no-op's if run on little endian hosts */ +#include "edonr_byteorder.h" + +#define hashState224(x) ((x)->pipe->p256) +#define hashState256(x) ((x)->pipe->p256) +#define hashState384(x) ((x)->pipe->p512) +#define hashState512(x) ((x)->pipe->p512) + +/* shift and rotate shortcuts */ +#define shl(x, n) ((x) << n) +#define shr(x, n) ((x) >> n) + +#define rotl32(x, n) (((x) << (n)) | ((x) >> (32 - (n)))) +#define rotr32(x, n) (((x) >> (n)) | ((x) << (32 - (n)))) + +#define rotl64(x, n) (((x) << (n)) | ((x) >> (64 - (n)))) +#define rotr64(x, n) (((x) >> (n)) | ((x) << (64 - (n)))) + +#if !defined(__C99_RESTRICT) +#define restrict /* restrict */ +#endif + +#define EDONR_VALID_HASHBITLEN(x) \ + ((x) == 512 || (x) == 384 || (x) == 256 || (x) == 224) + +/* EdonR224 initial double chaining pipe */ +static const uint32_t i224p2[16] = { + 0x00010203ul, 0x04050607ul, 0x08090a0bul, 0x0c0d0e0ful, + 0x10111213ul, 0x14151617ul, 0x18191a1bul, 0x1c1d1e1ful, + 0x20212223ul, 0x24252627ul, 0x28292a2bul, 0x2c2d2e2ful, + 0x30313233ul, 0x34353637ul, 0x38393a3bul, 0x3c3d3e3ful, +}; + +/* EdonR256 initial double chaining pipe */ +static const uint32_t i256p2[16] = { + 0x40414243ul, 0x44454647ul, 0x48494a4bul, 0x4c4d4e4ful, + 0x50515253ul, 0x54555657ul, 0x58595a5bul, 0x5c5d5e5ful, + 0x60616263ul, 0x64656667ul, 0x68696a6bul, 0x6c6d6e6ful, + 0x70717273ul, 0x74757677ul, 0x78797a7bul, 0x7c7d7e7ful, +}; + +/* EdonR384 initial double chaining pipe */ +static const uint64_t i384p2[16] = { + 0x0001020304050607ull, 0x08090a0b0c0d0e0full, + 0x1011121314151617ull, 0x18191a1b1c1d1e1full, + 0x2021222324252627ull, 0x28292a2b2c2d2e2full, + 0x3031323334353637ull, 0x38393a3b3c3d3e3full, + 0x4041424344454647ull, 0x48494a4b4c4d4e4full, + 0x5051525354555657ull, 0x58595a5b5c5d5e5full, + 0x6061626364656667ull, 0x68696a6b6c6d6e6full, + 0x7071727374757677ull, 0x78797a7b7c7d7e7full +}; + +/* EdonR512 initial double chaining pipe */ +static const uint64_t i512p2[16] = { + 0x8081828384858687ull, 0x88898a8b8c8d8e8full, + 0x9091929394959697ull, 0x98999a9b9c9d9e9full, + 0xa0a1a2a3a4a5a6a7ull, 0xa8a9aaabacadaeafull, + 0xb0b1b2b3b4b5b6b7ull, 0xb8b9babbbcbdbebfull, + 0xc0c1c2c3c4c5c6c7ull, 0xc8c9cacbcccdcecfull, + 0xd0d1d2d3d4d5d6d7ull, 0xd8d9dadbdcdddedfull, + 0xe0e1e2e3e4e5e6e7ull, 0xe8e9eaebecedeeefull, + 0xf0f1f2f3f4f5f6f7ull, 0xf8f9fafbfcfdfeffull +}; + +/* + * First Latin Square + * 0 7 1 3 2 4 6 5 + * 4 1 7 6 3 0 5 2 + * 7 0 4 2 5 3 1 6 + * 1 4 0 5 6 2 7 3 + * 2 3 6 7 1 5 0 4 + * 5 2 3 1 7 6 4 0 + * 3 6 5 0 4 7 2 1 + * 6 5 2 4 0 1 3 7 + */ +#define LS1_256(c, x0, x1, x2, x3, x4, x5, x6, x7) \ +{ \ + uint32_t x04, x17, x23, x56, x07, x26; \ + x04 = x0+x4, x17 = x1+x7, x07 = x04+x17; \ + s0 = c + x07 + x2; \ + s1 = rotl32(x07 + x3, 4); \ + s2 = rotl32(x07 + x6, 8); \ + x23 = x2 + x3; \ + s5 = rotl32(x04 + x23 + x5, 22); \ + x56 = x5 + x6; \ + s6 = rotl32(x17 + x56 + x0, 24); \ + x26 = x23+x56; \ + s3 = rotl32(x26 + x7, 13); \ + s4 = rotl32(x26 + x1, 17); \ + s7 = rotl32(x26 + x4, 29); \ +} + +#define LS1_512(c, x0, x1, x2, x3, x4, x5, x6, x7) \ +{ \ + uint64_t x04, x17, x23, x56, x07, x26; \ + x04 = x0+x4, x17 = x1+x7, x07 = x04+x17; \ + s0 = c + x07 + x2; \ + s1 = rotl64(x07 + x3, 5); \ + s2 = rotl64(x07 + x6, 15); \ + x23 = x2 + x3; \ + s5 = rotl64(x04 + x23 + x5, 40); \ + x56 = x5 + x6; \ + s6 = rotl64(x17 + x56 + x0, 50); \ + x26 = x23+x56; \ + s3 = rotl64(x26 + x7, 22); \ + s4 = rotl64(x26 + x1, 31); \ + s7 = rotl64(x26 + x4, 59); \ +} + +/* + * Second Orthogonal Latin Square + * 0 4 2 3 1 6 5 7 + * 7 6 3 2 5 4 1 0 + * 5 3 1 6 0 2 7 4 + * 1 0 5 4 3 7 2 6 + * 2 1 0 7 4 5 6 3 + * 3 5 7 0 6 1 4 2 + * 4 7 6 1 2 0 3 5 + * 6 2 4 5 7 3 0 1 + */ +#define LS2_256(c, y0, y1, y2, y3, y4, y5, y6, y7) \ +{ \ + uint32_t y01, y25, y34, y67, y04, y05, y27, y37; \ + y01 = y0+y1, y25 = y2+y5, y05 = y01+y25; \ + t0 = ~c + y05 + y7; \ + t2 = rotl32(y05 + y3, 9); \ + y34 = y3+y4, y04 = y01+y34; \ + t1 = rotl32(y04 + y6, 5); \ + t4 = rotl32(y04 + y5, 15); \ + y67 = y6+y7, y37 = y34+y67; \ + t3 = rotl32(y37 + y2, 11); \ + t7 = rotl32(y37 + y0, 27); \ + y27 = y25+y67; \ + t5 = rotl32(y27 + y4, 20); \ + t6 = rotl32(y27 + y1, 25); \ +} + +#define LS2_512(c, y0, y1, y2, y3, y4, y5, y6, y7) \ +{ \ + uint64_t y01, y25, y34, y67, y04, y05, y27, y37; \ + y01 = y0+y1, y25 = y2+y5, y05 = y01+y25; \ + t0 = ~c + y05 + y7; \ + t2 = rotl64(y05 + y3, 19); \ + y34 = y3+y4, y04 = y01+y34; \ + t1 = rotl64(y04 + y6, 10); \ + t4 = rotl64(y04 + y5, 36); \ + y67 = y6+y7, y37 = y34+y67; \ + t3 = rotl64(y37 + y2, 29); \ + t7 = rotl64(y37 + y0, 55); \ + y27 = y25+y67; \ + t5 = rotl64(y27 + y4, 44); \ + t6 = rotl64(y27 + y1, 48); \ +} + +#define quasi_exform256(r0, r1, r2, r3, r4, r5, r6, r7) \ +{ \ + uint32_t s04, s17, s23, s56, t01, t25, t34, t67; \ + s04 = s0 ^ s4, t01 = t0 ^ t1; \ + r0 = (s04 ^ s1) + (t01 ^ t5); \ + t67 = t6 ^ t7; \ + r1 = (s04 ^ s7) + (t2 ^ t67); \ + s23 = s2 ^ s3; \ + r7 = (s23 ^ s5) + (t4 ^ t67); \ + t34 = t3 ^ t4; \ + r3 = (s23 ^ s4) + (t0 ^ t34); \ + s56 = s5 ^ s6; \ + r5 = (s3 ^ s56) + (t34 ^ t6); \ + t25 = t2 ^ t5; \ + r6 = (s2 ^ s56) + (t25 ^ t7); \ + s17 = s1 ^ s7; \ + r4 = (s0 ^ s17) + (t1 ^ t25); \ + r2 = (s17 ^ s6) + (t01 ^ t3); \ +} + +#define quasi_exform512(r0, r1, r2, r3, r4, r5, r6, r7) \ +{ \ + uint64_t s04, s17, s23, s56, t01, t25, t34, t67; \ + s04 = s0 ^ s4, t01 = t0 ^ t1; \ + r0 = (s04 ^ s1) + (t01 ^ t5); \ + t67 = t6 ^ t7; \ + r1 = (s04 ^ s7) + (t2 ^ t67); \ + s23 = s2 ^ s3; \ + r7 = (s23 ^ s5) + (t4 ^ t67); \ + t34 = t3 ^ t4; \ + r3 = (s23 ^ s4) + (t0 ^ t34); \ + s56 = s5 ^ s6; \ + r5 = (s3 ^ s56) + (t34 ^ t6); \ + t25 = t2 ^ t5; \ + r6 = (s2 ^ s56) + (t25 ^ t7); \ + s17 = s1 ^ s7; \ + r4 = (s0 ^ s17) + (t1 ^ t25); \ + r2 = (s17 ^ s6) + (t01 ^ t3); \ +} + +static size_t +Q256(size_t bitlen, const uint32_t *data, uint32_t *restrict p) +{ + size_t bl; + + for (bl = bitlen; bl >= EdonR256_BLOCK_BITSIZE; + bl -= EdonR256_BLOCK_BITSIZE, data += 16) { + uint32_t s0, s1, s2, s3, s4, s5, s6, s7, t0, t1, t2, t3, t4, + t5, t6, t7; + uint32_t p0, p1, p2, p3, p4, p5, p6, p7, q0, q1, q2, q3, q4, + q5, q6, q7; + const uint32_t defix = 0xaaaaaaaa; +#if defined(MACHINE_IS_BIG_ENDIAN) + uint32_t swp0, swp1, swp2, swp3, swp4, swp5, swp6, swp7, swp8, + swp9, swp10, swp11, swp12, swp13, swp14, swp15; +#define d(j) swp ## j +#define s32(j) ld_swap32((uint32_t *)data + j, swp ## j) +#else +#define d(j) data[j] +#endif + + /* First row of quasigroup e-transformations */ +#if defined(MACHINE_IS_BIG_ENDIAN) + s32(8); + s32(9); + s32(10); + s32(11); + s32(12); + s32(13); + s32(14); + s32(15); +#endif + LS1_256(defix, d(15), d(14), d(13), d(12), d(11), d(10), d(9), + d(8)); +#if defined(MACHINE_IS_BIG_ENDIAN) + s32(0); + s32(1); + s32(2); + s32(3); + s32(4); + s32(5); + s32(6); + s32(7); +#undef s32 +#endif + LS2_256(defix, d(0), d(1), d(2), d(3), d(4), d(5), d(6), d(7)); + quasi_exform256(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_256(defix, d(8), d(9), d(10), d(11), d(12), d(13), d(14), + d(15)); + quasi_exform256(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Second row of quasigroup e-transformations */ + LS1_256(defix, p[8], p[9], p[10], p[11], p[12], p[13], p[14], + p[15]); + LS2_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + quasi_exform256(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_256(defix, q0, q1, q2, q3, q4, q5, q6, q7); + quasi_exform256(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Third row of quasigroup e-transformations */ + LS1_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_256(defix, p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7]); + quasi_exform256(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_256(defix, q0, q1, q2, q3, q4, q5, q6, q7); + LS2_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + quasi_exform256(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Fourth row of quasigroup e-transformations */ + LS1_256(defix, d(7), d(6), d(5), d(4), d(3), d(2), d(1), d(0)); + LS2_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + quasi_exform256(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_256(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_256(defix, q0, q1, q2, q3, q4, q5, q6, q7); + quasi_exform256(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Edon-R tweak on the original SHA-3 Edon-R submission. */ + p[0] ^= d(8) ^ p0; + p[1] ^= d(9) ^ p1; + p[2] ^= d(10) ^ p2; + p[3] ^= d(11) ^ p3; + p[4] ^= d(12) ^ p4; + p[5] ^= d(13) ^ p5; + p[6] ^= d(14) ^ p6; + p[7] ^= d(15) ^ p7; + p[8] ^= d(0) ^ q0; + p[9] ^= d(1) ^ q1; + p[10] ^= d(2) ^ q2; + p[11] ^= d(3) ^ q3; + p[12] ^= d(4) ^ q4; + p[13] ^= d(5) ^ q5; + p[14] ^= d(6) ^ q6; + p[15] ^= d(7) ^ q7; + } + +#undef d + return (bitlen - bl); +} + +#if defined(__IBMC__) && defined(_AIX) && defined(__64BIT__) +static inline size_t +#else +static size_t +#endif +Q512(size_t bitlen, const uint64_t *data, uint64_t *restrict p) +{ + size_t bl; + + for (bl = bitlen; bl >= EdonR512_BLOCK_BITSIZE; + bl -= EdonR512_BLOCK_BITSIZE, data += 16) { + uint64_t s0, s1, s2, s3, s4, s5, s6, s7, t0, t1, t2, t3, t4, + t5, t6, t7; + uint64_t p0, p1, p2, p3, p4, p5, p6, p7, q0, q1, q2, q3, q4, + q5, q6, q7; + const uint64_t defix = 0xaaaaaaaaaaaaaaaaull; +#if defined(MACHINE_IS_BIG_ENDIAN) + uint64_t swp0, swp1, swp2, swp3, swp4, swp5, swp6, swp7, swp8, + swp9, swp10, swp11, swp12, swp13, swp14, swp15; +#define d(j) swp##j +#define s64(j) ld_swap64((uint64_t *)data+j, swp##j) +#else +#define d(j) data[j] +#endif + + /* First row of quasigroup e-transformations */ +#if defined(MACHINE_IS_BIG_ENDIAN) + s64(8); + s64(9); + s64(10); + s64(11); + s64(12); + s64(13); + s64(14); + s64(15); +#endif + LS1_512(defix, d(15), d(14), d(13), d(12), d(11), d(10), d(9), + d(8)); +#if defined(MACHINE_IS_BIG_ENDIAN) + s64(0); + s64(1); + s64(2); + s64(3); + s64(4); + s64(5); + s64(6); + s64(7); +#undef s64 +#endif + LS2_512(defix, d(0), d(1), d(2), d(3), d(4), d(5), d(6), d(7)); + quasi_exform512(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_512(defix, d(8), d(9), d(10), d(11), d(12), d(13), d(14), + d(15)); + quasi_exform512(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Second row of quasigroup e-transformations */ + LS1_512(defix, p[8], p[9], p[10], p[11], p[12], p[13], p[14], + p[15]); + LS2_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + quasi_exform512(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_512(defix, q0, q1, q2, q3, q4, q5, q6, q7); + quasi_exform512(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Third row of quasigroup e-transformations */ + LS1_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_512(defix, p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7]); + quasi_exform512(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_512(defix, q0, q1, q2, q3, q4, q5, q6, q7); + LS2_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + quasi_exform512(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Fourth row of quasigroup e-transformations */ + LS1_512(defix, d(7), d(6), d(5), d(4), d(3), d(2), d(1), d(0)); + LS2_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + quasi_exform512(p0, p1, p2, p3, p4, p5, p6, p7); + + LS1_512(defix, p0, p1, p2, p3, p4, p5, p6, p7); + LS2_512(defix, q0, q1, q2, q3, q4, q5, q6, q7); + quasi_exform512(q0, q1, q2, q3, q4, q5, q6, q7); + + /* Edon-R tweak on the original SHA-3 Edon-R submission. */ + p[0] ^= d(8) ^ p0; + p[1] ^= d(9) ^ p1; + p[2] ^= d(10) ^ p2; + p[3] ^= d(11) ^ p3; + p[4] ^= d(12) ^ p4; + p[5] ^= d(13) ^ p5; + p[6] ^= d(14) ^ p6; + p[7] ^= d(15) ^ p7; + p[8] ^= d(0) ^ q0; + p[9] ^= d(1) ^ q1; + p[10] ^= d(2) ^ q2; + p[11] ^= d(3) ^ q3; + p[12] ^= d(4) ^ q4; + p[13] ^= d(5) ^ q5; + p[14] ^= d(6) ^ q6; + p[15] ^= d(7) ^ q7; + } + +#undef d + return (bitlen - bl); +} + +void +EdonRInit(EdonRState *state, size_t hashbitlen) +{ + ASSERT(EDONR_VALID_HASHBITLEN(hashbitlen)); + switch (hashbitlen) { + case 224: + state->hashbitlen = 224; + state->bits_processed = 0; + state->unprocessed_bits = 0; + bcopy(i224p2, hashState224(state)->DoublePipe, + 16 * sizeof (uint32_t)); + break; + + case 256: + state->hashbitlen = 256; + state->bits_processed = 0; + state->unprocessed_bits = 0; + bcopy(i256p2, hashState256(state)->DoublePipe, + 16 * sizeof (uint32_t)); + break; + + case 384: + state->hashbitlen = 384; + state->bits_processed = 0; + state->unprocessed_bits = 0; + bcopy(i384p2, hashState384(state)->DoublePipe, + 16 * sizeof (uint64_t)); + break; + + case 512: + state->hashbitlen = 512; + state->bits_processed = 0; + state->unprocessed_bits = 0; + bcopy(i512p2, hashState224(state)->DoublePipe, + 16 * sizeof (uint64_t)); + break; + } +} + + +void +EdonRUpdate(EdonRState *state, const uint8_t *data, size_t databitlen) +{ + uint32_t *data32; + uint64_t *data64; + + size_t bits_processed; + + ASSERT(EDONR_VALID_HASHBITLEN(state->hashbitlen)); + switch (state->hashbitlen) { + case 224: + case 256: + if (state->unprocessed_bits > 0) { + /* LastBytes = databitlen / 8 */ + int LastBytes = (int)databitlen >> 3; + + ASSERT(state->unprocessed_bits + databitlen <= + EdonR256_BLOCK_SIZE * 8); + + bcopy(data, hashState256(state)->LastPart + + (state->unprocessed_bits >> 3), LastBytes); + state->unprocessed_bits += (int)databitlen; + databitlen = state->unprocessed_bits; + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data32 = (uint32_t *)hashState256(state)->LastPart; + } else + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data32 = (uint32_t *)data; + + bits_processed = Q256(databitlen, data32, + hashState256(state)->DoublePipe); + state->bits_processed += bits_processed; + databitlen -= bits_processed; + state->unprocessed_bits = (int)databitlen; + if (databitlen > 0) { + /* LastBytes = Ceil(databitlen / 8) */ + int LastBytes = + ((~(((-(int)databitlen) >> 3) & 0x01ff)) + + 1) & 0x01ff; + + data32 += bits_processed >> 5; /* byte size update */ + bcopy(data32, hashState256(state)->LastPart, LastBytes); + } + break; + + case 384: + case 512: + if (state->unprocessed_bits > 0) { + /* LastBytes = databitlen / 8 */ + int LastBytes = (int)databitlen >> 3; + + ASSERT(state->unprocessed_bits + databitlen <= + EdonR512_BLOCK_SIZE * 8); + + bcopy(data, hashState512(state)->LastPart + + (state->unprocessed_bits >> 3), LastBytes); + state->unprocessed_bits += (int)databitlen; + databitlen = state->unprocessed_bits; + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data64 = (uint64_t *)hashState512(state)->LastPart; + } else + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data64 = (uint64_t *)data; + + bits_processed = Q512(databitlen, data64, + hashState512(state)->DoublePipe); + state->bits_processed += bits_processed; + databitlen -= bits_processed; + state->unprocessed_bits = (int)databitlen; + if (databitlen > 0) { + /* LastBytes = Ceil(databitlen / 8) */ + int LastBytes = + ((~(((-(int)databitlen) >> 3) & 0x03ff)) + + 1) & 0x03ff; + + data64 += bits_processed >> 6; /* byte size update */ + bcopy(data64, hashState512(state)->LastPart, LastBytes); + } + break; + } +} + +void +EdonRFinal(EdonRState *state, uint8_t *hashval) +{ + uint32_t *data32; + uint64_t *data64, num_bits; + + size_t databitlen; + int LastByte, PadOnePosition; + + num_bits = state->bits_processed + state->unprocessed_bits; + ASSERT(EDONR_VALID_HASHBITLEN(state->hashbitlen)); + switch (state->hashbitlen) { + case 224: + case 256: + LastByte = (int)state->unprocessed_bits >> 3; + PadOnePosition = 7 - (state->unprocessed_bits & 0x07); + hashState256(state)->LastPart[LastByte] = + (hashState256(state)->LastPart[LastByte] + & (0xff << (PadOnePosition + 1))) ^ + (0x01 << PadOnePosition); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data64 = (uint64_t *)hashState256(state)->LastPart; + + if (state->unprocessed_bits < 448) { + (void) memset((hashState256(state)->LastPart) + + LastByte + 1, 0x00, + EdonR256_BLOCK_SIZE - LastByte - 9); + databitlen = EdonR256_BLOCK_SIZE * 8; +#if defined(MACHINE_IS_BIG_ENDIAN) + st_swap64(num_bits, data64 + 7); +#else + data64[7] = num_bits; +#endif + } else { + (void) memset((hashState256(state)->LastPart) + + LastByte + 1, 0x00, + EdonR256_BLOCK_SIZE * 2 - LastByte - 9); + databitlen = EdonR256_BLOCK_SIZE * 16; +#if defined(MACHINE_IS_BIG_ENDIAN) + st_swap64(num_bits, data64 + 15); +#else + data64[15] = num_bits; +#endif + } + + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data32 = (uint32_t *)hashState256(state)->LastPart; + state->bits_processed += Q256(databitlen, data32, + hashState256(state)->DoublePipe); + break; + + case 384: + case 512: + LastByte = (int)state->unprocessed_bits >> 3; + PadOnePosition = 7 - (state->unprocessed_bits & 0x07); + hashState512(state)->LastPart[LastByte] = + (hashState512(state)->LastPart[LastByte] + & (0xff << (PadOnePosition + 1))) ^ + (0x01 << PadOnePosition); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + data64 = (uint64_t *)hashState512(state)->LastPart; + + if (state->unprocessed_bits < 960) { + (void) memset((hashState512(state)->LastPart) + + LastByte + 1, 0x00, + EdonR512_BLOCK_SIZE - LastByte - 9); + databitlen = EdonR512_BLOCK_SIZE * 8; +#if defined(MACHINE_IS_BIG_ENDIAN) + st_swap64(num_bits, data64 + 15); +#else + data64[15] = num_bits; +#endif + } else { + (void) memset((hashState512(state)->LastPart) + + LastByte + 1, 0x00, + EdonR512_BLOCK_SIZE * 2 - LastByte - 9); + databitlen = EdonR512_BLOCK_SIZE * 16; +#if defined(MACHINE_IS_BIG_ENDIAN) + st_swap64(num_bits, data64 + 31); +#else + data64[31] = num_bits; +#endif + } + + state->bits_processed += Q512(databitlen, data64, + hashState512(state)->DoublePipe); + break; + } + + switch (state->hashbitlen) { + case 224: { +#if defined(MACHINE_IS_BIG_ENDIAN) + uint32_t *d32 = (uint32_t *)hashval; + uint32_t *s32 = hashState224(state)->DoublePipe + 9; + int j; + + for (j = 0; j < EdonR224_DIGEST_SIZE >> 2; j++) + st_swap32(s32[j], d32 + j); +#else + bcopy(hashState256(state)->DoublePipe + 9, hashval, + EdonR224_DIGEST_SIZE); +#endif + break; + } + case 256: { +#if defined(MACHINE_IS_BIG_ENDIAN) + uint32_t *d32 = (uint32_t *)hashval; + uint32_t *s32 = hashState224(state)->DoublePipe + 8; + int j; + + for (j = 0; j < EdonR256_DIGEST_SIZE >> 2; j++) + st_swap32(s32[j], d32 + j); +#else + bcopy(hashState256(state)->DoublePipe + 8, hashval, + EdonR256_DIGEST_SIZE); +#endif + break; + } + case 384: { +#if defined(MACHINE_IS_BIG_ENDIAN) + uint64_t *d64 = (uint64_t *)hashval; + uint64_t *s64 = hashState384(state)->DoublePipe + 10; + int j; + + for (j = 0; j < EdonR384_DIGEST_SIZE >> 3; j++) + st_swap64(s64[j], d64 + j); +#else + bcopy(hashState384(state)->DoublePipe + 10, hashval, + EdonR384_DIGEST_SIZE); +#endif + break; + } + case 512: { +#if defined(MACHINE_IS_BIG_ENDIAN) + uint64_t *d64 = (uint64_t *)hashval; + uint64_t *s64 = hashState512(state)->DoublePipe + 8; + int j; + + for (j = 0; j < EdonR512_DIGEST_SIZE >> 3; j++) + st_swap64(s64[j], d64 + j); +#else + bcopy(hashState512(state)->DoublePipe + 8, hashval, + EdonR512_DIGEST_SIZE); +#endif + break; + } + } +} + + +void +EdonRHash(size_t hashbitlen, const uint8_t *data, size_t databitlen, + uint8_t *hashval) +{ + EdonRState state; + + EdonRInit(&state, hashbitlen); + EdonRUpdate(&state, data, databitlen); + EdonRFinal(&state, hashval); +} Added: vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1,219 @@ +/* + * IDI,NTNU + * + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * Copyright (C) 2009, 2010, Jorn Amundsen + * + * C header file to determine compile machine byte order. Take care when cross + * compiling. + * + * $Id: byteorder.h 517 2013-02-17 20:34:39Z joern $ + */ +/* + * Portions copyright (c) 2013, Saso Kiselkov, All rights reserved + */ + +#ifndef _CRYPTO_EDONR_BYTEORDER_H +#define _CRYPTO_EDONR_BYTEORDER_H + +#if defined(__linux) +#include +#else +#include +#endif + +#if defined(__BYTE_ORDER) +#if (__BYTE_ORDER == __BIG_ENDIAN) +#define MACHINE_IS_BIG_ENDIAN +#elif (__BYTE_ORDER == __LITTLE_ENDIAN) +#define MACHINE_IS_LITTLE_ENDIAN +#endif +#elif defined(BYTE_ORDER) +#if (BYTE_ORDER == BIG_ENDIAN) +#define MACHINE_IS_BIG_ENDIAN +#elif (BYTE_ORDER == LITTLE_ENDIAN) +#define MACHINE_IS_LITTLE_ENDIAN +#endif +#endif /* __BYTE_ORDER || BYTE_ORDER */ + +#if !defined(MACHINE_IS_BIG_ENDIAN) && !defined(MACHINE_IS_LITTLE_ENDIAN) +#if defined(_BIG_ENDIAN) || defined(_MIPSEB) +#define MACHINE_IS_BIG_ENDIAN +#endif +#if defined(_LITTLE_ENDIAN) || defined(_MIPSEL) +#define MACHINE_IS_LITTLE_ENDIAN +#endif +#endif /* !MACHINE_IS_BIG_ENDIAN && !MACHINE_IS_LITTLE_ENDIAN */ + +#if !defined(MACHINE_IS_BIG_ENDIAN) && !defined(MACHINE_IS_LITTLE_ENDIAN) +#error unknown machine byte sex +#endif + +#define BYTEORDER_INCLUDED + +#if defined(MACHINE_IS_BIG_ENDIAN) +/* + * Byte swapping macros for big endian architectures and compilers, + * add as appropriate for other architectures and/or compilers. + * + * ld_swap64(src,dst) : uint64_t dst = *(src) + * st_swap64(src,dst) : *(dst) = uint64_t src + */ + +#if defined(__PPC__) || defined(_ARCH_PPC) + +#if defined(__64BIT__) +#if defined(_ARCH_PWR7) +#define aix_ld_swap64(s64, d64)\ + __asm__("ldbrx %0,0,%1" : "=r"(d64) : "r"(s64)) +#define aix_st_swap64(s64, d64)\ + __asm__ volatile("stdbrx %1,0,%0" : : "r"(d64), "r"(s64)) +#else +#define aix_ld_swap64(s64, d64) \ +{ \ + uint64_t *s4, h; \ + \ + __asm__("addi %0,%3,4;lwbrx %1,0,%3;lwbrx %2,0,%0;rldimi %1,%2,32,0"\ + : "+r"(s4), "=r"(d64), "=r"(h) : "b"(s64)); \ +} + +#define aix_st_swap64(s64, d64) \ +{ \ + uint64_t *s4, h; \ + h = (s64) >> 32; \ + __asm__ volatile("addi %0,%3,4;stwbrx %1,0,%3;stwbrx %2,0,%0" \ + : "+r"(s4) : "r"(s64), "r"(h), "b"(d64)); \ +} +#endif /* 64BIT && PWR7 */ +#else +#define aix_ld_swap64(s64, d64) \ +{ \ + uint32_t *s4, h, l; \ + __asm__("addi %0,%3,4;lwbrx %1,0,%3;lwbrx %2,0,%0" \ + : "+r"(s4), "=r"(l), "=r"(h) : "b"(s64)); \ + d64 = ((uint64_t)h<<32) | l; \ +} + +#define aix_st_swap64(s64, d64) \ +{ \ + uint32_t *s4, h, l; \ + l = (s64) & 0xfffffffful, h = (s64) >> 32; \ + __asm__ volatile("addi %0,%3,4;stwbrx %1,0,%3;stwbrx %2,0,%0" \ + : "+r"(s4) : "r"(l), "r"(h), "b"(d64)); \ +} +#endif /* __64BIT__ */ +#define aix_ld_swap32(s32, d32)\ + __asm__("lwbrx %0,0,%1" : "=r"(d32) : "r"(s32)) +#define aix_st_swap32(s32, d32)\ + __asm__ volatile("stwbrx %1,0,%0" : : "r"(d32), "r"(s32)) +#define ld_swap32(s, d) aix_ld_swap32(s, d) +#define st_swap32(s, d) aix_st_swap32(s, d) +#define ld_swap64(s, d) aix_ld_swap64(s, d) +#define st_swap64(s, d) aix_st_swap64(s, d) +#endif /* __PPC__ || _ARCH_PPC */ + +#if defined(__sparc) +#if !defined(__arch64__) && !defined(__sparcv8) && defined(__sparcv9) +#define __arch64__ +#endif +#if defined(__GNUC__) || (defined(__SUNPRO_C) && __SUNPRO_C > 0x590) +/* need Sun Studio C 5.10 and above for GNU inline assembly */ +#if defined(__arch64__) +#define sparc_ld_swap64(s64, d64) \ + __asm__("ldxa [%1]0x88,%0" : "=r"(d64) : "r"(s64)) +#define sparc_st_swap64(s64, d64) \ + __asm__ volatile("stxa %0,[%1]0x88" : : "r"(s64), "r"(d64)) +#define st_swap64(s, d) sparc_st_swap64(s, d) +#else +#define sparc_ld_swap64(s64, d64) \ +{ \ + uint32_t *s4, h, l; \ + __asm__("add %3,4,%0\n\tlda [%3]0x88,%1\n\tlda [%0]0x88,%2" \ + : "+r"(s4), "=r"(l), "=r"(h) : "r"(s64)); \ + d64 = ((uint64_t)h<<32) | l; \ +} +#define sparc_st_swap64(s64, d64) \ +{ \ + uint32_t *s4, h, l; \ + l = (s64) & 0xfffffffful, h = (s64) >> 32; \ + __asm__ volatile("add %3,4,%0\n\tsta %1,[%3]0x88\n\tsta %2,[%0]0x88"\ + : "+r"(s4) : "r"(l), "r"(h), "r"(d64)); \ +} +#endif /* sparc64 */ +#define sparc_ld_swap32(s32, d32)\ + __asm__("lda [%1]0x88,%0" : "=r"(d32) : "r"(s32)) +#define sparc_st_swap32(s32, d32)\ + __asm__ volatile("sta %0,[%1]0x88" : : "r"(s32), "r"(d32)) +#define ld_swap32(s, d) sparc_ld_swap32(s, d) +#define st_swap32(s, d) sparc_st_swap32(s, d) +#define ld_swap64(s, d) sparc_ld_swap64(s, d) +#define st_swap64(s, d) sparc_st_swap64(s, d) +#endif /* GCC || Sun Studio C > 5.9 */ +#endif /* sparc */ + +/* GCC fallback */ +#if ((__GNUC__ >= 4) || defined(__PGIC__)) && !defined(ld_swap32) +#define ld_swap32(s, d) (d = __builtin_bswap32(*(s))) +#define st_swap32(s, d) (*(d) = __builtin_bswap32(s)) +#endif /* GCC4/PGIC && !swap32 */ +#if ((__GNUC__ >= 4) || defined(__PGIC__)) && !defined(ld_swap64) +#define ld_swap64(s, d) (d = __builtin_bswap64(*(s))) +#define st_swap64(s, d) (*(d) = __builtin_bswap64(s)) +#endif /* GCC4/PGIC && !swap64 */ + +/* generic fallback */ +#if !defined(ld_swap32) +#define ld_swap32(s, d) \ + (d = (*(s) >> 24) | (*(s) >> 8 & 0xff00) | \ + (*(s) << 8 & 0xff0000) | (*(s) << 24)) +#define st_swap32(s, d) \ + (*(d) = ((s) >> 24) | ((s) >> 8 & 0xff00) | \ + ((s) << 8 & 0xff0000) | ((s) << 24)) +#endif +#if !defined(ld_swap64) +#define ld_swap64(s, d) \ + (d = (*(s) >> 56) | (*(s) >> 40 & 0xff00) | \ + (*(s) >> 24 & 0xff0000) | (*(s) >> 8 & 0xff000000) | \ + (*(s) & 0xff000000) << 8 | (*(s) & 0xff0000) << 24 | \ + (*(s) & 0xff00) << 40 | *(s) << 56) +#define st_swap64(s, d) \ + (*(d) = ((s) >> 56) | ((s) >> 40 & 0xff00) | \ + ((s) >> 24 & 0xff0000) | ((s) >> 8 & 0xff000000) | \ + ((s) & 0xff000000) << 8 | ((s) & 0xff0000) << 24 | \ + ((s) & 0xff00) << 40 | (s) << 56) +#endif + +#endif /* MACHINE_IS_BIG_ENDIAN */ + + +#if defined(MACHINE_IS_LITTLE_ENDIAN) +/* replace swaps with simple assignments on little endian systems */ +#undef ld_swap32 +#undef st_swap32 +#define ld_swap32(s, d) (d = *(s)) +#define st_swap32(s, d) (*(d) = s) +#undef ld_swap64 +#undef st_swap64 +#define ld_swap64(s, d) (d = *(s)) +#define st_swap64(s, d) (*(d) = s) +#endif /* MACHINE_IS_LITTLE_ENDIAN */ + +#endif /* _CRYPTO_EDONR_BYTEORDER_H */ Added: vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1,3 @@ +Implementation of the Skein hash function. +Source code author: Doug Whiting, 2008. +This algorithm and source code is released to the public domain. Added: vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1 @@ +LICENSE TERMS OF SKEIN HASH ALGORITHM IMPLEMENTATION Added: vendor/illumos/dist/common/crypto/skein/skein.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/illumos/dist/common/crypto/skein/skein.c Wed Oct 14 11:12:47 2015 (r289310) @@ -0,0 +1,914 @@ +/* + * Implementation of the Skein hash function. + * Source code author: Doug Whiting, 2008. + * This algorithm and source code is released to the public domain. + */ +/* Copyright 2013 Doug Whiting. This code is released to the public domain. */ + +#define SKEIN_PORT_CODE /* instantiate any code in skein_port.h */ + +#include +#include +#include /* get the Skein API definitions */ +#include "skein_impl.h" /* get internal definitions */ + +/* External function to process blkCnt (nonzero) full block(s) of data. */ +void Skein_256_Process_Block(Skein_256_Ctxt_t *ctx, const uint8_t *blkPtr, + size_t blkCnt, size_t byteCntAdd); +void Skein_512_Process_Block(Skein_512_Ctxt_t *ctx, const uint8_t *blkPtr, + size_t blkCnt, size_t byteCntAdd); +void Skein1024_Process_Block(Skein1024_Ctxt_t *ctx, const uint8_t *blkPtr, + size_t blkCnt, size_t byteCntAdd); + +/* 256-bit Skein */ +/* init the context for a straight hashing operation */ +int +Skein_256_Init(Skein_256_Ctxt_t *ctx, size_t hashBitLen) +{ + union { + uint8_t b[SKEIN_256_STATE_BYTES]; + uint64_t w[SKEIN_256_STATE_WORDS]; + } cfg; /* config block */ + + Skein_Assert(hashBitLen > 0, SKEIN_BAD_HASHLEN); *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Oct 14 11:33:08 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A09CBA12CF7; Wed, 14 Oct 2015 11:33:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77D2C1AF9; Wed, 14 Oct 2015 11:33:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EBX7l1028128; Wed, 14 Oct 2015 11:33:07 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EBX7rq028126; Wed, 14 Oct 2015 11:33:07 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141133.t9EBX7rq028126@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 11:33:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289311 - in vendor/illumos/dist: lib/libzfs/common man/man1m X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 11:33:08 -0000 Author: mav Date: Wed Oct 14 11:33:07 2015 New Revision: 289311 URL: https://svnweb.freebsd.org/changeset/base/289311 Log: 5764 "zfs send -nv" directs output to stderr Reviewed by: Matthew Ahrens Reviewed by: Paul Dagnelie Reviewed by: Basil Crow Reviewed by: Steven Hartland Reviewed by: Bayard Bell Approved by: Dan McDonald Author: Manoj Joseph illumos/illumos-gate@dc5f28a3c341db7c241bba77ddc109c141072f27 Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c Wed Oct 14 11:12:47 2015 (r289310) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c Wed Oct 14 11:33:07 2015 (r289311) @@ -809,7 +809,8 @@ typedef struct send_dump_data { char prevsnap[ZFS_MAXNAMELEN]; uint64_t prevsnap_obj; boolean_t seenfrom, seento, replicate, doall, fromorigin; - boolean_t verbose, dryrun, parsable, progress, embed_data, large_block; + boolean_t verbose, dryrun, parsable, progress, embed_data, std_out; + boolean_t large_block; int outfd; boolean_t err; nvlist_t *fss; @@ -1037,6 +1038,7 @@ dump_snapshot(zfs_handle_t *zhp, void *a int err; boolean_t isfromsnap, istosnap, fromorigin; boolean_t exclude = B_FALSE; + FILE *fout = sdd->std_out ? stdout : stderr; err = 0; thissnap = strchr(zhp->zfs_name, '@') + 1; @@ -1111,30 +1113,30 @@ dump_snapshot(zfs_handle_t *zhp, void *a if (sdd->parsable) { if (sdd->prevsnap[0] != '\0') { - (void) fprintf(stderr, "incremental\t%s\t%s", + (void) fprintf(fout, "incremental\t%s\t%s", sdd->prevsnap, zhp->zfs_name); } else { - (void) fprintf(stderr, "full\t%s", + (void) fprintf(fout, "full\t%s", zhp->zfs_name); } } else { - (void) fprintf(stderr, dgettext(TEXT_DOMAIN, + (void) fprintf(fout, dgettext(TEXT_DOMAIN, "send from @%s to %s"), sdd->prevsnap, zhp->zfs_name); } if (err == 0) { if (sdd->parsable) { - (void) fprintf(stderr, "\t%llu\n", + (void) fprintf(fout, "\t%llu\n", (longlong_t)size); } else { char buf[16]; zfs_nicenum(size, buf, sizeof (buf)); - (void) fprintf(stderr, dgettext(TEXT_DOMAIN, + (void) fprintf(fout, dgettext(TEXT_DOMAIN, " estimated size is %s\n"), buf); } sdd->size += size; } else { - (void) fprintf(stderr, "\n"); + (void) fprintf(fout, "\n"); } } @@ -1378,6 +1380,7 @@ zfs_send(zfs_handle_t *zhp, const char * int pipefd[2]; dedup_arg_t dda = { 0 }; int featureflags = 0; + FILE *fout; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, "cannot send '%s'"), zhp->zfs_name); @@ -1505,6 +1508,9 @@ zfs_send(zfs_handle_t *zhp, const char * sdd.filter_cb_arg = cb_arg; if (debugnvp) sdd.debugnv = *debugnvp; + if (sdd.verbose && sdd.dryrun) + sdd.std_out = B_TRUE; + fout = sdd.std_out ? stdout : stderr; /* * Some flags require that we place user holds on the datasets that are @@ -1544,12 +1550,12 @@ zfs_send(zfs_handle_t *zhp, const char * if (flags->verbose) { if (flags->parsable) { - (void) fprintf(stderr, "size\t%llu\n", + (void) fprintf(fout, "size\t%llu\n", (longlong_t)sdd.size); } else { char buf[16]; zfs_nicenum(sdd.size, buf, sizeof (buf)); - (void) fprintf(stderr, dgettext(TEXT_DOMAIN, + (void) fprintf(fout, dgettext(TEXT_DOMAIN, "total estimated size is %s\n"), buf); } } Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Wed Oct 14 11:12:47 2015 (r289310) +++ vendor/illumos/dist/man/man1m/zfs.1m Wed Oct 14 11:33:07 2015 (r289311) @@ -3061,7 +3061,9 @@ Include the dataset's properties in the .RS 4n Do a dry-run ("No-op") send. Do not generate any actual send data. This is useful in conjunction with the \fB-v\fR or \fB-P\fR flags to determine what -data will be sent. +data will be sent. In this case, the verbose output will be written to +standard output (contrast with a non-dry-run, where the stream is written +to standard output and the verbose output goes to standard error). .RE .sp From owner-svn-src-vendor@freebsd.org Wed Oct 14 11:36:57 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DA27A12E72; Wed, 14 Oct 2015 11:36:57 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25EE21DF0; Wed, 14 Oct 2015 11:36:57 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EBauBp028417; Wed, 14 Oct 2015 11:36:56 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EBat1O028409; Wed, 14 Oct 2015 11:36:55 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141136.t9EBat1O028409@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 11:36:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289312 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/ill... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 11:36:57 -0000 Author: mav Date: Wed Oct 14 11:36:55 2015 New Revision: 289312 URL: https://svnweb.freebsd.org/changeset/base/289312 Log: 2605 want to resume interrupted zfs send Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Richard Elling Reviewed by: Xin Li Reviewed by: Arne Jansen Approved by: Dan McDonald Author: Matthew Ahrens illumos/illumos-gate@9c3fd1216fa7fb02cfbc78a2518a686d54b48ab8 Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/cmd/zstreamdump/zstreamdump.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_send.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_traverse.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Oct 14 11:36:55 2015 (r289312) @@ -246,10 +246,11 @@ get_usage(zfs_help_t idx) case HELP_PROMOTE: return (gettext("\tpromote \n")); case HELP_RECEIVE: - return (gettext("\treceive [-vnFu] \n" - "\treceive [-vnFu] [-o origin=] [-d | -e] " - "\n")); + "\treceive [-vnsFu] [-o origin=] [-d | -e] " + "\n" + "\treceive -A \n")); case HELP_RENAME: return (gettext("\trename [-f] " "\n" @@ -261,7 +262,8 @@ get_usage(zfs_help_t idx) return (gettext("\tsend [-DnPpRvLe] [-[iI] snapshot] " "\n" "\tsend [-Le] [-i snapshot|bookmark] " - "\n")); + "\n" + "\tsend [-nvPe] -t \n")); case HELP_SET: return (gettext("\tset " " ...\n")); @@ -3640,6 +3642,7 @@ zfs_do_send(int argc, char **argv) { char *fromname = NULL; char *toname = NULL; + char *resume_token = NULL; char *cp; zfs_handle_t *zhp; sendflags_t flags = { 0 }; @@ -3648,7 +3651,7 @@ zfs_do_send(int argc, char **argv) boolean_t extraverbose = B_FALSE; /* check options */ - while ((c = getopt(argc, argv, ":i:I:RDpvnPLe")) != -1) { + while ((c = getopt(argc, argv, ":i:I:RDpvnPLet:")) != -1) { switch (c) { case 'i': if (fromname) @@ -3689,6 +3692,9 @@ zfs_do_send(int argc, char **argv) case 'e': flags.embed_data = B_TRUE; break; + case 't': + resume_token = optarg; + break; case ':': (void) fprintf(stderr, gettext("missing argument for " "'%c' option\n"), optopt); @@ -3704,14 +3710,28 @@ zfs_do_send(int argc, char **argv) argc -= optind; argv += optind; - /* check number of arguments */ - if (argc < 1) { - (void) fprintf(stderr, gettext("missing snapshot argument\n")); - usage(B_FALSE); - } - if (argc > 1) { - (void) fprintf(stderr, gettext("too many arguments\n")); - usage(B_FALSE); + if (resume_token != NULL) { + if (fromname != NULL || flags.replicate || flags.props || + flags.dedup) { + (void) fprintf(stderr, + gettext("invalid flags combined with -t\n")); + usage(B_FALSE); + } + if (argc != 0) { + (void) fprintf(stderr, gettext("no additional " + "arguments are permitted with -t\n")); + usage(B_FALSE); + } + } else { + if (argc < 1) { + (void) fprintf(stderr, + gettext("missing snapshot argument\n")); + usage(B_FALSE); + } + if (argc > 1) { + (void) fprintf(stderr, gettext("too many arguments\n")); + usage(B_FALSE); + } } if (!flags.dryrun && isatty(STDOUT_FILENO)) { @@ -3721,6 +3741,11 @@ zfs_do_send(int argc, char **argv) return (1); } + if (resume_token != NULL) { + return (zfs_send_resume(g_zfs, &flags, STDOUT_FILENO, + resume_token)); + } + /* * Special case sending a filesystem, or from a bookmark. */ @@ -3826,8 +3851,6 @@ zfs_do_send(int argc, char **argv) } /* - * zfs receive [-vnFu] [-d | -e] - * * Restore a backup stream from stdin. */ static int @@ -3835,6 +3858,8 @@ zfs_do_receive(int argc, char **argv) { int c, err; recvflags_t flags = { 0 }; + boolean_t abort_resumable = B_FALSE; + nvlist_t *props; nvpair_t *nvp = NULL; @@ -3842,7 +3867,7 @@ zfs_do_receive(int argc, char **argv) nomem(); /* check options */ - while ((c = getopt(argc, argv, ":o:denuvF")) != -1) { + while ((c = getopt(argc, argv, ":o:denuvFsA")) != -1) { switch (c) { case 'o': if (parseprop(props, optarg) != 0) @@ -3864,9 +3889,15 @@ zfs_do_receive(int argc, char **argv) case 'v': flags.verbose = B_TRUE; break; + case 's': + flags.resumable = B_TRUE; + break; case 'F': flags.force = B_TRUE; break; + case 'A': + abort_resumable = B_TRUE; + break; case ':': (void) fprintf(stderr, gettext("missing argument for " "'%c' option\n"), optopt); @@ -3899,6 +3930,44 @@ zfs_do_receive(int argc, char **argv) } } + if (abort_resumable) { + if (flags.isprefix || flags.istail || flags.dryrun || + flags.resumable || flags.nomount) { + (void) fprintf(stderr, gettext("invalid option")); + usage(B_FALSE); + } + + char namebuf[ZFS_MAXNAMELEN]; + (void) snprintf(namebuf, sizeof (namebuf), + "%s/%%recv", argv[0]); + + if (zfs_dataset_exists(g_zfs, namebuf, + ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME)) { + zfs_handle_t *zhp = zfs_open(g_zfs, + namebuf, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME); + if (zhp == NULL) + return (1); + err = zfs_destroy(zhp, B_FALSE); + } else { + zfs_handle_t *zhp = zfs_open(g_zfs, + argv[0], ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME); + if (zhp == NULL) + usage(B_FALSE); + if (!zfs_prop_get_int(zhp, ZFS_PROP_INCONSISTENT) || + zfs_prop_get(zhp, ZFS_PROP_RECEIVE_RESUME_TOKEN, + NULL, 0, NULL, NULL, 0, B_TRUE) == -1) { + (void) fprintf(stderr, + gettext("'%s' does not have any " + "resumable receive state to abort\n"), + argv[0]); + return (1); + } + err = zfs_destroy(zhp, B_FALSE); + } + + return (err != 0); + } + if (isatty(STDIN_FILENO)) { (void) fprintf(stderr, gettext("Error: Backup stream can not be read " @@ -3906,7 +3975,6 @@ zfs_do_receive(int argc, char **argv) "You must redirect standard input.\n")); return (1); } - err = zfs_receive(g_zfs, argv[0], props, &flags, STDIN_FILENO, NULL); return (err != 0); @@ -5728,6 +5796,24 @@ share_mount_one(zfs_handle_t *zhp, int o } /* + * If this filesystem is inconsistent and has a receive resume + * token, we can not mount it. + */ + if (zfs_prop_get_int(zhp, ZFS_PROP_INCONSISTENT) && + zfs_prop_get(zhp, ZFS_PROP_RECEIVE_RESUME_TOKEN, + NULL, 0, NULL, NULL, 0, B_TRUE) == 0) { + if (!explicit) + return (0); + + (void) fprintf(stderr, gettext("cannot %s '%s': " + "Contains partially-completed state from " + "\"zfs receive -r\", which can be resumed with " + "\"zfs send -t\"\n"), + cmdname, zfs_get_name(zhp)); + return (1); + } + + /* * At this point, we have verified that the mountpoint and/or * shareopts are appropriate for auto management. If the * filesystem is already mounted or shared, return (failing Modified: vendor/illumos/dist/cmd/zstreamdump/zstreamdump.c ============================================================================== --- vendor/illumos/dist/cmd/zstreamdump/zstreamdump.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor/illumos/dist/cmd/zstreamdump/zstreamdump.c Wed Oct 14 11:36:55 2015 (r289312) @@ -125,7 +125,7 @@ read_hdr(dmu_replay_record_t *drr, zio_c saved_cksum.zc_word[1], saved_cksum.zc_word[2], saved_cksum.zc_word[3]); - exit(1); + return (0); } return (sizeof (*drr)); } @@ -346,8 +346,7 @@ main(int argc, char *argv[]) if (verbose) (void) printf("\n"); - if ((DMU_GET_STREAM_HDRTYPE(drrb->drr_versioninfo) == - DMU_COMPOUNDSTREAM) && drr->drr_payloadlen != 0) { + if (drr->drr_payloadlen != 0) { nvlist_t *nv; int sz = drr->drr_payloadlen; Modified: vendor/illumos/dist/lib/libzfs/common/libzfs.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Oct 14 11:33:07 2015 (r289311) +++ vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Oct 14 11:36:55 2015 (r289312) @@ -603,6 +603,10 @@ typedef boolean_t (snapfilter_cb_t)(zfs_ extern int zfs_send(zfs_handle_t *, const char *, const char *, sendflags_t *, int, snapfilter_cb_t, void *, nvlist_t **); extern int zfs_send_one(zfs_handle_t *, const char *, int, enum lzc_send_flags); +extern int zfs_send_resume(libzfs_handle_t *, sendflags_t *, int outfd, + const char *); +extern nvlist_t *zfs_send_resume_token_to_nvlist(libzfs_handle_t *hdl, + const char *token); extern int zfs_promote(zfs_handle_t *); extern int zfs_hold(zfs_handle_t *, const char *, const char *, @@ -643,6 +647,12 @@ typedef struct recvflags { /* set "canmount=off" on all modified filesystems */ boolean_t canmountoff; + /* + * Mark the file systems as "resumable" and do not destroy them if the + * receive is interrupted + */ + boolean_t resumable; + /* byteswap flag is used internally; callers need not specify */ boolean_t byteswap; Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Oct 14 11:36:55 2015 (r289312) @@ -1735,22 +1735,21 @@ getprop_uint64(zfs_handle_t *zhp, zfs_pr return (value); } -static char * +static const char * getprop_string(zfs_handle_t *zhp, zfs_prop_t prop, char **source) { nvlist_t *nv; - char *value; + const char *value; *source = NULL; if (nvlist_lookup_nvlist(zhp->zfs_props, zfs_prop_to_name(prop), &nv) == 0) { - verify(nvlist_lookup_string(nv, ZPROP_VALUE, &value) == 0); + value = fnvlist_lookup_string(nv, ZPROP_VALUE); (void) nvlist_lookup_string(nv, ZPROP_SOURCE, source); } else { verify(!zhp->zfs_props_table || zhp->zfs_props_table[prop] == B_TRUE); - if ((value = (char *)zfs_prop_default_string(prop)) == NULL) - value = ""; + value = zfs_prop_default_string(prop); *source = ""; } @@ -2152,7 +2151,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop { char *source = NULL; uint64_t val; - char *str; + const char *str; const char *strval; boolean_t received = zfs_is_recvd_props_mode(zhp); @@ -2257,14 +2256,10 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop break; case ZFS_PROP_ORIGIN: - (void) strlcpy(propbuf, getprop_string(zhp, prop, &source), - proplen); - /* - * If there is no parent at all, return failure to indicate that - * it doesn't apply to this dataset. - */ - if (propbuf[0] == '\0') + str = getprop_string(zhp, prop, &source); + if (str == NULL) return (-1); + (void) strlcpy(propbuf, str, proplen); break; case ZFS_PROP_CLONES: @@ -2441,8 +2436,10 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop break; case PROP_TYPE_STRING: - (void) strlcpy(propbuf, - getprop_string(zhp, prop, &source), proplen); + str = getprop_string(zhp, prop, &source); + if (str == NULL) + return (-1); + (void) strlcpy(propbuf, str, proplen); break; case PROP_TYPE_INDEX: Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Wed Oct 14 11:36:55 2015 (r289312) @@ -1015,6 +1015,17 @@ mount_cb(zfs_handle_t *zhp, void *data) return (0); } + /* + * If this filesystem is inconsistent and has a receive resume + * token, we can not mount it. + */ + if (zfs_prop_get_int(zhp, ZFS_PROP_INCONSISTENT) && + zfs_prop_get(zhp, ZFS_PROP_RECEIVE_RESUME_TOKEN, + NULL, 0, NULL, NULL, 0, B_TRUE) == 0) { + zfs_close(zhp); + return (0); + } + libzfs_add_handle(cbp, zhp); if (zfs_iter_filesystems(zhp, mount_cb, cbp) != 0) { zfs_close(zhp); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c Wed Oct 14 11:36:55 2015 (r289312) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2014 by Delphix. All rights reserved. + * Copyright (c) 2011, 2015 by Delphix. All rights reserved. * Copyright (c) 2012, Joyent, Inc. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. */ @@ -48,6 +48,7 @@ #include "zfs_prop.h" #include "zfs_fletcher.h" #include "libzfs_impl.h" +#include #include #include #include @@ -58,6 +59,8 @@ extern void zfs_setprop_error(libzfs_han static int zfs_receive_impl(libzfs_handle_t *, const char *, const char *, recvflags_t *, int, const char *, nvlist_t *, avl_tree_t *, char **, int, uint64_t *); +static int guid_to_name(libzfs_handle_t *, const char *, + uint64_t, boolean_t, char *); static const zio_cksum_t zero_cksum = { 0 }; @@ -275,8 +278,7 @@ cksummer(void *arg) DMU_BACKUP_FEATURE_DEDUPPROPS); DMU_SET_FEATUREFLAGS(drrb->drr_versioninfo, fflags); - if (DMU_GET_STREAM_HDRTYPE(drrb->drr_versioninfo) == - DMU_COMPOUNDSTREAM && drr->drr_payloadlen != 0) { + if (drr->drr_payloadlen != 0) { sz = drr->drr_payloadlen; if (sz > SPA_MAXBLOCKSIZE) { @@ -985,17 +987,14 @@ static void * send_progress_thread(void *arg) { progress_arg_t *pa = arg; - zfs_cmd_t zc = { 0 }; zfs_handle_t *zhp = pa->pa_zhp; libzfs_handle_t *hdl = zhp->zfs_hdl; unsigned long long bytes; char buf[16]; - time_t t; struct tm *tm; - assert(zhp->zfs_type == ZFS_TYPE_SNAPSHOT); (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); if (!pa->pa_parsable) @@ -1028,6 +1027,51 @@ send_progress_thread(void *arg) } } +static void +send_print_verbose(FILE *fout, const char *tosnap, const char *fromsnap, + uint64_t size, boolean_t parsable) +{ + if (parsable) { + if (fromsnap != NULL) { + (void) fprintf(fout, "incremental\t%s\t%s", + fromsnap, tosnap); + } else { + (void) fprintf(fout, "full\t%s", + tosnap); + } + } else { + if (fromsnap != NULL) { + if (strchr(fromsnap, '@') == NULL && + strchr(fromsnap, '#') == NULL) { + (void) fprintf(fout, dgettext(TEXT_DOMAIN, + "send from @%s to %s"), + fromsnap, tosnap); + } else { + (void) fprintf(fout, dgettext(TEXT_DOMAIN, + "send from %s to %s"), + fromsnap, tosnap); + } + } else { + (void) fprintf(fout, dgettext(TEXT_DOMAIN, + "full send of %s"), + tosnap); + } + } + + if (size != 0) { + if (parsable) { + (void) fprintf(fout, "\t%llu", + (longlong_t)size); + } else { + char buf[16]; + zfs_nicenum(size, buf, sizeof (buf)); + (void) fprintf(fout, dgettext(TEXT_DOMAIN, + " estimated size is %s"), buf); + } + } + (void) fprintf(fout, "\n"); +} + static int dump_snapshot(zfs_handle_t *zhp, void *arg) { @@ -1107,37 +1151,14 @@ dump_snapshot(zfs_handle_t *zhp, void *a (sdd->fromorigin || sdd->replicate); if (sdd->verbose) { - uint64_t size; - err = estimate_ioctl(zhp, sdd->prevsnap_obj, + uint64_t size = 0; + (void) estimate_ioctl(zhp, sdd->prevsnap_obj, fromorigin, &size); - if (sdd->parsable) { - if (sdd->prevsnap[0] != '\0') { - (void) fprintf(fout, "incremental\t%s\t%s", - sdd->prevsnap, zhp->zfs_name); - } else { - (void) fprintf(fout, "full\t%s", - zhp->zfs_name); - } - } else { - (void) fprintf(fout, dgettext(TEXT_DOMAIN, - "send from @%s to %s"), - sdd->prevsnap, zhp->zfs_name); - } - if (err == 0) { - if (sdd->parsable) { - (void) fprintf(fout, "\t%llu\n", - (longlong_t)size); - } else { - char buf[16]; - zfs_nicenum(size, buf, sizeof (buf)); - (void) fprintf(fout, dgettext(TEXT_DOMAIN, - " estimated size is %s\n"), buf); - } - sdd->size += size; - } else { - (void) fprintf(fout, "\n"); - } + send_print_verbose(fout, zhp->zfs_name, + sdd->prevsnap[0] ? sdd->prevsnap : NULL, + size, sdd->parsable); + sdd->size += size; } if (!sdd->dryrun) { @@ -1348,6 +1369,231 @@ again: return (0); } +nvlist_t * +zfs_send_resume_token_to_nvlist(libzfs_handle_t *hdl, const char *token) +{ + unsigned int version; + int nread; + unsigned long long checksum, packed_len; + + /* + * Decode token header, which is: + * -- + * Note that the only supported token version is 1. + */ + nread = sscanf(token, "%u-%llx-%llx-", + &version, &checksum, &packed_len); + if (nread != 3) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt (invalid format)")); + return (NULL); + } + + if (version != ZFS_SEND_RESUME_TOKEN_VERSION) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt (invalid version %u)"), + version); + return (NULL); + } + + /* convert hexadecimal representation to binary */ + token = strrchr(token, '-') + 1; + int len = strlen(token) / 2; + unsigned char *compressed = zfs_alloc(hdl, len); + for (int i = 0; i < len; i++) { + nread = sscanf(token + i * 2, "%2hhx", compressed + i); + if (nread != 1) { + free(compressed); + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt " + "(payload is not hex-encoded)")); + return (NULL); + } + } + + /* verify checksum */ + zio_cksum_t cksum; + fletcher_4_native(compressed, len, NULL, &cksum); + if (cksum.zc_word[0] != checksum) { + free(compressed); + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt (incorrect checksum)")); + return (NULL); + } + + /* uncompress */ + void *packed = zfs_alloc(hdl, packed_len); + uLongf packed_len_long = packed_len; + if (uncompress(packed, &packed_len_long, compressed, len) != Z_OK || + packed_len_long != packed_len) { + free(packed); + free(compressed); + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt (decompression failed)")); + return (NULL); + } + + /* unpack nvlist */ + nvlist_t *nv; + int error = nvlist_unpack(packed, packed_len, &nv, KM_SLEEP); + free(packed); + free(compressed); + if (error != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt (nvlist_unpack failed)")); + return (NULL); + } + return (nv); +} + +int +zfs_send_resume(libzfs_handle_t *hdl, sendflags_t *flags, int outfd, + const char *resume_token) +{ + char errbuf[1024]; + char *toname; + char *fromname = NULL; + uint64_t resumeobj, resumeoff, toguid, fromguid, bytes; + zfs_handle_t *zhp; + int error = 0; + char name[ZFS_MAXNAMELEN]; + enum lzc_send_flags lzc_flags = 0; + + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, + "cannot resume send")); + + nvlist_t *resume_nvl = + zfs_send_resume_token_to_nvlist(hdl, resume_token); + if (resume_nvl == NULL) { + /* + * zfs_error_aux has already been set by + * zfs_send_resume_token_to_nvlist + */ + return (zfs_error(hdl, EZFS_FAULT, errbuf)); + } + if (flags->verbose) { + (void) fprintf(stderr, dgettext(TEXT_DOMAIN, + "resume token contents:\n")); + nvlist_print(stderr, resume_nvl); + } + + if (nvlist_lookup_string(resume_nvl, "toname", &toname) != 0 || + nvlist_lookup_uint64(resume_nvl, "object", &resumeobj) != 0 || + nvlist_lookup_uint64(resume_nvl, "offset", &resumeoff) != 0 || + nvlist_lookup_uint64(resume_nvl, "bytes", &bytes) != 0 || + nvlist_lookup_uint64(resume_nvl, "toguid", &toguid) != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "resume token is corrupt")); + return (zfs_error(hdl, EZFS_FAULT, errbuf)); + } + fromguid = 0; + (void) nvlist_lookup_uint64(resume_nvl, "fromguid", &fromguid); + + if (flags->embed_data || nvlist_exists(resume_nvl, "embedok")) + lzc_flags |= LZC_SEND_FLAG_EMBED_DATA; + + if (guid_to_name(hdl, toname, toguid, B_FALSE, name) != 0) { + if (zfs_dataset_exists(hdl, toname, ZFS_TYPE_DATASET)) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'%s' is no longer the same snapshot used in " + "the initial send"), toname); + } else { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'%s' used in the initial send no longer exists"), + toname); + } + return (zfs_error(hdl, EZFS_BADPATH, errbuf)); + } + zhp = zfs_open(hdl, name, ZFS_TYPE_DATASET); + if (zhp == NULL) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "unable to access '%s'"), name); + return (zfs_error(hdl, EZFS_BADPATH, errbuf)); + } + + if (fromguid != 0) { + if (guid_to_name(hdl, toname, fromguid, B_TRUE, name) != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "incremental source %#llx no longer exists"), + (longlong_t)fromguid); + return (zfs_error(hdl, EZFS_BADPATH, errbuf)); + } + fromname = name; + } + + if (flags->verbose) { + uint64_t size = 0; + error = lzc_send_space(zhp->zfs_name, fromname, &size); + if (error == 0) + size = MAX(0, (int64_t)(size - bytes)); + send_print_verbose(stderr, zhp->zfs_name, fromname, + size, flags->parsable); + } + + if (!flags->dryrun) { + progress_arg_t pa = { 0 }; + pthread_t tid; + /* + * If progress reporting is requested, spawn a new thread to + * poll ZFS_IOC_SEND_PROGRESS at a regular interval. + */ + if (flags->progress) { + pa.pa_zhp = zhp; + pa.pa_fd = outfd; + pa.pa_parsable = flags->parsable; + + error = pthread_create(&tid, NULL, + send_progress_thread, &pa); + if (error != 0) { + zfs_close(zhp); + return (error); + } + } + + error = lzc_send_resume(zhp->zfs_name, fromname, outfd, + lzc_flags, resumeobj, resumeoff); + + if (flags->progress) { + (void) pthread_cancel(tid); + (void) pthread_join(tid, NULL); + } + + char errbuf[1024]; + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, + "warning: cannot send '%s'"), zhp->zfs_name); + + zfs_close(zhp); + + switch (error) { + case 0: + return (0); + case EXDEV: + case ENOENT: + case EDQUOT: + case EFBIG: + case EIO: + case ENOLINK: + case ENOSPC: + case ENOSTR: + case ENXIO: + case EPIPE: + case ERANGE: + case EFAULT: + case EROFS: + zfs_error_aux(hdl, strerror(errno)); + return (zfs_error(hdl, EZFS_BADBACKUP, errbuf)); + + default: + return (zfs_standard_error(hdl, errno, errbuf)); + } + } + + + zfs_close(zhp); + + return (error); +} + /* * Generate a send stream for the dataset identified by the argument zhp. * @@ -1885,6 +2131,7 @@ recv_destroy(libzfs_handle_t *hdl, const typedef struct guid_to_name_data { uint64_t guid; + boolean_t bookmark_ok; char *name; char *skip; } guid_to_name_data_t; @@ -1893,20 +2140,25 @@ static int guid_to_name_cb(zfs_handle_t *zhp, void *arg) { guid_to_name_data_t *gtnd = arg; + const char *slash; int err; if (gtnd->skip != NULL && - strcmp(zhp->zfs_name, gtnd->skip) == 0) { + (slash = strrchr(zhp->zfs_name, '/')) != NULL && + strcmp(slash + 1, gtnd->skip) == 0) { + zfs_close(zhp); return (0); } - if (zhp->zfs_dmustats.dds_guid == gtnd->guid) { + if (zfs_prop_get_int(zhp, ZFS_PROP_GUID) == gtnd->guid) { (void) strcpy(gtnd->name, zhp->zfs_name); zfs_close(zhp); return (EEXIST); } err = zfs_iter_children(zhp, guid_to_name_cb, gtnd); + if (err != EEXIST && gtnd->bookmark_ok) + err = zfs_iter_bookmarks(zhp, guid_to_name_cb, gtnd); zfs_close(zhp); return (err); } @@ -1920,45 +2172,48 @@ guid_to_name_cb(zfs_handle_t *zhp, void */ static int guid_to_name(libzfs_handle_t *hdl, const char *parent, uint64_t guid, - char *name) + boolean_t bookmark_ok, char *name) { - /* exhaustive search all local snapshots */ char pname[ZFS_MAXNAMELEN]; guid_to_name_data_t gtnd; - int err = 0; - zfs_handle_t *zhp; - char *cp; gtnd.guid = guid; + gtnd.bookmark_ok = bookmark_ok; gtnd.name = name; gtnd.skip = NULL; - (void) strlcpy(pname, parent, sizeof (pname)); - /* - * Search progressively larger portions of the hierarchy. This will + * Search progressively larger portions of the hierarchy, starting + * with the filesystem specified by 'parent'. This will * select the "most local" version of the origin snapshot in the case * that there are multiple matching snapshots in the system. */ - while ((cp = strrchr(pname, '/')) != NULL) { - + (void) strlcpy(pname, parent, sizeof (pname)); + char *cp = strrchr(pname, '@'); + if (cp == NULL) + cp = strchr(pname, '\0'); + for (; cp != NULL; cp = strrchr(pname, '/')) { /* Chop off the last component and open the parent */ *cp = '\0'; - zhp = make_dataset_handle(hdl, pname); + zfs_handle_t *zhp = make_dataset_handle(hdl, pname); if (zhp == NULL) continue; - - err = zfs_iter_children(zhp, guid_to_name_cb, >nd); + int err = guid_to_name_cb(zfs_handle_dup(zhp), >nd); + if (err != EEXIST) + err = zfs_iter_children(zhp, guid_to_name_cb, >nd); + if (err != EEXIST && bookmark_ok) + err = zfs_iter_bookmarks(zhp, guid_to_name_cb, >nd); zfs_close(zhp); if (err == EEXIST) return (0); /* - * Remember the dataset that we already searched, so we - * skip it next time through. + * Remember the last portion of the dataset so we skip it next + * time through (as we've already searched that portion of the + * hierarchy). */ - gtnd.skip = pname; + gtnd.skip = strrchr(pname, '/') + 1; } return (ENOENT); @@ -2529,11 +2784,9 @@ recv_skip(libzfs_handle_t *hdl, int fd, switch (drr->drr_type) { case DRR_BEGIN: - /* NB: not to be used on v2 stream packages */ if (drr->drr_payloadlen != 0) { - zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, - "invalid substream header")); - return (zfs_error(hdl, EZFS_BADSTREAM, errbuf)); + (void) recv_read(hdl, fd, buf, + drr->drr_payloadlen, B_FALSE, NULL); } break; @@ -2594,6 +2847,40 @@ recv_skip(libzfs_handle_t *hdl, int fd, return (-1); } +static void +recv_ecksum_set_aux(libzfs_handle_t *hdl, const char *target_snap, + boolean_t resumable) +{ + char target_fs[ZFS_MAXNAMELEN]; + + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "checksum mismatch or incomplete stream")); + + if (!resumable) + return; + (void) strlcpy(target_fs, target_snap, sizeof (target_fs)); + *strchr(target_fs, '@') = '\0'; + zfs_handle_t *zhp = zfs_open(hdl, target_fs, + ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME); + if (zhp == NULL) + return; + + char token_buf[ZFS_MAXPROPLEN]; + int error = zfs_prop_get(zhp, ZFS_PROP_RECEIVE_RESUME_TOKEN, + token_buf, sizeof (token_buf), + NULL, NULL, 0, B_TRUE); + if (error == 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "checksum mismatch or incomplete stream.\n" + "Partially received snapshot is saved.\n" + "A resuming stream can be generated on the sending " + "system by running:\n" + " zfs send -t %s"), + token_buf); + } + zfs_close(zhp); +} + /* * Restores a backup of tosnap from the file descriptor specified by infd. */ @@ -2748,7 +3035,7 @@ zfs_receive_one(libzfs_handle_t *hdl, in */ if (drrb->drr_flags & DRR_FLAG_CLONE) { if (guid_to_name(hdl, zc.zc_value, - drrb->drr_fromguid, zc.zc_string) != 0) { + drrb->drr_fromguid, B_FALSE, zc.zc_string) != 0) { zcmd_free_nvlists(&zc); zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "local origin for clone %s does not exist"), @@ -2764,8 +3051,10 @@ zfs_receive_one(libzfs_handle_t *hdl, in zc.zc_string); } + boolean_t resuming = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo) & + DMU_BACKUP_FEATURE_RESUMING; stream_wantsnewfs = (drrb->drr_fromguid == NULL || - (drrb->drr_flags & DRR_FLAG_CLONE) || originsnap); + (drrb->drr_flags & DRR_FLAG_CLONE) || originsnap) && !resuming; if (stream_wantsnewfs) { /* @@ -2784,7 +3073,7 @@ zfs_receive_one(libzfs_handle_t *hdl, in char suffix[ZFS_MAXNAMELEN]; (void) strcpy(suffix, strrchr(zc.zc_value, '/')); if (guid_to_name(hdl, zc.zc_name, parent_snapguid, - zc.zc_value) == 0) { + B_FALSE, zc.zc_value) == 0) { *strchr(zc.zc_value, '@') = '\0'; (void) strcat(zc.zc_value, suffix); } @@ -2811,7 +3100,7 @@ zfs_receive_one(libzfs_handle_t *hdl, in char snap[ZFS_MAXNAMELEN]; (void) strcpy(snap, strchr(zc.zc_value, '@')); if (guid_to_name(hdl, zc.zc_name, drrb->drr_fromguid, - zc.zc_value) == 0) { + B_FALSE, zc.zc_value) == 0) { *strchr(zc.zc_value, '@') = '\0'; (void) strcat(zc.zc_value, snap); } @@ -2825,11 +3114,12 @@ zfs_receive_one(libzfs_handle_t *hdl, in zfs_handle_t *zhp; /* - * Destination fs exists. Therefore this should either - * be an incremental, or the stream specifies a new fs - * (full stream or clone) and they want us to blow it - * away (and have therefore specified -F and removed any - * snapshots). + * Destination fs exists. It must be one of these cases: + * - an incremental send stream + * - the stream specifies a new fs (full stream or clone) + * and they want us to blow away the existing fs (and + * have therefore specified -F and removed any snapshots) + * - we are resuming a failed receive. */ if (stream_wantsnewfs) { if (!flags->force) { @@ -2884,6 +3174,18 @@ zfs_receive_one(libzfs_handle_t *hdl, in return (-1); } } + + /* + * If we are resuming a newfs, set newfs here so that we will + * mount it if the recv succeeds this time. We can tell + * that it was a newfs on the first recv because the fs + * itself will be inconsistent (if the fs existed when we + * did the first recv, we would have received it into + * .../%recv). + */ + if (resuming && zfs_prop_get_int(zhp, ZFS_PROP_INCONSISTENT)) + newfs = B_TRUE; + zfs_close(zhp); } else { /* @@ -2916,9 +3218,10 @@ zfs_receive_one(libzfs_handle_t *hdl, in newfs = B_TRUE; } - zc.zc_begin_record = drr_noswap->drr_u.drr_begin; + zc.zc_begin_record = *drr_noswap; zc.zc_cookie = infd; zc.zc_guid = flags->force; + zc.zc_resumable = flags->resumable; if (flags->verbose) { (void) printf("%s %s stream of %s into %s\n", flags->dryrun ? "would receive" : "receiving", @@ -3055,8 +3358,7 @@ zfs_receive_one(libzfs_handle_t *hdl, in (void) zfs_error(hdl, EZFS_BADSTREAM, errbuf); break; case ECKSUM: - zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, - "invalid stream (checksum mismatch)")); + recv_ecksum_set_aux(hdl, zc.zc_value, flags->resumable); (void) zfs_error(hdl, EZFS_BADSTREAM, errbuf); break; case ENOTSUP: @@ -3258,7 +3560,8 @@ zfs_receive_impl(libzfs_handle_t *hdl, c * Restores a backup of tosnap from the file descriptor specified by infd. *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Wed Oct 14 11:36:56 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7E6CA12E6B; Wed, 14 Oct 2015 11:36:56 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6DC841DEF; Wed, 14 Oct 2015 11:36:56 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9EBat03028404; Wed, 14 Oct 2015 11:36:55 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9EBatb8028399; Wed, 14 Oct 2015 11:36:55 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510141136.t9EBatb8028399@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 14 Oct 2015 11:36:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289312 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/ill... X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Oct 2015 11:36:56 -0000 Author: mav Date: Wed Oct 14 11:36:55 2015 New Revision: 289312 URL: https://svnweb.freebsd.org/changeset/base/289312 Log: 2605 want to resume interrupted zfs send Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Richard Elling Reviewed by: Xin Li Reviewed by: Arne Jansen Approved by: Dan McDonald Author: Matthew Ahrens illumos/illumos-gate@9c3fd1216fa7fb02cfbc78a2518a686d54b48ab8 Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_send.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_traverse.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/cmd/zstreamdump/zstreamdump.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_prop.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor-sys/illumos/dist/common/zfs/zfs_prop.c Wed Oct 14 11:36:55 2015 (r289312) @@ -342,6 +342,10 @@ zfs_prop_init(void) zprop_register_string(ZFS_PROP_MLSLABEL, "mlslabel", ZFS_MLSLABEL_DEFAULT, PROP_INHERIT, ZFS_TYPE_DATASET, "", "MLSLABEL"); + zprop_register_string(ZFS_PROP_RECEIVE_RESUME_TOKEN, + "receive_resume_token", + NULL, PROP_READONLY, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, + "", "RESUMETOK"); /* readonly number properties */ zprop_register_number(ZFS_PROP_USED, "used", 0, PROP_READONLY, Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Wed Oct 14 11:36:55 2015 (r289312) @@ -362,6 +362,17 @@ dmu_objset_open_impl(spa_t *spa, dsl_dat * checksum/compression/copies. */ if (ds != NULL) { + boolean_t needlock = B_FALSE; + + /* + * Note: it's valid to open the objset if the dataset is + * long-held, in which case the pool_config lock will not + * be held. + */ + if (!dsl_pool_config_held(dmu_objset_pool(os))) { + needlock = B_TRUE; + dsl_pool_config_enter(dmu_objset_pool(os), FTAG); + } err = dsl_prop_register(ds, zfs_prop_to_name(ZFS_PROP_PRIMARYCACHE), primary_cache_changed_cb, os); @@ -413,6 +424,8 @@ dmu_objset_open_impl(spa_t *spa, dsl_dat recordsize_changed_cb, os); } } + if (needlock) + dsl_pool_config_exit(dmu_objset_pool(os), FTAG); if (err != 0) { VERIFY(arc_buf_remove_ref(os->os_phys_buf, &os->os_phys_buf)); @@ -469,6 +482,13 @@ dmu_objset_from_ds(dsl_dataset_t *ds, ob { int err = 0; + /* + * We shouldn't be doing anything with dsl_dataset_t's unless the + * pool_config lock is held, or the dataset is long-held. + */ + ASSERT(dsl_pool_config_held(ds->ds_dir->dd_pool) || + dsl_dataset_long_held(ds)); + mutex_enter(&ds->ds_opening_lock); if (ds->ds_objset == NULL) { objset_t *os; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Wed Oct 14 11:36:55 2015 (r289312) @@ -60,12 +60,14 @@ int zfs_send_queue_length = 16 * 1024 * int zfs_recv_queue_length = 16 * 1024 * 1024; static char *dmu_recv_tag = "dmu_recv_tag"; -static const char *recv_clone_name = "%recv"; +const char *recv_clone_name = "%recv"; #define BP_SPAN(datablkszsec, indblkshift, level) \ (((uint64_t)datablkszsec) << (SPA_MINBLOCKSHIFT + \ (level) * (indblkshift - SPA_BLKPTRSHIFT))) +static void byteswap_record(dmu_replay_record_t *drr); + struct send_thread_arg { bqueue_t q; dsl_dataset_t *ds; /* Dataset to traverse */ @@ -73,6 +75,7 @@ struct send_thread_arg { int flags; /* flags to pass to traverse_dataset */ int error_code; boolean_t cancel; + zbookmark_phys_t resume; }; struct send_block_record { @@ -87,7 +90,7 @@ struct send_block_record { static int dump_bytes(dmu_sendarg_t *dsp, void *buf, int len) { - dsl_dataset_t *ds = dsp->dsa_os->os_dsl_dataset; + dsl_dataset_t *ds = dmu_objset_ds(dsp->dsa_os); ssize_t resid; /* have to get resid to get detailed errno */ ASSERT0(len % 8); @@ -145,7 +148,7 @@ dump_free(dmu_sendarg_t *dsp, uint64_t o * that the receiving system doesn't have any dbufs in the range * being freed. This is always true because there is a one-record * constraint: we only send one WRITE record for any given - * object+offset. We know that the one-record constraint is + * object,offset. We know that the one-record constraint is * true because we always send data in increasing order by * object,offset. * @@ -394,6 +397,19 @@ dump_dnode(dmu_sendarg_t *dsp, uint64_t { struct drr_object *drro = &(dsp->dsa_drr->drr_u.drr_object); + if (object < dsp->dsa_resume_object) { + /* + * Note: when resuming, we will visit all the dnodes in + * the block of dnodes that we are resuming from. In + * this case it's unnecessary to send the dnodes prior to + * the one we are resuming from. We should be at most one + * block's worth of dnodes behind the resume point. + */ + ASSERT3U(dsp->dsa_resume_object - object, <, + 1 << (DNODE_BLOCK_SHIFT - DNODE_SHIFT)); + return (0); + } + if (dnp == NULL || dnp->dn_type == DMU_OT_NONE) return (dump_freeobjects(dsp, object, 1)); @@ -474,6 +490,9 @@ send_cb(spa_t *spa, zilog_t *zilog, cons uint64_t record_size; int err = 0; + ASSERT(zb->zb_object == DMU_META_DNODE_OBJECT || + zb->zb_object >= sta->resume.zb_object); + if (sta->cancel) return (SET_ERROR(EINTR)); @@ -510,8 +529,10 @@ send_traverse_thread(void *arg) struct send_block_record *data; if (st_arg->ds != NULL) { - err = traverse_dataset(st_arg->ds, st_arg->fromtxg, - st_arg->flags, send_cb, arg); + err = traverse_dataset_resume(st_arg->ds, + st_arg->fromtxg, &st_arg->resume, + st_arg->flags, send_cb, st_arg); + if (err != EINTR) st_arg->error_code = err; } @@ -539,6 +560,9 @@ do_dump(dmu_sendarg_t *dsa, struct send_ ASSERT3U(zb->zb_level, >=, 0); + ASSERT(zb->zb_object == DMU_META_DNODE_OBJECT || + zb->zb_object >= dsa->dsa_resume_object); + if (zb->zb_object != DMU_META_DNODE_OBJECT && DMU_OBJECT_IS_SPECIAL(zb->zb_object)) { return (0); @@ -599,6 +623,10 @@ do_dump(dmu_sendarg_t *dsa, struct send_ uint64_t offset; ASSERT0(zb->zb_level); + ASSERT(zb->zb_object > dsa->dsa_resume_object || + (zb->zb_object == dsa->dsa_resume_object && + zb->zb_blkid * blksz >= dsa->dsa_resume_offset)); + if (arc_read(NULL, spa, bp, arc_getbuf_func, &abuf, ZIO_PRIORITY_ASYNC_READ, ZIO_FLAG_CANFAIL, &aflags, zb) != 0) { @@ -659,8 +687,10 @@ get_next_record(bqueue_t *bq, struct sen */ static int dmu_send_impl(void *tag, dsl_pool_t *dp, dsl_dataset_t *to_ds, - zfs_bookmark_phys_t *ancestor_zb, boolean_t is_clone, boolean_t embedok, - boolean_t large_block_ok, int outfd, vnode_t *vp, offset_t *off) + zfs_bookmark_phys_t *ancestor_zb, + boolean_t is_clone, boolean_t embedok, boolean_t large_block_ok, int outfd, + uint64_t resumeobj, uint64_t resumeoff, + vnode_t *vp, offset_t *off) { objset_t *os; dmu_replay_record_t *drr; @@ -668,7 +698,7 @@ dmu_send_impl(void *tag, dsl_pool_t *dp, int err; uint64_t fromtxg = 0; uint64_t featureflags = 0; - struct send_thread_arg to_arg; + struct send_thread_arg to_arg = { 0 }; err = dmu_objset_from_ds(to_ds, &os); if (err != 0) { @@ -705,6 +735,10 @@ dmu_send_impl(void *tag, dsl_pool_t *dp, featureflags |= DMU_BACKUP_FEATURE_EMBED_DATA_LZ4; } + if (resumeobj != 0 || resumeoff != 0) { + featureflags |= DMU_BACKUP_FEATURE_RESUMING; + } + DMU_SET_FEATUREFLAGS(drr->drr_u.drr_begin.drr_versioninfo, featureflags); @@ -740,6 +774,8 @@ dmu_send_impl(void *tag, dsl_pool_t *dp, dsp->dsa_pending_op = PENDING_NONE; dsp->dsa_incremental = (ancestor_zb != NULL); dsp->dsa_featureflags = featureflags; + dsp->dsa_resume_object = resumeobj; + dsp->dsa_resume_offset = resumeoff; mutex_enter(&to_ds->ds_sendstream_lock); list_insert_head(&to_ds->ds_sendstreams, dsp); @@ -748,7 +784,27 @@ dmu_send_impl(void *tag, dsl_pool_t *dp, dsl_dataset_long_hold(to_ds, FTAG); dsl_pool_rele(dp, tag); - if (dump_record(dsp, NULL, 0) != 0) { + void *payload = NULL; + size_t payload_len = 0; + if (resumeobj != 0 || resumeoff != 0) { + dmu_object_info_t to_doi; + err = dmu_object_info(os, resumeobj, &to_doi); + if (err != 0) + goto out; + SET_BOOKMARK(&to_arg.resume, to_ds->ds_object, resumeobj, 0, + resumeoff / to_doi.doi_data_block_size); + + nvlist_t *nvl = fnvlist_alloc(); + fnvlist_add_uint64(nvl, "resume_object", resumeobj); + fnvlist_add_uint64(nvl, "resume_offset", resumeoff); + payload = fnvlist_pack(nvl, &payload_len); + drr->drr_payloadlen = payload_len; + fnvlist_free(nvl); + } + + err = dump_record(dsp, payload, payload_len); + fnvlist_pack_free(payload, payload_len); + if (err != 0) { err = dsp->dsa_err; goto out; } @@ -859,19 +915,19 @@ dmu_send_obj(const char *pool, uint64_t is_clone = (fromds->ds_dir != ds->ds_dir); dsl_dataset_rele(fromds, FTAG); err = dmu_send_impl(FTAG, dp, ds, &zb, is_clone, - embedok, large_block_ok, outfd, vp, off); + embedok, large_block_ok, outfd, 0, 0, vp, off); } else { err = dmu_send_impl(FTAG, dp, ds, NULL, B_FALSE, - embedok, large_block_ok, outfd, vp, off); + embedok, large_block_ok, outfd, 0, 0, vp, off); } dsl_dataset_rele(ds, FTAG); return (err); } int -dmu_send(const char *tosnap, const char *fromsnap, - boolean_t embedok, boolean_t large_block_ok, - int outfd, vnode_t *vp, offset_t *off) +dmu_send(const char *tosnap, const char *fromsnap, boolean_t embedok, + boolean_t large_block_ok, int outfd, uint64_t resumeobj, uint64_t resumeoff, + vnode_t *vp, offset_t *off) { dsl_pool_t *dp; dsl_dataset_t *ds; @@ -938,10 +994,12 @@ dmu_send(const char *tosnap, const char return (err); } err = dmu_send_impl(FTAG, dp, ds, &zb, is_clone, - embedok, large_block_ok, outfd, vp, off); + embedok, large_block_ok, + outfd, resumeobj, resumeoff, vp, off); } else { err = dmu_send_impl(FTAG, dp, ds, NULL, B_FALSE, - embedok, large_block_ok, outfd, vp, off); + embedok, large_block_ok, + outfd, resumeobj, resumeoff, vp, off); } if (owned) dsl_dataset_disown(ds, FTAG); @@ -1184,6 +1242,7 @@ dmu_recv_begin_check(void *arg, dmu_tx_t /* already checked */ ASSERT3U(drrb->drr_magic, ==, DMU_BACKUP_MAGIC); + ASSERT(!(featureflags & DMU_BACKUP_FEATURE_RESUMING)); if (DMU_GET_STREAM_HDRTYPE(drrb->drr_versioninfo) == DMU_COMPOUNDSTREAM || @@ -1196,6 +1255,10 @@ dmu_recv_begin_check(void *arg, dmu_tx_t spa_version(dp->dp_spa) < SPA_VERSION_SA) return (SET_ERROR(ENOTSUP)); + if (drba->drba_cookie->drc_resumable && + !spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_EXTENSIBLE_DATASET)) + return (SET_ERROR(ENOTSUP)); + /* * The receiving code doesn't know how to translate a WRITE_EMBEDDED * record to a plan WRITE record, so the pool must have the @@ -1299,15 +1362,16 @@ dmu_recv_begin_sync(void *arg, dmu_tx_t { dmu_recv_begin_arg_t *drba = arg; dsl_pool_t *dp = dmu_tx_pool(tx); + objset_t *mos = dp->dp_meta_objset; struct drr_begin *drrb = drba->drba_cookie->drc_drrb; const char *tofs = drba->drba_cookie->drc_tofs; dsl_dataset_t *ds, *newds; uint64_t dsobj; int error; - uint64_t crflags; + uint64_t crflags = 0; - crflags = (drrb->drr_flags & DRR_FLAG_CI_DATA) ? - DS_FLAG_CI_DATASET : 0; + if (drrb->drr_flags & DRR_FLAG_CI_DATA) + crflags |= DS_FLAG_CI_DATASET; error = dsl_dataset_hold(dp, tofs, FTAG, &ds); if (error == 0) { @@ -1345,6 +1409,31 @@ dmu_recv_begin_sync(void *arg, dmu_tx_t } VERIFY0(dsl_dataset_own_obj(dp, dsobj, dmu_recv_tag, &newds)); + if (drba->drba_cookie->drc_resumable) { + dsl_dataset_zapify(newds, tx); + if (drrb->drr_fromguid != 0) { + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_FROMGUID, + 8, 1, &drrb->drr_fromguid, tx)); + } + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_TOGUID, + 8, 1, &drrb->drr_toguid, tx)); + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_TONAME, + 1, strlen(drrb->drr_toname) + 1, drrb->drr_toname, tx)); + uint64_t one = 1; + uint64_t zero = 0; + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_OBJECT, + 8, 1, &one, tx)); + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_OFFSET, + 8, 1, &zero, tx)); + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_BYTES, + 8, 1, &zero, tx)); + if (DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo) & + DMU_BACKUP_FEATURE_EMBED_DATA) { + VERIFY0(zap_add(mos, dsobj, DS_FIELD_RESUME_EMBEDOK, + 8, 1, &one, tx)); + } + } + dmu_buf_will_dirty(newds->ds_dbuf, tx); dsl_dataset_phys(newds)->ds_flags |= DS_FLAG_INCONSISTENT; @@ -1362,56 +1451,192 @@ dmu_recv_begin_sync(void *arg, dmu_tx_t spa_history_log_internal_ds(newds, "receive", tx, ""); } +static int +dmu_recv_resume_begin_check(void *arg, dmu_tx_t *tx) +{ + dmu_recv_begin_arg_t *drba = arg; + dsl_pool_t *dp = dmu_tx_pool(tx); + struct drr_begin *drrb = drba->drba_cookie->drc_drrb; + int error; + uint64_t featureflags = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo); + dsl_dataset_t *ds; + const char *tofs = drba->drba_cookie->drc_tofs; + + /* already checked */ + ASSERT3U(drrb->drr_magic, ==, DMU_BACKUP_MAGIC); + ASSERT(featureflags & DMU_BACKUP_FEATURE_RESUMING); + + if (DMU_GET_STREAM_HDRTYPE(drrb->drr_versioninfo) == + DMU_COMPOUNDSTREAM || + drrb->drr_type >= DMU_OST_NUMTYPES) + return (SET_ERROR(EINVAL)); + + /* Verify pool version supports SA if SA_SPILL feature set */ + if ((featureflags & DMU_BACKUP_FEATURE_SA_SPILL) && + spa_version(dp->dp_spa) < SPA_VERSION_SA) + return (SET_ERROR(ENOTSUP)); + + /* + * The receiving code doesn't know how to translate a WRITE_EMBEDDED + * record to a plain WRITE record, so the pool must have the + * EMBEDDED_DATA feature enabled if the stream has WRITE_EMBEDDED + * records. Same with WRITE_EMBEDDED records that use LZ4 compression. + */ + if ((featureflags & DMU_BACKUP_FEATURE_EMBED_DATA) && + !spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_EMBEDDED_DATA)) + return (SET_ERROR(ENOTSUP)); + if ((featureflags & DMU_BACKUP_FEATURE_EMBED_DATA_LZ4) && + !spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_LZ4_COMPRESS)) + return (SET_ERROR(ENOTSUP)); + + char recvname[ZFS_MAXNAMELEN]; + + (void) snprintf(recvname, sizeof (recvname), "%s/%s", + tofs, recv_clone_name); + + if (dsl_dataset_hold(dp, recvname, FTAG, &ds) != 0) { + /* %recv does not exist; continue in tofs */ + error = dsl_dataset_hold(dp, tofs, FTAG, &ds); + if (error != 0) + return (error); + } + + /* check that ds is marked inconsistent */ + if (!DS_IS_INCONSISTENT(ds)) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* check that there is resuming data, and that the toguid matches */ + if (!dsl_dataset_is_zapified(ds)) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EINVAL)); + } + uint64_t val; + error = zap_lookup(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_TOGUID, sizeof (val), 1, &val); + if (error != 0 || drrb->drr_toguid != val) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* + * Check if the receive is still running. If so, it will be owned. + * Note that nothing else can own the dataset (e.g. after the receive + * fails) because it will be marked inconsistent. + */ + if (dsl_dataset_has_owner(ds)) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EBUSY)); + } + + /* There should not be any snapshots of this fs yet. */ + if (ds->ds_prev != NULL && ds->ds_prev->ds_dir == ds->ds_dir) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* + * Note: resume point will be checked when we process the first WRITE + * record. + */ + + /* check that the origin matches */ + val = 0; + (void) zap_lookup(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_FROMGUID, sizeof (val), 1, &val); + if (drrb->drr_fromguid != val) { + dsl_dataset_rele(ds, FTAG); + return (SET_ERROR(EINVAL)); + } + + dsl_dataset_rele(ds, FTAG); + return (0); +} + +static void +dmu_recv_resume_begin_sync(void *arg, dmu_tx_t *tx) +{ + dmu_recv_begin_arg_t *drba = arg; + dsl_pool_t *dp = dmu_tx_pool(tx); + const char *tofs = drba->drba_cookie->drc_tofs; + dsl_dataset_t *ds; + uint64_t dsobj; + char recvname[ZFS_MAXNAMELEN]; + + (void) snprintf(recvname, sizeof (recvname), "%s/%s", + tofs, recv_clone_name); + + if (dsl_dataset_hold(dp, recvname, FTAG, &ds) != 0) { + /* %recv does not exist; continue in tofs */ + VERIFY0(dsl_dataset_hold(dp, tofs, FTAG, &ds)); + drba->drba_cookie->drc_newfs = B_TRUE; + } + + /* clear the inconsistent flag so that we can own it */ + ASSERT(DS_IS_INCONSISTENT(ds)); + dmu_buf_will_dirty(ds->ds_dbuf, tx); + dsl_dataset_phys(ds)->ds_flags &= ~DS_FLAG_INCONSISTENT; + dsobj = ds->ds_object; + dsl_dataset_rele(ds, FTAG); + + VERIFY0(dsl_dataset_own_obj(dp, dsobj, dmu_recv_tag, &ds)); + + dmu_buf_will_dirty(ds->ds_dbuf, tx); + dsl_dataset_phys(ds)->ds_flags |= DS_FLAG_INCONSISTENT; + + ASSERT(!BP_IS_HOLE(dsl_dataset_get_blkptr(ds))); + + drba->drba_cookie->drc_ds = ds; + + spa_history_log_internal_ds(ds, "resume receive", tx, ""); +} + /* * NB: callers *MUST* call dmu_recv_stream() if dmu_recv_begin() * succeeds; otherwise we will leak the holds on the datasets. */ int -dmu_recv_begin(char *tofs, char *tosnap, struct drr_begin *drrb, - boolean_t force, char *origin, dmu_recv_cookie_t *drc) +dmu_recv_begin(char *tofs, char *tosnap, dmu_replay_record_t *drr_begin, + boolean_t force, boolean_t resumable, char *origin, dmu_recv_cookie_t *drc) { dmu_recv_begin_arg_t drba = { 0 }; - dmu_replay_record_t *drr; bzero(drc, sizeof (dmu_recv_cookie_t)); - drc->drc_drrb = drrb; + drc->drc_drr_begin = drr_begin; + drc->drc_drrb = &drr_begin->drr_u.drr_begin; drc->drc_tosnap = tosnap; drc->drc_tofs = tofs; drc->drc_force = force; + drc->drc_resumable = resumable; drc->drc_cred = CRED(); - if (drrb->drr_magic == BSWAP_64(DMU_BACKUP_MAGIC)) + if (drc->drc_drrb->drr_magic == BSWAP_64(DMU_BACKUP_MAGIC)) { drc->drc_byteswap = B_TRUE; - else if (drrb->drr_magic != DMU_BACKUP_MAGIC) - return (SET_ERROR(EINVAL)); - - drr = kmem_zalloc(sizeof (dmu_replay_record_t), KM_SLEEP); - drr->drr_type = DRR_BEGIN; - drr->drr_u.drr_begin = *drc->drc_drrb; - if (drc->drc_byteswap) { - fletcher_4_incremental_byteswap(drr, + fletcher_4_incremental_byteswap(drr_begin, sizeof (dmu_replay_record_t), &drc->drc_cksum); - } else { - fletcher_4_incremental_native(drr, + byteswap_record(drr_begin); + } else if (drc->drc_drrb->drr_magic == DMU_BACKUP_MAGIC) { + fletcher_4_incremental_native(drr_begin, sizeof (dmu_replay_record_t), &drc->drc_cksum); - } - kmem_free(drr, sizeof (dmu_replay_record_t)); - - if (drc->drc_byteswap) { - drrb->drr_magic = BSWAP_64(drrb->drr_magic); - drrb->drr_versioninfo = BSWAP_64(drrb->drr_versioninfo); - drrb->drr_creation_time = BSWAP_64(drrb->drr_creation_time); - drrb->drr_type = BSWAP_32(drrb->drr_type); - drrb->drr_toguid = BSWAP_64(drrb->drr_toguid); - drrb->drr_fromguid = BSWAP_64(drrb->drr_fromguid); + } else { + return (SET_ERROR(EINVAL)); } drba.drba_origin = origin; drba.drba_cookie = drc; drba.drba_cred = CRED(); - return (dsl_sync_task(tofs, dmu_recv_begin_check, dmu_recv_begin_sync, - &drba, 5, ZFS_SPACE_CHECK_NORMAL)); + if (DMU_GET_FEATUREFLAGS(drc->drc_drrb->drr_versioninfo) & + DMU_BACKUP_FEATURE_RESUMING) { + return (dsl_sync_task(tofs, + dmu_recv_resume_begin_check, dmu_recv_resume_begin_sync, + &drba, 5, ZFS_SPACE_CHECK_NORMAL)); + } else { + return (dsl_sync_task(tofs, + dmu_recv_begin_check, dmu_recv_begin_sync, + &drba, 5, ZFS_SPACE_CHECK_NORMAL)); + } } struct receive_record_arg { @@ -1423,6 +1648,7 @@ struct receive_record_arg { */ arc_buf_t *write_buf; int payload_size; + uint64_t bytes_read; /* bytes read from stream when record created */ boolean_t eos_marker; /* Marks the end of the stream */ bqueue_node_t node; }; @@ -1431,6 +1657,7 @@ struct receive_writer_arg { objset_t *os; boolean_t byteswap; bqueue_t q; + /* * These three args are used to signal to the main thread that we're * done. @@ -1438,15 +1665,20 @@ struct receive_writer_arg { kmutex_t mutex; kcondvar_t cv; boolean_t done; + int err; /* A map from guid to dataset to help handle dedup'd streams. */ avl_tree_t *guid_to_ds_map; + boolean_t resumable; + uint64_t last_object, last_offset; + uint64_t bytes_read; /* bytes read when current record created */ }; struct receive_arg { objset_t *os; vnode_t *vp; /* The vnode to read the stream from */ uint64_t voff; /* The current offset in the stream */ + uint64_t bytes_read; /* * A record that has had its payload read in, but hasn't yet been handed * off to the worker thread. @@ -1518,14 +1750,21 @@ receive_read(struct receive_arg *ra, int ra->voff, UIO_SYSSPACE, FAPPEND, RLIM64_INFINITY, CRED(), &resid); - if (resid == len - done) - ra->err = SET_ERROR(EINVAL); + if (resid == len - done) { + /* + * Note: ECKSUM indicates that the receive + * was interrupted and can potentially be resumed. + */ + ra->err = SET_ERROR(ECKSUM); + } ra->voff += len - done - resid; done = len - resid; if (ra->err != 0) return (ra->err); } + ra->bytes_read += len; + ASSERT3U(done, ==, len); return (0); } @@ -1626,6 +1865,43 @@ deduce_nblkptr(dmu_object_type_t bonus_t } } +static void +save_resume_state(struct receive_writer_arg *rwa, + uint64_t object, uint64_t offset, dmu_tx_t *tx) +{ + int txgoff = dmu_tx_get_txg(tx) & TXG_MASK; + + if (!rwa->resumable) + return; + + /* + * We use ds_resume_bytes[] != 0 to indicate that we need to + * update this on disk, so it must not be 0. + */ + ASSERT(rwa->bytes_read != 0); + + /* + * We only resume from write records, which have a valid + * (non-meta-dnode) object number. + */ + ASSERT(object != 0); + + /* + * For resuming to work correctly, we must receive records in order, + * sorted by object,offset. This is checked by the callers, but + * assert it here for good measure. + */ + ASSERT3U(object, >=, rwa->os->os_dsl_dataset->ds_resume_object[txgoff]); + ASSERT(object != rwa->os->os_dsl_dataset->ds_resume_object[txgoff] || + offset >= rwa->os->os_dsl_dataset->ds_resume_offset[txgoff]); + ASSERT3U(rwa->bytes_read, >=, + rwa->os->os_dsl_dataset->ds_resume_bytes[txgoff]); + + rwa->os->os_dsl_dataset->ds_resume_object[txgoff] = object; + rwa->os->os_dsl_dataset->ds_resume_offset[txgoff] = offset; + rwa->os->os_dsl_dataset->ds_resume_bytes[txgoff] = rwa->bytes_read; +} + static int receive_object(struct receive_writer_arg *rwa, struct drr_object *drro, void *data) @@ -1722,6 +1998,7 @@ receive_object(struct receive_writer_arg dmu_buf_rele(db, FTAG); } dmu_tx_commit(tx); + return (0); } @@ -1747,6 +2024,7 @@ receive_freeobjects(struct receive_write if (err != 0) return (err); } + return (0); } @@ -1761,6 +2039,18 @@ receive_write(struct receive_writer_arg !DMU_OT_IS_VALID(drrw->drr_type)) return (SET_ERROR(EINVAL)); + /* + * For resuming to work, records must be in increasing order + * by (object, offset). + */ + if (drrw->drr_object < rwa->last_object || + (drrw->drr_object == rwa->last_object && + drrw->drr_offset < rwa->last_offset)) { + return (SET_ERROR(EINVAL)); + } + rwa->last_object = drrw->drr_object; + rwa->last_offset = drrw->drr_offset; + if (dmu_object_info(rwa->os, drrw->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); @@ -1784,8 +2074,17 @@ receive_write(struct receive_writer_arg if (dmu_bonus_hold(rwa->os, drrw->drr_object, FTAG, &bonus) != 0) return (SET_ERROR(EINVAL)); dmu_assign_arcbuf(bonus, drrw->drr_offset, abuf, tx); + + /* + * Note: If the receive fails, we want the resume stream to start + * with the same record that we last successfully received (as opposed + * to the next record), so that we can verify that we are + * resuming from the correct location. + */ + save_resume_state(rwa, drrw->drr_object, drrw->drr_offset, tx); dmu_tx_commit(tx); dmu_buf_rele(bonus, FTAG); + return (0); } @@ -1844,43 +2143,48 @@ receive_write_byref(struct receive_write dmu_write(rwa->os, drrwbr->drr_object, drrwbr->drr_offset, drrwbr->drr_length, dbp->db_data, tx); dmu_buf_rele(dbp, FTAG); + + /* See comment in restore_write. */ + save_resume_state(rwa, drrwbr->drr_object, drrwbr->drr_offset, tx); dmu_tx_commit(tx); return (0); } static int receive_write_embedded(struct receive_writer_arg *rwa, - struct drr_write_embedded *drrwnp, void *data) + struct drr_write_embedded *drrwe, void *data) { dmu_tx_t *tx; int err; - if (drrwnp->drr_offset + drrwnp->drr_length < drrwnp->drr_offset) + if (drrwe->drr_offset + drrwe->drr_length < drrwe->drr_offset) return (EINVAL); - if (drrwnp->drr_psize > BPE_PAYLOAD_SIZE) + if (drrwe->drr_psize > BPE_PAYLOAD_SIZE) return (EINVAL); - if (drrwnp->drr_etype >= NUM_BP_EMBEDDED_TYPES) + if (drrwe->drr_etype >= NUM_BP_EMBEDDED_TYPES) return (EINVAL); - if (drrwnp->drr_compression >= ZIO_COMPRESS_FUNCTIONS) + if (drrwe->drr_compression >= ZIO_COMPRESS_FUNCTIONS) return (EINVAL); tx = dmu_tx_create(rwa->os); - dmu_tx_hold_write(tx, drrwnp->drr_object, - drrwnp->drr_offset, drrwnp->drr_length); + dmu_tx_hold_write(tx, drrwe->drr_object, + drrwe->drr_offset, drrwe->drr_length); err = dmu_tx_assign(tx, TXG_WAIT); if (err != 0) { dmu_tx_abort(tx); return (err); } - dmu_write_embedded(rwa->os, drrwnp->drr_object, - drrwnp->drr_offset, data, drrwnp->drr_etype, - drrwnp->drr_compression, drrwnp->drr_lsize, drrwnp->drr_psize, + dmu_write_embedded(rwa->os, drrwe->drr_object, + drrwe->drr_offset, data, drrwe->drr_etype, + drrwe->drr_compression, drrwe->drr_lsize, drrwe->drr_psize, rwa->byteswap ^ ZFS_HOST_BYTEORDER, tx); + /* See comment in restore_write. */ + save_resume_state(rwa, drrwe->drr_object, drrwe->drr_offset, tx); dmu_tx_commit(tx); return (0); } @@ -1954,10 +2258,16 @@ receive_free(struct receive_writer_arg * static void dmu_recv_cleanup_ds(dmu_recv_cookie_t *drc) { - char name[MAXNAMELEN]; - dsl_dataset_name(drc->drc_ds, name); - dsl_dataset_disown(drc->drc_ds, dmu_recv_tag); - (void) dsl_destroy_head(name); + if (drc->drc_resumable) { + /* wait for our resume state to be written to disk */ + txg_wait_synced(drc->drc_ds->ds_dir->dd_pool, 0); + dsl_dataset_disown(drc->drc_ds, dmu_recv_tag); + } else { + char name[MAXNAMELEN]; + dsl_dataset_name(drc->drc_ds, name); + dsl_dataset_disown(drc->drc_ds, dmu_recv_tag); + (void) dsl_destroy_head(name); + } } static void @@ -1984,12 +2294,17 @@ receive_read_payload_and_next_header(str if (len != 0) { ASSERT3U(len, <=, SPA_MAXBLOCKSIZE); - ra->rrd->payload = buf; - ra->rrd->payload_size = len; - err = receive_read(ra, len, ra->rrd->payload); + err = receive_read(ra, len, buf); if (err != 0) return (err); - receive_cksum(ra, len, ra->rrd->payload); + receive_cksum(ra, len, buf); + + /* note: rrd is NULL when reading the begin record's payload */ + if (ra->rrd != NULL) { + ra->rrd->payload = buf; + ra->rrd->payload_size = len; + ra->rrd->bytes_read = ra->bytes_read; + } } ra->prev_cksum = ra->cksum; @@ -1997,6 +2312,7 @@ receive_read_payload_and_next_header(str ra->next_rrd = kmem_zalloc(sizeof (*ra->next_rrd), KM_SLEEP); err = receive_read(ra, sizeof (ra->next_rrd->header), &ra->next_rrd->header); + ra->next_rrd->bytes_read = ra->bytes_read; if (err != 0) { kmem_free(ra->next_rrd, sizeof (*ra->next_rrd)); ra->next_rrd = NULL; @@ -2176,7 +2492,7 @@ receive_read_record(struct receive_arg * { struct drr_end *drre = &ra->rrd->header.drr_u.drr_end; if (!ZIO_CHECKSUM_EQUAL(ra->prev_cksum, drre->drr_checksum)) - return (SET_ERROR(EINVAL)); + return (SET_ERROR(ECKSUM)); return (0); } case DRR_SPILL: @@ -2203,6 +2519,10 @@ receive_process_record(struct receive_wr { int err; + /* Processing in order, therefore bytes_read should be increasing. */ + ASSERT3U(rrd->bytes_read, >=, rwa->bytes_read); + rwa->bytes_read = rrd->bytes_read; + switch (rrd->header.drr_type) { case DRR_OBJECT: { @@ -2297,6 +2617,33 @@ receive_writer_thread(void *arg) mutex_exit(&rwa->mutex); } +static int +resume_check(struct receive_arg *ra, nvlist_t *begin_nvl) +{ + uint64_t val; + objset_t *mos = dmu_objset_pool(ra->os)->dp_meta_objset; + uint64_t dsobj = dmu_objset_id(ra->os); + uint64_t resume_obj, resume_off; + + if (nvlist_lookup_uint64(begin_nvl, + "resume_object", &resume_obj) != 0 || + nvlist_lookup_uint64(begin_nvl, + "resume_offset", &resume_off) != 0) { + return (SET_ERROR(EINVAL)); + } + VERIFY0(zap_lookup(mos, dsobj, + DS_FIELD_RESUME_OBJECT, sizeof (val), 1, &val)); + if (resume_obj != val) + return (SET_ERROR(EINVAL)); + VERIFY0(zap_lookup(mos, dsobj, + DS_FIELD_RESUME_OFFSET, sizeof (val), 1, &val)); + if (resume_off != val) + return (SET_ERROR(EINVAL)); + + return (0); +} + + /* * Read in the stream's records, one by one, and apply them to the pool. There * are two threads involved; the thread that calls this function will spin up a @@ -2317,11 +2664,19 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, struct receive_arg ra = { 0 }; struct receive_writer_arg rwa = { 0 }; int featureflags; + nvlist_t *begin_nvl = NULL; ra.byteswap = drc->drc_byteswap; ra.cksum = drc->drc_cksum; ra.vp = vp; ra.voff = *voffp; + + if (dsl_dataset_is_zapified(drc->drc_ds)) { + (void) zap_lookup(drc->drc_ds->ds_dir->dd_pool->dp_meta_objset, + drc->drc_ds->ds_object, DS_FIELD_RESUME_BYTES, + sizeof (ra.bytes_read), 1, &ra.bytes_read); + } + list_create(&ra.ignore_obj_list, sizeof (struct receive_ign_obj_node), offsetof(struct receive_ign_obj_node, node)); @@ -2374,9 +2729,29 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, drc->drc_guid_to_ds_map = rwa.guid_to_ds_map; } - err = receive_read_payload_and_next_header(&ra, 0, NULL); - if (err) + uint32_t payloadlen = drc->drc_drr_begin->drr_payloadlen; + void *payload = NULL; + if (payloadlen != 0) + payload = kmem_alloc(payloadlen, KM_SLEEP); + + err = receive_read_payload_and_next_header(&ra, payloadlen, payload); + if (err != 0) { + if (payloadlen != 0) + kmem_free(payload, payloadlen); goto out; + } + if (payloadlen != 0) { + err = nvlist_unpack(payload, payloadlen, &begin_nvl, KM_SLEEP); + kmem_free(payload, payloadlen); + if (err != 0) + goto out; + } + + if (featureflags & DMU_BACKUP_FEATURE_RESUMING) { + err = resume_check(&ra, begin_nvl); + if (err != 0) + goto out; + } (void) bqueue_init(&rwa.q, zfs_recv_queue_length, offsetof(struct receive_record_arg, node)); @@ -2384,6 +2759,7 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, mutex_init(&rwa.mutex, NULL, MUTEX_DEFAULT, NULL); rwa.os = ra.os; rwa.byteswap = drc->drc_byteswap; + rwa.resumable = drc->drc_resumable; (void) thread_create(NULL, 0, receive_writer_thread, &rwa, 0, curproc, TS_RUN, minclsyspri); @@ -2442,13 +2818,15 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, err = rwa.err; out: + nvlist_free(begin_nvl); if ((featureflags & DMU_BACKUP_FEATURE_DEDUP) && (cleanup_fd != -1)) zfs_onexit_fd_rele(cleanup_fd); if (err != 0) { /* - * destroy what we created, so we don't leave it in the - * inconsistent restoring state. + * Clean up references. If receive is not resumable, + * destroy what we created, so we don't leave it in + * the inconsistent state. */ dmu_recv_cleanup_ds(drc); } @@ -2608,6 +2986,20 @@ dmu_recv_end_sync(void *arg, dmu_tx_t *t dmu_buf_will_dirty(ds->ds_dbuf, tx); dsl_dataset_phys(ds)->ds_flags &= ~DS_FLAG_INCONSISTENT; + if (dsl_dataset_has_resume_receive_state(ds)) { + (void) zap_remove(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_FROMGUID, tx); + (void) zap_remove(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_OBJECT, tx); + (void) zap_remove(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_OFFSET, tx); + (void) zap_remove(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_BYTES, tx); + (void) zap_remove(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_TOGUID, tx); + (void) zap_remove(dp->dp_meta_objset, ds->ds_object, + DS_FIELD_RESUME_TONAME, tx); + } } drc->drc_newsnapobj = dsl_dataset_phys(drc->drc_ds)->ds_prev_snap_obj; /* Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c Wed Oct 14 11:33:07 2015 (r289311) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c Wed Oct 14 11:36:55 2015 (r289312) @@ -47,6 +47,7 @@ typedef struct prefetch_data { int pd_flags; boolean_t pd_cancel; boolean_t pd_exited; + zbookmark_phys_t pd_resume; } prefetch_data_t; typedef struct traverse_data { @@ -311,33 +312,30 @@ traverse_visitbp(traverse_data_t *td, co ZIO_PRIORITY_ASYNC_READ, ZIO_FLAG_CANFAIL, &flags, zb); if (err != 0) *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@freebsd.org Thu Oct 15 15:20:13 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16184A15032; Thu, 15 Oct 2015 15:20:13 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B546E1E38; Thu, 15 Oct 2015 15:20:08 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA05156; Thu, 15 Oct 2015 18:19:59 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZmkJz-000ML9-HX; Thu, 15 Oct 2015 18:19:59 +0300 Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... To: Alexander Motin , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org References: <201510141112.t9EBClT3022215@repo.freebsd.org> From: Andriy Gapon Message-ID: <561FC3EC.7020706@FreeBSD.org> Date: Thu, 15 Oct 2015 17:19:08 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <201510141112.t9EBClT3022215@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 15:20:13 -0000 On 14/10/2015 13:12, Alexander Motin wrote: > Author: mav > Date: Wed Oct 14 11:12:47 2015 > New Revision: 289310 > URL: https://svnweb.freebsd.org/changeset/base/289310 > > Log: > 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R Thank you very much for importing this complex commit. I have one question below. > Reviewed by: George Wilson > Reviewed by: Prakash Surya > Reviewed by: Saso Kiselkov > Reviewed by: Richard Lowe > Approved by: Garrett D'Amore > Author: Matthew Ahrens > > illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f > > Added: > vendor-sys/illumos/dist/uts/common/crypto/ > vendor-sys/illumos/dist/uts/common/crypto/io/ > vendor-sys/illumos/dist/uts/common/crypto/io/edonr_mod.c (contents, props changed) > vendor-sys/illumos/dist/uts/common/crypto/io/skein_mod.c (contents, props changed) Do we actually need these two file in the vendor area? They look like illumos crypto drivers to me, so I think that it is unlikely that we will have a use for them. Or do you have some big plans about that? :-) > vendor-sys/illumos/dist/uts/common/fs/zfs/edonr_zfs.c (contents, props changed) > vendor-sys/illumos/dist/uts/common/fs/zfs/skein_zfs.c (contents, props changed) > vendor-sys/illumos/dist/uts/common/sys/edonr.h (contents, props changed) > vendor-sys/illumos/dist/uts/common/sys/skein.h (contents, props changed) -- Andriy Gapon From owner-svn-src-vendor@freebsd.org Thu Oct 15 15:27:18 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13036A152B8; Thu, 15 Oct 2015 15:27:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3B525A93; Thu, 15 Oct 2015 15:27:16 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA05269; Thu, 15 Oct 2015 18:27:14 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZmkR0-000MM4-5w; Thu, 15 Oct 2015 18:27:14 +0300 Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... To: Alexander Motin , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org References: <201510141112.t9EBCmFZ022230@repo.freebsd.org> From: Andriy Gapon X-Enigmail-Draft-Status: N1110 Message-ID: <561FC5A0.3040909@FreeBSD.org> Date: Thu, 15 Oct 2015 17:26:24 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <201510141112.t9EBCmFZ022230@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 15:27:18 -0000 On 14/10/2015 13:12, Alexander Motin wrote: > Author: mav > Date: Wed Oct 14 11:12:47 2015 > New Revision: 289310 > URL: https://svnweb.freebsd.org/changeset/base/289310 > > Log: > 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R > > Reviewed by: George Wilson > Reviewed by: Prakash Surya > Reviewed by: Saso Kiselkov > Reviewed by: Richard Lowe > Approved by: Garrett D'Amore > Author: Matthew Ahrens > > illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f > > Added: > vendor/illumos/dist/common/crypto/ > vendor/illumos/dist/common/crypto/edonr/ > vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) > vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) > vendor/illumos/dist/common/crypto/skein/ > vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) > vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) > vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) > vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) > vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) > vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) > vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) It seems that these are pieces of code that can be used by both the userland and the kernel. I think that previously we used to import such code into illumos-sys and then integrate it into sys/cddl (e.g. common/zfs/). Perhaps it is worthwhile following that convention in this case as well? > Modified: > vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c > vendor/illumos/dist/man/man5/zpool-features.5 Also, we probably need to synchronize man/man1m/zfs.1m in our vendor area with the latest upstream version. Even though we do not directly use that file, it still would be easier to make the future imports and also to compare (and to keep in sync) our version of the manual with the upstream version. Thank you for doing all the heavy lifting with this change! -- Andriy Gapon From owner-svn-src-vendor@freebsd.org Thu Oct 15 15:41:10 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6D70A15684; Thu, 15 Oct 2015 15:41:10 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qk0-x235.google.com (mail-qk0-x235.google.com [IPv6:2607:f8b0:400d:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F0B31CDE; Thu, 15 Oct 2015 15:41:07 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by qkas79 with SMTP id s79so40673099qka.0; Thu, 15 Oct 2015 08:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=KkR5hMPZ4NV5lAb9UjLHdvYoH+Axy7zKlbzrhUo2Wj8=; b=cozQg4vgjX33ezSQl5lsxraf/Nm087kbzOAamH4agRji3gKZbF6hUP7y3JkVS4dg6n 9NKdVrsORXcGM4DZNJUYj2vgBUfuir5sDd65H14Jqu7vYFqFJPQjg1nEQVtPmfxooB+T cCXc4Zmw7yy9nTw6UbRy++0Oi8g0zlbruGXrhV++oeVFefccPwft4nDCMhghn9WXnNnF o6A2Q9JFSP1svvDsneC1MJU/UVkbOgjcyYqbPn8t15x7CDdNyHItHqeM5jRvjAFUSOqy Q6vmqLBHwSMS4GFcb44dCaInHYvMQeRVgi6Siyor8DihalHUlsxYBpQxGNNYYrHAxil0 3jBg== X-Received: by 10.194.11.71 with SMTP id o7mr11514434wjb.75.1444923666389; Thu, 15 Oct 2015 08:41:06 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([2a01:d0:c0a9:3:c685:8ff:fe11:1aa2]) by smtp.googlemail.com with ESMTPSA id i10sm17163361wjz.41.2015.10.15.08.41.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Oct 2015 08:41:05 -0700 (PDT) Sender: Alexander Motin Message-ID: <561FC90F.4090706@FreeBSD.org> Date: Thu, 15 Oct 2015 18:41:03 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Andriy Gapon , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... References: <201510141112.t9EBCmFZ022230@repo.freebsd.org> <561FC5A0.3040909@FreeBSD.org> In-Reply-To: <561FC5A0.3040909@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 15:41:11 -0000 On 15.10.2015 18:26, Andriy Gapon wrote: > On 14/10/2015 13:12, Alexander Motin wrote: >> Author: mav >> Date: Wed Oct 14 11:12:47 2015 >> New Revision: 289310 >> URL: https://svnweb.freebsd.org/changeset/base/289310 >> >> Log: >> 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R >> >> Reviewed by: George Wilson >> Reviewed by: Prakash Surya >> Reviewed by: Saso Kiselkov >> Reviewed by: Richard Lowe >> Approved by: Garrett D'Amore >> Author: Matthew Ahrens >> >> illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f >> >> Added: >> vendor/illumos/dist/common/crypto/ >> vendor/illumos/dist/common/crypto/edonr/ >> vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) >> vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/ >> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) >> vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) > > It seems that these are pieces of code that can be used by both the > userland and the kernel. I think that previously we used to import such > code into illumos-sys and then integrate it into sys/cddl (e.g. > common/zfs/). Perhaps it is worthwhile following that convention in > this case as well? I haven't decided what to do with this specific case. As I understand, now we are using FreeBSD's native crypto code instead of illumos' one. I was thinking about importing this commit only in infrastructural parts, until respective algorithms are implemented in our native crypto. I think it should not be a problem for Skein, since according to comments it is in public domain. About Edon-R I am not sure, since it seems to be CDDL. -- Alexander Motin From owner-svn-src-vendor@freebsd.org Thu Oct 15 15:44:12 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 745A5A1581B; Thu, 15 Oct 2015 15:44:12 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C994BFC; Thu, 15 Oct 2015 15:44:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA05522; Thu, 15 Oct 2015 18:44:09 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1ZmkhM-000MNr-S1; Thu, 15 Oct 2015 18:44:08 +0300 Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... To: Alexander Motin , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org References: <201510141112.t9EBCmFZ022230@repo.freebsd.org> <561FC5A0.3040909@FreeBSD.org> <561FC90F.4090706@FreeBSD.org> From: Andriy Gapon Message-ID: <561FC996.3020207@FreeBSD.org> Date: Thu, 15 Oct 2015 17:43:18 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <561FC90F.4090706@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 15:44:12 -0000 On 15/10/2015 17:41, Alexander Motin wrote: > On 15.10.2015 18:26, Andriy Gapon wrote: >> On 14/10/2015 13:12, Alexander Motin wrote: >>> Author: mav >>> Date: Wed Oct 14 11:12:47 2015 >>> New Revision: 289310 >>> URL: https://svnweb.freebsd.org/changeset/base/289310 >>> >>> Log: >>> 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R >>> >>> Reviewed by: George Wilson >>> Reviewed by: Prakash Surya >>> Reviewed by: Saso Kiselkov >>> Reviewed by: Richard Lowe >>> Approved by: Garrett D'Amore >>> Author: Matthew Ahrens >>> >>> illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f >>> >>> Added: >>> vendor/illumos/dist/common/crypto/ >>> vendor/illumos/dist/common/crypto/edonr/ >>> vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) >>> vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/ >>> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) >>> vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) >> >> It seems that these are pieces of code that can be used by both the >> userland and the kernel. I think that previously we used to import such >> code into illumos-sys and then integrate it into sys/cddl (e.g. >> common/zfs/). Perhaps it is worthwhile following that convention in >> this case as well? > > I haven't decided what to do with this specific case. As I understand, > now we are using FreeBSD's native crypto code instead of illumos' one. I > was thinking about importing this commit only in infrastructural parts, > until respective algorithms are implemented in our native crypto. I > think it should not be a problem for Skein, since according to comments > it is in public domain. About Edon-R I am not sure, since it seems to be > CDDL. > This is confusing... It seems that like you replied to my question in another email, not the quoted one :-) -- Andriy Gapon From owner-svn-src-vendor@freebsd.org Thu Oct 15 15:48:56 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC7CFA1588E; Thu, 15 Oct 2015 15:48:55 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-lf0-x22f.google.com (mail-lf0-x22f.google.com [IPv6:2a00:1450:4010:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E0262AA; Thu, 15 Oct 2015 15:48:55 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by lfeh64 with SMTP id h64so32734632lfe.3; Thu, 15 Oct 2015 08:48:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=/MmDOb0XE/onoZsRyjwrB80KsCDRsESvN4IKf5SpkQk=; b=dmcs7avlMzVfsv6jNtfVqZ9UMV2SeaQ+7vLt4RgY3HrZu5Km5rlNBeE9ya3XUkv0qQ 2QLbWTESokbLGOhH6jvkR1yH1ceD2fwdRAPzx8uyd62KIGsib1RLVe/zV7/SGS8TUan2 7coOilBQXfRliieq2eWTEdMafq18hUAf7unhF5dd55MtDWW4xWr7L9fbNfTt6B+7FZWP fNszSU2AAeIZcT7Ds2QbvwZr1/2p5aJPj0XLeMqJxsis/bFJa/WARXJThBCkYg9tpt/y wQOcBJkf6vICBq+xhQhiM2SlFr5mfsuoNJ5fYsh0GYXwexKo0vz0bpZf6ly/i4eEiuDW xvig== X-Received: by 10.180.20.1 with SMTP id j1mr27954638wie.78.1444924131751; Thu, 15 Oct 2015 08:48:51 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([2a01:d0:c0a9:3:c685:8ff:fe11:1aa2]) by smtp.googlemail.com with ESMTPSA id xa5sm17196280wjc.20.2015.10.15.08.48.50 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Oct 2015 08:48:51 -0700 (PDT) Sender: Alexander Motin Message-ID: <561FCAE1.1040504@FreeBSD.org> Date: Thu, 15 Oct 2015 18:48:49 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Andriy Gapon , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... References: <201510141112.t9EBCmFZ022230@repo.freebsd.org> <561FC5A0.3040909@FreeBSD.org> <561FC90F.4090706@FreeBSD.org> <561FC996.3020207@FreeBSD.org> In-Reply-To: <561FC996.3020207@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 15:48:56 -0000 On 15.10.2015 18:43, Andriy Gapon wrote: > On 15/10/2015 17:41, Alexander Motin wrote: >> On 15.10.2015 18:26, Andriy Gapon wrote: >>> On 14/10/2015 13:12, Alexander Motin wrote: >>>> Author: mav >>>> Date: Wed Oct 14 11:12:47 2015 >>>> New Revision: 289310 >>>> URL: https://svnweb.freebsd.org/changeset/base/289310 >>>> >>>> Log: >>>> 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R >>>> >>>> Reviewed by: George Wilson >>>> Reviewed by: Prakash Surya >>>> Reviewed by: Saso Kiselkov >>>> Reviewed by: Richard Lowe >>>> Approved by: Garrett D'Amore >>>> Author: Matthew Ahrens >>>> >>>> illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f >>>> >>>> Added: >>>> vendor/illumos/dist/common/crypto/ >>>> vendor/illumos/dist/common/crypto/edonr/ >>>> vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) >>>> vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/ >>>> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) >>>> vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) >>> >>> It seems that these are pieces of code that can be used by both the >>> userland and the kernel. I think that previously we used to import such >>> code into illumos-sys and then integrate it into sys/cddl (e.g. >>> common/zfs/). Perhaps it is worthwhile following that convention in >>> this case as well? >> >> I haven't decided what to do with this specific case. As I understand, >> now we are using FreeBSD's native crypto code instead of illumos' one. I >> was thinking about importing this commit only in infrastructural parts, >> until respective algorithms are implemented in our native crypto. I >> think it should not be a problem for Skein, since according to comments >> it is in public domain. About Edon-R I am not sure, since it seems to be >> CDDL. > > This is confusing... It seems that like you replied to my question in > another email, not the quoted one :-) What other email? You've told me that I put code into the wrong place, and I replied that it may be irrelevant, since I am not sure we should import this code to FreeBSD as-is to its place in illumos. -- Alexander Motin From owner-svn-src-vendor@freebsd.org Thu Oct 15 15:49:13 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E3ADA158E2 for ; Thu, 15 Oct 2015 15:49:13 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qk0-f172.google.com (mail-qk0-f172.google.com [209.85.220.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFB6961A for ; Thu, 15 Oct 2015 15:49:12 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by qkht68 with SMTP id t68so40833971qkh.3 for ; Thu, 15 Oct 2015 08:49:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=a8gfLratcTHOa0fJRjfmQlH3ZY2EGc+xLYvGHDFRwHE=; b=aPHJl8nTe940nNXivcToQ6dJGel6YR553xOqAOiOkjX2h5xLEnA2JoJuOu6llkss63 E63pzihGorCbPzjW+8/nTSTABJjCBQRQrB4+f3uU0GO24daD8ptdaMs8P/i4MZWILN3R xKf8aZHTD1HeYSwvA/XyJBM9oSdCMENbdsOS57nz+e1aoqBHFOo4JDtYiffCfYqxFA/y YJnUDpgR5s/EgTYT8UkHCw4a1eS4sYQamyla/LLjc2GSEsBxCK+6sNsq5gSGPYfCt25J r3Zc6AFIzvhoBOg8otY3+ofYpn1nHu2JC4DLUmyBrF5IYGtqVbTFDBJX/6Ak1/2kpbZh VTYA== X-Gm-Message-State: ALoCoQnXsCazPlD4ICB96DmfMZbZQqpjzplqaZqoH76QmqcCtqXpWeY+Qjahky3MSWnJiFjUgjBg X-Received: by 10.55.25.141 with SMTP id 13mr12742739qkz.105.1444924145818; Thu, 15 Oct 2015 08:49:05 -0700 (PDT) Received: from mutt-hardenedbsd ([63.88.83.104]) by smtp.gmail.com with ESMTPSA id q32sm5711120qkq.1.2015.10.15.08.49.04 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Oct 2015 08:49:05 -0700 (PDT) Date: Thu, 15 Oct 2015 11:49:02 -0400 From: Shawn Webb To: Andriy Gapon Cc: Alexander Motin , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... Message-ID: <20151015154853.GA60326@mutt-hardenedbsd> References: <201510141112.t9EBClT3022215@repo.freebsd.org> <561FC3EC.7020706@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="envbJBWh7q8WU6mo" Content-Disposition: inline In-Reply-To: <561FC3EC.7020706@FreeBSD.org> X-Operating-System: FreeBSD mutt-hardenedbsd 11.0-CURRENT-HBSD FreeBSD 11.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 15:49:13 -0000 --envbJBWh7q8WU6mo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 15, 2015 at 05:19:08PM +0200, Andriy Gapon wrote: > On 14/10/2015 13:12, Alexander Motin wrote: > > Author: mav > > Date: Wed Oct 14 11:12:47 2015 > > New Revision: 289310 > > URL: https://svnweb.freebsd.org/changeset/base/289310 > >=20 > > Log: > > 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R >=20 > Thank you very much for importing this complex commit. > I have one question below. >=20 > > Reviewed by: George Wilson > > Reviewed by: Prakash Surya > > Reviewed by: Saso Kiselkov > > Reviewed by: Richard Lowe > > Approved by: Garrett D'Amore > > Author: Matthew Ahrens > > =20 > > illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f > >=20 > > Added: > > vendor-sys/illumos/dist/uts/common/crypto/ > > vendor-sys/illumos/dist/uts/common/crypto/io/ > > vendor-sys/illumos/dist/uts/common/crypto/io/edonr_mod.c (contents,= props changed) > > vendor-sys/illumos/dist/uts/common/crypto/io/skein_mod.c (contents,= props changed) >=20 > Do we actually need these two file in the vendor area? > They look like illumos crypto drivers to me, so I think that it is > unlikely that we will have a use for them. Or do you have some big > plans about that? :-) =46rom a securitiy engineering perspective, it would be extremely nice to be able to use these additional hashing algorithms from elsewhere. Would it be possible to genericize them and integrate the ZFS support with that? HardenedBSD has a feature called Integriforce, which allows us to validate executables against a pre-computed list of hashes loaded into the kernel. It'd be extremely nice to add support for these other hashing algorithms. Thanks, --=20 Shawn Webb HardenedBSD GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --envbJBWh7q8WU6mo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWH8rjAAoJEGqEZY9SRW7uIcEP/R8+znW/japTaYM6m74NizEJ tJAQ1HeewOtiI4ujQU+fGs8WkEyQNIa/hPI3IUOVBNKZDp31L3cHZnr6Eeo1UWHM pZUjIv1SzYHL1RhXVqNPWIruL8IoidOaYk/aftifMb/2nqhBfkEpOutPGwWx42UB 2s/ZaUtVzL/Ulq5mudxm3uZJtcIxlUP+tjTlWAm+62I4DxiH0PCEWzSNa/4JIjua g7YIp3u6fIZyMB0CPAoQsGFbudXAA55ETcPaAoSmc1+tCLAr2Dd8JWyW5pkBVeCm tiQSybyBZQK4h/ti/ei8EODi9Q5Rb18SRjgVzfHknP9/rBRzgo9fsLtgGlGFSc7N k1JEF9HMsZ85MIE2H+sqmKF7l/s8pg4rSqMQ6gzDWyzeYEkjoHhY8/krI4SLJJ1x afAIuHChqiBCEy04+w4Z8CQ1SXGrziKhD//akF+/WPJ0N5G8uPTMLNzDqd/XUYJI cKC34i4/625Fq51Aaf/olvoI+GUHQ0zGbawTEtafo3tFNcao4MNxS8hQPef3DIov jG0aaU7thVC2wYX0nQ/3SLTpLSW0zp1yYMgVv8jT+QN8nMk9gwbp/PsNnS2qYv5a hW8Qo8Ra+aoYN8h3Ym9dQVpd5Ga7nMRDvGs/uY2HPmbl0oemGLiOQBvVbk36MYth CKicxOBdalhE4wM7qzRy =o8so -----END PGP SIGNATURE----- --envbJBWh7q8WU6mo-- From owner-svn-src-vendor@freebsd.org Thu Oct 15 18:46:49 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53A5AA151F2; Thu, 15 Oct 2015 18:46:49 +0000 (UTC) (envelope-from mahrens@gmail.com) Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com [IPv6:2a00:1450:4010:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CBACE13D6; Thu, 15 Oct 2015 18:46:48 +0000 (UTC) (envelope-from mahrens@gmail.com) Received: by lffy185 with SMTP id y185so42006187lff.2; Thu, 15 Oct 2015 11:46:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=XxvQQnBB0fXDjQGwNkpJrszAOP1bJa7pXP04JVWC3e4=; b=TioSlg+3aAxgniuW61vXm0nNek8kzoJazs9u74ntj1vyw0+zUX8bvYRbTYxSAVqdzI hcRwsl/rgSE1Vk6Jxhi04uzcA8kppNXZwbWEC8mxkse0HZM1yiasprE4nYNT4J9R2Njq Ij95x7QYljzdcHwOUHgWDw/QEm3ZJJGTL9g6pH5Vd0ang5Hmz2BfhpcF8awRI88g0iur joldovJvLvccLCYaJJpTG5bkut2apURzPZFWEDEzsYWwPFpdH3CXDW/Abotnyj2ywMAE y1CsrC/pqkRsJncyfzydpFuNwkqMHSbiqBws5zFIw9qQcOUwllfD4+MnnFcmo7nrfUOF l5Xg== MIME-Version: 1.0 X-Received: by 10.25.23.69 with SMTP id n66mr2037497lfi.52.1444934806731; Thu, 15 Oct 2015 11:46:46 -0700 (PDT) Sender: mahrens@gmail.com Received: by 10.112.4.8 with HTTP; Thu, 15 Oct 2015 11:46:46 -0700 (PDT) In-Reply-To: <561FC90F.4090706@FreeBSD.org> References: <201510141112.t9EBCmFZ022230@repo.freebsd.org> <561FC5A0.3040909@FreeBSD.org> <561FC90F.4090706@FreeBSD.org> Date: Thu, 15 Oct 2015 11:46:46 -0700 X-Google-Sender-Auth: 7LKecjwO-zzr263MSyWRMiPZuL4 Message-ID: Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... From: Matthew Ahrens To: Alexander Motin Cc: Andriy Gapon , "src-committers@freebsd.org" , svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 18:46:49 -0000 On Thu, Oct 15, 2015 at 8:41 AM, Alexander Motin wrote: > > I haven't decided what to do with this specific case. As I understand, > now we are using FreeBSD's native crypto code instead of illumos' one. I > was thinking about importing this commit only in infrastructural parts, > until respective algorithms are implemented in our native crypto. I > think it should not be a problem for Skein, since according to comments > it is in public domain. About Edon-R I am not sure, since it seems to be > CDDL. > > In terms of Edon-R license, we requested the author to relicense under CDDL. He might be amenable to also license under BSD. Some googling turned up a separate implementation which is permissively licensed -- not sure if it is binary compatible or not. https://github.com/rhash/RHash/blob/master/librhash/edonr.c I'll forward you the email from the Edon-R author with license info. --matt From owner-svn-src-vendor@freebsd.org Fri Oct 16 16:27:00 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89271A16C7A; Fri, 16 Oct 2015 16:27:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E918EC46; Fri, 16 Oct 2015 16:26:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA25724; Fri, 16 Oct 2015 19:26:50 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Zn7qE-000O7s-4F; Fri, 16 Oct 2015 19:26:50 +0300 Subject: Re: svn commit: r289310 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/crypto vendor-sys/illumos/dist/uts/common/crypto/io vendor-sys/illumos... To: Alexander Motin , src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-vendor@FreeBSD.org References: <201510141112.t9EBCmFZ022230@repo.freebsd.org> <561FC5A0.3040909@FreeBSD.org> <561FC90F.4090706@FreeBSD.org> <561FC996.3020207@FreeBSD.org> <561FCAE1.1040504@FreeBSD.org> From: Andriy Gapon Message-ID: <56212524.2020301@FreeBSD.org> Date: Fri, 16 Oct 2015 18:26:12 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <561FCAE1.1040504@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2015 16:27:00 -0000 On 15/10/2015 17:48, Alexander Motin wrote: > On 15.10.2015 18:43, Andriy Gapon wrote: >> On 15/10/2015 17:41, Alexander Motin wrote: >>> On 15.10.2015 18:26, Andriy Gapon wrote: >>>> On 14/10/2015 13:12, Alexander Motin wrote: >>>>> Author: mav >>>>> Date: Wed Oct 14 11:12:47 2015 >>>>> New Revision: 289310 >>>>> URL: https://svnweb.freebsd.org/changeset/base/289310 >>>>> >>>>> Log: >>>>> 4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R >>>>> >>>>> Reviewed by: George Wilson >>>>> Reviewed by: Prakash Surya >>>>> Reviewed by: Saso Kiselkov >>>>> Reviewed by: Richard Lowe >>>>> Approved by: Garrett D'Amore >>>>> Author: Matthew Ahrens >>>>> >>>>> illumos/illumos-gate@45818ee124adeaaf947698996b4f4c722afc6d1f >>>>> >>>>> Added: >>>>> vendor/illumos/dist/common/crypto/ >>>>> vendor/illumos/dist/common/crypto/edonr/ >>>>> vendor/illumos/dist/common/crypto/edonr/edonr.c (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/edonr/edonr_byteorder.h (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/ >>>>> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/THIRDPARTYLICENSE.descrip (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/skein.c (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/skein_block.c (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/skein_impl.h (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/skein_iv.c (contents, props changed) >>>>> vendor/illumos/dist/common/crypto/skein/skein_port.h (contents, props changed) >>>> >>>> It seems that these are pieces of code that can be used by both the >>>> userland and the kernel. I think that previously we used to import such >>>> code into illumos-sys and then integrate it into sys/cddl (e.g. >>>> common/zfs/). Perhaps it is worthwhile following that convention in >>>> this case as well? >>> >>> I haven't decided what to do with this specific case. As I understand, >>> now we are using FreeBSD's native crypto code instead of illumos' one. I >>> was thinking about importing this commit only in infrastructural parts, >>> until respective algorithms are implemented in our native crypto. I >>> think it should not be a problem for Skein, since according to comments >>> it is in public domain. About Edon-R I am not sure, since it seems to be >>> CDDL. >> >> This is confusing... It seems that like you replied to my question in >> another email, not the quoted one :-) > > What other email? So, I was confused, I got an impression that you had replied to this one http://article.gmane.org/gmane.os.freebsd.devel.cvs/534648 Actually, it would be nice if you do :) > You've told me that I put code into the wrong place, > and I replied that it may be irrelevant, since I am not sure we should > import this code to FreeBSD as-is to its place in illumos. Even if the code turns out to be irrelevant for us, it still would be nice to put it into the correct / conventional place in the vendor area. There is no reason to create the confusion for no good cause. -- Andriy Gapon From owner-svn-src-vendor@freebsd.org Fri Oct 16 19:13:35 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0131FA1662D; Fri, 16 Oct 2015 19:13:35 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2AAD186A; Fri, 16 Oct 2015 19:13:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9GJDX2n041338; Fri, 16 Oct 2015 19:13:33 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9GJDX8h041337; Fri, 16 Oct 2015 19:13:33 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510161913.t9GJDX8h041337@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 16 Oct 2015 19:13:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289428 - vendor-sys/illumos/dist/uts/common/crypto vendor/illumos/dist/common/crypto X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2015 19:13:35 -0000 Author: mav Date: Fri Oct 16 19:13:33 2015 New Revision: 289428 URL: https://svnweb.freebsd.org/changeset/base/289428 Log: Remove parts not directly mergeable to FreeBSD. Illumos kernel modures are just not applicable, while hashing algorithms would better be BSD-licensed and put into proper sys/crypto place. Requested by: avg Deleted: vendor/illumos/dist/common/crypto/ Changes in other areas also in this revision: Deleted: vendor-sys/illumos/dist/uts/common/crypto/ From owner-svn-src-vendor@freebsd.org Fri Oct 16 19:13:34 2015 Return-Path: Delivered-To: svn-src-vendor@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E19ADA1662C; Fri, 16 Oct 2015 19:13:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A9A581869; Fri, 16 Oct 2015 19:13:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id t9GJDXSY041333; Fri, 16 Oct 2015 19:13:33 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id t9GJDXkV041332; Fri, 16 Oct 2015 19:13:33 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201510161913.t9GJDXkV041332@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 16 Oct 2015 19:13:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r289428 - vendor-sys/illumos/dist/uts/common/crypto vendor/illumos/dist/common/crypto X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2015 19:13:35 -0000 Author: mav Date: Fri Oct 16 19:13:33 2015 New Revision: 289428 URL: https://svnweb.freebsd.org/changeset/base/289428 Log: Remove parts not directly mergeable to FreeBSD. Illumos kernel modures are just not applicable, while hashing algorithms would better be BSD-licensed and put into proper sys/crypto place. Requested by: avg Deleted: vendor-sys/illumos/dist/uts/common/crypto/ Changes in other areas also in this revision: Deleted: vendor/illumos/dist/common/crypto/