From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:12:48 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7EEAFA6B; Wed, 20 Nov 2013 10:12:48 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6DB332DD5; Wed, 20 Nov 2013 10:12:48 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKACm9e042711; Wed, 20 Nov 2013 10:12:48 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKACmts042710; Wed, 20 Nov 2013 10:12:48 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201012.rAKACmts042710@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:12:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258368 - vendor-sys/illumos/dist/uts/common vendor/illumos/dist/lib/libdtrace/common X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:12:48 -0000 Author: avg Date: Wed Nov 20 10:12:47 2013 New Revision: 258368 URL: http://svnweb.freebsd.org/changeset/base/258368 Log: 3933 contract adoption can race illumos/illumos-gate@a81df0a5d715363cc1841810a87818dfa95675c0 Note we now reference illumos commits in the github repository as the mercurial mirror seems to be neglected. For connectivity: r255255 referenced 14164:dceb17481b99 which is illumos/illumos-gate@69962b5647e4a8b9b14998733b765925381b727e r255258 referenced 14176:15e9457c12ec which is illumos/illumos-gate@cf746768a898264fa491791a8252865e5338dc67 Modified: vendor/illumos/dist/lib/libdtrace/common/dt_print.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/Makefile.files Modified: vendor/illumos/dist/lib/libdtrace/common/dt_print.c ============================================================================== --- vendor/illumos/dist/lib/libdtrace/common/dt_print.c Wed Nov 20 09:03:48 2013 (r258367) +++ vendor/illumos/dist/lib/libdtrace/common/dt_print.c Wed Nov 20 10:12:47 2013 (r258368) @@ -493,9 +493,11 @@ dt_print_enum(ctf_id_t base, ulong_t off value = *(uint8_t *)addr; break; case sizeof (uint16_t): + /* LINTED - alignment */ value = *(uint16_t *)addr; break; case sizeof (int32_t): + /* LINTED - alignment */ value = *(int32_t *)addr; break; default: From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:12:48 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 446FDA6A; Wed, 20 Nov 2013 10:12:48 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 196142DD4; Wed, 20 Nov 2013 10:12:48 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAClnm042705; Wed, 20 Nov 2013 10:12:47 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAClVD042704; Wed, 20 Nov 2013 10:12:47 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201012.rAKAClVD042704@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:12:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258368 - vendor-sys/illumos/dist/uts/common vendor/illumos/dist/lib/libdtrace/common X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:12:48 -0000 Author: avg Date: Wed Nov 20 10:12:47 2013 New Revision: 258368 URL: http://svnweb.freebsd.org/changeset/base/258368 Log: 3933 contract adoption can race illumos/illumos-gate@a81df0a5d715363cc1841810a87818dfa95675c0 Note we now reference illumos commits in the github repository as the mercurial mirror seems to be neglected. For connectivity: r255255 referenced 14164:dceb17481b99 which is illumos/illumos-gate@69962b5647e4a8b9b14998733b765925381b727e r255258 referenced 14176:15e9457c12ec which is illumos/illumos-gate@cf746768a898264fa491791a8252865e5338dc67 Modified: vendor-sys/illumos/dist/uts/common/Makefile.files Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libdtrace/common/dt_print.c Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 09:03:48 2013 (r258367) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:12:47 2013 (r258368) @@ -146,7 +146,7 @@ GENUNIX_OBJS += \ ddi_intr_impl.o \ ddi_intr_irm.o \ ddi_nodeid.o \ - ddi_timer.o \ + ddi_periodic.o \ devcfg.o \ devcache.o \ device.o \ @@ -1467,6 +1467,23 @@ RPCMOD_OBJS += rpcmod.o clnt_cots.o clnt rpcsys.o xdr_sizeof.o clnt_rdma.o svc_rdma.o \ xdr_rdma.o rdma_subr.o xdrrdma_sizeof.o +KLMMOD_OBJS += klmmod.o \ + nlm_impl.o \ + nlm_rpc_handle.o \ + nlm_dispatch.o \ + nlm_rpc_svc.o \ + nlm_client.o \ + nlm_service.o \ + nlm_prot_clnt.o \ + nlm_prot_xdr.o \ + nlm_rpc_clnt.o \ + nsm_addr_clnt.o \ + nsm_addr_xdr.o \ + sm_inter_clnt.o \ + sm_inter_xdr.o + +KLMOPS_OBJS += klmops.o + TLIMOD_OBJS += tlimod.o t_kalloc.o t_kbind.o t_kclose.o \ t_kconnect.o t_kfree.o t_kgtstate.o t_kopen.o \ t_krcvudat.o t_ksndudat.o t_kspoll.o t_kunbind.o \ @@ -1646,10 +1663,7 @@ K5_KRB= kfree.o copy_key.o \ K5_OS= timeofday.o toffset.o \ init_os_ctx.o c_ustime.o -SEAL= -# EXPORT DELETE START SEAL= seal.o unseal.o -# EXPORT DELETE END MECH= delete_sec_context.o \ import_sec_context.o \ From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:35:52 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAEB7230; Wed, 20 Nov 2013 10:35:52 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AA5072F37; Wed, 20 Nov 2013 10:35:52 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAZqN4049577; Wed, 20 Nov 2013 10:35:52 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAZqoL049576; Wed, 20 Nov 2013 10:35:52 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201035.rAKAZqoL049576@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:35:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258369 - vendor-sys/illumos/dist/uts/common X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:35:52 -0000 Author: avg Date: Wed Nov 20 10:35:52 2013 New Revision: 258369 URL: http://svnweb.freebsd.org/changeset/base/258369 Log: 3881 want device driver for HP SmartArray RAID controllers illumos/illumos-gate@80c94ecd7a524eb933a4bb221a9618b9dc490e76 Modified: vendor-sys/illumos/dist/uts/common/Makefile.files Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:12:47 2013 (r258368) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:35:52 2013 (r258369) @@ -2029,6 +2029,14 @@ MEGA_SAS_OBJS = megaraid_sas.o MR_SAS_OBJS = ld_pd_map.o mr_sas.o mr_sas_tbolt.o mr_sas_list.o # +# CPQARY3 module +# +CPQARY3_OBJS = cpqary3.o cpqary3_noe.o cpqary3_talk2ctlr.o \ + cpqary3_isr.o cpqary3_transport.o cpqary3_mem.o \ + cpqary3_scsi.o cpqary3_util.o cpqary3_ioctl.o \ + cpqary3_bd.o + +# # ISCSI_INITIATOR module # ISCSI_INITIATOR_OBJS = chap.o iscsi_io.o iscsi_thread.o \ From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:38:38 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 709FA468; Wed, 20 Nov 2013 10:38:38 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4638A2F5E; Wed, 20 Nov 2013 10:38:38 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAccX3049980; Wed, 20 Nov 2013 10:38:38 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAccwm049979; Wed, 20 Nov 2013 10:38:38 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201038.rAKAccwm049979@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:38:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258370 - vendor-sys/illumos/dist/uts/common X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:38:38 -0000 Author: avg Date: Wed Nov 20 10:38:37 2013 New Revision: 258370 URL: http://svnweb.freebsd.org/changeset/base/258370 Log: 4091 e1000g I217/I218 support illumos/illumos-gate@75eba5b6d79ed4d2ce3daf7b2806306b6b69a938 Modified: vendor-sys/illumos/dist/uts/common/Makefile.files Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:35:52 2013 (r258369) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:38:37 2013 (r258370) @@ -1923,21 +1923,31 @@ XGE_HAL_OBJS = xgehal-channel.o xgehal- xge-queue.o xgehal-mgmt.o xgehal-mgmtaux.o # -# e1000g module +# e1000/igb common objs +# +# Historically e1000g and igb had separate copies of all of the common +# code. At this time while they are now sharing the same copy of it, they +# are building it into their own modules which is due to the differences +# in the osdep and debug portions of their code. # -E1000G_OBJS += e1000_80003es2lan.o e1000_82540.o e1000_82541.o e1000_82542.o \ +E1000API_OBJS += e1000_80003es2lan.o e1000_82540.o e1000_82541.o e1000_82542.o \ e1000_82543.o e1000_82571.o e1000_api.o e1000_ich8lan.o \ - e1000_mac.o e1000_manage.o e1000_nvm.o e1000_osdep.o \ - e1000_phy.o e1000g_debug.o e1000g_main.o e1000g_alloc.o \ - e1000g_tx.o e1000g_rx.o e1000g_stat.o + e1000_mac.o e1000_manage.o e1000_nvm.o e1000_phy.o \ + e1000_82575.o e1000_i210.o e1000_mbx.o e1000_vf.o + +# +# e1000g module +# +E1000G_OBJS += e1000g_debug.o e1000g_main.o e1000g_alloc.o \ + e1000g_tx.o e1000g_rx.o e1000g_stat.o \ + e1000g_osdep.o e1000g_workarounds.o + # # Intel 82575 1G NIC driver module # -IGB_OBJS = igb_82575.o igb_api.o igb_mac.o igb_manage.o \ - igb_nvm.o igb_osdep.o igb_phy.o igb_buf.o \ - igb_debug.o igb_gld.o igb_log.o igb_main.o \ - igb_rx.o igb_stat.o igb_tx.o +IGB_OBJS = igb_buf.o igb_debug.o igb_gld.o igb_log.o igb_main.o \ + igb_rx.o igb_stat.o igb_tx.o igb_osdep.o # # Intel Pro/100 NIC driver module From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:41:12 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3C3E5CD; Wed, 20 Nov 2013 10:41:12 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 916D72FA1; Wed, 20 Nov 2013 10:41:12 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAfCPO052406; Wed, 20 Nov 2013 10:41:12 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAfCDU052402; Wed, 20 Nov 2013 10:41:12 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201041.rAKAfCDU052402@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:41:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258371 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c vendor-s... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:41:12 -0000 Author: avg Date: Wed Nov 20 10:41:10 2013 New Revision: 258371 URL: http://svnweb.freebsd.org/changeset/base/258371 Log: 4101 metaslab_debug should allow for fine-grained control 4102 space_maps should store more information about themselves 4103 space map object blocksize should be increased 4104 ::spa_space no longer works 4105 removing a mirrored log device results in a leaked object 4106 asynchronously load metaslab Revision illumos/illumos-gate@0713e232b7712cd27d99e1e935ebb8d5de61c57d Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/man/man5/zpool-features.5 Changes in other areas also in this revision: Added: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c/ vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c/range_tree.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h/ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h/space_reftree.h (contents, props changed) Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Wed Nov 20 10:38:37 2013 (r258370) +++ vendor/illumos/dist/cmd/zdb/zdb.c Wed Nov 20 10:41:10 2013 (r258371) @@ -246,7 +246,7 @@ const char histo_stars[] = "************ const int histo_width = sizeof (histo_stars) - 1; static void -dump_histogram(const uint64_t *histo, int size) +dump_histogram(const uint64_t *histo, int size, int offset) { int i; int minidx = size - 1; @@ -267,7 +267,7 @@ dump_histogram(const uint64_t *histo, in for (i = minidx; i <= maxidx; i++) { (void) printf("\t\t\t%3u: %6llu %s\n", - i, (u_longlong_t)histo[i], + i + offset, (u_longlong_t)histo[i], &histo_stars[(max - histo[i]) * histo_width / max]); } } @@ -320,19 +320,19 @@ dump_zap_stats(objset_t *os, uint64_t ob (u_longlong_t)zs.zs_salt); (void) printf("\t\tLeafs with 2^n pointers:\n"); - dump_histogram(zs.zs_leafs_with_2n_pointers, ZAP_HISTOGRAM_SIZE); + dump_histogram(zs.zs_leafs_with_2n_pointers, ZAP_HISTOGRAM_SIZE, 0); (void) printf("\t\tBlocks with n*5 entries:\n"); - dump_histogram(zs.zs_blocks_with_n5_entries, ZAP_HISTOGRAM_SIZE); + dump_histogram(zs.zs_blocks_with_n5_entries, ZAP_HISTOGRAM_SIZE, 0); (void) printf("\t\tBlocks n/10 full:\n"); - dump_histogram(zs.zs_blocks_n_tenths_full, ZAP_HISTOGRAM_SIZE); + dump_histogram(zs.zs_blocks_n_tenths_full, ZAP_HISTOGRAM_SIZE, 0); (void) printf("\t\tEntries with n chunks:\n"); - dump_histogram(zs.zs_entries_using_n_chunks, ZAP_HISTOGRAM_SIZE); + dump_histogram(zs.zs_entries_using_n_chunks, ZAP_HISTOGRAM_SIZE, 0); (void) printf("\t\tBuckets with n entries:\n"); - dump_histogram(zs.zs_buckets_with_n_entries, ZAP_HISTOGRAM_SIZE); + dump_histogram(zs.zs_buckets_with_n_entries, ZAP_HISTOGRAM_SIZE, 0); } /*ARGSUSED*/ @@ -521,26 +521,85 @@ dump_zpldir(objset_t *os, uint64_t objec zap_cursor_fini(&zc); } +int +get_dtl_refcount(vdev_t *vd) +{ + int refcount = 0; + + if (vd->vdev_ops->vdev_op_leaf) { + space_map_t *sm = vd->vdev_dtl_sm; + + if (sm != NULL && + sm->sm_dbuf->db_size == sizeof (space_map_phys_t)) + return (1); + return (0); + } + + for (int c = 0; c < vd->vdev_children; c++) + refcount += get_dtl_refcount(vd->vdev_child[c]); + return (refcount); +} + +int +get_metaslab_refcount(vdev_t *vd) +{ + int refcount = 0; + + if (vd->vdev_top == vd) { + for (int m = 0; m < vd->vdev_ms_count; m++) { + space_map_t *sm = vd->vdev_ms[m]->ms_sm; + + if (sm != NULL && + sm->sm_dbuf->db_size == sizeof (space_map_phys_t)) + refcount++; + } + } + for (int c = 0; c < vd->vdev_children; c++) + refcount += get_metaslab_refcount(vd->vdev_child[c]); + + return (refcount); +} + +static int +verify_spacemap_refcounts(spa_t *spa) +{ + int expected_refcount, actual_refcount; + + expected_refcount = spa_feature_get_refcount(spa, + &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM]); + actual_refcount = get_dtl_refcount(spa->spa_root_vdev); + actual_refcount += get_metaslab_refcount(spa->spa_root_vdev); + + if (expected_refcount != actual_refcount) { + (void) printf("space map refcount mismatch: expected %d != " + "actual %d\n", expected_refcount, actual_refcount); + return (2); + } + return (0); +} + static void -dump_spacemap(objset_t *os, space_map_obj_t *smo, space_map_t *sm) +dump_spacemap(objset_t *os, space_map_t *sm) { uint64_t alloc, offset, entry; - uint8_t mapshift = sm->sm_shift; - uint64_t mapstart = sm->sm_start; char *ddata[] = { "ALLOC", "FREE", "CONDENSE", "INVALID", "INVALID", "INVALID", "INVALID", "INVALID" }; - if (smo->smo_object == 0) + if (sm == NULL) return; /* * Print out the freelist entries in both encoded and decoded form. */ alloc = 0; - for (offset = 0; offset < smo->smo_objsize; offset += sizeof (entry)) { - VERIFY3U(0, ==, dmu_read(os, smo->smo_object, offset, + for (offset = 0; offset < space_map_length(sm); + offset += sizeof (entry)) { + uint8_t mapshift = sm->sm_shift; + + VERIFY0(dmu_read(os, space_map_object(sm), offset, sizeof (entry), &entry, DMU_READ_PREFETCH)); if (SM_DEBUG_DECODE(entry)) { + (void) printf("\t [%6llu] %s: txg %llu, pass %llu\n", (u_longlong_t)(offset / sizeof (entry)), ddata[SM_DEBUG_ACTION_DECODE(entry)], @@ -552,10 +611,10 @@ dump_spacemap(objset_t *os, space_map_ob (u_longlong_t)(offset / sizeof (entry)), SM_TYPE_DECODE(entry) == SM_ALLOC ? 'A' : 'F', (u_longlong_t)((SM_OFFSET_DECODE(entry) << - mapshift) + mapstart), + mapshift) + sm->sm_start), (u_longlong_t)((SM_OFFSET_DECODE(entry) << - mapshift) + mapstart + (SM_RUN_DECODE(entry) << - mapshift)), + mapshift) + sm->sm_start + + (SM_RUN_DECODE(entry) << mapshift)), (u_longlong_t)(SM_RUN_DECODE(entry) << mapshift)); if (SM_TYPE_DECODE(entry) == SM_ALLOC) alloc += SM_RUN_DECODE(entry) << mapshift; @@ -563,10 +622,10 @@ dump_spacemap(objset_t *os, space_map_ob alloc -= SM_RUN_DECODE(entry) << mapshift; } } - if (alloc != smo->smo_alloc) { + if (alloc != space_map_allocated(sm)) { (void) printf("space_map_object alloc (%llu) INCONSISTENT " "with space map summary (%llu)\n", - (u_longlong_t)smo->smo_alloc, (u_longlong_t)alloc); + (u_longlong_t)space_map_allocated(sm), (u_longlong_t)alloc); } } @@ -574,15 +633,17 @@ static void dump_metaslab_stats(metaslab_t *msp) { char maxbuf[32]; - space_map_t *sm = msp->ms_map; - avl_tree_t *t = sm->sm_pp_root; - int free_pct = sm->sm_space * 100 / sm->sm_size; + range_tree_t *rt = msp->ms_tree; + avl_tree_t *t = &msp->ms_size_tree; + int free_pct = range_tree_space(rt) * 100 / msp->ms_size; - zdb_nicenum(space_map_maxsize(sm), maxbuf); + zdb_nicenum(metaslab_block_maxsize(msp), maxbuf); (void) printf("\t %25s %10lu %7s %6s %4s %4d%%\n", "segments", avl_numnodes(t), "maxsize", maxbuf, "freepct", free_pct); + (void) printf("\tIn-memory histogram:\n"); + dump_histogram(rt->rt_histogram, RANGE_TREE_HISTOGRAM_SIZE, 0); } static void @@ -590,33 +651,45 @@ dump_metaslab(metaslab_t *msp) { vdev_t *vd = msp->ms_group->mg_vd; spa_t *spa = vd->vdev_spa; - space_map_t *sm = msp->ms_map; - space_map_obj_t *smo = &msp->ms_smo; + space_map_t *sm = msp->ms_sm; char freebuf[32]; - zdb_nicenum(sm->sm_size - smo->smo_alloc, freebuf); + zdb_nicenum(msp->ms_size - space_map_allocated(sm), freebuf); (void) printf( "\tmetaslab %6llu offset %12llx spacemap %6llu free %5s\n", - (u_longlong_t)(sm->sm_start / sm->sm_size), - (u_longlong_t)sm->sm_start, (u_longlong_t)smo->smo_object, freebuf); + (u_longlong_t)msp->ms_id, (u_longlong_t)msp->ms_start, + (u_longlong_t)space_map_object(sm), freebuf); - if (dump_opt['m'] > 1 && !dump_opt['L']) { + if (dump_opt['m'] > 2 && !dump_opt['L']) { mutex_enter(&msp->ms_lock); - space_map_load_wait(sm); - if (!sm->sm_loaded) - VERIFY(space_map_load(sm, zfs_metaslab_ops, - SM_FREE, smo, spa->spa_meta_objset) == 0); + metaslab_load_wait(msp); + if (!msp->ms_loaded) { + VERIFY0(metaslab_load(msp)); + range_tree_stat_verify(msp->ms_tree); + } dump_metaslab_stats(msp); - space_map_unload(sm); + metaslab_unload(msp); mutex_exit(&msp->ms_lock); } - if (dump_opt['d'] > 5 || dump_opt['m'] > 2) { - ASSERT(sm->sm_size == (1ULL << vd->vdev_ms_shift)); + if (dump_opt['m'] > 1 && sm != NULL && + spa_feature_is_active(spa, + &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM])) { + /* + * The space map histogram represents free space in chunks + * of sm_shift (i.e. bucket 0 refers to 2^sm_shift). + */ + (void) printf("\tOn-disk histogram:\n"); + dump_histogram(sm->sm_phys->smp_histogram, + SPACE_MAP_HISTOGRAM_SIZE(sm), sm->sm_shift); + } + + if (dump_opt['d'] > 5 || dump_opt['m'] > 3) { + ASSERT(msp->ms_size == (1ULL << vd->vdev_ms_shift)); mutex_enter(&msp->ms_lock); - dump_spacemap(spa->spa_meta_objset, smo, sm); + dump_spacemap(spa->spa_meta_objset, msp->ms_sm); mutex_exit(&msp->ms_lock); } } @@ -803,9 +876,9 @@ dump_all_ddts(spa_t *spa) } static void -dump_dtl_seg(space_map_t *sm, uint64_t start, uint64_t size) +dump_dtl_seg(void *arg, uint64_t start, uint64_t size) { - char *prefix = (void *)sm; + char *prefix = arg; (void) printf("%s [%llu,%llu) length %llu\n", prefix, @@ -835,17 +908,16 @@ dump_dtl(vdev_t *vd, int indent) required ? "DTL-required" : "DTL-expendable"); for (int t = 0; t < DTL_TYPES; t++) { - space_map_t *sm = &vd->vdev_dtl[t]; - if (sm->sm_space == 0) + range_tree_t *rt = vd->vdev_dtl[t]; + if (range_tree_space(rt) == 0) continue; (void) snprintf(prefix, sizeof (prefix), "\t%*s%s", indent + 2, "", name[t]); - mutex_enter(sm->sm_lock); - space_map_walk(sm, dump_dtl_seg, (void *)prefix); - mutex_exit(sm->sm_lock); + mutex_enter(rt->rt_lock); + range_tree_walk(rt, dump_dtl_seg, prefix); + mutex_exit(rt->rt_lock); if (dump_opt['d'] > 5 && vd->vdev_children == 0) - dump_spacemap(spa->spa_meta_objset, - &vd->vdev_dtl_smo, sm); + dump_spacemap(spa->spa_meta_objset, vd->vdev_dtl_sm); } for (int c = 0; c < vd->vdev_children; c++) @@ -2221,39 +2293,17 @@ zdb_blkptr_cb(spa_t *spa, zilog_t *zilog } static void -zdb_leak(space_map_t *sm, uint64_t start, uint64_t size) +zdb_leak(void *arg, uint64_t start, uint64_t size) { - vdev_t *vd = sm->sm_ppd; + vdev_t *vd = arg; (void) printf("leaked space: vdev %llu, offset 0x%llx, size %llu\n", (u_longlong_t)vd->vdev_id, (u_longlong_t)start, (u_longlong_t)size); } -/* ARGSUSED */ -static void -zdb_space_map_load(space_map_t *sm) -{ -} - -static void -zdb_space_map_unload(space_map_t *sm) -{ - space_map_vacate(sm, zdb_leak, sm); -} - -/* ARGSUSED */ -static void -zdb_space_map_claim(space_map_t *sm, uint64_t start, uint64_t size) -{ -} - -static space_map_ops_t zdb_space_map_ops = { - zdb_space_map_load, - zdb_space_map_unload, +static metaslab_ops_t zdb_metaslab_ops = { NULL, /* alloc */ - zdb_space_map_claim, - NULL, /* free */ - NULL /* maxsize */ + NULL /* fragmented */ }; static void @@ -2308,11 +2358,21 @@ zdb_leak_init(spa_t *spa, zdb_cb_t *zcb) for (int m = 0; m < vd->vdev_ms_count; m++) { metaslab_t *msp = vd->vdev_ms[m]; mutex_enter(&msp->ms_lock); - space_map_unload(msp->ms_map); - VERIFY(space_map_load(msp->ms_map, - &zdb_space_map_ops, SM_ALLOC, &msp->ms_smo, - spa->spa_meta_objset) == 0); - msp->ms_map->sm_ppd = vd; + metaslab_unload(msp); + + /* + * For leak detection, we overload the metaslab + * ms_tree to contain allocated segments + * instead of free segments. As a result, + * we can't use the normal metaslab_load/unload + * interfaces. + */ + if (msp->ms_sm != NULL) { + msp->ms_ops = &zdb_metaslab_ops; + VERIFY0(space_map_load(msp->ms_sm, + msp->ms_tree, SM_ALLOC)); + msp->ms_loaded = B_TRUE; + } mutex_exit(&msp->ms_lock); } } @@ -2335,7 +2395,20 @@ zdb_leak_fini(spa_t *spa) for (int m = 0; m < vd->vdev_ms_count; m++) { metaslab_t *msp = vd->vdev_ms[m]; mutex_enter(&msp->ms_lock); - space_map_unload(msp->ms_map); + + /* + * The ms_tree has been overloaded to + * contain allocated segments. Now that we + * finished traversing all blocks, any + * block that remains in the ms_tree + * represents an allocated block that we + * did not claim during the traversal. + * Claimed blocks would have been removed + * from the ms_tree. + */ + range_tree_vacate(msp->ms_tree, zdb_leak, vd); + msp->ms_loaded = B_FALSE; + mutex_exit(&msp->ms_lock); } } @@ -2563,7 +2636,7 @@ dump_block_stats(spa_t *spa) "(in 512-byte sectors): " "number of blocks\n"); dump_histogram(zb->zb_psize_histogram, - PSIZE_HISTO_SIZE); + PSIZE_HISTO_SIZE, 0); } } } @@ -2733,6 +2806,9 @@ dump_zpool(spa_t *spa) if (dump_opt['b'] || dump_opt['c']) rc = dump_block_stats(spa); + if (rc == 0) + rc = verify_spacemap_refcounts(spa); + if (dump_opt['s']) show_pool_stats(spa); Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Wed Nov 20 10:38:37 2013 (r258370) +++ vendor/illumos/dist/cmd/ztest/ztest.c Wed Nov 20 10:41:10 2013 (r258371) @@ -5336,7 +5336,7 @@ ztest_deadman_thread(void *arg) * vdev_deadman() discovers that there hasn't been any recent * I/Os then it will end up aborting the tests. */ - if (spa_suspended(spa)) { + if (spa_suspended(spa) || spa->spa_root_vdev == NULL) { fatal(0, "aborting test after %llu seconds because " "pool has transitioned to a suspended state.", zfs_deadman_synctime_ms / 1000); Modified: vendor/illumos/dist/man/man5/zpool-features.5 ============================================================================== --- vendor/illumos/dist/man/man5/zpool-features.5 Wed Nov 20 10:38:37 2013 (r258370) +++ vendor/illumos/dist/man/man5/zpool-features.5 Wed Nov 20 10:41:10 2013 (r258371) @@ -1,5 +1,5 @@ '\" te -.\" Copyright (c) 2012 by Delphix. All rights reserved. +.\" Copyright (c) 2013 by Delphix. All rights reserved. .\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved. .\" Copyright (c) 2013, Joyent, Inc. All rights reserved. .\" The contents of this file are subject to the terms of the Common Development @@ -14,7 +14,7 @@ .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your .\" own identifying information: .\" Portions Copyright [yyyy] [name of copyright owner] -.TH ZPOOL-FEATURES 5 "Mar 16, 2012" +.TH ZPOOL-FEATURES 5 "Aug 27, 2013" .SH NAME zpool\-features \- ZFS pool feature descriptions .SH DESCRIPTION @@ -229,7 +229,26 @@ read-only compatible, this operation wil on systems without support for the \fBlz4_compress\fR feature. At the moment, this operation cannot be reversed. Booting off of \fBlz4\fR-compressed root pools is supported. +.RE + +.sp +.ne 2 +.na +\fB\fBspacemap_histogram\fR\fR +.ad +.RS 4n +.TS +l l . +GUID com.delphix:spacemap_histogram +READ\-ONLY COMPATIBLE yes +DEPENDENCIES none +.TE +This features allows ZFS to maintain more information about how free space +is organized within the pool. If this feature is \fBenabled\fR, ZFS will +set this feature to \fBactive\fR when a new space map object is created or +an existing space map is upgraded to the new format. Once the feature is +\fBactive\fR, it will remain in that state until the pool is destroyed. .RE .sp From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:41:12 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04FE35CC; Wed, 20 Nov 2013 10:41:12 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E58EB2FA0; Wed, 20 Nov 2013 10:41:11 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAfB3k052396; Wed, 20 Nov 2013 10:41:11 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAfBHw052390; Wed, 20 Nov 2013 10:41:11 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201041.rAKAfBHw052390@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:41:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258371 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c vendor-s... X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:41:12 -0000 Author: avg Date: Wed Nov 20 10:41:10 2013 New Revision: 258371 URL: http://svnweb.freebsd.org/changeset/base/258371 Log: 4101 metaslab_debug should allow for fine-grained control 4102 space_maps should store more information about themselves 4103 space map object blocksize should be increased 4104 ::spa_space no longer works 4105 removing a mirrored log device results in a leaked object 4106 asynchronously load metaslab Revision illumos/illumos-gate@0713e232b7712cd27d99e1e935ebb8d5de61c57d Added: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c/ vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c/range_tree.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h/ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h/space_reftree.h (contents, props changed) Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/man/man5/zpool-features.5 Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.c Wed Nov 20 10:38:37 2013 (r258370) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.c Wed Nov 20 10:41:10 2013 (r258371) @@ -20,7 +20,7 @@ */ /* - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ @@ -164,4 +164,7 @@ zpool_feature_init(void) zfeature_register(SPA_FEATURE_MULTI_VDEV_CRASH_DUMP, "com.joyent:multi_vdev_crash_dump", "multi_vdev_crash_dump", "Crash dumps to multiple vdev pools.", B_FALSE, B_FALSE, NULL); + zfeature_register(SPA_FEATURE_SPACEMAP_HISTOGRAM, + "com.delphix:spacemap_histogram", "spacemap_histogram", + "Spacemaps maintain space histograms.", B_TRUE, B_FALSE, NULL); } Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.h ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.h Wed Nov 20 10:38:37 2013 (r258370) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.h Wed Nov 20 10:41:10 2013 (r258371) @@ -20,7 +20,7 @@ */ /* - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ @@ -56,6 +56,7 @@ enum spa_feature { SPA_FEATURE_EMPTY_BPOBJ, SPA_FEATURE_LZ4_COMPRESS, SPA_FEATURE_MULTI_VDEV_CRASH_DUMP, + SPA_FEATURE_SPACEMAP_HISTOGRAM, SPA_FEATURES } spa_feature_t; Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:38:37 2013 (r258370) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:41:10 2013 (r258371) @@ -22,7 +22,7 @@ # # Copyright (c) 1991, 2010, Oracle and/or its affiliates. All rights reserved. # Copyright (c) 2012 Nexenta Systems, Inc. All rights reserved. -# Copyright (c) 2012 by Delphix. All rights reserved. +# Copyright (c) 2013 by Delphix. All rights reserved. # Copyright (c) 2013 by Saso Kiselkov. All rights reserved. # @@ -1359,6 +1359,7 @@ ZFS_COMMON_OBJS += \ lz4.o \ lzjb.o \ metaslab.o \ + range_tree.o \ refcount.o \ rrwlock.o \ sa.o \ @@ -1369,6 +1370,7 @@ ZFS_COMMON_OBJS += \ spa_history.o \ spa_misc.o \ space_map.o \ + space_reftree.o \ txg.o \ uberblock.o \ unique.o \ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Wed Nov 20 10:38:37 2013 (r258370) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Wed Nov 20 10:41:10 2013 (r258371) @@ -1334,7 +1334,7 @@ dnode_set_blksz(dnode_t *dn, uint64_t si rw_enter(&dn->dn_struct_rwlock, RW_WRITER); /* Check for any allocated blocks beyond the first */ - if (dn->dn_phys->dn_maxblkid != 0) + if (dn->dn_maxblkid != 0) goto fail; mutex_enter(&dn->dn_dbufs_mtx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Wed Nov 20 10:38:37 2013 (r258370) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Wed Nov 20 10:41:10 2013 (r258371) @@ -31,6 +31,7 @@ #include #include #include +#include /* * Allow allocations to switch to gang blocks quickly. We do this to @@ -44,6 +45,11 @@ (!((flags) & (METASLAB_GANG_CHILD | METASLAB_GANG_HEADER | \ METASLAB_GANG_AVOID))) +#define METASLAB_WEIGHT_PRIMARY (1ULL << 63) +#define METASLAB_WEIGHT_SECONDARY (1ULL << 62) +#define METASLAB_ACTIVE_MASK \ + (METASLAB_WEIGHT_PRIMARY | METASLAB_WEIGHT_SECONDARY) + uint64_t metaslab_aliquot = 512ULL << 10; uint64_t metaslab_gang_bang = SPA_MAXBLOCKSIZE + 1; /* force gang blocks */ @@ -79,9 +85,14 @@ int zfs_mg_alloc_failures = 0; int zfs_mg_noalloc_threshold = 0; /* - * Metaslab debugging: when set, keeps all space maps in core to verify frees. + * When set will load all metaslabs when pool is first opened. */ -static int metaslab_debug = 0; +int metaslab_debug_load = 0; + +/* + * When set will prevent metaslabs from being unloaded. + */ +int metaslab_debug_unload = 0; /* * Minimum size which forces the dynamic allocator to change @@ -106,14 +117,16 @@ int metaslab_df_free_pct = 4; uint64_t metaslab_min_alloc_size = DMU_MAX_ACCESS; /* - * Max number of space_maps to prefetch. + * Percentage of all cpus that can be used by the metaslab taskq. */ -int metaslab_prefetch_limit = SPA_DVAS_PER_BP; +int metaslab_load_pct = 50; /* - * Percentage bonus multiplier for metaslabs that are in the bonus area. + * Determines how many txgs a metaslab may remain loaded without having any + * allocations from it. As long as a metaslab continues to be used we will + * keep it loaded. */ -int metaslab_smo_bonus_pct = 150; +int metaslab_unload_delay = TXG_SIZE * 2; /* * Should we be willing to write data to degraded vdevs? @@ -121,12 +134,28 @@ int metaslab_smo_bonus_pct = 150; boolean_t zfs_write_to_degraded = B_FALSE; /* + * Max number of metaslabs per group to preload. + */ +int metaslab_preload_limit = SPA_DVAS_PER_BP; + +/* + * Enable/disable preloading of metaslab. + */ +boolean_t metaslab_preload_enabled = B_TRUE; + +/* + * Enable/disable additional weight factor for each metaslab. + */ +boolean_t metaslab_weight_factor_enable = B_FALSE; + + +/* * ========================================================================== * Metaslab classes * ========================================================================== */ metaslab_class_t * -metaslab_class_create(spa_t *spa, space_map_ops_t *ops) +metaslab_class_create(spa_t *spa, metaslab_ops_t *ops) { metaslab_class_t *mc; @@ -230,9 +259,9 @@ metaslab_compare(const void *x1, const v /* * If the weights are identical, use the offset to force uniqueness. */ - if (m1->ms_map->sm_start < m2->ms_map->sm_start) + if (m1->ms_start < m2->ms_start) return (-1); - if (m1->ms_map->sm_start > m2->ms_map->sm_start) + if (m1->ms_start > m2->ms_start) return (1); ASSERT3P(m1, ==, m2); @@ -300,6 +329,9 @@ metaslab_group_create(metaslab_class_t * mg->mg_class = mc; mg->mg_activation_count = 0; + mg->mg_taskq = taskq_create("metaslab_group_tasksq", metaslab_load_pct, + minclsyspri, 10, INT_MAX, TASKQ_THREADS_CPU_PCT); + return (mg); } @@ -368,6 +400,8 @@ metaslab_group_passivate(metaslab_group_ return; } + taskq_wait(mg->mg_taskq); + mgprev = mg->mg_prev; mgnext = mg->mg_next; @@ -447,130 +481,200 @@ metaslab_group_allocatable(metaslab_grou /* * ========================================================================== - * Common allocator routines + * Range tree callbacks * ========================================================================== */ + +/* + * Comparison function for the private size-ordered tree. Tree is sorted + * by size, larger sizes at the end of the tree. + */ static int -metaslab_segsize_compare(const void *x1, const void *x2) +metaslab_rangesize_compare(const void *x1, const void *x2) { - const space_seg_t *s1 = x1; - const space_seg_t *s2 = x2; - uint64_t ss_size1 = s1->ss_end - s1->ss_start; - uint64_t ss_size2 = s2->ss_end - s2->ss_start; + const range_seg_t *r1 = x1; + const range_seg_t *r2 = x2; + uint64_t rs_size1 = r1->rs_end - r1->rs_start; + uint64_t rs_size2 = r2->rs_end - r2->rs_start; - if (ss_size1 < ss_size2) + if (rs_size1 < rs_size2) return (-1); - if (ss_size1 > ss_size2) + if (rs_size1 > rs_size2) return (1); - if (s1->ss_start < s2->ss_start) + if (r1->rs_start < r2->rs_start) return (-1); - if (s1->ss_start > s2->ss_start) + + if (r1->rs_start > r2->rs_start) return (1); return (0); } /* - * This is a helper function that can be used by the allocator to find - * a suitable block to allocate. This will search the specified AVL - * tree looking for a block that matches the specified criteria. + * Create any block allocator specific components. The current allocators + * rely on using both a size-ordered range_tree_t and an array of uint64_t's. */ -static uint64_t -metaslab_block_picker(avl_tree_t *t, uint64_t *cursor, uint64_t size, - uint64_t align) +static void +metaslab_rt_create(range_tree_t *rt, void *arg) { - space_seg_t *ss, ssearch; - avl_index_t where; - - ssearch.ss_start = *cursor; - ssearch.ss_end = *cursor + size; - - ss = avl_find(t, &ssearch, &where); - if (ss == NULL) - ss = avl_nearest(t, where, AVL_AFTER); + metaslab_t *msp = arg; - while (ss != NULL) { - uint64_t offset = P2ROUNDUP(ss->ss_start, align); - - if (offset + size <= ss->ss_end) { - *cursor = offset + size; - return (offset); - } - ss = AVL_NEXT(t, ss); - } + ASSERT3P(rt->rt_arg, ==, msp); + ASSERT(msp->ms_tree == NULL); - /* - * If we know we've searched the whole map (*cursor == 0), give up. - * Otherwise, reset the cursor to the beginning and try again. - */ - if (*cursor == 0) - return (-1ULL); - - *cursor = 0; - return (metaslab_block_picker(t, cursor, size, align)); + avl_create(&msp->ms_size_tree, metaslab_rangesize_compare, + sizeof (range_seg_t), offsetof(range_seg_t, rs_pp_node)); } +/* + * Destroy the block allocator specific components. + */ static void -metaslab_pp_load(space_map_t *sm) +metaslab_rt_destroy(range_tree_t *rt, void *arg) { - space_seg_t *ss; + metaslab_t *msp = arg; - ASSERT(sm->sm_ppd == NULL); - sm->sm_ppd = kmem_zalloc(64 * sizeof (uint64_t), KM_SLEEP); + ASSERT3P(rt->rt_arg, ==, msp); + ASSERT3P(msp->ms_tree, ==, rt); + ASSERT0(avl_numnodes(&msp->ms_size_tree)); - sm->sm_pp_root = kmem_alloc(sizeof (avl_tree_t), KM_SLEEP); - avl_create(sm->sm_pp_root, metaslab_segsize_compare, - sizeof (space_seg_t), offsetof(struct space_seg, ss_pp_node)); - - for (ss = avl_first(&sm->sm_root); ss; ss = AVL_NEXT(&sm->sm_root, ss)) - avl_add(sm->sm_pp_root, ss); + avl_destroy(&msp->ms_size_tree); } static void -metaslab_pp_unload(space_map_t *sm) +metaslab_rt_add(range_tree_t *rt, range_seg_t *rs, void *arg) { - void *cookie = NULL; - - kmem_free(sm->sm_ppd, 64 * sizeof (uint64_t)); - sm->sm_ppd = NULL; + metaslab_t *msp = arg; - while (avl_destroy_nodes(sm->sm_pp_root, &cookie) != NULL) { - /* tear down the tree */ - } - - avl_destroy(sm->sm_pp_root); - kmem_free(sm->sm_pp_root, sizeof (avl_tree_t)); - sm->sm_pp_root = NULL; + ASSERT3P(rt->rt_arg, ==, msp); + ASSERT3P(msp->ms_tree, ==, rt); + VERIFY(!msp->ms_condensing); + avl_add(&msp->ms_size_tree, rs); } -/* ARGSUSED */ static void -metaslab_pp_claim(space_map_t *sm, uint64_t start, uint64_t size) +metaslab_rt_remove(range_tree_t *rt, range_seg_t *rs, void *arg) { - /* No need to update cursor */ + metaslab_t *msp = arg; + + ASSERT3P(rt->rt_arg, ==, msp); + ASSERT3P(msp->ms_tree, ==, rt); + VERIFY(!msp->ms_condensing); + avl_remove(&msp->ms_size_tree, rs); } -/* ARGSUSED */ static void -metaslab_pp_free(space_map_t *sm, uint64_t start, uint64_t size) +metaslab_rt_vacate(range_tree_t *rt, void *arg) { - /* No need to update cursor */ + metaslab_t *msp = arg; + + ASSERT3P(rt->rt_arg, ==, msp); + ASSERT3P(msp->ms_tree, ==, rt); + + /* + * Normally one would walk the tree freeing nodes along the way. + * Since the nodes are shared with the range trees we can avoid + * walking all nodes and just reinitialize the avl tree. The nodes + * will be freed by the range tree, so we don't want to free them here. + */ + avl_create(&msp->ms_size_tree, metaslab_rangesize_compare, + sizeof (range_seg_t), offsetof(range_seg_t, rs_pp_node)); } +static range_tree_ops_t metaslab_rt_ops = { + metaslab_rt_create, + metaslab_rt_destroy, + metaslab_rt_add, + metaslab_rt_remove, + metaslab_rt_vacate +}; + +/* + * ========================================================================== + * Metaslab block operations + * ========================================================================== + */ + /* * Return the maximum contiguous segment within the metaslab. */ uint64_t -metaslab_pp_maxsize(space_map_t *sm) +metaslab_block_maxsize(metaslab_t *msp) { - avl_tree_t *t = sm->sm_pp_root; - space_seg_t *ss; + avl_tree_t *t = &msp->ms_size_tree; + range_seg_t *rs; - if (t == NULL || (ss = avl_last(t)) == NULL) + if (t == NULL || (rs = avl_last(t)) == NULL) return (0ULL); - return (ss->ss_end - ss->ss_start); + return (rs->rs_end - rs->rs_start); +} + +uint64_t +metaslab_block_alloc(metaslab_t *msp, uint64_t size) +{ + uint64_t start; + range_tree_t *rt = msp->ms_tree; + + VERIFY(!msp->ms_condensing); + + start = msp->ms_ops->msop_alloc(msp, size); + if (start != -1ULL) { + vdev_t *vd = msp->ms_group->mg_vd; + + VERIFY0(P2PHASE(start, 1ULL << vd->vdev_ashift)); + VERIFY0(P2PHASE(size, 1ULL << vd->vdev_ashift)); + VERIFY3U(range_tree_space(rt) - size, <=, msp->ms_size); + range_tree_remove(rt, start, size); + } + return (start); +} + +/* + * ========================================================================== + * Common allocator routines + * ========================================================================== + */ + +/* + * This is a helper function that can be used by the allocator to find + * a suitable block to allocate. This will search the specified AVL + * tree looking for a block that matches the specified criteria. + */ +static uint64_t +metaslab_block_picker(avl_tree_t *t, uint64_t *cursor, uint64_t size, + uint64_t align) +{ + range_seg_t *rs, rsearch; + avl_index_t where; + + rsearch.rs_start = *cursor; + rsearch.rs_end = *cursor + size; + + rs = avl_find(t, &rsearch, &where); + if (rs == NULL) + rs = avl_nearest(t, where, AVL_AFTER); + + while (rs != NULL) { + uint64_t offset = P2ROUNDUP(rs->rs_start, align); + + if (offset + size <= rs->rs_end) { + *cursor = offset + size; + return (offset); + } + rs = AVL_NEXT(t, rs); + } + + /* + * If we know we've searched the whole map (*cursor == 0), give up. + * Otherwise, reset the cursor to the beginning and try again. + */ + if (*cursor == 0) + return (-1ULL); + + *cursor = 0; + return (metaslab_block_picker(t, cursor, size, align)); } /* @@ -579,29 +683,31 @@ metaslab_pp_maxsize(space_map_t *sm) * ========================================================================== */ static uint64_t -metaslab_ff_alloc(space_map_t *sm, uint64_t size) +metaslab_ff_alloc(metaslab_t *msp, uint64_t size) { - avl_tree_t *t = &sm->sm_root; + /* + * Find the largest power of 2 block size that evenly divides the + * requested size. This is used to try to allocate blocks with similar + * alignment from the same area of the metaslab (i.e. same cursor + * bucket) but it does not guarantee that other allocations sizes + * may exist in the same region. + */ uint64_t align = size & -size; - uint64_t *cursor = (uint64_t *)sm->sm_ppd + highbit(align) - 1; + uint64_t *cursor = &msp->ms_lbas[highbit(align) - 1]; + avl_tree_t *t = &msp->ms_tree->rt_root; return (metaslab_block_picker(t, cursor, size, align)); } /* ARGSUSED */ -boolean_t -metaslab_ff_fragmented(space_map_t *sm) +static boolean_t +metaslab_ff_fragmented(metaslab_t *msp) { return (B_TRUE); } -static space_map_ops_t metaslab_ff_ops = { - metaslab_pp_load, - metaslab_pp_unload, +static metaslab_ops_t metaslab_ff_ops = { metaslab_ff_alloc, - metaslab_pp_claim, - metaslab_pp_free, - metaslab_pp_maxsize, metaslab_ff_fragmented }; @@ -614,16 +720,24 @@ static space_map_ops_t metaslab_ff_ops = * ========================================================================== */ static uint64_t -metaslab_df_alloc(space_map_t *sm, uint64_t size) +metaslab_df_alloc(metaslab_t *msp, uint64_t size) { - avl_tree_t *t = &sm->sm_root; + /* + * Find the largest power of 2 block size that evenly divides the + * requested size. This is used to try to allocate blocks with similar + * alignment from the same area of the metaslab (i.e. same cursor + * bucket) but it does not guarantee that other allocations sizes + * may exist in the same region. + */ uint64_t align = size & -size; - uint64_t *cursor = (uint64_t *)sm->sm_ppd + highbit(align) - 1; - uint64_t max_size = metaslab_pp_maxsize(sm); - int free_pct = sm->sm_space * 100 / sm->sm_size; + uint64_t *cursor = &msp->ms_lbas[highbit(align) - 1]; + range_tree_t *rt = msp->ms_tree; + avl_tree_t *t = &rt->rt_root; + uint64_t max_size = metaslab_block_maxsize(msp); + int free_pct = range_tree_space(rt) * 100 / msp->ms_size; - ASSERT(MUTEX_HELD(sm->sm_lock)); - ASSERT3U(avl_numnodes(&sm->sm_root), ==, avl_numnodes(sm->sm_pp_root)); + ASSERT(MUTEX_HELD(&msp->ms_lock)); + ASSERT3U(avl_numnodes(t), ==, avl_numnodes(&msp->ms_size_tree)); if (max_size < size) return (-1ULL); @@ -634,7 +748,7 @@ metaslab_df_alloc(space_map_t *sm, uint6 */ if (max_size < metaslab_df_alloc_threshold || free_pct < metaslab_df_free_pct) { - t = sm->sm_pp_root; + t = &msp->ms_size_tree; *cursor = 0; } @@ -642,10 +756,11 @@ metaslab_df_alloc(space_map_t *sm, uint6 } static boolean_t -metaslab_df_fragmented(space_map_t *sm) +metaslab_df_fragmented(metaslab_t *msp) { - uint64_t max_size = metaslab_pp_maxsize(sm); - int free_pct = sm->sm_space * 100 / sm->sm_size; + range_tree_t *rt = msp->ms_tree; + uint64_t max_size = metaslab_block_maxsize(msp); + int free_pct = range_tree_space(rt) * 100 / msp->ms_size; if (max_size >= metaslab_df_alloc_threshold && free_pct >= metaslab_df_free_pct) @@ -654,182 +769,228 @@ metaslab_df_fragmented(space_map_t *sm) return (B_TRUE); } -static space_map_ops_t metaslab_df_ops = { - metaslab_pp_load, - metaslab_pp_unload, +static metaslab_ops_t metaslab_df_ops = { metaslab_df_alloc, - metaslab_pp_claim, - metaslab_pp_free, - metaslab_pp_maxsize, metaslab_df_fragmented }; /* * ========================================================================== - * Other experimental allocators + * Cursor fit block allocator - + * Select the largest region in the metaslab, set the cursor to the beginning + * of the range and the cursor_end to the end of the range. As allocations + * are made advance the cursor. Continue allocating from the cursor until + * the range is exhausted and then find a new range. * ========================================================================== */ static uint64_t -metaslab_cdf_alloc(space_map_t *sm, uint64_t size) +metaslab_cf_alloc(metaslab_t *msp, uint64_t size) { - avl_tree_t *t = &sm->sm_root; - uint64_t *cursor = (uint64_t *)sm->sm_ppd; - uint64_t *extent_end = (uint64_t *)sm->sm_ppd + 1; - uint64_t max_size = metaslab_pp_maxsize(sm); - uint64_t rsize = size; + range_tree_t *rt = msp->ms_tree; + avl_tree_t *t = &msp->ms_size_tree; + uint64_t *cursor = &msp->ms_lbas[0]; + uint64_t *cursor_end = &msp->ms_lbas[1]; uint64_t offset = 0; - ASSERT(MUTEX_HELD(sm->sm_lock)); - ASSERT3U(avl_numnodes(&sm->sm_root), ==, avl_numnodes(sm->sm_pp_root)); - - if (max_size < size) - return (-1ULL); + ASSERT(MUTEX_HELD(&msp->ms_lock)); + ASSERT3U(avl_numnodes(t), ==, avl_numnodes(&rt->rt_root)); - ASSERT3U(*extent_end, >=, *cursor); + ASSERT3U(*cursor_end, >=, *cursor); - /* - * If we're running low on space switch to using the size - * sorted AVL tree (best-fit). - */ - if ((*cursor + size) > *extent_end) { + if ((*cursor + size) > *cursor_end) { + range_seg_t *rs; - t = sm->sm_pp_root; - *cursor = *extent_end = 0; + rs = avl_last(&msp->ms_size_tree); + if (rs == NULL || (rs->rs_end - rs->rs_start) < size) + return (-1ULL); - if (max_size > 2 * SPA_MAXBLOCKSIZE) - rsize = MIN(metaslab_min_alloc_size, max_size); - offset = metaslab_block_picker(t, extent_end, rsize, 1ULL); - if (offset != -1) - *cursor = offset + size; - } else { - offset = metaslab_block_picker(t, cursor, rsize, 1ULL); + *cursor = rs->rs_start; + *cursor_end = rs->rs_end; } - ASSERT3U(*cursor, <=, *extent_end); + + offset = *cursor; + *cursor += size; + return (offset); } static boolean_t -metaslab_cdf_fragmented(space_map_t *sm) +metaslab_cf_fragmented(metaslab_t *msp) { - uint64_t max_size = metaslab_pp_maxsize(sm); - - if (max_size > (metaslab_min_alloc_size * 10)) - return (B_FALSE); - return (B_TRUE); + return (metaslab_block_maxsize(msp) < metaslab_min_alloc_size); } -static space_map_ops_t metaslab_cdf_ops = { - metaslab_pp_load, - metaslab_pp_unload, - metaslab_cdf_alloc, - metaslab_pp_claim, - metaslab_pp_free, - metaslab_pp_maxsize, - metaslab_cdf_fragmented +static metaslab_ops_t metaslab_cf_ops = { + metaslab_cf_alloc, + metaslab_cf_fragmented }; +/* + * ========================================================================== + * New dynamic fit allocator - + * Select a region that is large enough to allocate 2^metaslab_ndf_clump_shift + * contiguous blocks. If no region is found then just use the largest segment + * that remains. + * ========================================================================== + */ + +/* + * Determines desired number of contiguous blocks (2^metaslab_ndf_clump_shift) + * to request from the allocator. + */ uint64_t metaslab_ndf_clump_shift = 4; static uint64_t -metaslab_ndf_alloc(space_map_t *sm, uint64_t size) +metaslab_ndf_alloc(metaslab_t *msp, uint64_t size) { - avl_tree_t *t = &sm->sm_root; + avl_tree_t *t = &msp->ms_tree->rt_root; avl_index_t where; - space_seg_t *ss, ssearch; + range_seg_t *rs, rsearch; uint64_t hbit = highbit(size); - uint64_t *cursor = (uint64_t *)sm->sm_ppd + hbit - 1; - uint64_t max_size = metaslab_pp_maxsize(sm); + uint64_t *cursor = &msp->ms_lbas[hbit - 1]; + uint64_t max_size = metaslab_block_maxsize(msp); - ASSERT(MUTEX_HELD(sm->sm_lock)); - ASSERT3U(avl_numnodes(&sm->sm_root), ==, avl_numnodes(sm->sm_pp_root)); + ASSERT(MUTEX_HELD(&msp->ms_lock)); + ASSERT3U(avl_numnodes(t), ==, avl_numnodes(&msp->ms_size_tree)); if (max_size < size) return (-1ULL); - ssearch.ss_start = *cursor; - ssearch.ss_end = *cursor + size; + rsearch.rs_start = *cursor; + rsearch.rs_end = *cursor + size; - ss = avl_find(t, &ssearch, &where); - if (ss == NULL || (ss->ss_start + size > ss->ss_end)) { - t = sm->sm_pp_root; + rs = avl_find(t, &rsearch, &where); + if (rs == NULL || (rs->rs_end - rs->rs_start) < size) { + t = &msp->ms_size_tree; - ssearch.ss_start = 0; - ssearch.ss_end = MIN(max_size, + rsearch.rs_start = 0; + rsearch.rs_end = MIN(max_size, 1ULL << (hbit + metaslab_ndf_clump_shift)); - ss = avl_find(t, &ssearch, &where); - if (ss == NULL) - ss = avl_nearest(t, where, AVL_AFTER); - ASSERT(ss != NULL); + rs = avl_find(t, &rsearch, &where); + if (rs == NULL) + rs = avl_nearest(t, where, AVL_AFTER); + ASSERT(rs != NULL); } - if (ss != NULL) { - if (ss->ss_start + size <= ss->ss_end) { - *cursor = ss->ss_start + size; - return (ss->ss_start); - } + if ((rs->rs_end - rs->rs_start) >= size) { + *cursor = rs->rs_start + size; + return (rs->rs_start); } return (-1ULL); } static boolean_t -metaslab_ndf_fragmented(space_map_t *sm) +metaslab_ndf_fragmented(metaslab_t *msp) { - uint64_t max_size = metaslab_pp_maxsize(sm); - - if (max_size > (metaslab_min_alloc_size << metaslab_ndf_clump_shift)) - return (B_FALSE); - return (B_TRUE); + return (metaslab_block_maxsize(msp) <= + (metaslab_min_alloc_size << metaslab_ndf_clump_shift)); } - -static space_map_ops_t metaslab_ndf_ops = { - metaslab_pp_load, - metaslab_pp_unload, +static metaslab_ops_t metaslab_ndf_ops = { metaslab_ndf_alloc, - metaslab_pp_claim, - metaslab_pp_free, - metaslab_pp_maxsize, metaslab_ndf_fragmented }; -space_map_ops_t *zfs_metaslab_ops = &metaslab_df_ops; +metaslab_ops_t *zfs_metaslab_ops = &metaslab_df_ops; /* * ========================================================================== * Metaslabs * ========================================================================== */ + +/* + * Wait for any in-progress metaslab loads to complete. + */ +void +metaslab_load_wait(metaslab_t *msp) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + + while (msp->ms_loading) { + ASSERT(!msp->ms_loaded); + cv_wait(&msp->ms_load_cv, &msp->ms_lock); + } +} + +int +metaslab_load(metaslab_t *msp) +{ + int error = 0; + + ASSERT(MUTEX_HELD(&msp->ms_lock)); + ASSERT(!msp->ms_loaded); + ASSERT(!msp->ms_loading); + + msp->ms_loading = B_TRUE; + + /* + * If the space map has not been allocated yet, then treat + * all the space in the metaslab as free and add it to the + * ms_tree. + */ + if (msp->ms_sm != NULL) + error = space_map_load(msp->ms_sm, msp->ms_tree, SM_FREE); + else + range_tree_add(msp->ms_tree, msp->ms_start, msp->ms_size); + + msp->ms_loaded = (error == 0); + msp->ms_loading = B_FALSE; + + if (msp->ms_loaded) { + for (int t = 0; t < TXG_DEFER_SIZE; t++) { + range_tree_walk(msp->ms_defertree[t], + range_tree_remove, msp->ms_tree); + } + } + cv_broadcast(&msp->ms_load_cv); + return (error); +} + +void +metaslab_unload(metaslab_t *msp) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + range_tree_vacate(msp->ms_tree, NULL, NULL); + msp->ms_loaded = B_FALSE; + msp->ms_weight &= ~METASLAB_ACTIVE_MASK; +} + metaslab_t * -metaslab_init(metaslab_group_t *mg, space_map_obj_t *smo, - uint64_t start, uint64_t size, uint64_t txg) +metaslab_init(metaslab_group_t *mg, uint64_t id, uint64_t object, uint64_t txg) { vdev_t *vd = mg->mg_vd; + objset_t *mos = vd->vdev_spa->spa_meta_objset; metaslab_t *msp; msp = kmem_zalloc(sizeof (metaslab_t), KM_SLEEP); mutex_init(&msp->ms_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&msp->ms_load_cv, NULL, CV_DEFAULT, NULL); + msp->ms_id = id; + msp->ms_start = id << vd->vdev_ms_shift; + msp->ms_size = 1ULL << vd->vdev_ms_shift; - msp->ms_smo_syncing = *smo; + /* + * We only open space map objects that already exist. All others + * will be opened when we finally allocate an object for it. + */ + if (object != 0) { + VERIFY0(space_map_open(&msp->ms_sm, mos, object, msp->ms_start, + msp->ms_size, vd->vdev_ashift, &msp->ms_lock)); + ASSERT(msp->ms_sm != NULL); + } /* - * We create the main space map here, but we don't create the - * allocmaps and freemaps until metaslab_sync_done(). This serves + * We create the main range tree here, but we don't create the + * alloctree and freetree until metaslab_sync_done(). This serves * two purposes: it allows metaslab_sync_done() to detect the * addition of new space; and for debugging, it ensures that we'd * data fault on any attempt to use this metaslab before it's ready. */ - msp->ms_map = kmem_zalloc(sizeof (space_map_t), KM_SLEEP); - space_map_create(msp->ms_map, start, size, - vd->vdev_ashift, &msp->ms_lock); - + msp->ms_tree = range_tree_create(&metaslab_rt_ops, msp, &msp->ms_lock); metaslab_group_add(mg, msp); - if (metaslab_debug && smo->smo_object != 0) { - mutex_enter(&msp->ms_lock); - VERIFY(space_map_load(msp->ms_map, mg->mg_class->mc_ops, - SM_FREE, smo, spa_meta_objset(vd->vdev_spa)) == 0); - mutex_exit(&msp->ms_lock); - } + msp->ms_ops = mg->mg_class->mc_ops; /* * If we're opening an existing pool (txg == 0) or creating @@ -840,6 +1001,17 @@ metaslab_init(metaslab_group_t *mg, spac if (txg <= TXG_INITIAL) metaslab_sync_done(msp, 0); + /* + * If metaslab_debug_load is set and we're initializing a metaslab + * that has an allocated space_map object then load the its space + * map so that can verify frees. + */ + if (metaslab_debug_load && msp->ms_sm != NULL) { + mutex_enter(&msp->ms_lock); + VERIFY0(metaslab_load(msp)); + mutex_exit(&msp->ms_lock); + } + if (txg != 0) { vdev_dirty(vd, 0, NULL, txg); vdev_dirty(vd, VDD_METASLAB, msp, txg); @@ -853,48 +1025,103 @@ metaslab_fini(metaslab_t *msp) { metaslab_group_t *mg = msp->ms_group; - vdev_space_update(mg->mg_vd, - -msp->ms_smo.smo_alloc, 0, -msp->ms_map->sm_size); - metaslab_group_remove(mg, msp); mutex_enter(&msp->ms_lock); - space_map_unload(msp->ms_map); - space_map_destroy(msp->ms_map); - kmem_free(msp->ms_map, sizeof (*msp->ms_map)); + VERIFY(msp->ms_group == NULL); + vdev_space_update(mg->mg_vd, -space_map_allocated(msp->ms_sm), + 0, -msp->ms_size); + space_map_close(msp->ms_sm); + + metaslab_unload(msp); + range_tree_destroy(msp->ms_tree); for (int t = 0; t < TXG_SIZE; t++) { - space_map_destroy(msp->ms_allocmap[t]); - space_map_destroy(msp->ms_freemap[t]); - kmem_free(msp->ms_allocmap[t], sizeof (*msp->ms_allocmap[t])); - kmem_free(msp->ms_freemap[t], sizeof (*msp->ms_freemap[t])); + range_tree_destroy(msp->ms_alloctree[t]); + range_tree_destroy(msp->ms_freetree[t]); } for (int t = 0; t < TXG_DEFER_SIZE; t++) { - space_map_destroy(msp->ms_defermap[t]); - kmem_free(msp->ms_defermap[t], sizeof (*msp->ms_defermap[t])); + range_tree_destroy(msp->ms_defertree[t]); } ASSERT0(msp->ms_deferspace); mutex_exit(&msp->ms_lock); + cv_destroy(&msp->ms_load_cv); mutex_destroy(&msp->ms_lock); kmem_free(msp, sizeof (metaslab_t)); } -#define METASLAB_WEIGHT_PRIMARY (1ULL << 63) -#define METASLAB_WEIGHT_SECONDARY (1ULL << 62) -#define METASLAB_ACTIVE_MASK \ - (METASLAB_WEIGHT_PRIMARY | METASLAB_WEIGHT_SECONDARY) +/* + * Apply a weighting factor based on the histogram information for this + * metaslab. The current weighting factor is somewhat arbitrary and requires + * additional investigation. The implementation provides a measure of + * "weighted" free space and gives a higher weighting for larger contiguous + * regions. The weighting factor is determined by counting the number of + * sm_shift sectors that exist in each region represented by the histogram. + * That value is then multiplied by the power of 2 exponent and the sm_shift + * value. + * + * For example, assume the 2^21 histogram bucket has 4 2MB regions and the + * metaslab has an sm_shift value of 9 (512B): + * + * 1) calculate the number of sm_shift sectors in the region: + * 2^21 / 2^9 = 2^12 = 4096 * 4 (number of regions) = 16384 + * 2) multiply by the power of 2 exponent and the sm_shift value: + * 16384 * 21 * 9 = 3096576 + * This value will be added to the weighting of the metaslab. + */ +static uint64_t +metaslab_weight_factor(metaslab_t *msp) +{ + uint64_t factor = 0; + uint64_t sectors; + int i; + + /* + * A null space map means that the entire metaslab is free, + * calculate a weight factor that spans the entire size of the + * metaslab. *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:50:55 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5554DA63; Wed, 20 Nov 2013 10:50:55 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 43A3A2061; Wed, 20 Nov 2013 10:50:55 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAotbX055785; Wed, 20 Nov 2013 10:50:55 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAosl4055779; Wed, 20 Nov 2013 10:50:54 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201050.rAKAosl4055779@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:50:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258372 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:50:55 -0000 Author: avg Date: Wed Nov 20 10:50:54 2013 New Revision: 258372 URL: http://svnweb.freebsd.org/changeset/base/258372 Log: fix a screw up in r258371 Pointyhat to: avg Added: vendor-sys/illumos/dist/uts/common/fs/zfs/space_reftree.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h (contents, props changed) Replaced: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h (contents, props changed) Added: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c Wed Nov 20 10:50:54 2013 (r258372) @@ -0,0 +1,391 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ +/* + * Copyright (c) 2013 by Delphix. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include + +static kmem_cache_t *range_seg_cache; + +void +range_tree_init(void) +{ + ASSERT(range_seg_cache == NULL); + range_seg_cache = kmem_cache_create("range_seg_cache", + sizeof (range_seg_t), 0, NULL, NULL, NULL, NULL, NULL, 0); +} + +void +range_tree_fini(void) +{ + kmem_cache_destroy(range_seg_cache); + range_seg_cache = NULL; +} + +void +range_tree_stat_verify(range_tree_t *rt) +{ + range_seg_t *rs; + uint64_t hist[RANGE_TREE_HISTOGRAM_SIZE] = { 0 }; + int i; + + for (rs = avl_first(&rt->rt_root); rs != NULL; + rs = AVL_NEXT(&rt->rt_root, rs)) { + uint64_t size = rs->rs_end - rs->rs_start; + int idx = highbit(size) - 1; + + hist[idx]++; + ASSERT3U(hist[idx], !=, 0); + } + + for (i = 0; i < RANGE_TREE_HISTOGRAM_SIZE; i++) { + if (hist[i] != rt->rt_histogram[i]) { + zfs_dbgmsg("i=%d, hist=%p, hist=%llu, rt_hist=%llu", + i, hist, hist[i], rt->rt_histogram[i]); + } + VERIFY3U(hist[i], ==, rt->rt_histogram[i]); + } +} + +static void +range_tree_stat_incr(range_tree_t *rt, range_seg_t *rs) +{ + uint64_t size = rs->rs_end - rs->rs_start; + int idx = highbit(size) - 1; + + ASSERT3U(idx, <, + sizeof (rt->rt_histogram) / sizeof (*rt->rt_histogram)); + + ASSERT(MUTEX_HELD(rt->rt_lock)); + rt->rt_histogram[idx]++; + ASSERT3U(rt->rt_histogram[idx], !=, 0); +} + +static void +range_tree_stat_decr(range_tree_t *rt, range_seg_t *rs) +{ + uint64_t size = rs->rs_end - rs->rs_start; + int idx = highbit(size) - 1; + + ASSERT3U(idx, <, + sizeof (rt->rt_histogram) / sizeof (*rt->rt_histogram)); + + ASSERT(MUTEX_HELD(rt->rt_lock)); + ASSERT3U(rt->rt_histogram[idx], !=, 0); + rt->rt_histogram[idx]--; +} + +/* + * NOTE: caller is responsible for all locking. + */ +static int +range_tree_seg_compare(const void *x1, const void *x2) +{ + const range_seg_t *r1 = x1; + const range_seg_t *r2 = x2; + + if (r1->rs_start < r2->rs_start) { + if (r1->rs_end > r2->rs_start) + return (0); + return (-1); + } + if (r1->rs_start > r2->rs_start) { + if (r1->rs_start < r2->rs_end) + return (0); + return (1); + } + return (0); +} + +range_tree_t * +range_tree_create(range_tree_ops_t *ops, void *arg, kmutex_t *lp) +{ + range_tree_t *rt; + + rt = kmem_zalloc(sizeof (range_tree_t), KM_SLEEP); + + avl_create(&rt->rt_root, range_tree_seg_compare, + sizeof (range_seg_t), offsetof(range_seg_t, rs_node)); + + rt->rt_lock = lp; + rt->rt_ops = ops; + rt->rt_arg = arg; + + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_create(rt, rt->rt_arg); + + return (rt); +} + +void +range_tree_destroy(range_tree_t *rt) +{ + VERIFY0(rt->rt_space); + + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_destroy(rt, rt->rt_arg); + + avl_destroy(&rt->rt_root); + kmem_free(rt, sizeof (*rt)); +} + +void +range_tree_add(void *arg, uint64_t start, uint64_t size) +{ + range_tree_t *rt = arg; + avl_index_t where; + range_seg_t rsearch, *rs_before, *rs_after, *rs; + uint64_t end = start + size; + boolean_t merge_before, merge_after; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + VERIFY(size != 0); + + rsearch.rs_start = start; + rsearch.rs_end = end; + rs = avl_find(&rt->rt_root, &rsearch, &where); + + if (rs != NULL && rs->rs_start <= start && rs->rs_end >= end) { + zfs_panic_recover("zfs: allocating allocated segment" + "(offset=%llu size=%llu)\n", + (longlong_t)start, (longlong_t)size); + return; + } + + /* Make sure we don't overlap with either of our neighbors */ + VERIFY(rs == NULL); + + rs_before = avl_nearest(&rt->rt_root, where, AVL_BEFORE); + rs_after = avl_nearest(&rt->rt_root, where, AVL_AFTER); + + merge_before = (rs_before != NULL && rs_before->rs_end == start); + merge_after = (rs_after != NULL && rs_after->rs_start == end); + + if (merge_before && merge_after) { + avl_remove(&rt->rt_root, rs_before); + if (rt->rt_ops != NULL) { + rt->rt_ops->rtop_remove(rt, rs_before, rt->rt_arg); + rt->rt_ops->rtop_remove(rt, rs_after, rt->rt_arg); + } + + range_tree_stat_decr(rt, rs_before); + range_tree_stat_decr(rt, rs_after); + + rs_after->rs_start = rs_before->rs_start; + kmem_cache_free(range_seg_cache, rs_before); + rs = rs_after; + } else if (merge_before) { + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_remove(rt, rs_before, rt->rt_arg); + + range_tree_stat_decr(rt, rs_before); + + rs_before->rs_end = end; + rs = rs_before; + } else if (merge_after) { + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_remove(rt, rs_after, rt->rt_arg); + + range_tree_stat_decr(rt, rs_after); + + rs_after->rs_start = start; + rs = rs_after; + } else { + rs = kmem_cache_alloc(range_seg_cache, KM_SLEEP); + rs->rs_start = start; + rs->rs_end = end; + avl_insert(&rt->rt_root, rs, where); + } + + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_add(rt, rs, rt->rt_arg); + + range_tree_stat_incr(rt, rs); + rt->rt_space += size; +} + +void +range_tree_remove(void *arg, uint64_t start, uint64_t size) +{ + range_tree_t *rt = arg; + avl_index_t where; + range_seg_t rsearch, *rs, *newseg; + uint64_t end = start + size; + boolean_t left_over, right_over; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + VERIFY3U(size, !=, 0); + VERIFY3U(size, <=, rt->rt_space); + + rsearch.rs_start = start; + rsearch.rs_end = end; + rs = avl_find(&rt->rt_root, &rsearch, &where); + + /* Make sure we completely overlap with someone */ + if (rs == NULL) { + zfs_panic_recover("zfs: freeing free segment " + "(offset=%llu size=%llu)", + (longlong_t)start, (longlong_t)size); + return; + } + VERIFY3U(rs->rs_start, <=, start); + VERIFY3U(rs->rs_end, >=, end); + + left_over = (rs->rs_start != start); + right_over = (rs->rs_end != end); + + range_tree_stat_decr(rt, rs); + + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_remove(rt, rs, rt->rt_arg); + + if (left_over && right_over) { + newseg = kmem_cache_alloc(range_seg_cache, KM_SLEEP); + newseg->rs_start = end; + newseg->rs_end = rs->rs_end; + range_tree_stat_incr(rt, newseg); + + rs->rs_end = start; + + avl_insert_here(&rt->rt_root, newseg, rs, AVL_AFTER); + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_add(rt, newseg, rt->rt_arg); + } else if (left_over) { + rs->rs_end = start; + } else if (right_over) { + rs->rs_start = end; + } else { + avl_remove(&rt->rt_root, rs); + kmem_cache_free(range_seg_cache, rs); + rs = NULL; + } + + if (rs != NULL) { + range_tree_stat_incr(rt, rs); + + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_add(rt, rs, rt->rt_arg); + } + + rt->rt_space -= size; +} + +static range_seg_t * +range_tree_find(range_tree_t *rt, uint64_t start, uint64_t size, + avl_index_t *wherep) +{ + range_seg_t rsearch, *rs; + uint64_t end = start + size; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + VERIFY(size != 0); + + rsearch.rs_start = start; + rsearch.rs_end = end; + rs = avl_find(&rt->rt_root, &rsearch, wherep); + + if (rs != NULL && rs->rs_start <= start && rs->rs_end >= end) + return (rs); + return (NULL); +} + +void +range_tree_verify(range_tree_t *rt, uint64_t off, uint64_t size) +{ + range_seg_t *rs; + avl_index_t where; + + mutex_enter(rt->rt_lock); + rs = range_tree_find(rt, off, size, &where); + if (rs != NULL) + panic("freeing free block; rs=%p", (void *)rs); + mutex_exit(rt->rt_lock); +} + +boolean_t +range_tree_contains(range_tree_t *rt, uint64_t start, uint64_t size) +{ + avl_index_t where; + + return (range_tree_find(rt, start, size, &where) != NULL); +} + +void +range_tree_swap(range_tree_t **rtsrc, range_tree_t **rtdst) +{ + range_tree_t *rt; + + ASSERT(MUTEX_HELD((*rtsrc)->rt_lock)); + ASSERT0(range_tree_space(*rtdst)); + ASSERT0(avl_numnodes(&(*rtdst)->rt_root)); + + rt = *rtsrc; + *rtsrc = *rtdst; + *rtdst = rt; +} + +void +range_tree_vacate(range_tree_t *rt, range_tree_func_t *func, void *arg) +{ + range_seg_t *rs; + void *cookie = NULL; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + + if (rt->rt_ops != NULL) + rt->rt_ops->rtop_vacate(rt, rt->rt_arg); + + while ((rs = avl_destroy_nodes(&rt->rt_root, &cookie)) != NULL) { + if (func != NULL) + func(arg, rs->rs_start, rs->rs_end - rs->rs_start); + kmem_cache_free(range_seg_cache, rs); + } + + bzero(rt->rt_histogram, sizeof (rt->rt_histogram)); + rt->rt_space = 0; +} + +void +range_tree_walk(range_tree_t *rt, range_tree_func_t *func, void *arg) +{ + range_seg_t *rs; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + + for (rs = avl_first(&rt->rt_root); rs; rs = AVL_NEXT(&rt->rt_root, rs)) + func(arg, rs->rs_start, rs->rs_end - rs->rs_start); +} + +uint64_t +range_tree_space(range_tree_t *rt) +{ + return (rt->rt_space); +} Added: vendor-sys/illumos/dist/uts/common/fs/zfs/space_reftree.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/space_reftree.c Wed Nov 20 10:50:54 2013 (r258372) @@ -0,0 +1,159 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ +/* + * Copyright (c) 2013 by Delphix. All rights reserved. + */ + +#include +#include +#include + +/* + * Space reference trees. + * + * A range tree is a collection of integers. Every integer is either + * in the tree, or it's not. A space reference tree generalizes + * the idea: it allows its members to have arbitrary reference counts, + * as opposed to the implicit reference count of 0 or 1 in a range tree. + * This representation comes in handy when computing the union or + * intersection of multiple space maps. For example, the union of + * N range trees is the subset of the reference tree with refcnt >= 1. + * The intersection of N range trees is the subset with refcnt >= N. + * + * [It's very much like a Fourier transform. Unions and intersections + * are hard to perform in the 'range tree domain', so we convert the trees + * into the 'reference count domain', where it's trivial, then invert.] + * + * vdev_dtl_reassess() uses computations of this form to determine + * DTL_MISSING and DTL_OUTAGE for interior vdevs -- e.g. a RAID-Z vdev + * has an outage wherever refcnt >= vdev_nparity + 1, and a mirror vdev + * has an outage wherever refcnt >= vdev_children. + */ +static int +space_reftree_compare(const void *x1, const void *x2) +{ + const space_ref_t *sr1 = x1; + const space_ref_t *sr2 = x2; + + if (sr1->sr_offset < sr2->sr_offset) + return (-1); + if (sr1->sr_offset > sr2->sr_offset) + return (1); + + if (sr1 < sr2) + return (-1); + if (sr1 > sr2) + return (1); + + return (0); +} + +void +space_reftree_create(avl_tree_t *t) +{ + avl_create(t, space_reftree_compare, + sizeof (space_ref_t), offsetof(space_ref_t, sr_node)); +} + +void +space_reftree_destroy(avl_tree_t *t) +{ + space_ref_t *sr; + void *cookie = NULL; + + while ((sr = avl_destroy_nodes(t, &cookie)) != NULL) + kmem_free(sr, sizeof (*sr)); + + avl_destroy(t); +} + +static void +space_reftree_add_node(avl_tree_t *t, uint64_t offset, int64_t refcnt) +{ + space_ref_t *sr; + + sr = kmem_alloc(sizeof (*sr), KM_SLEEP); + sr->sr_offset = offset; + sr->sr_refcnt = refcnt; + + avl_add(t, sr); +} + +void +space_reftree_add_seg(avl_tree_t *t, uint64_t start, uint64_t end, + int64_t refcnt) +{ + space_reftree_add_node(t, start, refcnt); + space_reftree_add_node(t, end, -refcnt); +} + +/* + * Convert (or add) a range tree into a reference tree. + */ +void +space_reftree_add_map(avl_tree_t *t, range_tree_t *rt, int64_t refcnt) +{ + range_seg_t *rs; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + + for (rs = avl_first(&rt->rt_root); rs; rs = AVL_NEXT(&rt->rt_root, rs)) + space_reftree_add_seg(t, rs->rs_start, rs->rs_end, refcnt); +} + +/* + * Convert a reference tree into a range tree. The range tree will contain + * all members of the reference tree for which refcnt >= minref. + */ +void +space_reftree_generate_map(avl_tree_t *t, range_tree_t *rt, int64_t minref) +{ + uint64_t start = -1ULL; + int64_t refcnt = 0; + space_ref_t *sr; + + ASSERT(MUTEX_HELD(rt->rt_lock)); + + range_tree_vacate(rt, NULL, NULL); + + for (sr = avl_first(t); sr != NULL; sr = AVL_NEXT(t, sr)) { + refcnt += sr->sr_refcnt; + if (refcnt >= minref) { + if (start == -1ULL) { + start = sr->sr_offset; + } + } else { + if (start != -1ULL) { + uint64_t end = sr->sr_offset; + ASSERT(start <= end); + if (end > start) + range_tree_add(rt, start, end - start); + start = -1ULL; + } + } + } + ASSERT(refcnt == 0); + ASSERT(start == -1ULL); +} Added: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h Wed Nov 20 10:50:54 2013 (r258372) @@ -0,0 +1,96 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Copyright (c) 2013 by Delphix. All rights reserved. + */ + +#ifndef _SYS_RANGE_TREE_H +#define _SYS_RANGE_TREE_H + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define RANGE_TREE_HISTOGRAM_SIZE 64 + +typedef struct range_tree_ops range_tree_ops_t; + +typedef struct range_tree { + avl_tree_t rt_root; /* offset-ordered segment AVL tree */ + uint64_t rt_space; /* sum of all segments in the map */ + range_tree_ops_t *rt_ops; + void *rt_arg; + + /* + * The rt_histogram maintains a histogram of ranges. Each bucket, + * rt_histogram[i], contains the number of ranges whose size is: + * 2^i <= size of range in bytes < 2^(i+1) + */ + uint64_t rt_histogram[RANGE_TREE_HISTOGRAM_SIZE]; + kmutex_t *rt_lock; /* pointer to lock that protects map */ +} range_tree_t; + +typedef struct range_seg { + avl_node_t rs_node; /* AVL node */ + avl_node_t rs_pp_node; /* AVL picker-private node */ + uint64_t rs_start; /* starting offset of this segment */ + uint64_t rs_end; /* ending offset (non-inclusive) */ +} range_seg_t; + +struct range_tree_ops { + void (*rtop_create)(range_tree_t *rt, void *arg); + void (*rtop_destroy)(range_tree_t *rt, void *arg); + void (*rtop_add)(range_tree_t *rt, range_seg_t *rs, void *arg); + void (*rtop_remove)(range_tree_t *rt, range_seg_t *rs, void *arg); + void (*rtop_vacate)(range_tree_t *rt, void *arg); +}; + +typedef void range_tree_func_t(void *arg, uint64_t start, uint64_t size); + +void range_tree_init(void); +void range_tree_fini(void); +range_tree_t *range_tree_create(range_tree_ops_t *ops, void *arg, kmutex_t *lp); +void range_tree_destroy(range_tree_t *rt); +boolean_t range_tree_contains(range_tree_t *rt, uint64_t start, uint64_t size); +uint64_t range_tree_space(range_tree_t *rt); +void range_tree_verify(range_tree_t *rt, uint64_t start, uint64_t size); +void range_tree_swap(range_tree_t **rtsrc, range_tree_t **rtdst); +void range_tree_stat_verify(range_tree_t *rt); + +void range_tree_add(void *arg, uint64_t start, uint64_t size); +void range_tree_remove(void *arg, uint64_t start, uint64_t size); + +void range_tree_vacate(range_tree_t *rt, range_tree_func_t *func, void *arg); +void range_tree_walk(range_tree_t *rt, range_tree_func_t *func, void *arg); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_RANGE_TREE_H */ Added: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_reftree.h Wed Nov 20 10:50:54 2013 (r258372) @@ -0,0 +1,57 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Copyright (c) 2013 by Delphix. All rights reserved. + */ + +#ifndef _SYS_SPACE_REFTREE_H +#define _SYS_SPACE_REFTREE_H + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct space_ref { + avl_node_t sr_node; /* AVL node */ + uint64_t sr_offset; /* range offset (start or end) */ + int64_t sr_refcnt; /* associated reference count */ +} space_ref_t; + +void space_reftree_create(avl_tree_t *t); +void space_reftree_destroy(avl_tree_t *t); +void space_reftree_add_seg(avl_tree_t *t, uint64_t start, uint64_t end, + int64_t refcnt); +void space_reftree_add_map(avl_tree_t *t, range_tree_t *rt, int64_t refcnt); +void space_reftree_generate_map(avl_tree_t *t, range_tree_t *rt, + int64_t minref); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_SPACE_REFTREE_H */ From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:52:49 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F567BDE; Wed, 20 Nov 2013 10:52:49 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8DEA82088; Wed, 20 Nov 2013 10:52:49 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAqn1C056031; Wed, 20 Nov 2013 10:52:49 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAqnHX056029; Wed, 20 Nov 2013 10:52:49 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201052.rAKAqnHX056029@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:52:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258373 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/zhack X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:52:49 -0000 Author: avg Date: Wed Nov 20 10:52:48 2013 New Revision: 258373 URL: http://svnweb.freebsd.org/changeset/base/258373 Log: 4168 ztest assertion failure in dbuf_undirty 4169 verbatim import causes zdb to segfa 4170 zhack leaves pool in ACTIVE state illumos/illumos-gate@7fdd916c474ea52896c671bbe7b56ba34a1ca132 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zhack/zhack.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed Nov 20 10:50:54 2013 (r258372) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Wed Nov 20 10:52:48 2013 (r258373) @@ -1334,14 +1334,6 @@ dbuf_undirty(dmu_buf_impl_t *db, dmu_tx_ DB_DNODE_ENTER(db); dn = DB_DNODE(db); - /* - * Note: This code will probably work even if there are concurrent - * holders, but it is untested in that scenerio, as the ZPL and - * ztest have additional locking (the range locks) that prevents - * that type of concurrent access. - */ - ASSERT3U(refcount_count(&db->db_holds), ==, db->db_dirtycnt); - dprintf_dbuf(db, "size=%llx\n", (u_longlong_t)db->db.db_size); ASSERT(db->db.db_size != 0); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Wed Nov 20 10:50:54 2013 (r258372) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Wed Nov 20 10:52:48 2013 (r258373) @@ -3842,8 +3842,6 @@ spa_import(const char *pool, nvlist_t *c spa_config_sync(spa, B_FALSE, B_TRUE); mutex_exit(&spa_namespace_lock); - spa_history_log_version(spa, "import"); - return (0); } From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:52:50 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E885BDF; Wed, 20 Nov 2013 10:52:50 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E2E772089; Wed, 20 Nov 2013 10:52:49 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAqnxM056037; Wed, 20 Nov 2013 10:52:49 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAqnI9056036; Wed, 20 Nov 2013 10:52:49 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201052.rAKAqnI9056036@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:52:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258373 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/zhack X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:52:50 -0000 Author: avg Date: Wed Nov 20 10:52:48 2013 New Revision: 258373 URL: http://svnweb.freebsd.org/changeset/base/258373 Log: 4168 ztest assertion failure in dbuf_undirty 4169 verbatim import causes zdb to segfa 4170 zhack leaves pool in ACTIVE state illumos/illumos-gate@7fdd916c474ea52896c671bbe7b56ba34a1ca132 Modified: vendor/illumos/dist/cmd/zhack/zhack.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Modified: vendor/illumos/dist/cmd/zhack/zhack.c ============================================================================== --- vendor/illumos/dist/cmd/zhack/zhack.c Wed Nov 20 10:50:54 2013 (r258372) +++ vendor/illumos/dist/cmd/zhack/zhack.c Wed Nov 20 10:52:48 2013 (r258373) @@ -20,7 +20,7 @@ */ /* - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. */ @@ -85,10 +85,15 @@ usage(void) static void -fatal(const char *fmt, ...) +fatal(spa_t *spa, void *tag, const char *fmt, ...) { va_list ap; + if (spa != NULL) { + spa_close(spa, tag); + (void) spa_export(g_pool, NULL, B_TRUE, B_FALSE); + } + va_start(ap, fmt); (void) fprintf(stderr, "%s: ", cmdname); (void) vfprintf(stderr, fmt, ap); @@ -159,13 +164,14 @@ import_pool(const char *target, boolean_ g_importargs.can_be_active = B_TRUE; if (zpool_search_import(g_zfs, &g_importargs) != NULL || spa_open(target, &spa, FTAG) == 0) { - fatal("cannot import '%s': pool is active; run " - "\"zpool export %s\" first\n", - g_pool, g_pool); + fatal(spa, FTAG, "cannot import '%s': pool is " + "active; run " "\"zpool export %s\" " + "first\n", g_pool, g_pool); } } - fatal("cannot import '%s': no such pool available\n", g_pool); + fatal(NULL, FTAG, "cannot import '%s': no such pool " + "available\n", g_pool); } elem = nvlist_next_nvpair(pools, NULL); @@ -186,7 +192,8 @@ import_pool(const char *target, boolean_ error = 0; if (error) - fatal("can't import '%s': %s", name, strerror(error)); + fatal(NULL, FTAG, "can't import '%s': %s", name, + strerror(error)); } static void @@ -201,10 +208,11 @@ zhack_spa_open(const char *target, boole zfeature_checks_disable = B_FALSE; if (err != 0) - fatal("cannot open '%s': %s", target, strerror(err)); + fatal(*spa, FTAG, "cannot open '%s': %s", target, + strerror(err)); if (spa_version(*spa) < SPA_VERSION_FEATURES) { - fatal("'%s' has version %d, features not enabled", target, - (int)spa_version(*spa)); + fatal(*spa, FTAG, "'%s' has version %d, features not enabled", + target, (int)spa_version(*spa)); } } @@ -336,15 +344,16 @@ zhack_do_feature_enable(int argc, char * feature.fi_guid = argv[1]; if (!zfeature_is_valid_guid(feature.fi_guid)) - fatal("invalid feature guid: %s", feature.fi_guid); + fatal(NULL, FTAG, "invalid feature guid: %s", feature.fi_guid); zhack_spa_open(target, B_FALSE, FTAG, &spa); mos = spa->spa_meta_objset; if (0 == zfeature_lookup_guid(feature.fi_guid, NULL)) - fatal("'%s' is a real feature, will not enable"); + fatal(spa, FTAG, "'%s' is a real feature, will not enable"); if (0 == zap_contains(mos, spa->spa_feat_desc_obj, feature.fi_guid)) - fatal("feature already enabled: %s", feature.fi_guid); + fatal(spa, FTAG, "feature already enabled: %s", + feature.fi_guid); VERIFY0(dsl_sync_task(spa_name(spa), NULL, feature_enable_sync, &feature, 5)); @@ -423,13 +432,14 @@ zhack_do_feature_ref(int argc, char **ar feature.fi_guid = argv[1]; if (!zfeature_is_valid_guid(feature.fi_guid)) - fatal("invalid feature guid: %s", feature.fi_guid); + fatal(NULL, FTAG, "invalid feature guid: %s", feature.fi_guid); zhack_spa_open(target, B_FALSE, FTAG, &spa); mos = spa->spa_meta_objset; if (0 == zfeature_lookup_guid(feature.fi_guid, NULL)) - fatal("'%s' is a real feature, will not change refcount"); + fatal(spa, FTAG, "'%s' is a real feature, will not change " + "refcount"); if (0 == zap_contains(mos, spa->spa_feat_for_read_obj, feature.fi_guid)) { @@ -438,11 +448,12 @@ zhack_do_feature_ref(int argc, char **ar feature.fi_guid)) { feature.fi_can_readonly = B_TRUE; } else { - fatal("feature is not enabled: %s", feature.fi_guid); + fatal(spa, FTAG, "feature is not enabled: %s", feature.fi_guid); } if (decr && !spa_feature_is_active(spa, &feature)) - fatal("feature refcount already 0: %s", feature.fi_guid); + fatal(spa, FTAG, "feature refcount already 0: %s", + feature.fi_guid); VERIFY0(dsl_sync_task(spa_name(spa), NULL, decr ? feature_decr_sync : feature_incr_sync, &feature, 5)); @@ -530,8 +541,8 @@ main(int argc, char **argv) usage(); } - if (!g_readonly && spa_export(g_pool, NULL, B_TRUE, B_TRUE) != 0) { - fatal("pool export failed; " + if (!g_readonly && spa_export(g_pool, NULL, B_TRUE, B_FALSE) != 0) { + fatal(NULL, FTAG, "pool export failed; " "changes may not be committed to disk\n"); } From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:54:10 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0A07E6A; Wed, 20 Nov 2013 10:54:10 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ADA4620AF; Wed, 20 Nov 2013 10:54:10 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAsAds056252; Wed, 20 Nov 2013 10:54:10 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAs63j056222; Wed, 20 Nov 2013 10:54:06 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201054.rAKAs63j056222@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:54:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258374 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/... X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:54:10 -0000 Author: avg Date: Wed Nov 20 10:54:06 2013 New Revision: 258374 URL: http://svnweb.freebsd.org/changeset/base/258374 Log: 4171 clean up spa_feature_*() interfaces 4172 implement extensible_dataset feature for use by other zpool features illumos/illumos-gate@2acef22db7808606888f8f92715629ff3ba555b9 Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zap.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/zhack/zhack.c vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c vendor/illumos/dist/man/man5/zpool-features.5 Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.c Wed Nov 20 10:54:06 2013 (r258374) @@ -92,32 +92,22 @@ zfeature_is_supported(const char *guid) if (zfeature_checks_disable) return (B_TRUE); - return (0 == zfeature_lookup_guid(guid, NULL)); -} - -int -zfeature_lookup_guid(const char *guid, zfeature_info_t **res) -{ - for (int i = 0; i < SPA_FEATURES; i++) { + for (spa_feature_t i = 0; i < SPA_FEATURES; i++) { zfeature_info_t *feature = &spa_feature_table[i]; - if (strcmp(guid, feature->fi_guid) == 0) { - if (res != NULL) - *res = feature; - return (0); - } + if (strcmp(guid, feature->fi_guid) == 0) + return (B_TRUE); } - - return (ENOENT); + return (B_FALSE); } int -zfeature_lookup_name(const char *name, zfeature_info_t **res) +zfeature_lookup_name(const char *name, spa_feature_t *res) { - for (int i = 0; i < SPA_FEATURES; i++) { + for (spa_feature_t i = 0; i < SPA_FEATURES; i++) { zfeature_info_t *feature = &spa_feature_table[i]; if (strcmp(name, feature->fi_uname) == 0) { if (res != NULL) - *res = feature; + *res = i; return (0); } } @@ -126,11 +116,12 @@ zfeature_lookup_name(const char *name, z } static void -zfeature_register(int fid, const char *guid, const char *name, const char *desc, - boolean_t readonly, boolean_t mos, zfeature_info_t **deps) +zfeature_register(spa_feature_t fid, const char *guid, const char *name, + const char *desc, boolean_t readonly, boolean_t mos, + const spa_feature_t *deps) { zfeature_info_t *feature = &spa_feature_table[fid]; - static zfeature_info_t *nodeps[] = { NULL }; + static spa_feature_t nodeps[] = { SPA_FEATURE_NONE }; ASSERT(name != NULL); ASSERT(desc != NULL); @@ -141,6 +132,7 @@ zfeature_register(int fid, const char *g if (deps == NULL) deps = nodeps; + feature->fi_feature = fid; feature->fi_guid = guid; feature->fi_uname = name; feature->fi_desc = desc; @@ -167,4 +159,8 @@ zpool_feature_init(void) zfeature_register(SPA_FEATURE_SPACEMAP_HISTOGRAM, "com.delphix:spacemap_histogram", "spacemap_histogram", "Spacemaps maintain space histograms.", B_TRUE, B_FALSE, NULL); + zfeature_register(SPA_FEATURE_EXTENSIBLE_DATASET, + "com.delphix:extensible_dataset", "extensible_dataset", + "Enhanced dataset functionality, used by other features.", + B_FALSE, B_FALSE, NULL); } Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.h ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.h Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.h Wed Nov 20 10:54:06 2013 (r258374) @@ -38,35 +38,38 @@ extern "C" { struct zfeature_info; +typedef enum spa_feature { + SPA_FEATURE_NONE = -1, + SPA_FEATURE_ASYNC_DESTROY, + SPA_FEATURE_EMPTY_BPOBJ, + SPA_FEATURE_LZ4_COMPRESS, + SPA_FEATURE_MULTI_VDEV_CRASH_DUMP, + SPA_FEATURE_SPACEMAP_HISTOGRAM, + SPA_FEATURE_EXTENSIBLE_DATASET, + SPA_FEATURES +} spa_feature_t; + typedef struct zfeature_info { + spa_feature_t fi_feature; const char *fi_uname; /* User-facing feature name */ const char *fi_guid; /* On-disk feature identifier */ const char *fi_desc; /* Feature description */ boolean_t fi_can_readonly; /* Can open pool readonly w/o support? */ boolean_t fi_mos; /* Is the feature necessary to read the MOS? */ - struct zfeature_info **fi_depends; /* array; null terminated */ + /* array of dependencies, terminated by SPA_FEATURE_NONE */ + const spa_feature_t *fi_depends; } zfeature_info_t; typedef int (zfeature_func_t)(zfeature_info_t *fi, void *arg); #define ZFS_FEATURE_DEBUG -enum spa_feature { - SPA_FEATURE_ASYNC_DESTROY, - SPA_FEATURE_EMPTY_BPOBJ, - SPA_FEATURE_LZ4_COMPRESS, - SPA_FEATURE_MULTI_VDEV_CRASH_DUMP, - SPA_FEATURE_SPACEMAP_HISTOGRAM, - SPA_FEATURES -} spa_feature_t; - extern zfeature_info_t spa_feature_table[SPA_FEATURES]; extern boolean_t zfeature_is_valid_guid(const char *); extern boolean_t zfeature_is_supported(const char *); -extern int zfeature_lookup_guid(const char *, zfeature_info_t **res); -extern int zfeature_lookup_name(const char *, zfeature_info_t **res); +extern int zfeature_lookup_name(const char *name, spa_feature_t *res); extern void zpool_feature_init(void); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c Wed Nov 20 10:54:06 2013 (r258374) @@ -36,13 +36,11 @@ uint64_t bpobj_alloc_empty(objset_t *os, int blocksize, dmu_tx_t *tx) { - zfeature_info_t *empty_bpobj_feat = - &spa_feature_table[SPA_FEATURE_EMPTY_BPOBJ]; spa_t *spa = dmu_objset_spa(os); dsl_pool_t *dp = dmu_objset_pool(os); - if (spa_feature_is_enabled(spa, empty_bpobj_feat)) { - if (!spa_feature_is_active(spa, empty_bpobj_feat)) { + if (spa_feature_is_enabled(spa, SPA_FEATURE_EMPTY_BPOBJ)) { + if (!spa_feature_is_active(spa, SPA_FEATURE_EMPTY_BPOBJ)) { ASSERT0(dp->dp_empty_bpobj); dp->dp_empty_bpobj = bpobj_alloc(os, SPA_MAXBLOCKSIZE, tx); @@ -51,7 +49,7 @@ bpobj_alloc_empty(objset_t *os, int bloc DMU_POOL_EMPTY_BPOBJ, sizeof (uint64_t), 1, &dp->dp_empty_bpobj, tx) == 0); } - spa_feature_incr(spa, empty_bpobj_feat, tx); + spa_feature_incr(spa, SPA_FEATURE_EMPTY_BPOBJ, tx); ASSERT(dp->dp_empty_bpobj != 0); return (dp->dp_empty_bpobj); } else { @@ -62,12 +60,11 @@ bpobj_alloc_empty(objset_t *os, int bloc void bpobj_decr_empty(objset_t *os, dmu_tx_t *tx) { - zfeature_info_t *empty_bpobj_feat = - &spa_feature_table[SPA_FEATURE_EMPTY_BPOBJ]; dsl_pool_t *dp = dmu_objset_pool(os); - spa_feature_decr(dmu_objset_spa(os), empty_bpobj_feat, tx); - if (!spa_feature_is_active(dmu_objset_spa(os), empty_bpobj_feat)) { + spa_feature_decr(dmu_objset_spa(os), SPA_FEATURE_EMPTY_BPOBJ, tx); + if (!spa_feature_is_active(dmu_objset_spa(os), + SPA_FEATURE_EMPTY_BPOBJ)) { VERIFY3U(0, ==, zap_remove(dp->dp_meta_objset, DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_EMPTY_BPOBJ, tx)); @@ -265,6 +262,7 @@ bpobj_iterate_impl(bpobj_t *bpo, bpobj_i mutex_exit(&bpo->bpo_lock); return (err); } + ASSERT3U(doi.doi_type, ==, DMU_OT_BPOBJ_SUBOBJ); epb = doi.doi_data_block_size / sizeof (uint64_t); for (i = bpo->bpo_phys->bpo_num_subobjs - 1; i >= 0; i--) { Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Wed Nov 20 10:54:06 2013 (r258374) @@ -27,6 +27,8 @@ #include #include #include +#include +#include uint64_t dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, @@ -195,3 +197,54 @@ dmu_object_next(objset_t *os, uint64_t * return (error); } + +/* + * Turn this object from old_type into DMU_OTN_ZAP_METADATA, and bump the + * refcount on SPA_FEATURE_EXTENSIBLE_DATASET. + * + * Only for use from syncing context, on MOS objects. + */ +void +dmu_object_zapify(objset_t *mos, uint64_t object, dmu_object_type_t old_type, + dmu_tx_t *tx) +{ + dnode_t *dn; + + ASSERT(dmu_tx_is_syncing(tx)); + + VERIFY0(dnode_hold(mos, object, FTAG, &dn)); + if (dn->dn_type == DMU_OTN_ZAP_METADATA) { + dnode_rele(dn, FTAG); + return; + } + ASSERT3U(dn->dn_type, ==, old_type); + ASSERT0(dn->dn_maxblkid); + dn->dn_next_type[tx->tx_txg & TXG_MASK] = dn->dn_type = + DMU_OTN_ZAP_METADATA; + dnode_setdirty(dn, tx); + dnode_rele(dn, FTAG); + + mzap_create_impl(mos, object, 0, 0, tx); + + spa_feature_incr(dmu_objset_spa(mos), + SPA_FEATURE_EXTENSIBLE_DATASET, tx); +} + +void +dmu_object_free_zapified(objset_t *mos, uint64_t object, dmu_tx_t *tx) +{ + dnode_t *dn; + dmu_object_type_t t; + + ASSERT(dmu_tx_is_syncing(tx)); + + VERIFY0(dnode_hold(mos, object, FTAG, &dn)); + t = dn->dn_type; + dnode_rele(dn, FTAG); + + if (t == DMU_OTN_ZAP_METADATA) { + spa_feature_decr(dmu_objset_spa(mos), + SPA_FEATURE_EXTENSIBLE_DATASET, tx); + } + VERIFY0(dmu_object_free(mos, object, tx)); +} Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c Wed Nov 20 10:54:06 2013 (r258374) @@ -605,7 +605,7 @@ traverse_pool(spa_t *spa, uint64_t txg_s continue; } - if (doi.doi_type == DMU_OT_DSL_DATASET) { + if (doi.doi_bonus_type == DMU_OT_DSL_DATASET) { dsl_dataset_t *ds; uint64_t txg = txg_start; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Wed Nov 20 10:54:06 2013 (r258374) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. */ #include @@ -574,7 +574,12 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) BP_GET_LSIZE(&dnp->dn_blkptr[0]) == dnp->dn_datablkszsec << SPA_MINBLOCKSHIFT); - if (dn->dn_next_blksz[txgoff]) { + if (dn->dn_next_type[txgoff] != 0) { + dnp->dn_type = dn->dn_type; + dn->dn_next_type[txgoff] = 0; + } + + if (dn->dn_next_blksz[txgoff] != 0) { ASSERT(P2PHASE(dn->dn_next_blksz[txgoff], SPA_MINBLOCKSIZE) == 0); ASSERT(BP_IS_HOLE(&dnp->dn_blkptr[0]) || @@ -587,7 +592,7 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) dn->dn_next_blksz[txgoff] = 0; } - if (dn->dn_next_bonuslen[txgoff]) { + if (dn->dn_next_bonuslen[txgoff] != 0) { if (dn->dn_next_bonuslen[txgoff] == DN_ZERO_BONUSLEN) dnp->dn_bonuslen = 0; else @@ -596,7 +601,7 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) dn->dn_next_bonuslen[txgoff] = 0; } - if (dn->dn_next_bonustype[txgoff]) { + if (dn->dn_next_bonustype[txgoff] != 0) { ASSERT(DMU_OT_IS_VALID(dn->dn_next_bonustype[txgoff])); dnp->dn_bonustype = dn->dn_next_bonustype[txgoff]; dn->dn_next_bonustype[txgoff] = 0; @@ -614,7 +619,7 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) dn->dn_rm_spillblk[txgoff] = 0; } - if (dn->dn_next_indblkshift[txgoff]) { + if (dn->dn_next_indblkshift[txgoff] != 0) { ASSERT(dnp->dn_nlevels == 1); dnp->dn_indblkshift = dn->dn_next_indblkshift[txgoff]; dn->dn_next_indblkshift[txgoff] = 0; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c Wed Nov 20 10:54:06 2013 (r258374) @@ -352,7 +352,7 @@ dsl_dataset_hold_obj(dsl_pool_t *dp, uin /* Make sure dsobj has the correct object type. */ dmu_object_info_from_db(dbuf, &doi); - if (doi.doi_type != DMU_OT_DSL_DATASET) { + if (doi.doi_bonus_type != DMU_OT_DSL_DATASET) { dmu_buf_rele(dbuf, tag); return (SET_ERROR(EINVAL)); } @@ -2972,3 +2972,11 @@ dsl_dataset_is_before(dsl_dataset_t *lat dsl_dataset_rele(origin, FTAG); return (ret); } + + +void +dsl_dataset_zapify(dsl_dataset_t *ds, dmu_tx_t *tx) +{ + objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; + dmu_object_zapify(mos, ds->ds_object, DMU_OT_DSL_DATASET, tx); +} Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c Wed Nov 20 10:54:06 2013 (r258374) @@ -38,6 +38,7 @@ #include #include #include +#include typedef struct dmu_snapshots_destroy_arg { nvlist_t *dsda_snaps; @@ -448,7 +449,7 @@ dsl_destroy_snapshot_sync_impl(dsl_datas VERIFY0(zap_destroy(mos, ds->ds_phys->ds_userrefs_obj, tx)); dsl_dir_rele(ds->ds_dir, ds); ds->ds_dir = NULL; - VERIFY0(dmu_object_free(mos, obj, tx)); + dmu_object_free_zapified(mos, obj, tx); } static void @@ -671,7 +672,7 @@ dsl_dir_destroy_sync(uint64_t ddobj, dmu dd->dd_parent->dd_phys->dd_child_dir_zapobj, dd->dd_myname, tx)); dsl_dir_rele(dd, FTAG); - VERIFY0(dmu_object_free(mos, ddobj, tx)); + dmu_object_free_zapified(mos, ddobj, tx); } void @@ -724,10 +725,6 @@ dsl_destroy_head_sync_impl(dsl_dataset_t ds->ds_prev->ds_phys->ds_num_children--; } - zfeature_info_t *async_destroy = - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY]; - objset_t *os; - /* * Destroy the deadlist. Unless it's a clone, the * deadlist should be empty. (If it's a clone, it's @@ -738,9 +735,10 @@ dsl_destroy_head_sync_impl(dsl_dataset_t dmu_buf_will_dirty(ds->ds_dbuf, tx); ds->ds_phys->ds_deadlist_obj = 0; + objset_t *os; VERIFY0(dmu_objset_from_ds(ds, &os)); - if (!spa_feature_is_enabled(dp->dp_spa, async_destroy)) { + if (!spa_feature_is_enabled(dp->dp_spa, SPA_FEATURE_ASYNC_DESTROY)) { old_synchronous_dataset_destroy(ds, tx); } else { /* @@ -751,10 +749,11 @@ dsl_destroy_head_sync_impl(dsl_dataset_t zil_destroy_sync(dmu_objset_zil(os), tx); - if (!spa_feature_is_active(dp->dp_spa, async_destroy)) { + if (!spa_feature_is_active(dp->dp_spa, + SPA_FEATURE_ASYNC_DESTROY)) { dsl_scan_t *scn = dp->dp_scan; - - spa_feature_incr(dp->dp_spa, async_destroy, tx); + spa_feature_incr(dp->dp_spa, SPA_FEATURE_ASYNC_DESTROY, + tx); dp->dp_bptree_obj = bptree_alloc(mos, tx); VERIFY0(zap_add(mos, DMU_POOL_DIRECTORY_OBJECT, @@ -814,7 +813,7 @@ dsl_destroy_head_sync_impl(dsl_dataset_t ASSERT0(ds->ds_phys->ds_userrefs_obj); dsl_dir_rele(ds->ds_dir, ds); ds->ds_dir = NULL; - VERIFY0(dmu_object_free(mos, obj, tx)); + dmu_object_free_zapified(mos, obj, tx); dsl_dir_destroy_sync(ddobj, tx); @@ -870,8 +869,7 @@ dsl_destroy_head(const char *name) error = spa_open(name, &spa, FTAG); if (error != 0) return (error); - isenabled = spa_feature_is_enabled(spa, - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY]); + isenabled = spa_feature_is_enabled(spa, SPA_FEATURE_ASYNC_DESTROY); spa_close(spa, FTAG); ddha.ddha_name = name; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Wed Nov 20 10:54:06 2013 (r258374) @@ -32,6 +32,7 @@ #include #include #include +#include #include #include #include @@ -88,7 +89,7 @@ dsl_dir_hold_obj(dsl_pool_t *dp, uint64_ { dmu_object_info_t doi; dmu_object_info_from_db(dbuf, &doi); - ASSERT3U(doi.doi_type, ==, DMU_OT_DSL_DIR); + ASSERT3U(doi.doi_bonus_type, ==, DMU_OT_DSL_DIR); ASSERT3U(doi.doi_bonus_size, >=, sizeof (dsl_dir_phys_t)); } #endif @@ -1352,3 +1353,10 @@ dsl_dir_snap_cmtime_update(dsl_dir_t *dd dd->dd_snap_cmtime = t; mutex_exit(&dd->dd_lock); } + +void +dsl_dir_zapify(dsl_dir_t *dd, dmu_tx_t *tx) +{ + objset_t *mos = dd->dd_pool->dp_meta_objset; + dmu_object_zapify(mos, dd->dd_object, DMU_OT_DSL_DIR, tx); +} Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c Wed Nov 20 10:54:06 2013 (r258374) @@ -250,8 +250,7 @@ dsl_pool_open(dsl_pool_t *dp) dp->dp_meta_objset, obj)); } - if (spa_feature_is_active(dp->dp_spa, - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY])) { + if (spa_feature_is_active(dp->dp_spa, SPA_FEATURE_ASYNC_DESTROY)) { err = zap_lookup(dp->dp_meta_objset, DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_BPTREE_OBJ, sizeof (uint64_t), 1, &dp->dp_bptree_obj); @@ -259,8 +258,7 @@ dsl_pool_open(dsl_pool_t *dp) goto out; } - if (spa_feature_is_active(dp->dp_spa, - &spa_feature_table[SPA_FEATURE_EMPTY_BPOBJ])) { + if (spa_feature_is_active(dp->dp_spa, SPA_FEATURE_EMPTY_BPOBJ)) { err = zap_lookup(dp->dp_meta_objset, DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_EMPTY_BPOBJ, sizeof (uint64_t), 1, &dp->dp_empty_bpobj); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Wed Nov 20 10:54:06 2013 (r258374) @@ -102,7 +102,7 @@ dsl_scan_init(dsl_pool_t *dp, uint64_t t */ ASSERT(!scn->scn_async_destroying); scn->scn_async_destroying = spa_feature_is_active(dp->dp_spa, - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY]); + SPA_FEATURE_ASYNC_DESTROY); err = zap_lookup(dp->dp_meta_objset, DMU_POOL_DIRECTORY_OBJECT, "scrub_func", sizeof (uint64_t), 1, &f); @@ -1353,7 +1353,6 @@ dsl_scan_active(dsl_scan_t *scn) return (B_FALSE); if (spa_shutting_down(spa)) return (B_FALSE); - if (scn->scn_phys.scn_state == DSS_SCANNING || scn->scn_async_destroying) return (B_TRUE); @@ -1412,7 +1411,7 @@ dsl_scan_sync(dsl_pool_t *dp, dmu_tx_t * VERIFY3U(0, ==, zio_wait(scn->scn_zio_root)); if (err == 0 && spa_feature_is_active(spa, - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY])) { + SPA_FEATURE_ASYNC_DESTROY)) { ASSERT(scn->scn_async_destroying); scn->scn_is_bptree = B_TRUE; scn->scn_zio_root = zio_root(dp->dp_spa, NULL, @@ -1423,11 +1422,11 @@ dsl_scan_sync(dsl_pool_t *dp, dmu_tx_t * VERIFY0(zio_wait(scn->scn_zio_root)); if (err == 0) { - zfeature_info_t *feat = &spa_feature_table - [SPA_FEATURE_ASYNC_DESTROY]; /* finished; deactivate async destroy feature */ - spa_feature_decr(spa, feat, tx); - ASSERT(!spa_feature_is_active(spa, feat)); + spa_feature_decr(spa, SPA_FEATURE_ASYNC_DESTROY, + tx); + ASSERT(!spa_feature_is_active(spa, + SPA_FEATURE_ASYNC_DESTROY)); VERIFY0(zap_remove(dp->dp_meta_objset, DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_BPTREE_OBJ, tx)); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Wed Nov 20 10:54:06 2013 (r258374) @@ -2300,14 +2300,12 @@ spa_load_impl(spa_t *spa, uint64_t pool_ enabled_feat = fnvlist_alloc(); unsup_feat = fnvlist_alloc(); - if (!feature_is_supported(spa->spa_meta_objset, - spa->spa_feat_for_read_obj, spa->spa_feat_desc_obj, + if (!spa_features_check(spa, B_FALSE, unsup_feat, enabled_feat)) missing_feat_read = B_TRUE; if (spa_writeable(spa) || state == SPA_LOAD_TRYIMPORT) { - if (!feature_is_supported(spa->spa_meta_objset, - spa->spa_feat_for_write_obj, spa->spa_feat_desc_obj, + if (!spa_features_check(spa, B_TRUE, unsup_feat, enabled_feat)) { missing_feat_write = B_TRUE; } @@ -5914,7 +5912,7 @@ spa_sync_props(void *arg, dmu_tx_t *tx) zpool_prop_t prop; const char *propname; zprop_type_t proptype; - zfeature_info_t *feature; + spa_feature_t fid; switch (prop = zpool_name_to_prop(nvpair_name(elem))) { case ZPROP_INVAL: @@ -5924,9 +5922,9 @@ spa_sync_props(void *arg, dmu_tx_t *tx) ASSERT(zpool_prop_feature(nvpair_name(elem))); fname = strchr(nvpair_name(elem), '@') + 1; - VERIFY0(zfeature_lookup_name(fname, &feature)); + VERIFY0(zfeature_lookup_name(fname, &fid)); - spa_feature_enable(spa, feature, tx); + spa_feature_enable(spa, fid, tx); spa_history_log_internal(spa, "set", tx, "%s=enabled", nvpair_name(elem)); break; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Wed Nov 20 10:54:06 2013 (r258374) @@ -1143,15 +1143,17 @@ spa_vdev_state_exit(spa_t *spa, vdev_t * void spa_activate_mos_feature(spa_t *spa, const char *feature) { - (void) nvlist_add_boolean(spa->spa_label_features, feature); - vdev_config_dirty(spa->spa_root_vdev); + if (!nvlist_exists(spa->spa_label_features, feature)) { + fnvlist_add_boolean(spa->spa_label_features, feature); + vdev_config_dirty(spa->spa_root_vdev); + } } void spa_deactivate_mos_feature(spa_t *spa, const char *feature) { - (void) nvlist_remove_all(spa->spa_label_features, feature); - vdev_config_dirty(spa->spa_root_vdev); + if (nvlist_remove_all(spa->spa_label_features, feature) == 0) + vdev_config_dirty(spa->spa_root_vdev); } /* Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Wed Nov 20 10:54:06 2013 (r258374) @@ -474,8 +474,6 @@ space_map_truncate(space_map_t *sm, dmu_ { objset_t *os = sm->sm_os; spa_t *spa = dmu_objset_spa(os); - zfeature_info_t *space_map_histogram = - &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM]; dmu_object_info_t doi; int bonuslen; @@ -485,7 +483,7 @@ space_map_truncate(space_map_t *sm, dmu_ VERIFY0(dmu_free_range(os, space_map_object(sm), 0, -1ULL, tx)); dmu_object_info_from_db(sm->sm_dbuf, &doi); - if (spa_feature_is_enabled(spa, space_map_histogram)) { + if (spa_feature_is_enabled(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM)) { bonuslen = sizeof (space_map_phys_t); ASSERT3U(bonuslen, <=, dmu_bonus_max()); } else { @@ -525,13 +523,11 @@ uint64_t space_map_alloc(objset_t *os, dmu_tx_t *tx) { spa_t *spa = dmu_objset_spa(os); - zfeature_info_t *space_map_histogram = - &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM]; uint64_t object; int bonuslen; - if (spa_feature_is_enabled(spa, space_map_histogram)) { - spa_feature_incr(spa, space_map_histogram, tx); + if (spa_feature_is_enabled(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM)) { + spa_feature_incr(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM, tx); bonuslen = sizeof (space_map_phys_t); ASSERT3U(bonuslen, <=, dmu_bonus_max()); } else { @@ -549,20 +545,20 @@ void space_map_free(space_map_t *sm, dmu_tx_t *tx) { spa_t *spa; - zfeature_info_t *space_map_histogram = - &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM]; if (sm == NULL) return; spa = dmu_objset_spa(sm->sm_os); - if (spa_feature_is_enabled(spa, space_map_histogram)) { + if (spa_feature_is_enabled(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM)) { dmu_object_info_t doi; dmu_object_info_from_db(sm->sm_dbuf, &doi); if (doi.doi_bonus_size != SPACE_MAP_SIZE_V0) { - VERIFY(spa_feature_is_active(spa, space_map_histogram)); - spa_feature_decr(spa, space_map_histogram, tx); + VERIFY(spa_feature_is_active(spa, + SPA_FEATURE_SPACEMAP_HISTOGRAM)); + spa_feature_decr(spa, + SPA_FEATURE_SPACEMAP_HISTOGRAM, tx); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h Wed Nov 20 10:54:06 2013 (r258374) @@ -298,6 +298,8 @@ typedef struct dmu_sendarg { uint64_t dsa_last_data_offset; } dmu_sendarg_t; +void dmu_object_zapify(objset_t *, uint64_t, dmu_object_type_t, dmu_tx_t *); +void dmu_object_free_zapified(objset_t *, uint64_t, dmu_tx_t *); #ifdef __cplusplus } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h Wed Nov 20 10:54:06 2013 (r258374) @@ -178,6 +178,7 @@ typedef struct dnode { uint16_t dn_datablkszsec; /* in 512b sectors */ uint32_t dn_datablksz; /* in bytes */ uint64_t dn_maxblkid; + uint8_t dn_next_type[TXG_SIZE]; uint8_t dn_next_nblkptr[TXG_SIZE]; uint8_t dn_next_nlevels[TXG_SIZE]; uint8_t dn_next_indblkshift[TXG_SIZE]; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h Wed Nov 20 10:54:06 2013 (r258374) @@ -49,9 +49,9 @@ struct dsl_pool; #define DS_FLAG_INCONSISTENT (1ULL<<0) #define DS_IS_INCONSISTENT(ds) \ ((ds)->ds_phys->ds_flags & DS_FLAG_INCONSISTENT) + /* - * Note: nopromote can not yet be set, but we want support for it in this - * on-disk version, so that we don't need to upgrade for it later. + * Do not allow this dataset to be promoted. */ #define DS_FLAG_NOPROMOTE (1ULL<<1) @@ -71,6 +71,11 @@ struct dsl_pool; ((ds)->ds_phys->ds_flags & DS_FLAG_DEFER_DESTROY) /* + * DS_FIELD_* are strings that are used in the "extensified" dataset zap object. + * They should be of the format :. + */ + +/* * DS_FLAG_CI_DATASET is set if the dataset contains a file system whose * name lookups should be performed case-insensitively. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zap.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zap.h Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zap.h Wed Nov 20 10:54:06 2013 (r258374) @@ -141,6 +141,12 @@ uint64_t zap_create_link(objset_t *os, d uint64_t parent_obj, const char *name, dmu_tx_t *tx); /* + * Initialize an already-allocated object. + */ +void mzap_create_impl(objset_t *os, uint64_t obj, int normflags, + zap_flags_t flags, dmu_tx_t *tx); + +/* * Create a new zapobj with no attributes from the given (unallocated) * object number. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h Wed Nov 20 10:54:06 2013 (r258374) @@ -27,6 +27,7 @@ #define _SYS_ZFEATURE_H #include +#include #include "zfeature_common.h" #ifdef __cplusplus @@ -37,17 +38,25 @@ struct spa; struct dmu_tx; struct objset; -extern boolean_t feature_is_supported(struct objset *os, uint64_t obj, - uint64_t desc_obj, nvlist_t *unsup_feat, nvlist_t *enabled_feat); - extern void spa_feature_create_zap_objects(struct spa *, struct dmu_tx *); -extern void spa_feature_enable(struct spa *, zfeature_info_t *, +extern void spa_feature_enable(struct spa *, spa_feature_t, + struct dmu_tx *); +extern void spa_feature_incr(struct spa *, spa_feature_t, struct dmu_tx *); +extern void spa_feature_decr(struct spa *, spa_feature_t, struct dmu_tx *); +extern boolean_t spa_feature_is_enabled(struct spa *, spa_feature_t); +extern boolean_t spa_feature_is_active(struct spa *, spa_feature_t); +extern uint64_t spa_feature_refcount(spa_t *, spa_feature_t, uint64_t); +extern boolean_t spa_features_check(spa_t *, boolean_t, nvlist_t *, nvlist_t *); + +/* + * These functions are only exported for zhack and zdb; normal callers should + * use the above interfaces. + */ +extern int feature_get_refcount(struct spa *, zfeature_info_t *, uint64_t *); +extern void feature_enable_sync(struct spa *, zfeature_info_t *, + struct dmu_tx *); +extern void feature_sync(struct spa *, zfeature_info_t *, uint64_t, struct dmu_tx *); -extern void spa_feature_incr(struct spa *, zfeature_info_t *, struct dmu_tx *); -extern void spa_feature_decr(struct spa *, zfeature_info_t *, struct dmu_tx *); -extern boolean_t spa_feature_is_enabled(struct spa *, zfeature_info_t *); -extern boolean_t spa_feature_is_active(struct spa *, zfeature_info_t *); -extern int spa_feature_get_refcount(struct spa *, zfeature_info_t *); #ifdef __cplusplus } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Wed Nov 20 10:54:06 2013 (r258374) @@ -572,7 +572,7 @@ mzap_upgrade(zap_t **zapp, dmu_tx_t *tx, return (err); } -static void +void mzap_create_impl(objset_t *os, uint64_t obj, int normflags, zap_flags_t flags, dmu_tx_t *tx) { @@ -862,8 +862,8 @@ zap_lookup_uint64(objset_t *os, uint64_t int zap_contains(objset_t *os, uint64_t zapobj, const char *name) { - int err = (zap_lookup_norm(os, zapobj, name, 0, - 0, NULL, MT_EXACT, NULL, 0, NULL)); + int err = zap_lookup_norm(os, zapobj, name, 0, + 0, NULL, MT_EXACT, NULL, 0, NULL); if (err == EOVERFLOW || err == EINVAL) err = 0; /* found, but skipped reading the value */ return (err); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c Wed Nov 20 10:54:06 2013 (r258374) @@ -161,23 +161,25 @@ */ typedef enum { - FEATURE_ACTION_ENABLE, FEATURE_ACTION_INCR, FEATURE_ACTION_DECR, } feature_action_t; /* - * Checks that the features active in the specified object are supported by + * Checks that the active features in the pool are supported by * this software. Adds each unsupported feature (name -> description) to * the supplied nvlist. */ boolean_t -feature_is_supported(objset_t *os, uint64_t obj, uint64_t desc_obj, +spa_features_check(spa_t *spa, boolean_t for_write, nvlist_t *unsup_feat, nvlist_t *enabled_feat) { + objset_t *os = spa->spa_meta_objset; boolean_t supported; zap_cursor_t zc; zap_attribute_t za; + uint64_t obj = for_write ? + spa->spa_feat_for_write_obj : spa->spa_feat_for_read_obj; supported = B_TRUE; for (zap_cursor_init(&zc, os, obj); @@ -199,8 +201,8 @@ feature_is_supported(objset_t *os, uint6 char *desc = ""; char buf[MAXPATHLEN]; - if (zap_lookup(os, desc_obj, za.za_name, - 1, sizeof (buf), buf) == 0) + if (zap_lookup(os, spa->spa_feat_desc_obj, + za.za_name, 1, sizeof (buf), buf) == 0) desc = buf; VERIFY(nvlist_add_string(unsup_feat, za.za_name, @@ -213,13 +215,18 @@ feature_is_supported(objset_t *os, uint6 return (supported); } -static int -feature_get_refcount(objset_t *os, uint64_t read_obj, uint64_t write_obj, - zfeature_info_t *feature, uint64_t *res) +/* + * Note: well-designed features will not need to use this; they should + * use spa_feature_is_enabled() and spa_feature_is_active() instead. + * However, this is non-static for zdb and zhack. + */ +int +feature_get_refcount(spa_t *spa, zfeature_info_t *feature, uint64_t *res) { int err; uint64_t refcount; - uint64_t zapobj = feature->fi_can_readonly ? write_obj : read_obj; + uint64_t zapobj = feature->fi_can_readonly ? + spa->spa_feat_for_write_obj : spa->spa_feat_for_read_obj; /* * If the pool is currently being created, the feature objects may not @@ -228,8 +235,8 @@ feature_get_refcount(objset_t *os, uint6 if (zapobj == 0) return (SET_ERROR(ENOTSUP)); - err = zap_lookup(os, zapobj, feature->fi_guid, sizeof (uint64_t), 1, - &refcount); + err = zap_lookup(spa->spa_meta_objset, zapobj, + feature->fi_guid, sizeof (uint64_t), 1, &refcount); if (err != 0) { if (err == ENOENT) return (SET_ERROR(ENOTSUP)); @@ -240,49 +247,81 @@ feature_get_refcount(objset_t *os, uint6 return (0); } -static int -feature_do_action(objset_t *os, uint64_t read_obj, uint64_t write_obj, - uint64_t desc_obj, zfeature_info_t *feature, feature_action_t action, +/* + * This function is non-static for zhack; it should otherwise not be used + * outside this file. + */ +void +feature_sync(spa_t *spa, zfeature_info_t *feature, uint64_t refcount, dmu_tx_t *tx) { - int error; - uint64_t refcount; - uint64_t zapobj = feature->fi_can_readonly ? write_obj : read_obj; + uint64_t zapobj = feature->fi_can_readonly ? + spa->spa_feat_for_write_obj : spa->spa_feat_for_read_obj; + + VERIFY0(zap_update(spa->spa_meta_objset, zapobj, feature->fi_guid, + sizeof (uint64_t), 1, &refcount, tx)); + + if (refcount == 0) + spa_deactivate_mos_feature(spa, feature->fi_guid); + else if (feature->fi_mos) + spa_activate_mos_feature(spa, feature->fi_guid); +} + +/* + * This function is non-static for zhack; it should otherwise not be used + * outside this file. + */ +void +feature_enable_sync(spa_t *spa, zfeature_info_t *feature, dmu_tx_t *tx) +{ + uint64_t zapobj = feature->fi_can_readonly ? + spa->spa_feat_for_write_obj : spa->spa_feat_for_read_obj; ASSERT(0 != zapobj); ASSERT(zfeature_is_valid_guid(feature->fi_guid)); - - error = zap_lookup(os, zapobj, feature->fi_guid, - sizeof (uint64_t), 1, &refcount); + ASSERT3U(spa_version(spa), >=, SPA_VERSION_FEATURES); /* - * If we can't ascertain the status of the specified feature, an I/O - * error occurred. + * If the feature is already enabled, ignore the request. */ - if (error != 0 && error != ENOENT) - return (error); + if (zap_contains(spa->spa_meta_objset, zapobj, feature->fi_guid) == 0) + return; + + for (int i = 0; feature->fi_depends[i] != SPA_FEATURE_NONE; i++) + spa_feature_enable(spa, feature->fi_depends[i], tx); + + VERIFY0(zap_update(spa->spa_meta_objset, spa->spa_feat_desc_obj, + feature->fi_guid, 1, strlen(feature->fi_desc) + 1, + feature->fi_desc, tx)); + feature_sync(spa, feature, 0, tx); +} + +static void +feature_do_action(spa_t *spa, spa_feature_t fid, feature_action_t action, + dmu_tx_t *tx) +{ + uint64_t refcount; + zfeature_info_t *feature = &spa_feature_table[fid]; + uint64_t zapobj = feature->fi_can_readonly ? + spa->spa_feat_for_write_obj : spa->spa_feat_for_read_obj; + + ASSERT3U(fid, <, SPA_FEATURES); + ASSERT(0 != zapobj); + ASSERT(zfeature_is_valid_guid(feature->fi_guid)); + + ASSERT(dmu_tx_is_syncing(tx)); + ASSERT3U(spa_version(spa), >=, SPA_VERSION_FEATURES); + + VERIFY0(zap_lookup(spa->spa_meta_objset, zapobj, feature->fi_guid, + sizeof (uint64_t), 1, &refcount)); switch (action) { - case FEATURE_ACTION_ENABLE: - /* - * If the feature is already enabled, ignore the request. - */ - if (error == 0) - return (0); - refcount = 0; - break; case FEATURE_ACTION_INCR: - if (error == ENOENT) - return (SET_ERROR(ENOTSUP)); - if (refcount == UINT64_MAX) - return (SET_ERROR(EOVERFLOW)); + VERIFY3U(refcount, !=, UINT64_MAX); refcount++; break; case FEATURE_ACTION_DECR: - if (error == ENOENT) - return (SET_ERROR(ENOTSUP)); - if (refcount == 0) - return (SET_ERROR(EOVERFLOW)); + VERIFY3U(refcount, !=, 0); refcount--; break; default: @@ -290,42 +329,7 @@ feature_do_action(objset_t *os, uint64_t break; } - if (action == FEATURE_ACTION_ENABLE) { - int i; - - for (i = 0; feature->fi_depends[i] != NULL; i++) { - zfeature_info_t *dep = feature->fi_depends[i]; - - error = feature_do_action(os, read_obj, write_obj, - desc_obj, dep, FEATURE_ACTION_ENABLE, tx); - if (error != 0) - return (error); - } - } - - error = zap_update(os, zapobj, feature->fi_guid, - sizeof (uint64_t), 1, &refcount, tx); - if (error != 0) - return (error); - - if (action == FEATURE_ACTION_ENABLE) { - error = zap_update(os, desc_obj, - feature->fi_guid, 1, strlen(feature->fi_desc) + 1, - feature->fi_desc, tx); - if (error != 0) - return (error); - } - - if (action == FEATURE_ACTION_INCR && refcount == 1 && feature->fi_mos) { - spa_activate_mos_feature(dmu_objset_spa(os), feature->fi_guid); - } - - if (action == FEATURE_ACTION_DECR && refcount == 0) { - spa_deactivate_mos_feature(dmu_objset_spa(os), - feature->fi_guid); - } - - return (0); + feature_sync(spa, feature, refcount, tx); } void @@ -353,82 +357,51 @@ spa_feature_create_zap_objects(spa_t *sp * Enable any required dependencies, then enable the requested feature. *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:54:12 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE3EFE6B; Wed, 20 Nov 2013 10:54:11 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DB89E20B1; Wed, 20 Nov 2013 10:54:11 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAsBgt056262; Wed, 20 Nov 2013 10:54:11 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAsAvn056257; Wed, 20 Nov 2013 10:54:10 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201054.rAKAsAvn056257@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:54:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258374 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:54:12 -0000 Author: avg Date: Wed Nov 20 10:54:06 2013 New Revision: 258374 URL: http://svnweb.freebsd.org/changeset/base/258374 Log: 4171 clean up spa_feature_*() interfaces 4172 implement extensible_dataset feature for use by other zpool features illumos/illumos-gate@2acef22db7808606888f8f92715629ff3ba555b9 Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/zhack/zhack.c vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c vendor/illumos/dist/man/man5/zpool-features.5 Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_traverse.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dataset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_destroy.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_pool.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_dataset.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zap.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfeature.h vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfeature.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor/illumos/dist/cmd/zdb/zdb.c Wed Nov 20 10:54:06 2013 (r258374) @@ -563,16 +563,20 @@ get_metaslab_refcount(vdev_t *vd) static int verify_spacemap_refcounts(spa_t *spa) { - int expected_refcount, actual_refcount; + uint64_t expected_refcount = 0; + uint64_t actual_refcount; - expected_refcount = spa_feature_get_refcount(spa, - &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM]); + (void) feature_get_refcount(spa, + &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM], + &expected_refcount); actual_refcount = get_dtl_refcount(spa->spa_root_vdev); actual_refcount += get_metaslab_refcount(spa->spa_root_vdev); if (expected_refcount != actual_refcount) { - (void) printf("space map refcount mismatch: expected %d != " - "actual %d\n", expected_refcount, actual_refcount); + (void) printf("space map refcount mismatch: expected %lld != " + "actual %lld\n", + (longlong_t)expected_refcount, + (longlong_t)actual_refcount); return (2); } return (0); @@ -674,8 +678,7 @@ dump_metaslab(metaslab_t *msp) } if (dump_opt['m'] > 1 && sm != NULL && - spa_feature_is_active(spa, - &spa_feature_table[SPA_FEATURE_SPACEMAP_HISTOGRAM])) { + spa_feature_is_active(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM)) { /* * The space map histogram represents free space in chunks * of sm_shift (i.e. bucket 0 refers to 2^sm_shift). @@ -2466,8 +2469,7 @@ dump_block_stats(spa_t *spa) (void) bpobj_iterate_nofree(&spa->spa_dsl_pool->dp_free_bpobj, count_block_cb, &zcb, NULL); } - if (spa_feature_is_active(spa, - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY])) { + if (spa_feature_is_active(spa, SPA_FEATURE_ASYNC_DESTROY)) { VERIFY3U(0, ==, bptree_iterate(spa->spa_meta_objset, spa->spa_dsl_pool->dp_bptree_obj, B_FALSE, count_block_cb, &zcb, NULL)); @@ -2793,7 +2795,7 @@ dump_zpool(spa_t *spa) } if (spa_feature_is_active(spa, - &spa_feature_table[SPA_FEATURE_ASYNC_DESTROY])) { + SPA_FEATURE_ASYNC_DESTROY)) { dump_bptree(spa->spa_meta_objset, spa->spa_dsl_pool->dp_bptree_obj, "Pool dataset frees"); Modified: vendor/illumos/dist/cmd/zhack/zhack.c ============================================================================== --- vendor/illumos/dist/cmd/zhack/zhack.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor/illumos/dist/cmd/zhack/zhack.c Wed Nov 20 10:54:06 2013 (r258374) @@ -283,12 +283,13 @@ zhack_do_feature_stat(int argc, char **a } static void -feature_enable_sync(void *arg, dmu_tx_t *tx) +zhack_feature_enable_sync(void *arg, dmu_tx_t *tx) { spa_t *spa = dmu_tx_pool(tx)->dp_spa; zfeature_info_t *feature = arg; - spa_feature_enable(spa, feature, tx); + feature_enable_sync(spa, feature, tx); + spa_history_log_internal(spa, "zhack enable feature", tx, "guid=%s can_readonly=%u", feature->fi_guid, feature->fi_can_readonly); @@ -302,7 +303,7 @@ zhack_do_feature_enable(int argc, char * spa_t *spa; objset_t *mos; zfeature_info_t feature; - zfeature_info_t *nodeps[] = { NULL }; + spa_feature_t nodeps[] = { SPA_FEATURE_NONE }; /* * Features are not added to the pool's label until their refcounts @@ -349,14 +350,14 @@ zhack_do_feature_enable(int argc, char * zhack_spa_open(target, B_FALSE, FTAG, &spa); mos = spa->spa_meta_objset; - if (0 == zfeature_lookup_guid(feature.fi_guid, NULL)) + if (zfeature_is_supported(feature.fi_guid)) fatal(spa, FTAG, "'%s' is a real feature, will not enable"); if (0 == zap_contains(mos, spa->spa_feat_desc_obj, feature.fi_guid)) fatal(spa, FTAG, "feature already enabled: %s", feature.fi_guid); VERIFY0(dsl_sync_task(spa_name(spa), NULL, - feature_enable_sync, &feature, 5)); + zhack_feature_enable_sync, &feature, 5)); spa_close(spa, FTAG); @@ -368,8 +369,10 @@ feature_incr_sync(void *arg, dmu_tx_t *t { spa_t *spa = dmu_tx_pool(tx)->dp_spa; zfeature_info_t *feature = arg; + uint64_t refcount; - spa_feature_incr(spa, feature, tx); + VERIFY0(feature_get_refcount(spa, feature, &refcount)); + feature_sync(spa, feature, refcount + 1, tx); spa_history_log_internal(spa, "zhack feature incr", tx, "guid=%s", feature->fi_guid); } @@ -379,8 +382,10 @@ feature_decr_sync(void *arg, dmu_tx_t *t { spa_t *spa = dmu_tx_pool(tx)->dp_spa; zfeature_info_t *feature = arg; + uint64_t refcount; - spa_feature_decr(spa, feature, tx); + VERIFY0(feature_get_refcount(spa, feature, &refcount)); + feature_sync(spa, feature, refcount - 1, tx); spa_history_log_internal(spa, "zhack feature decr", tx, "guid=%s", feature->fi_guid); } @@ -394,7 +399,7 @@ zhack_do_feature_ref(int argc, char **ar spa_t *spa; objset_t *mos; zfeature_info_t feature; - zfeature_info_t *nodeps[] = { NULL }; + spa_feature_t nodeps[] = { SPA_FEATURE_NONE }; /* * fi_desc does not matter here because it was written to disk @@ -437,9 +442,10 @@ zhack_do_feature_ref(int argc, char **ar zhack_spa_open(target, B_FALSE, FTAG, &spa); mos = spa->spa_meta_objset; - if (0 == zfeature_lookup_guid(feature.fi_guid, NULL)) - fatal(spa, FTAG, "'%s' is a real feature, will not change " - "refcount"); + if (zfeature_is_supported(feature.fi_guid)) { + fatal(spa, FTAG, + "'%s' is a real feature, will not change refcount"); + } if (0 == zap_contains(mos, spa->spa_feat_for_read_obj, feature.fi_guid)) { @@ -451,9 +457,14 @@ zhack_do_feature_ref(int argc, char **ar fatal(spa, FTAG, "feature is not enabled: %s", feature.fi_guid); } - if (decr && !spa_feature_is_active(spa, &feature)) - fatal(spa, FTAG, "feature refcount already 0: %s", - feature.fi_guid); + if (decr) { + uint64_t count; + if (feature_get_refcount(spa, &feature, &count) == 0 && + count != 0) { + fatal(spa, FTAG, "feature refcount already 0: %s", + feature.fi_guid); + } + } VERIFY0(dsl_sync_task(spa_name(spa), NULL, decr ? feature_decr_sync : feature_incr_sync, &feature, 5)); Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Wed Nov 20 10:54:06 2013 (r258374) @@ -22,7 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2012 by Frederik Wessels. All rights reserved. * Copyright (c) 2013 by Prasad Joshi (sTec). All rights reserved. */ @@ -876,7 +876,7 @@ zpool_do_create(int argc, char **argv) * Hand off to libzfs. */ if (enable_all_pool_feat) { - int i; + spa_feature_t i; for (i = 0; i < SPA_FEATURES; i++) { char propname[MAXPATHLEN]; zfeature_info_t *feat = &spa_feature_table[i]; Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Wed Nov 20 10:52:48 2013 (r258373) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Wed Nov 20 10:54:06 2013 (r258374) @@ -22,7 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ @@ -413,10 +413,9 @@ zpool_valid_proplist(libzfs_handle_t *hd prop = zpool_name_to_prop(propname); if (prop == ZPROP_INVAL && zpool_prop_feature(propname)) { int err; - zfeature_info_t *feature; char *fname = strchr(propname, '@') + 1; - err = zfeature_lookup_name(fname, &feature); + err = zfeature_lookup_name(fname, NULL); if (err != 0) { ASSERT3U(err, ==, ENOENT); zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, @@ -807,14 +806,14 @@ zpool_prop_get_feature(zpool_handle_t *z */ if (supported) { int ret; - zfeature_info_t *fi; + spa_feature_t fid; - ret = zfeature_lookup_name(feature, &fi); + ret = zfeature_lookup_name(feature, &fid); if (ret != 0) { (void) strlcpy(buf, "-", len); return (ENOTSUP); } - feature = fi->fi_guid; + feature = spa_feature_table[fid].fi_guid; } if (nvlist_lookup_uint64(features, feature, &refcount) == 0) Modified: vendor/illumos/dist/man/man5/zpool-features.5 ============================================================================== --- vendor/illumos/dist/man/man5/zpool-features.5 Wed Nov 20 10:52:48 2013 (r258373) +++ vendor/illumos/dist/man/man5/zpool-features.5 Wed Nov 20 10:54:06 2013 (r258374) @@ -271,6 +271,29 @@ configuration. When the \fBmulti_vdev_crash_dump\fR feature is set to \fBenabled\fR, the administrator can use the \fBdumpadm\fR(1M) command to configure a dump device on a pool comprised of multiple vdevs. +.RE + +.sp +.ne 2 +.na +\fB\fBextensible_dataset\fR\fR +.ad +.RS 4n +.TS +l l . +GUID com.delphix:extensible_dataset +READ\-ONLY COMPATIBLE no +DEPENDENCIES none +.TE + +This feature allows more flexible use of internal ZFS data structures, +and exists for other features to depend on. + +This feature will be \fBactive\fR when the first dependent feature uses it, +and will be returned to the \fBenabled\fR state when all datasets that use +this feature are destroyed. + +.RE .SH "SEE ALSO" \fBzpool\fR(1M) From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:57:45 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8D451C9; Wed, 20 Nov 2013 10:57:45 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 989CC20CF; Wed, 20 Nov 2013 10:57:45 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAvjQf056713; Wed, 20 Nov 2013 10:57:45 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAvjIX056712; Wed, 20 Nov 2013 10:57:45 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201057.rAKAvjIX056712@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:57:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258375 - vendor-sys/illumos/dist/uts/common X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:57:45 -0000 Author: avg Date: Wed Nov 20 10:57:45 2013 New Revision: 258375 URL: http://svnweb.freebsd.org/changeset/base/258375 Log: 4166 EOF pcata 4167 EOF pcser 2995 pcan and pcwl wifi device drivers EOL illumos/illumos-gate@506aa7c68b127eefb0197e329af7e9abcc3ebc98 Modified: vendor-sys/illumos/dist/uts/common/Makefile.files Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:54:06 2013 (r258374) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 10:57:45 2013 (r258375) @@ -1724,14 +1724,6 @@ BUSRA_OBJS += busra.o PCS_OBJS += pcs.o -PCAN_OBJS += pcan.o - -PCATA_OBJS += pcide.o pcdisk.o pclabel.o pcata.o - -PCSER_OBJS += pcser.o pcser_cis.o - -PCWL_OBJS += pcwl.o - PSET_OBJS += pset.o OHCI_OBJS += ohci.o ohci_hub.o ohci_polled.o From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:58:27 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11C1A304; Wed, 20 Nov 2013 10:58:27 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DAED620DC; Wed, 20 Nov 2013 10:58:26 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAwQd4056832; Wed, 20 Nov 2013 10:58:26 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAwQH7056828; Wed, 20 Nov 2013 10:58:26 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201058.rAKAwQH7056828@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:58:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258376 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:58:27 -0000 Author: avg Date: Wed Nov 20 10:58:25 2013 New Revision: 258376 URL: http://svnweb.freebsd.org/changeset/base/258376 Log: 3964 L2ARC should always compress metadata buffers illumos/illumos-gate@e4be62a2b74a8f09bb669217a1a39eee069b13a1 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Wed Nov 20 10:57:45 2013 (r258375) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Wed Nov 20 10:58:25 2013 (r258376) @@ -329,7 +329,8 @@ boolean_t dbuf_is_metadata(dmu_buf_impl_ ((_db)->db_objset->os_secondary_cache == ZFS_CACHE_METADATA))) #define DBUF_IS_L2COMPRESSIBLE(_db) \ - ((_db)->db_objset->os_compress != ZIO_COMPRESS_OFF) + ((_db)->db_objset->os_compress != ZIO_COMPRESS_OFF || \ + (dbuf_is_metadata(_db) && zfs_mdcomp_disable == B_FALSE)) #ifdef ZFS_DEBUG Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Wed Nov 20 10:57:45 2013 (r258375) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Wed Nov 20 10:58:25 2013 (r258376) @@ -24,6 +24,7 @@ * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2012, Joyent, Inc. All rights reserved. + * Copyright 2013 DEY Storage Systems, Inc. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -806,6 +807,8 @@ int dmu_diff(const char *tosnap_name, co #define ZFS_CRC64_POLY 0xC96C5795D7870F42ULL /* ECMA-182, reflected form */ extern uint64_t zfs_crc64_table[256]; +extern int zfs_mdcomp_disable; + #ifdef __cplusplus } #endif Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Wed Nov 20 10:57:45 2013 (r258375) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Wed Nov 20 10:58:25 2013 (r258376) @@ -130,7 +130,7 @@ struct objset { ((os)->os_secondary_cache == ZFS_CACHE_ALL || \ (os)->os_secondary_cache == ZFS_CACHE_METADATA) -#define DMU_OS_IS_L2COMPRESSIBLE(os) ((os)->os_compress != ZIO_COMPRESS_OFF) +#define DMU_OS_IS_L2COMPRESSIBLE(os) (zfs_mdcomp_disable == B_FALSE) /* called from zpl */ int dmu_objset_hold(const char *name, void *tag, objset_t **osp); From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 10:59:42 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E5BF459; Wed, 20 Nov 2013 10:59:42 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3E33A20EC; Wed, 20 Nov 2013 10:59:42 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKAxgv7056988; Wed, 20 Nov 2013 10:59:42 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKAxgxF056986; Wed, 20 Nov 2013 10:59:42 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201059.rAKAxgxF056986@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 10:59:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258377 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 10:59:42 -0000 Author: avg Date: Wed Nov 20 10:59:41 2013 New Revision: 258377 URL: http://svnweb.freebsd.org/changeset/base/258377 Log: 4088 use after free in arc_release() illumos/illumos-gate@ccc22e130479b5bd7c0002267fee1e0602d3f772 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Nov 20 10:58:25 2013 (r258376) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Nov 20 10:59:41 2013 (r258377) @@ -3332,6 +3332,7 @@ arc_release(arc_buf_t *buf, void *tag) if (l2hdr) { mutex_enter(&l2arc_buflist_mtx); hdr->b_l2hdr = NULL; + list_remove(l2hdr->b_dev->l2ad_buflist, hdr); } buf_size = hdr->b_size; @@ -3415,7 +3416,6 @@ arc_release(arc_buf_t *buf, void *tag) if (l2hdr) { ARCSTAT_INCR(arcstat_l2_asize, -l2hdr->b_asize); - list_remove(l2hdr->b_dev->l2ad_buflist, hdr); kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t)); ARCSTAT_INCR(arcstat_l2_size, -buf_size); mutex_exit(&l2arc_buflist_mtx); From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:00:21 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20AC65D6; Wed, 20 Nov 2013 11:00:21 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 018522124; Wed, 20 Nov 2013 11:00:21 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB0Kdc057299; Wed, 20 Nov 2013 11:00:20 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB0KW3057298; Wed, 20 Nov 2013 11:00:20 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201100.rAKB0KW3057298@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:00:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258378 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:00:21 -0000 Author: avg Date: Wed Nov 20 11:00:20 2013 New Revision: 258378 URL: http://svnweb.freebsd.org/changeset/base/258378 Log: 4089 NULL pointer dereference in arc_read() illumos/illumos-gate@57815f6b95a743697e148327725b7f568e75e6ea Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Nov 20 10:59:41 2013 (r258377) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Wed Nov 20 11:00:20 2013 (r258378) @@ -20,10 +20,10 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2012, Joyent, Inc. All rights reserved. * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. */ /* @@ -2972,6 +2972,8 @@ top: vdev_t *vd = NULL; uint64_t addr = 0; boolean_t devw = B_FALSE; + enum zio_compress b_compress = ZIO_COMPRESS_OFF; + uint64_t b_asize = 0; if (hdr == NULL) { /* this block is not in the cache */ @@ -3041,10 +3043,12 @@ top: hdr->b_acb = acb; hdr->b_flags |= ARC_IO_IN_PROGRESS; - if (HDR_L2CACHE(hdr) && hdr->b_l2hdr != NULL && + if (hdr->b_l2hdr != NULL && (vd = hdr->b_l2hdr->b_dev->l2ad_vdev) != NULL) { devw = hdr->b_l2hdr->b_dev->l2ad_writing; addr = hdr->b_l2hdr->b_daddr; + b_compress = hdr->b_l2hdr->b_compress; + b_asize = hdr->b_l2hdr->b_asize; /* * Lock out device removal. */ @@ -3092,7 +3096,7 @@ top: cb->l2rcb_bp = *bp; cb->l2rcb_zb = *zb; cb->l2rcb_flags = zio_flags; - cb->l2rcb_compress = hdr->b_l2hdr->b_compress; + cb->l2rcb_compress = b_compress; ASSERT(addr >= VDEV_LABEL_START_SIZE && addr + size < vd->vdev_psize - @@ -3104,8 +3108,7 @@ top: * Issue a null zio if the underlying buffer * was squashed to zero size by compression. */ - if (hdr->b_l2hdr->b_compress == - ZIO_COMPRESS_EMPTY) { + if (b_compress == ZIO_COMPRESS_EMPTY) { rzio = zio_null(pio, spa, vd, l2arc_read_done, cb, zio_flags | ZIO_FLAG_DONT_CACHE | @@ -3114,8 +3117,8 @@ top: ZIO_FLAG_DONT_RETRY); } else { rzio = zio_read_phys(pio, vd, addr, - hdr->b_l2hdr->b_asize, - buf->b_data, ZIO_CHECKSUM_OFF, + b_asize, buf->b_data, + ZIO_CHECKSUM_OFF, l2arc_read_done, cb, priority, zio_flags | ZIO_FLAG_DONT_CACHE | ZIO_FLAG_CANFAIL | @@ -3124,8 +3127,7 @@ top: } DTRACE_PROBE2(l2arc__read, vdev_t *, vd, zio_t *, rzio); - ARCSTAT_INCR(arcstat_l2_read_bytes, - hdr->b_l2hdr->b_asize); + ARCSTAT_INCR(arcstat_l2_read_bytes, b_asize); if (*arc_flags & ARC_NOWAIT) { zio_nowait(rzio); From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:02:19 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F3E2729; Wed, 20 Nov 2013 11:02:19 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6D81F213E; Wed, 20 Nov 2013 11:02:19 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB2JLI059649; Wed, 20 Nov 2013 11:02:19 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB2J9H059648; Wed, 20 Nov 2013 11:02:19 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201102.rAKB2J9H059648@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:02:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258379 - vendor-sys/illumos/dist/uts/common/dtrace vendor/illumos/dist/lib/libdtrace/common X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:02:19 -0000 Author: avg Date: Wed Nov 20 11:02:18 2013 New Revision: 258379 URL: http://svnweb.freebsd.org/changeset/base/258379 Log: 4248 dtrace(1M) should never create DOF with empty probes section 4249 Only probes from the first DTrace object file will be included illumos/illumos-gate@54a20ab41aadcb81c53e72fc65886e964e9add59 Modified: vendor-sys/illumos/dist/uts/common/dtrace/dtrace.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libdtrace/common/dt_dof.c vendor/illumos/dist/lib/libdtrace/common/dt_error.c vendor/illumos/dist/lib/libdtrace/common/dt_impl.h Modified: vendor-sys/illumos/dist/uts/common/dtrace/dtrace.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/dtrace/dtrace.c Wed Nov 20 11:00:20 2013 (r258378) +++ vendor-sys/illumos/dist/uts/common/dtrace/dtrace.c Wed Nov 20 11:02:18 2013 (r258379) @@ -21,7 +21,7 @@ /* * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, Joyent, Inc. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2012 by Delphix. All rights reserved. */ @@ -13941,8 +13941,8 @@ dtrace_helper_provider_add(dof_helper_t * Check to make sure this isn't a duplicate. */ for (i = 0; i < help->dthps_nprovs; i++) { - if (dofhp->dofhp_addr == - help->dthps_provs[i]->dthp_prov.dofhp_addr) + if (dofhp->dofhp_dof == + help->dthps_provs[i]->dthp_prov.dofhp_dof) return (EALREADY); } From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:02:20 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43CB672A; Wed, 20 Nov 2013 11:02:20 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 31774213F; Wed, 20 Nov 2013 11:02:20 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB2Kew059663; Wed, 20 Nov 2013 11:02:20 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB2JW3059655; Wed, 20 Nov 2013 11:02:19 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201102.rAKB2JW3059655@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:02:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258379 - vendor-sys/illumos/dist/uts/common/dtrace vendor/illumos/dist/lib/libdtrace/common X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:02:20 -0000 Author: avg Date: Wed Nov 20 11:02:18 2013 New Revision: 258379 URL: http://svnweb.freebsd.org/changeset/base/258379 Log: 4248 dtrace(1M) should never create DOF with empty probes section 4249 Only probes from the first DTrace object file will be included illumos/illumos-gate@54a20ab41aadcb81c53e72fc65886e964e9add59 Modified: vendor/illumos/dist/lib/libdtrace/common/dt_dof.c vendor/illumos/dist/lib/libdtrace/common/dt_error.c vendor/illumos/dist/lib/libdtrace/common/dt_impl.h Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/dtrace/dtrace.c Modified: vendor/illumos/dist/lib/libdtrace/common/dt_dof.c ============================================================================== --- vendor/illumos/dist/lib/libdtrace/common/dt_dof.c Wed Nov 20 11:00:20 2013 (r258378) +++ vendor/illumos/dist/lib/libdtrace/common/dt_dof.c Wed Nov 20 11:02:18 2013 (r258379) @@ -22,6 +22,7 @@ /* * Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2011 by Delphix. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ #include @@ -482,7 +483,7 @@ dof_add_probe(dt_idhash_t *dhp, dt_ident return (0); } -static void +static int dof_add_provider(dt_dof_t *ddo, const dt_provider_t *pvp) { dtrace_hdl_t *dtp = ddo->ddo_hdl; @@ -493,8 +494,12 @@ dof_add_provider(dt_dof_t *ddo, const dt size_t sz; id_t i; - if (pvp->pv_flags & DT_PROVIDER_IMPL) - return; /* ignore providers that are exported by dtrace(7D) */ + if (pvp->pv_flags & DT_PROVIDER_IMPL) { + /* + * ignore providers that are exported by dtrace(7D) + */ + return (0); + } nxr = dt_popcb(pvp->pv_xrefs, pvp->pv_xrmax); dofs = alloca(sizeof (dof_secidx_t) * (nxr + 1)); @@ -521,6 +526,9 @@ dof_add_provider(dt_dof_t *ddo, const dt (void) dt_idhash_iter(pvp->pv_probes, dof_add_probe, ddo); + if (dt_buf_len(&ddo->ddo_probes) == 0) + return (dt_set_errno(dtp, EDT_NOPROBES)); + dofpv.dofpv_probes = dof_add_lsect(ddo, NULL, DOF_SECT_PROBES, sizeof (uint64_t), 0, sizeof (dof_probe_t), dt_buf_len(&ddo->ddo_probes)); @@ -575,6 +583,8 @@ dof_add_provider(dt_dof_t *ddo, const dt sizeof (dof_secidx_t), 0, sizeof (dof_secidx_t), sizeof (dof_secidx_t) * (nxr + 1)); } + + return (0); } static int @@ -818,8 +828,10 @@ dtrace_dof_create(dtrace_hdl_t *dtp, dtr */ if (flags & DTRACE_D_PROBES) { for (pvp = dt_list_next(&dtp->dt_provlist); - pvp != NULL; pvp = dt_list_next(pvp)) - dof_add_provider(ddo, pvp); + pvp != NULL; pvp = dt_list_next(pvp)) { + if (dof_add_provider(ddo, pvp) != 0) + return (NULL); + } } /* Modified: vendor/illumos/dist/lib/libdtrace/common/dt_error.c ============================================================================== --- vendor/illumos/dist/lib/libdtrace/common/dt_error.c Wed Nov 20 11:00:20 2013 (r258378) +++ vendor/illumos/dist/lib/libdtrace/common/dt_error.c Wed Nov 20 11:02:18 2013 (r258379) @@ -26,6 +26,7 @@ /* * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ #include @@ -108,7 +109,8 @@ static const struct { { EDT_BADSTACKPC, "Invalid stack program counter size" }, { EDT_BADAGGVAR, "Invalid aggregation variable identifier" }, { EDT_OVERSION, "Client requested deprecated version of library" }, - { EDT_ENABLING_ERR, "Failed to enable probe" } + { EDT_ENABLING_ERR, "Failed to enable probe" }, + { EDT_NOPROBES, "No probe sites found for declared provider" } }; static const int _dt_nerr = sizeof (_dt_errlist) / sizeof (_dt_errlist[0]); Modified: vendor/illumos/dist/lib/libdtrace/common/dt_impl.h ============================================================================== --- vendor/illumos/dist/lib/libdtrace/common/dt_impl.h Wed Nov 20 11:00:20 2013 (r258378) +++ vendor/illumos/dist/lib/libdtrace/common/dt_impl.h Wed Nov 20 11:02:18 2013 (r258379) @@ -25,7 +25,7 @@ */ /* - * Copyright (c) 2011, Joyent, Inc. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2012 by Delphix. All rights reserved. */ @@ -512,7 +512,8 @@ enum { EDT_BADSTACKPC, /* invalid stack program counter size */ EDT_BADAGGVAR, /* invalid aggregation variable identifier */ EDT_OVERSION, /* client is requesting deprecated version */ - EDT_ENABLING_ERR /* failed to enable probe */ + EDT_ENABLING_ERR, /* failed to enable probe */ + EDT_NOPROBES /* no probes sites for declared provider */ }; /* From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:03:11 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC9C698B; Wed, 20 Nov 2013 11:03:11 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AC18C214F; Wed, 20 Nov 2013 11:03:11 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB3Bik059824; Wed, 20 Nov 2013 11:03:11 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB3Boa059823; Wed, 20 Nov 2013 11:03:11 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201103.rAKB3Boa059823@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:03:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258380 - vendor-sys/illumos/dist/uts/common X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:03:11 -0000 Author: avg Date: Wed Nov 20 11:03:11 2013 New Revision: 258380 URL: http://svnweb.freebsd.org/changeset/base/258380 Log: 4236 Internet Packet Disturber 4237 net_* and hook_* man pages are wrong illumos/illumos-gate@fe77cc0407fb667ddc04e1a8f2e203bb7b9c80e1 Modified: vendor-sys/illumos/dist/uts/common/Makefile.files Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 11:02:18 2013 (r258379) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Wed Nov 20 11:03:11 2013 (r258380) @@ -1751,6 +1751,8 @@ IPF_OBJS += ip_fil_solaris.o fil.o solar ip_proxy.o ip_auth.o ip_pool.o ip_htable.o ip_lookup.o \ ip_log.o misc.o ip_compat.o ip_nat6.o drand48.o +IPD_OBJS += ipd.o + IBD_OBJS += ibd.o ibd_cm.o EIBNX_OBJS += enx_main.o enx_hdlrs.o enx_ibt.o enx_log.o enx_fip.o \ From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:04:15 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DC15AE3; Wed, 20 Nov 2013 11:04:15 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3D6F9215B; Wed, 20 Nov 2013 11:04:15 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB4Fjl060031; Wed, 20 Nov 2013 11:04:15 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB4FE8060029; Wed, 20 Nov 2013 11:04:15 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201104.rAKB4FE8060029@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:04:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258381 - vendor/illumos/dist/lib/libdtrace/common X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:04:15 -0000 Author: avg Date: Wed Nov 20 11:04:14 2013 New Revision: 258381 URL: http://svnweb.freebsd.org/changeset/base/258381 Log: 4251 libdtrace leaks open file handles 4250 libdtrace should use "F" specifier on fopen() illumos/illumos-gate@93ed8d0d4b068b95d0bb50d57bb854df462a8485 Modified: vendor/illumos/dist/lib/libdtrace/common/dt_cc.c Modified: vendor/illumos/dist/lib/libdtrace/common/dt_cc.c ============================================================================== --- vendor/illumos/dist/lib/libdtrace/common/dt_cc.c Wed Nov 20 11:03:11 2013 (r258380) +++ vendor/illumos/dist/lib/libdtrace/common/dt_cc.c Wed Nov 20 11:04:14 2013 (r258381) @@ -2149,7 +2149,7 @@ dt_load_libs_dir(dtrace_hdl_t *dtp, cons (void) snprintf(fname, sizeof (fname), "%s/%s", path, dp->d_name); - if ((fp = fopen(fname, "r")) == NULL) { + if ((fp = fopen(fname, "rF")) == NULL) { dt_dprintf("skipping library %s: %s\n", fname, strerror(errno)); continue; @@ -2171,12 +2171,15 @@ dt_load_libs_dir(dtrace_hdl_t *dtp, cons dt_dprintf("skipping library %s, already processed " "library with the same name: %s", dp->d_name, dld->dtld_library); + (void) fclose(fp); continue; } dtp->dt_filetag = fname; - if (dt_lib_depend_add(dtp, &dtp->dt_lib_dep, fname) != 0) + if (dt_lib_depend_add(dtp, &dtp->dt_lib_dep, fname) != 0) { + (void) fclose(fp); return (-1); /* preserve dt_errno */ + } rv = dt_compile(dtp, DT_CTX_DPROG, DTRACE_PROBESPEC_NAME, NULL, @@ -2184,8 +2187,10 @@ dt_load_libs_dir(dtrace_hdl_t *dtp, cons if (rv != NULL && dtp->dt_errno && (dtp->dt_errno != EDT_COMPILER || - dtp->dt_errtag != dt_errtag(D_PRAGMA_DEPEND))) + dtp->dt_errtag != dt_errtag(D_PRAGMA_DEPEND))) { + (void) fclose(fp); return (-1); /* preserve dt_errno */ + } if (dtp->dt_errno) dt_dprintf("error parsing library %s: %s\n", From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:05:08 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A2C9C44; Wed, 20 Nov 2013 11:05:08 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1A00A2169; Wed, 20 Nov 2013 11:05:08 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB57b4060208; Wed, 20 Nov 2013 11:05:07 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB57Zb060207; Wed, 20 Nov 2013 11:05:07 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201105.rAKB57Zb060207@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:05:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258382 - vendor/illumos/dist/lib/libzpool/common X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:05:08 -0000 Author: avg Date: Wed Nov 20 11:05:07 2013 New Revision: 258382 URL: http://svnweb.freebsd.org/changeset/base/258382 Log: 4161 deadlock between zfs_read() and zfs_putpage() illumos/illumos-gate@b3d32f0ceb59362ba287dcfd6de471e98bfc7fa9 Modified: vendor/illumos/dist/lib/libzpool/common/kernel.c Modified: vendor/illumos/dist/lib/libzpool/common/kernel.c ============================================================================== --- vendor/illumos/dist/lib/libzpool/common/kernel.c Wed Nov 20 11:04:14 2013 (r258381) +++ vendor/illumos/dist/lib/libzpool/common/kernel.c Wed Nov 20 11:05:07 2013 (r258382) @@ -21,6 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ #include @@ -221,10 +222,10 @@ rw_enter(krwlock_t *rwlp, krw_t rw) ASSERT(rwlp->rw_owner != (void *)-1UL); ASSERT(rwlp->rw_owner != curthread); - if (rw == RW_READER) - VERIFY(rw_rdlock(&rwlp->rw_lock) == 0); - else + if (rw == RW_WRITER) VERIFY(rw_wrlock(&rwlp->rw_lock) == 0); + else + VERIFY(rw_rdlock(&rwlp->rw_lock) == 0); rwlp->rw_owner = curthread; } @@ -247,10 +248,10 @@ rw_tryenter(krwlock_t *rwlp, krw_t rw) ASSERT(rwlp->initialized == B_TRUE); ASSERT(rwlp->rw_owner != (void *)-1UL); - if (rw == RW_READER) - rv = rw_tryrdlock(&rwlp->rw_lock); - else + if (rw == RW_WRITER) rv = rw_trywrlock(&rwlp->rw_lock); + else + rv = rw_tryrdlock(&rwlp->rw_lock); if (rv == 0) { rwlp->rw_owner = curthread; From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:06:03 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26679D8B; Wed, 20 Nov 2013 11:06:03 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 15D81217C; Wed, 20 Nov 2013 11:06:03 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB62PG060374; Wed, 20 Nov 2013 11:06:02 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB627j060368; Wed, 20 Nov 2013 11:06:02 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201106.rAKB627j060368@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:06:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258383 - in vendor-sys/illumos/dist/uts/common: fs fs/zfs sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:06:03 -0000 Author: avg Date: Wed Nov 20 11:06:02 2013 New Revision: 258383 URL: http://svnweb.freebsd.org/changeset/base/258383 Log: 3928 `tail -f ...` doesn't notice file truncation 3929 `man tail` doesn't mentioned "-F" option 3930 'tail -F ...' not resetting the offset of file rotation properly 3968 want FILE_TRUNC event for PORT_SOURCE_FILE illumos/illumos-gate@72102e7461c97dc268d21d9dd8f02da45f174acd Modified: vendor-sys/illumos/dist/uts/common/fs/vnode.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c vendor-sys/illumos/dist/uts/common/sys/vnode.h Modified: vendor-sys/illumos/dist/uts/common/fs/vnode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/vnode.c Wed Nov 20 11:05:07 2013 (r258382) +++ vendor-sys/illumos/dist/uts/common/fs/vnode.c Wed Nov 20 11:06:02 2013 (r258383) @@ -21,6 +21,7 @@ /* * Copyright (c) 1988, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ /* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ @@ -2582,6 +2583,15 @@ vnevent_mountedover(vnode_t *vp, caller_ (void) VOP_VNEVENT(vp, VE_MOUNTEDOVER, NULL, NULL, ct); } +void +vnevent_truncate(vnode_t *vp, caller_context_t *ct) +{ + if (vp == NULL || vp->v_femhead == NULL) { + return; + } + (void) VOP_VNEVENT(vp, VE_TRUNCATE, NULL, NULL, ct); +} + /* * Vnode accessors. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c Wed Nov 20 11:05:07 2013 (r258382) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c Wed Nov 20 11:06:02 2013 (r258383) @@ -2756,6 +2756,9 @@ top: ZFS_EXIT(zfsvfs); return (err); } + + if (vap->va_size == 0) + vnevent_truncate(ZTOV(zp), ct); } if (mask & (AT_ATIME|AT_MTIME) || @@ -4770,6 +4773,9 @@ zfs_space(vnode_t *vp, int cmd, flock64_ error = zfs_freesp(zp, off, len, flag, TRUE); + if (error == 0 && off == 0 && len == 0) + vnevent_truncate(ZTOV(zp), ct); + ZFS_EXIT(zfsvfs); return (error); } Modified: vendor-sys/illumos/dist/uts/common/sys/vnode.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/vnode.h Wed Nov 20 11:05:07 2013 (r258382) +++ vendor-sys/illumos/dist/uts/common/sys/vnode.h Wed Nov 20 11:06:02 2013 (r258383) @@ -21,6 +21,7 @@ /* * Copyright (c) 1988, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013, Joyent, Inc. All rights reserved. */ /* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ @@ -733,7 +734,8 @@ typedef enum vnevent { VE_CREATE = 5, /* Create with vnode's name which exists */ VE_LINK = 6, /* Link with vnode's name as source */ VE_RENAME_DEST_DIR = 7, /* Rename with vnode as target dir */ - VE_MOUNTEDOVER = 8 /* File or Filesystem got mounted over vnode */ + VE_MOUNTEDOVER = 8, /* File or Filesystem got mounted over vnode */ + VE_TRUNCATE = 9 /* Truncate */ } vnevent_t; /* @@ -1290,6 +1292,7 @@ void vnevent_create(vnode_t *, caller_co void vnevent_link(vnode_t *, caller_context_t *); void vnevent_rename_dest_dir(vnode_t *, caller_context_t *ct); void vnevent_mountedover(vnode_t *, caller_context_t *); +void vnevent_truncate(vnode_t *, caller_context_t *); int vnevent_support(vnode_t *, caller_context_t *); /* Vnode specific data */ From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:06:59 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18E72ED9; Wed, 20 Nov 2013 11:06:59 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ED238218B; Wed, 20 Nov 2013 11:06:58 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB6w34060532; Wed, 20 Nov 2013 11:06:58 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB6vAI060519; Wed, 20 Nov 2013 11:06:57 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201106.rAKB6vAI060519@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:06:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258384 - in vendor/illumos/dist: cmd/zfs lib/libzfs/common man/man1m X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:06:59 -0000 Author: avg Date: Wed Nov 20 11:06:57 2013 New Revision: 258384 URL: http://svnweb.freebsd.org/changeset/base/258384 Log: 2583 Add -p (parsable) option to zfs list illumos/illumos-gate@43d68d68c1ce08fb35026bebfb141af422e7082e Modified: vendor/illumos/dist/cmd/zfs/zfs_iter.c vendor/illumos/dist/cmd/zfs/zfs_iter.h vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor/illumos/dist/cmd/zfs/zfs_iter.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_iter.c Wed Nov 20 11:06:02 2013 (r258383) +++ vendor/illumos/dist/cmd/zfs/zfs_iter.c Wed Nov 20 11:06:57 2013 (r258384) @@ -18,8 +18,10 @@ * * CDDL HEADER END */ + /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. */ #include @@ -107,7 +109,8 @@ zfs_callback(zfs_handle_t *zhp, void *da cb->cb_props_table); if (zfs_expand_proplist(zhp, cb->cb_proplist, - (cb->cb_flags & ZFS_ITER_RECVD_PROPS)) + (cb->cb_flags & ZFS_ITER_RECVD_PROPS), + (cb->cb_flags & ZFS_ITER_LITERAL_PROPS)) != 0) { free(node); return (-1); Modified: vendor/illumos/dist/cmd/zfs/zfs_iter.h ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_iter.h Wed Nov 20 11:06:02 2013 (r258383) +++ vendor/illumos/dist/cmd/zfs/zfs_iter.h Wed Nov 20 11:06:57 2013 (r258384) @@ -18,9 +18,11 @@ * * CDDL HEADER END */ + /* * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. */ #ifndef ZFS_ITER_H @@ -43,6 +45,7 @@ typedef struct zfs_sort_column { #define ZFS_ITER_PROP_LISTSNAPS (1 << 2) #define ZFS_ITER_DEPTH_LIMIT (1 << 3) #define ZFS_ITER_RECVD_PROPS (1 << 4) +#define ZFS_ITER_LITERAL_PROPS (1 << 5) int zfs_for_each(int, char **, int options, zfs_type_t, zfs_sort_column_t *, zprop_list_t **, int, zfs_iter_f, void *); Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Nov 20 11:06:02 2013 (r258383) +++ vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Nov 20 11:06:57 2013 (r258384) @@ -21,11 +21,11 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2012 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2012 by Delphix. All rights reserved. * Copyright 2012 Milan Jurik. All rights reserved. * Copyright (c) 2012, Joyent, Inc. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. */ #include @@ -232,9 +232,8 @@ get_usage(zfs_help_t idx) return (gettext("\tupgrade [-v]\n" "\tupgrade [-r] [-V version] <-a | filesystem ...>\n")); case HELP_LIST: - return (gettext("\tlist [-rH][-d max] " - "[-o property[,...]] [-t type[,...]] [-s property] ...\n" - "\t [-S property] ... " + return (gettext("\tlist [-Hp] [-r|-d max] [-o property[,...]] " + "[-s property]...\n\t [-S property]... [-t type[,...]] " "[filesystem|volume|snapshot] ...\n")); case HELP_MOUNT: return (gettext("\tmount\n" @@ -291,12 +290,12 @@ get_usage(zfs_help_t idx) "\n")); case HELP_USERSPACE: return (gettext("\tuserspace [-Hinp] [-o field[,...]] " - "[-s field] ...\n\t[-S field] ... " - "[-t type[,...]] \n")); + "[-s field]...\n\t [-S field]... [-t type[,...]] " + "\n")); case HELP_GROUPSPACE: return (gettext("\tgroupspace [-Hinp] [-o field[,...]] " - "[-s field] ...\n\t[-S field] ... " - "[-t type[,...]] \n")); + "[-s field]...\n\t [-S field]... [-t type[,...]] " + "\n")); case HELP_HOLD: return (gettext("\thold [-r] ...\n")); case HELP_HOLDS: @@ -2104,7 +2103,7 @@ zfs_do_upgrade(int argc, char **argv) * -i Translate SID to POSIX ID. * -n Print numeric ID instead of user/group name. * -o Control which fields to display. - * -p Use exact (parseable) numeric output. + * -p Use exact (parsable) numeric output. * -s Specify sort columns, descending order. * -S Specify sort columns, ascending order. * -t Control which object types to display. @@ -2782,24 +2781,25 @@ zfs_do_userspace(int argc, char **argv) } /* - * list [-r][-d max] [-H] [-o property[,property]...] [-t type[,type]...] - * [-s property [-s property]...] [-S property [-S property]...] - * ... + * list [-Hp][-r|-d max] [-o property[,...]] [-s property] ... [-S property] ... + * [-t type[,...]] [filesystem|volume|snapshot] ... * - * -r Recurse over all children + * -H Scripted mode; elide headers and separate columns by tabs. + * -p Display values in parsable (literal) format. + * -r Recurse over all children. * -d Limit recursion by depth. - * -H Scripted mode; elide headers and separate columns by tabs * -o Control which fields to display. - * -t Control which object types to display. * -s Specify sort columns, descending order. * -S Specify sort columns, ascending order. + * -t Control which object types to display. * - * When given no arguments, lists all filesystems in the system. + * When given no arguments, list all filesystems in the system. * Otherwise, list the specified datasets, optionally recursing down them if * '-r' is specified. */ typedef struct list_cbdata { boolean_t cb_first; + boolean_t cb_literal; boolean_t cb_scripted; zprop_list_t *cb_proplist; } list_cbdata_t; @@ -2808,8 +2808,9 @@ typedef struct list_cbdata { * Given a list of columns to display, output appropriate headers for each one. */ static void -print_header(zprop_list_t *pl) +print_header(list_cbdata_t *cb) { + zprop_list_t *pl = cb->cb_proplist; char headerbuf[ZFS_MAXPROPLEN]; const char *header; int i; @@ -2850,19 +2851,19 @@ print_header(zprop_list_t *pl) * to the described layout. */ static void -print_dataset(zfs_handle_t *zhp, zprop_list_t *pl, boolean_t scripted) +print_dataset(zfs_handle_t *zhp, list_cbdata_t *cb) { + zprop_list_t *pl = cb->cb_proplist; boolean_t first = B_TRUE; char property[ZFS_MAXPROPLEN]; nvlist_t *userprops = zfs_get_user_props(zhp); nvlist_t *propval; char *propstr; boolean_t right_justify; - int width; for (; pl != NULL; pl = pl->pl_next) { if (!first) { - if (scripted) + if (cb->cb_scripted) (void) printf("\t"); else (void) printf(" "); @@ -2872,22 +2873,22 @@ print_dataset(zfs_handle_t *zhp, zprop_l if (pl->pl_prop != ZPROP_INVAL) { if (zfs_prop_get(zhp, pl->pl_prop, property, - sizeof (property), NULL, NULL, 0, B_FALSE) != 0) + sizeof (property), NULL, NULL, 0, + cb->cb_literal) != 0) propstr = "-"; else propstr = property; - right_justify = zfs_prop_align_right(pl->pl_prop); } else if (zfs_prop_userquota(pl->pl_user_prop)) { if (zfs_prop_get_userquota(zhp, pl->pl_user_prop, - property, sizeof (property), B_FALSE) != 0) + property, sizeof (property), cb->cb_literal) != 0) propstr = "-"; else propstr = property; right_justify = B_TRUE; } else if (zfs_prop_written(pl->pl_user_prop)) { if (zfs_prop_get_written(zhp, pl->pl_user_prop, - property, sizeof (property), B_FALSE) != 0) + property, sizeof (property), cb->cb_literal) != 0) propstr = "-"; else propstr = property; @@ -2902,19 +2903,17 @@ print_dataset(zfs_handle_t *zhp, zprop_l right_justify = B_FALSE; } - width = pl->pl_width; - /* * If this is being called in scripted mode, or if this is the * last column and it is left-justified, don't include a width * format specifier. */ - if (scripted || (pl->pl_next == NULL && !right_justify)) + if (cb->cb_scripted || (pl->pl_next == NULL && !right_justify)) (void) printf("%s", propstr); else if (right_justify) - (void) printf("%*s", width, propstr); + (void) printf("%*s", pl->pl_width, propstr); else - (void) printf("%-*s", width, propstr); + (void) printf("%-*s", pl->pl_width, propstr); } (void) printf("\n"); @@ -2930,11 +2929,11 @@ list_callback(zfs_handle_t *zhp, void *d if (cbp->cb_first) { if (!cbp->cb_scripted) - print_header(cbp->cb_proplist); + print_header(cbp); cbp->cb_first = B_FALSE; } - print_dataset(zhp, cbp->cb_proplist, cbp->cb_scripted); + print_dataset(zhp, cbp); return (0); } @@ -2943,7 +2942,6 @@ static int zfs_do_list(int argc, char **argv) { int c; - boolean_t scripted = B_FALSE; static char default_fields[] = "name,used,available,referenced,mountpoint"; int types = ZFS_TYPE_DATASET; @@ -2957,11 +2955,15 @@ zfs_do_list(int argc, char **argv) int flags = ZFS_ITER_PROP_LISTSNAPS | ZFS_ITER_ARGS_CAN_BE_PATHS; /* check options */ - while ((c = getopt(argc, argv, ":d:o:rt:Hs:S:")) != -1) { + while ((c = getopt(argc, argv, "HS:d:o:prs:t:")) != -1) { switch (c) { case 'o': fields = optarg; break; + case 'p': + cb.cb_literal = B_TRUE; + flags |= ZFS_ITER_LITERAL_PROPS; + break; case 'd': limit = parse_depth(optarg, &flags); break; @@ -2969,7 +2971,7 @@ zfs_do_list(int argc, char **argv) flags |= ZFS_ITER_RECURSE; break; case 'H': - scripted = B_TRUE; + cb.cb_scripted = B_TRUE; break; case 's': if (zfs_add_sort_column(&sortcol, optarg, @@ -3051,7 +3053,6 @@ zfs_do_list(int argc, char **argv) != 0) usage(B_FALSE); - cb.cb_scripted = scripted; cb.cb_first = B_TRUE; ret = zfs_for_each(argc, argv, flags, types, sortcol, &cb.cb_proplist, Modified: vendor/illumos/dist/lib/libzfs/common/libzfs.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Nov 20 11:06:02 2013 (r258383) +++ vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Nov 20 11:06:57 2013 (r258384) @@ -21,10 +21,10 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2012 by Delphix. All rights reserved. * Copyright (c) 2012, Joyent, Inc. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. */ #ifndef _LIBZFS_H @@ -458,7 +458,8 @@ typedef struct zprop_list { boolean_t pl_fixed; } zprop_list_t; -extern int zfs_expand_proplist(zfs_handle_t *, zprop_list_t **, boolean_t); +extern int zfs_expand_proplist(zfs_handle_t *, zprop_list_t **, boolean_t, + boolean_t); extern void zfs_prune_proplist(zfs_handle_t *, uint8_t *); #define ZFS_MOUNTPOINT_NONE "none" Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Nov 20 11:06:02 2013 (r258383) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Nov 20 11:06:57 2013 (r258384) @@ -23,9 +23,9 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright (c) 2012 DEY Storage Systems, Inc. All rights reserved. - * Copyright 2012 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2013 Martin Matuska. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. */ #include @@ -3798,7 +3798,8 @@ zfs_get_recvd_props(zfs_handle_t *zhp) * of the RECEIVED column. */ int -zfs_expand_proplist(zfs_handle_t *zhp, zprop_list_t **plp, boolean_t received) +zfs_expand_proplist(zfs_handle_t *zhp, zprop_list_t **plp, boolean_t received, + boolean_t literal) { libzfs_handle_t *hdl = zhp->zfs_hdl; zprop_list_t *entry; @@ -3860,18 +3861,18 @@ zfs_expand_proplist(zfs_handle_t *zhp, z * Now go through and check the width of any non-fixed columns */ for (entry = *plp; entry != NULL; entry = entry->pl_next) { - if (entry->pl_fixed) + if (entry->pl_fixed && !literal) continue; if (entry->pl_prop != ZPROP_INVAL) { if (zfs_prop_get(zhp, entry->pl_prop, - buf, sizeof (buf), NULL, NULL, 0, B_FALSE) == 0) { + buf, sizeof (buf), NULL, NULL, 0, literal) == 0) { if (strlen(buf) > entry->pl_width) entry->pl_width = strlen(buf); } if (received && zfs_prop_get_recvd(zhp, zfs_prop_to_name(entry->pl_prop), - buf, sizeof (buf), B_FALSE) == 0) + buf, sizeof (buf), literal) == 0) if (strlen(buf) > entry->pl_recvd_width) entry->pl_recvd_width = strlen(buf); } else { @@ -3884,7 +3885,7 @@ zfs_expand_proplist(zfs_handle_t *zhp, z } if (received && zfs_prop_get_recvd(zhp, entry->pl_user_prop, - buf, sizeof (buf), B_FALSE) == 0) + buf, sizeof (buf), literal) == 0) if (strlen(buf) > entry->pl_recvd_width) entry->pl_recvd_width = strlen(buf); } Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Wed Nov 20 11:06:02 2013 (r258383) +++ vendor/illumos/dist/man/man1m/zfs.1m Wed Nov 20 11:06:57 2013 (r258384) @@ -27,7 +27,7 @@ .\" Copyright 2013 Nexenta Systems, Inc. All Rights Reserved. .\" Copyright (c) 2013, Joyent, Inc. All rights reserved. .\" -.TH ZFS 1M "Jan 26, 2013" +.TH ZFS 1M "Oct 16, 2013" .SH NAME zfs \- configures ZFS file systems .SH SYNOPSIS @@ -95,7 +95,7 @@ zfs \- configures ZFS file systems .LP .nf -\fBzfs\fR \fBlist\fR [\fB-r\fR|\fB-d\fR \fIdepth\fR][\fB-H\fR][\fB-o\fR \fIproperty\fR[,\fIproperty\fR]...] [\fB-t\fR \fItype\fR[,\fItype\fR]...] +\fBzfs\fR \fBlist\fR [\fB-r\fR|\fB-d\fR \fIdepth\fR][\fB-Hp\fR][\fB-o\fR \fIproperty\fR[,\fIproperty\fR]...] [\fB-t\fR \fItype\fR[,\fItype\fR]...] [\fB-s\fR \fIproperty\fR]... [\fB-S\fR \fIproperty\fR]... [\fIfilesystem\fR|\fIvolume\fR|\fIsnapshot\fR]... .fi @@ -248,7 +248,6 @@ zfs \- configures ZFS file systems \fBzfs\fR \fBdiff\fR [\fB-FHt\fR] \fIsnapshot\fR \fIsnapshot|filesystem\fR .SH DESCRIPTION -.sp .LP The \fBzfs\fR command configures \fBZFS\fR datasets within a \fBZFS\fR storage pool, as described in \fBzpool\fR(1M). A dataset is identified by a unique path @@ -306,7 +305,6 @@ specified as \fIfilesystem@name\fR or \f .RE .SS "ZFS File System Hierarchy" -.sp .LP A \fBZFS\fR storage pool is a logical collection of devices that provide space for datasets. A storage pool is also the root of the \fBZFS\fR file system @@ -320,7 +318,6 @@ characteristics, however, are managed by .LP See \fBzpool\fR(1M) for more information on creating and administering pools. .SS "Snapshots" -.sp .LP A snapshot is a read-only copy of a file system or volume. Snapshots can be created extremely quickly, and initially consume no additional space within the @@ -337,7 +334,6 @@ in the root of the file system. Snapshot and may be unmounted at regular intervals. The visibility of the \fB\&.zfs\fR directory can be controlled by the \fBsnapdir\fR property. .SS "Clones" -.sp .LP A clone is a writable volume or file system whose initial contents are the same as another dataset. As with snapshots, creating a clone is nearly @@ -357,7 +353,6 @@ The clone parent-child dependency relati clone of the specified file system, which makes it possible to destroy the file system that the clone was created from. .SS "Mount Points" -.sp .LP Creating a \fBZFS\fR file system is a simple operation, so the number of file systems per system is likely to be numerous. To cope with this, \fBZFS\fR @@ -390,7 +385,6 @@ is set to \fBlegacy\fR, \fBZFS\fR makes and the administrator is responsible for mounting and unmounting the file system. .SS "Zones" -.sp .LP A \fBZFS\fR file system can be added to a non-global zone by using the \fBzonecfg\fR \fBadd fs\fR subcommand. A \fBZFS\fR file system that is added to @@ -428,7 +422,6 @@ The global administrator can forcibly cl this should be done with extreme care. The global administrator should verify that all the mount points are acceptable before clearing the property. .SS "Native Properties" -.sp .LP Properties are divided into two types, native properties and user-defined (or "user") properties. Native properties either export internal statistics or @@ -1536,7 +1529,6 @@ The \fBcasesensitivity\fR, \fBnormalizat are also new permissions that can be assigned to non-privileged users by using the \fBZFS\fR delegated administration feature. .SS "Temporary Mount Point Properties" -.sp .LP When a file system is mounted, either through \fBmount\fR(1M) for legacy mounts or the \fBzfs mount\fR command for normal file systems, its mount options are @@ -1565,7 +1557,6 @@ are reported as "temporary" by the \fBzf changed while the dataset is mounted, the new setting overrides any temporary settings. .SS "User Properties" -.sp .LP In addition to the standard native properties, \fBZFS\fR supports arbitrary user properties. User properties have no effect on \fBZFS\fR behavior, but @@ -1598,7 +1589,6 @@ to clear a user property . If the proper dataset, it is removed entirely. Property values are limited to 1024 characters. .SS "ZFS Volumes as Swap or Dump Devices" -.sp .LP During an initial installation a swap device and dump device are created on \fBZFS\fR volumes in the \fBZFS\fR root pool. By default, the swap area size is @@ -1614,7 +1604,6 @@ installed or upgraded, use the \fBswap\f If you need to change the size of your swap area or dump device, see the \fISolaris ZFS Administration Guide\fR. .SH SUBCOMMANDS -.sp .LP All subcommands that modify state are logged persistently to the pool in their original form. @@ -2117,7 +2106,7 @@ only dataset that can be renamed recursi .sp .ne 2 .na -\fB\fBzfs\fR \fBlist\fR [\fB-r\fR|\fB-d\fR \fIdepth\fR] [\fB-H\fR] [\fB-o\fR +\fB\fBzfs\fR \fBlist\fR [\fB-r\fR|\fB-d\fR \fIdepth\fR] [\fB-Hp\fR] [\fB-o\fR \fIproperty\fR[,\fIproperty\fR]...] [ \fB-t\fR \fItype\fR[,\fItype\fR]...] [ \fB-s\fR \fIproperty\fR ]... [ \fB-S\fR \fIproperty\fR ]... [\fIfilesystem\fR|\fIvolume\fR|\fIsnapshot\fR]...\fR @@ -2144,6 +2133,16 @@ tab instead of arbitrary white space. .sp .ne 2 .na +\fB\fB-p\fR\fR +.ad +.sp .6 +.RS 4n +Display numbers in parsable (exact) values. +.RE + +.sp +.ne 2 +.na \fB\fB-r\fR\fR .ad .sp .6 @@ -2380,7 +2379,7 @@ is all sources. .ad .sp .6 .RS 4n -Display numbers in parseable (exact) values. +Display numbers in parsable (exact) values. .RE .RE @@ -3392,7 +3391,7 @@ F Regular file .ad .sp .6 .RS 4n -Give more parseable tab-separated output, without header lines and without arrows. +Give more parsable tab-separated output, without header lines and without arrows. .RE .sp .ne 2 @@ -3953,7 +3952,6 @@ M F /tank/test/modified .sp .SH EXIT STATUS -.sp .LP The following exit values are returned: .sp @@ -3987,7 +3985,6 @@ Invalid command line options were specif .RE .SH ATTRIBUTES -.sp .LP See \fBattributes\fR(5) for descriptions of the following attributes: .sp @@ -4003,7 +4000,6 @@ Interface Stability Committed .TE .SH SEE ALSO -.sp .LP \fBssh\fR(1), \fBiscsitadm\fR(1M), \fBmount\fR(1M), \fBshare\fR(1M), \fBsharemgr\fR(1M), \fBunshare\fR(1M), \fBzonecfg\fR(1M), \fBzpool\fR(1M), From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:08:08 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24F0D98; Wed, 20 Nov 2013 11:08:08 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 04ED62198; Wed, 20 Nov 2013 11:08:08 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB87EI060753; Wed, 20 Nov 2013 11:08:07 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB87gf060750; Wed, 20 Nov 2013 11:08:07 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201108.rAKB87gf060750@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:08:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258385 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:08:08 -0000 Author: avg Date: Wed Nov 20 11:08:07 2013 New Revision: 258385 URL: http://svnweb.freebsd.org/changeset/base/258385 Log: 4128 disks in zpools never go away when pulled illumos/illumos-gate@39cddb10a31c1c2e66aed69e6871d09caa4c8147 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_disk.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_disk.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_disk.h Wed Nov 20 11:06:57 2013 (r258384) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_disk.h Wed Nov 20 11:08:07 2013 (r258385) @@ -22,6 +22,7 @@ * Copyright 2008 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. * Copyright (c) 2013 Joyent, Inc. All rights reserved. + * Copyright 2012 Nexenta Systems, Inc. All rights reserved. */ #ifndef _SYS_VDEV_DISK_H @@ -44,6 +45,8 @@ typedef struct vdev_disk { ddi_devid_t vd_devid; char *vd_minor; ldi_handle_t vd_lh; + list_t vd_ldi_cbs; + boolean_t vd_ldi_offline; } vdev_disk_t; #endif Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Wed Nov 20 11:06:57 2013 (r258384) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Wed Nov 20 11:08:07 2013 (r258385) @@ -22,7 +22,7 @@ * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013 by Delphix. All rights reserved. * Copyright 2013 Nexenta Systems, Inc. All rights reserved. - * Copyright 2013 Joyent, Inc. All rights reserved. + * Copyright (c) 2013 Joyent, Inc. All rights reserved. */ #include @@ -42,6 +42,146 @@ extern ldi_ident_t zfs_li; +static void vdev_disk_close(vdev_t *); + +typedef struct vdev_disk_ldi_cb { + list_node_t lcb_next; + ldi_callback_id_t lcb_id; +} vdev_disk_ldi_cb_t; + +static void +vdev_disk_alloc(vdev_t *vd) +{ + vdev_disk_t *dvd; + + dvd = vd->vdev_tsd = kmem_zalloc(sizeof (vdev_disk_t), KM_SLEEP); + /* + * Create the LDI event callback list. + */ + list_create(&dvd->vd_ldi_cbs, sizeof (vdev_disk_ldi_cb_t), + offsetof(vdev_disk_ldi_cb_t, lcb_next)); +} + +static void +vdev_disk_free(vdev_t *vd) +{ + vdev_disk_t *dvd = vd->vdev_tsd; + vdev_disk_ldi_cb_t *lcb; + + if (dvd == NULL) + return; + + /* + * We have already closed the LDI handle. Clean up the LDI event + * callbacks and free vd->vdev_tsd. + */ + while ((lcb = list_head(&dvd->vd_ldi_cbs)) != NULL) { + list_remove(&dvd->vd_ldi_cbs, lcb); + (void) ldi_ev_remove_callbacks(lcb->lcb_id); + kmem_free(lcb, sizeof (vdev_disk_ldi_cb_t)); + } + list_destroy(&dvd->vd_ldi_cbs); + kmem_free(dvd, sizeof (vdev_disk_t)); + vd->vdev_tsd = NULL; +} + +/* ARGSUSED */ +static int +vdev_disk_off_notify(ldi_handle_t lh, ldi_ev_cookie_t ecookie, void *arg, + void *ev_data) +{ + vdev_t *vd = (vdev_t *)arg; + vdev_disk_t *dvd = vd->vdev_tsd; + + /* + * Ignore events other than offline. + */ + if (strcmp(ldi_ev_get_type(ecookie), LDI_EV_OFFLINE) != 0) + return (LDI_EV_SUCCESS); + + /* + * All LDI handles must be closed for the state change to succeed, so + * call on vdev_disk_close() to do this. + * + * We inform vdev_disk_close that it is being called from offline + * notify context so it will defer cleanup of LDI event callbacks and + * freeing of vd->vdev_tsd to the offline finalize or a reopen. + */ + dvd->vd_ldi_offline = B_TRUE; + vdev_disk_close(vd); + + /* + * Now that the device is closed, request that the spa_async_thread + * mark the device as REMOVED and notify FMA of the removal. + */ + zfs_post_remove(vd->vdev_spa, vd); + vd->vdev_remove_wanted = B_TRUE; + spa_async_request(vd->vdev_spa, SPA_ASYNC_REMOVE); + + return (LDI_EV_SUCCESS); +} + +/* ARGSUSED */ +static void +vdev_disk_off_finalize(ldi_handle_t lh, ldi_ev_cookie_t ecookie, + int ldi_result, void *arg, void *ev_data) +{ + vdev_t *vd = (vdev_t *)arg; + + /* + * Ignore events other than offline. + */ + if (strcmp(ldi_ev_get_type(ecookie), LDI_EV_OFFLINE) != 0) + return; + + /* + * We have already closed the LDI handle in notify. + * Clean up the LDI event callbacks and free vd->vdev_tsd. + */ + vdev_disk_free(vd); + + /* + * Request that the vdev be reopened if the offline state change was + * unsuccessful. + */ + if (ldi_result != LDI_EV_SUCCESS) { + vd->vdev_probe_wanted = B_TRUE; + spa_async_request(vd->vdev_spa, SPA_ASYNC_PROBE); + } +} + +static ldi_ev_callback_t vdev_disk_off_callb = { + .cb_vers = LDI_EV_CB_VERS, + .cb_notify = vdev_disk_off_notify, + .cb_finalize = vdev_disk_off_finalize +}; + +/* ARGSUSED */ +static void +vdev_disk_dgrd_finalize(ldi_handle_t lh, ldi_ev_cookie_t ecookie, + int ldi_result, void *arg, void *ev_data) +{ + vdev_t *vd = (vdev_t *)arg; + + /* + * Ignore events other than degrade. + */ + if (strcmp(ldi_ev_get_type(ecookie), LDI_EV_DEGRADE) != 0) + return; + + /* + * Degrade events always succeed. Mark the vdev as degraded. + * This status is purely informative for the user. + */ + (void) vdev_degrade(vd->vdev_spa, vd->vdev_guid, 0); +} + +static ldi_ev_callback_t vdev_disk_dgrd_callb = { + .cb_vers = LDI_EV_CB_VERS, + .cb_notify = NULL, + .cb_finalize = vdev_disk_dgrd_finalize +}; + static void vdev_disk_hold(vdev_t *vd) { @@ -146,7 +286,9 @@ vdev_disk_open(vdev_t *vd, uint64_t *psi uint64_t *ashift) { spa_t *spa = vd->vdev_spa; - vdev_disk_t *dvd; + vdev_disk_t *dvd = vd->vdev_tsd; + ldi_ev_cookie_t ecookie; + vdev_disk_ldi_cb_t *lcb; union { struct dk_minfo_ext ude; struct dk_minfo ud; @@ -172,13 +314,25 @@ vdev_disk_open(vdev_t *vd, uint64_t *psi * Reopen the device if it's not currently open. Otherwise, * just update the physical size of the device. */ - if (vd->vdev_tsd != NULL) { - ASSERT(vd->vdev_reopening); - dvd = vd->vdev_tsd; - goto skip_open; + if (dvd != NULL) { + if (dvd->vd_ldi_offline && dvd->vd_lh == NULL) { + /* + * If we are opening a device in its offline notify + * context, the LDI handle was just closed. Clean + * up the LDI event callbacks and free vd->vdev_tsd. + */ + vdev_disk_free(vd); + } else { + ASSERT(vd->vdev_reopening); + goto skip_open; + } } - dvd = vd->vdev_tsd = kmem_zalloc(sizeof (vdev_disk_t), KM_SLEEP); + /* + * Create vd->vdev_tsd. + */ + vdev_disk_alloc(vd); + dvd = vd->vdev_tsd; /* * When opening a disk device, we want to preserve the user's original @@ -211,23 +365,28 @@ vdev_disk_open(vdev_t *vd, uint64_t *psi if (vd->vdev_wholedisk == -1ULL) { size_t len = strlen(vd->vdev_path) + 3; char *buf = kmem_alloc(len, KM_SLEEP); - ldi_handle_t lh; (void) snprintf(buf, len, "%ss0", vd->vdev_path); - if (ldi_open_by_name(buf, spa_mode(spa), kcred, - &lh, zfs_li) == 0) { + error = ldi_open_by_name(buf, spa_mode(spa), kcred, + &dvd->vd_lh, zfs_li); + if (error == 0) { spa_strfree(vd->vdev_path); vd->vdev_path = buf; vd->vdev_wholedisk = 1ULL; - (void) ldi_close(lh, spa_mode(spa), kcred); } else { kmem_free(buf, len); } } - error = ldi_open_by_name(vd->vdev_path, spa_mode(spa), kcred, - &dvd->vd_lh, zfs_li); + /* + * If we have not yet opened the device, try to open it by the + * specified path. + */ + if (error != 0) { + error = ldi_open_by_name(vd->vdev_path, spa_mode(spa), + kcred, &dvd->vd_lh, zfs_li); + } /* * Compare the devid to the stored value. @@ -334,6 +493,27 @@ vdev_disk_open(vdev_t *vd, uint64_t *psi kmem_free(physpath, MAXPATHLEN); } + /* + * Register callbacks for the LDI offline event. + */ + if (ldi_ev_get_cookie(dvd->vd_lh, LDI_EV_OFFLINE, &ecookie) == + LDI_EV_SUCCESS) { + lcb = kmem_zalloc(sizeof (vdev_disk_ldi_cb_t), KM_SLEEP); + list_insert_tail(&dvd->vd_ldi_cbs, lcb); + (void) ldi_ev_register_callbacks(dvd->vd_lh, ecookie, + &vdev_disk_off_callb, (void *) vd, &lcb->lcb_id); + } + + /* + * Register callbacks for the LDI degrade event. + */ + if (ldi_ev_get_cookie(dvd->vd_lh, LDI_EV_DEGRADE, &ecookie) == + LDI_EV_SUCCESS) { + lcb = kmem_zalloc(sizeof (vdev_disk_ldi_cb_t), KM_SLEEP); + list_insert_tail(&dvd->vd_ldi_cbs, lcb); + (void) ldi_ev_register_callbacks(dvd->vd_lh, ecookie, + &vdev_disk_dgrd_callb, (void *) vd, &lcb->lcb_id); + } skip_open: /* * Determine the actual size of the device. @@ -412,18 +592,31 @@ vdev_disk_close(vdev_t *vd) if (vd->vdev_reopening || dvd == NULL) return; - if (dvd->vd_minor != NULL) + if (dvd->vd_minor != NULL) { ddi_devid_str_free(dvd->vd_minor); + dvd->vd_minor = NULL; + } - if (dvd->vd_devid != NULL) + if (dvd->vd_devid != NULL) { ddi_devid_free(dvd->vd_devid); + dvd->vd_devid = NULL; + } - if (dvd->vd_lh != NULL) + if (dvd->vd_lh != NULL) { (void) ldi_close(dvd->vd_lh, spa_mode(vd->vdev_spa), kcred); + dvd->vd_lh = NULL; + } vd->vdev_delayed_close = B_FALSE; - kmem_free(dvd, sizeof (vdev_disk_t)); - vd->vdev_tsd = NULL; + /* + * If we closed the LDI handle due to an offline notify from LDI, + * don't free vd->vdev_tsd or unregister the callbacks here; + * the offline finalize callback or a reopen will take care of it. + */ + if (dvd->vd_ldi_offline) + return; + + vdev_disk_free(vd); } int @@ -432,6 +625,13 @@ vdev_disk_physio(vdev_t *vd, caddr_t dat { vdev_disk_t *dvd = vd->vdev_tsd; + /* + * If the vdev is closed, it's likely in the REMOVED or FAULTED state. + * Nothing to be done here but return failure. + */ + if (dvd == NULL || (dvd->vd_ldi_offline && dvd->vd_lh == NULL)) + return (EIO); + ASSERT(vd->vdev_ops == &vdev_disk_ops); /* @@ -527,6 +727,15 @@ vdev_disk_io_start(zio_t *zio) buf_t *bp; int error; + /* + * If the vdev is closed, it's likely in the REMOVED or FAULTED state. + * Nothing to be done here but return failure. + */ + if (dvd == NULL || (dvd->vd_ldi_offline && dvd->vd_lh == NULL)) { + zio->io_error = ENXIO; + return (ZIO_PIPELINE_CONTINUE); + } + if (zio->io_type == ZIO_TYPE_IOCTL) { /* XXPOLICY */ if (!vdev_readable(vd)) { From owner-svn-src-vendor@FreeBSD.ORG Wed Nov 20 11:09:13 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32A051EA; Wed, 20 Nov 2013 11:09:13 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 224B521A7; Wed, 20 Nov 2013 11:09:13 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rAKB9DUp060934; Wed, 20 Nov 2013 11:09:13 GMT (envelope-from avg@svn.freebsd.org) Received: (from avg@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rAKB9CEu060933; Wed, 20 Nov 2013 11:09:12 GMT (envelope-from avg@svn.freebsd.org) Message-Id: <201311201109.rAKB9CEu060933@svn.freebsd.org> From: Andriy Gapon Date: Wed, 20 Nov 2013 11:09:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258386 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 11:09:13 -0000 Author: avg Date: Wed Nov 20 11:09:12 2013 New Revision: 258386 URL: http://svnweb.freebsd.org/changeset/base/258386 Log: 4322 ZFS deadlock on dp_config_rwlock illumos/illumos-gate@c50d56f667f119d78fa3d94d6bef2c298ba556f6 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_userhold.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_userhold.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_userhold.c Wed Nov 20 11:08:07 2013 (r258385) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_userhold.c Wed Nov 20 11:09:12 2013 (r258386) @@ -564,21 +564,23 @@ dsl_dataset_user_release_impl(nvlist_t * ddura.ddura_holdfunc = dsl_dataset_hold_obj_string; pool = spa_name(tmpdp->dp_spa); #ifdef _KERNEL - dsl_pool_config_enter(tmpdp, FTAG); for (pair = nvlist_next_nvpair(holds, NULL); pair != NULL; pair = nvlist_next_nvpair(holds, pair)) { dsl_dataset_t *ds; + dsl_pool_config_enter(tmpdp, FTAG); error = dsl_dataset_hold_obj_string(tmpdp, nvpair_name(pair), FTAG, &ds); if (error == 0) { char name[MAXNAMELEN]; dsl_dataset_name(ds, name); + dsl_pool_config_exit(tmpdp, FTAG); dsl_dataset_rele(ds, FTAG); (void) zfs_unmount_snap(name); + } else { + dsl_pool_config_exit(tmpdp, FTAG); } } - dsl_pool_config_exit(tmpdp, FTAG); #endif } else { /* Non-temporary holds are specified by name. */ From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 21 19:16:53 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51C967FC; Thu, 21 Nov 2013 19:16:53 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 406F52E3B; Thu, 21 Nov 2013 19:16:53 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rALJGrEw020629; Thu, 21 Nov 2013 19:16:53 GMT (envelope-from brooks@svn.freebsd.org) Received: (from brooks@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rALJGqnp020626; Thu, 21 Nov 2013 19:16:52 GMT (envelope-from brooks@svn.freebsd.org) Message-Id: <201311211916.rALJGqnp020626@svn.freebsd.org> From: Brooks Davis Date: Thu, 21 Nov 2013 19:16:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258434 - vendor/NetBSD/mtree/dist X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2013 19:16:53 -0000 Author: brooks Date: Thu Nov 21 19:16:52 2013 New Revision: 258434 URL: http://svnweb.freebsd.org/changeset/base/258434 Log: Vendor import of NetBSD's mtree at 2013-11-21 Modified: vendor/NetBSD/mtree/dist/compare.c vendor/NetBSD/mtree/dist/create.c vendor/NetBSD/mtree/dist/spec.c Modified: vendor/NetBSD/mtree/dist/compare.c ============================================================================== --- vendor/NetBSD/mtree/dist/compare.c Thu Nov 21 19:04:59 2013 (r258433) +++ vendor/NetBSD/mtree/dist/compare.c Thu Nov 21 19:16:52 2013 (r258434) @@ -1,4 +1,4 @@ -/* $NetBSD: compare.c,v 1.56 2013/09/09 23:27:43 christos Exp $ */ +/* $NetBSD: compare.c,v 1.58 2013/11/21 18:39:50 christos Exp $ */ /*- * Copyright (c) 1989, 1993 @@ -38,7 +38,7 @@ #if 0 static char sccsid[] = "@(#)compare.c 8.1 (Berkeley) 6/6/93"; #else -__RCSID("$NetBSD: compare.c,v 1.56 2013/09/09 23:27:43 christos Exp $"); +__RCSID("$NetBSD: compare.c,v 1.58 2013/11/21 18:39:50 christos Exp $"); #endif #endif /* not lint */ @@ -48,6 +48,7 @@ __RCSID("$NetBSD: compare.c,v 1.56 2013/ #include #include #include +#include #include #include #include @@ -71,13 +72,18 @@ __RCSID("$NetBSD: compare.c,v 1.56 2013/ #define INDENTNAMELEN 8 #define MARK \ do { \ - len = printf("%s: ", RP(p)); \ - if (len > INDENTNAMELEN) { \ + if (flavor == F_FREEBSD9) { \ + len = printf("%s changed\n", RP(p)); \ tab = "\t"; \ - printf("\n"); \ } else { \ - tab = ""; \ - printf("%*s", INDENTNAMELEN - (int)len, ""); \ + len = printf("%s: ", RP(p)); \ + if (len > INDENTNAMELEN) { \ + tab = "\t"; \ + printf("\n"); \ + } else { \ + tab = ""; \ + printf("%*s", INDENTNAMELEN - (int)len, ""); \ + } \ } \ } while (0) #define LABEL if (!label++) MARK @@ -170,7 +176,8 @@ compare(NODE *s, FTSENT *p) break; #endif typeerr: LABEL; - printf("\ttype (%s, %s)\n", + printf(flavor == F_FREEBSD9 ? + "\ttype expected %s found %s\n" : "\ttype (%s, %s)\n", nodetype(s->type), inotype(p->fts_statp->st_mode)); return (label); } @@ -192,7 +199,9 @@ typeerr: LABEL; (s->type == F_BLOCK || s->type == F_CHAR) && s->st_rdev != p->fts_statp->st_rdev) { LABEL; - printf("%sdevice (%#jx, %#jx", + printf(flavor == F_FREEBSD9 ? + "%sdevice expected %#jx found %#jx" : + "%sdevice (%#jx, %#jx", tab, (uintmax_t)s->st_rdev, (uintmax_t)p->fts_statp->st_rdev); if (uflag) { @@ -202,10 +211,12 @@ typeerr: LABEL; s->st_rdev) == -1) || (lchown(p->fts_accpath, p->fts_statp->st_uid, p->fts_statp->st_gid) == -1) ) - printf(", not modified: %s)\n", - strerror(errno)); + printf(", not modified: %s%s\n", + strerror(errno), + flavor == F_FREEBSD9 ? "" : ")"); else - printf(", modified)\n"); + printf(", modified%s\n", + flavor == F_FREEBSD9 ? "" : ")"); } else printf(")\n"); tab = "\t"; @@ -213,28 +224,34 @@ typeerr: LABEL; /* Set the uid/gid first, then set the mode. */ if (s->flags & (F_UID | F_UNAME) && s->st_uid != p->fts_statp->st_uid) { LABEL; - printf("%suser (%lu, %lu", + printf(flavor == F_FREEBSD9 ? + "%suser expected %lu found %lu" : "%suser (%lu, %lu", tab, (u_long)s->st_uid, (u_long)p->fts_statp->st_uid); if (uflag) { if (lchown(p->fts_accpath, s->st_uid, -1)) - printf(", not modified: %s)\n", - strerror(errno)); + printf(", not modified: %s%s\n", + strerror(errno), + flavor == F_FREEBSD9 ? "" : ")"); else - printf(", modified)\n"); + printf(", modified%s\n", + flavor == F_FREEBSD9 ? "" : ")"); } else printf(")\n"); tab = "\t"; } if (s->flags & (F_GID | F_GNAME) && s->st_gid != p->fts_statp->st_gid) { LABEL; - printf("%sgid (%lu, %lu", + printf(flavor == F_FREEBSD9 ? + "%sgid expected %lu found %lu" : "%sgid (%lu, %lu", tab, (u_long)s->st_gid, (u_long)p->fts_statp->st_gid); if (uflag) { if (lchown(p->fts_accpath, -1, s->st_gid)) - printf(", not modified: %s)\n", - strerror(errno)); + printf(", not modified: %s%s\n", + strerror(errno), + flavor == F_FREEBSD9 ? "" : ")"); else - printf(", modified)\n"); + printf(", modified%s\n", + flavor == F_FREEBSD9 ? "" : ")"); } else printf(")\n"); @@ -259,15 +276,19 @@ typeerr: LABEL; } LABEL; - printf("%spermissions (%#lo, %#lo", + printf(flavor == F_FREEBSD9 ? + "%spermissions expcted %#lo found %#lo" : + "%spermissions (%#lo, %#lo", tab, (u_long)s->st_mode, (u_long)p->fts_statp->st_mode & MBITS); if (uflag) { if (lchmod(p->fts_accpath, s->st_mode)) - printf(", not modified: %s)\n", - strerror(errno)); + printf(", not modified: %s%s\n", + strerror(errno), + flavor == F_FREEBSD9 ? "" : ")"); else - printf(", modified)\n"); + printf(", modified%s\n", + flavor == F_FREEBSD9 ? "" : ")"); } else printf(")\n"); @@ -277,13 +298,16 @@ typeerr: LABEL; if (s->flags & F_NLINK && s->type != F_DIR && s->st_nlink != p->fts_statp->st_nlink) { LABEL; - printf("%slink count (%lu, %lu)\n", + printf(flavor == F_FREEBSD9 ? + "%slink count expected %lu found %lu\n" : + "%slink count (%lu, %lu)\n", tab, (u_long)s->st_nlink, (u_long)p->fts_statp->st_nlink); tab = "\t"; } if (s->flags & F_SIZE && s->st_size != p->fts_statp->st_size) { LABEL; - printf("%ssize (%ju, %ju)\n", + printf(flavor == F_FREEBSD9 ? + "%ssize expected %ju found %ju\n" : "%ssize (%ju, %ju)\n", tab, (uintmax_t)s->st_size, (uintmax_t)p->fts_statp->st_size); tab = "\t"; @@ -318,18 +342,22 @@ typeerr: LABEL; if (tv[0].tv_sec != tv[1].tv_sec || tv[0].tv_usec != tv[1].tv_usec) { LABEL; - printf("%smodification time (%.24s, ", + printf(flavor == F_FREEBSD9 ? + "%smodification time expected %.24s found " : + "%smodification time (%.24s, ", tab, ctime(&smtime)); printf("%.24s", ctime(&pmtime)); if (tflag) { tv[1] = tv[0]; if (utimes(p->fts_accpath, tv)) - printf(", not modified: %s)\n", - strerror(errno)); + printf(", not modified: %s%s\n", + strerror(errno), + flavor == F_FREEBSD9 ? "" : ")"); else - printf(", modified)\n"); + printf(", modified%s\n", + flavor == F_FREEBSD9 ? "" : ")"); } else - printf(")\n"); + printf("%s\n", flavor == F_FREEBSD9 ? "" : ")"); tab = "\t"; } } @@ -346,7 +374,9 @@ typeerr: LABEL; char *f_s; LABEL; f_s = flags_to_string(s->st_flags, "none"); - printf("%sflags (\"%s\" is not ", tab, f_s); + printf(flavor == F_FREEBSD9 ? + "%sflags expected \"%s\" found " : + "%sflags (\"%s\" is not ", tab, f_s); free(f_s); f_s = flags_to_string(p->fts_statp->st_flags, "none"); printf("\"%s\"", f_s); @@ -360,7 +390,7 @@ typeerr: LABEL; else SETFLAGS(0, (~SP_FLGS & CH_MASK)); } else - printf(")\n"); + printf("%s\n", flavor == F_FREEBSD9 ? "" : ")"); tab = "\t"; } #endif /* HAVE_STRUCT_STAT_ST_FLAGS */ @@ -386,7 +416,9 @@ typeerr: LABEL; close(fd); if (s->cksum != val) { LABEL; - printf("%scksum (%lu, %lu)\n", + printf(flavor == F_FREEBSD9 ? + "%scksum expected %lu found %lu\n" : + "%scksum (%lu, %lu)\n", tab, s->cksum, (unsigned long)val); } tab = "\t"; @@ -402,7 +434,9 @@ typeerr: LABEL; } else { if (strcmp(s->md5digest, digestbuf)) { LABEL; - printf("%s%s (0x%s, 0x%s)\n", + printf(flavor == F_FREEBSD9 ? + "%s%s expected %s found %s\n" : + "%s%s (0x%s, 0x%s)\n", tab, MD5KEY, s->md5digest, digestbuf); } tab = "\t"; @@ -420,7 +454,9 @@ typeerr: LABEL; } else { if (strcmp(s->rmd160digest, digestbuf)) { LABEL; - printf("%s%s (0x%s, 0x%s)\n", + printf(flavor == F_FREEBSD9 ? + "%s%s expected %s found %s\n" : + "%s%s (0x%s, 0x%s)\n", tab, RMD160KEY, s->rmd160digest, digestbuf); } tab = "\t"; @@ -438,7 +474,9 @@ typeerr: LABEL; } else { if (strcmp(s->sha1digest, digestbuf)) { LABEL; - printf("%s%s (0x%s, 0x%s)\n", + printf(flavor == F_FREEBSD9 ? + "%s%s expected %s found %s\n" : + "%s%s (0x%s, 0x%s)\n", tab, SHA1KEY, s->sha1digest, digestbuf); } tab = "\t"; @@ -456,7 +494,9 @@ typeerr: LABEL; } else { if (strcmp(s->sha256digest, digestbuf)) { LABEL; - printf("%s%s (0x%s, 0x%s)\n", + printf(flavor == F_FREEBSD9 ? + "%s%s expected %s found %s\n" : + "%s%s (0x%s, 0x%s)\n", tab, SHA256KEY, s->sha256digest, digestbuf); } tab = "\t"; @@ -473,7 +513,9 @@ typeerr: LABEL; } else { if (strcmp(s->sha384digest, digestbuf)) { LABEL; - printf("%s%s (0x%s, 0x%s)\n", + printf(flavor == F_FREEBSD9 ? + "%s%s expected %s found %s\n" : + "%s%s (0x%s, 0x%s)\n", tab, SHA384KEY, s->sha384digest, digestbuf); } tab = "\t"; @@ -490,7 +532,9 @@ typeerr: LABEL; } else { if (strcmp(s->sha512digest, digestbuf)) { LABEL; - printf("%s%s (0x%s, 0x%s)\n", + printf(flavor == F_FREEBSD9 ? + "%s%s expected %s found %s\n" : + "%s%s (0x%s, 0x%s)\n", tab, SHA512KEY, s->sha512digest, digestbuf); } tab = "\t"; @@ -501,16 +545,20 @@ typeerr: LABEL; if (s->flags & F_SLINK && strcmp(cp = rlink(p->fts_accpath), s->slink)) { LABEL; - printf("%slink ref (%s, %s", tab, cp, s->slink); + printf(flavor == F_FREEBSD9 ? + "%slink ref expected %s found %s" : + "%slink ref (%s, %s", tab, cp, s->slink); if (uflag) { if ((unlink(p->fts_accpath) == -1) || (symlink(s->slink, p->fts_accpath) == -1) ) - printf(", not modified: %s)\n", - strerror(errno)); + printf(", not modified: %s%s\n", + strerror(errno), + flavor == F_FREEBSD9 ? "" : ")"); else - printf(", modified)\n"); + printf(", modified%s\n", + flavor == F_FREEBSD9 ? "" : ")"); } else - printf(")\n"); + printf("%s\n", flavor == F_FREEBSD9 ? "" : ")"); } return (label); } Modified: vendor/NetBSD/mtree/dist/create.c ============================================================================== --- vendor/NetBSD/mtree/dist/create.c Thu Nov 21 19:04:59 2013 (r258433) +++ vendor/NetBSD/mtree/dist/create.c Thu Nov 21 19:16:52 2013 (r258434) @@ -1,4 +1,4 @@ -/* $NetBSD: create.c,v 1.71 2013/10/16 17:24:20 christos Exp $ */ +/* $NetBSD: create.c,v 1.72 2013/10/17 17:22:59 christos Exp $ */ /*- * Copyright (c) 1989, 1993 @@ -38,7 +38,7 @@ #if 0 static char sccsid[] = "@(#)create.c 8.1 (Berkeley) 6/6/93"; #else -__RCSID("$NetBSD: create.c,v 1.71 2013/10/16 17:24:20 christos Exp $"); +__RCSID("$NetBSD: create.c,v 1.72 2013/10/17 17:22:59 christos Exp $"); #endif #endif /* not lint */ @@ -55,6 +55,7 @@ __RCSID("$NetBSD: create.c,v 1.71 2013/1 #include #include #include +#include #include #include #include Modified: vendor/NetBSD/mtree/dist/spec.c ============================================================================== --- vendor/NetBSD/mtree/dist/spec.c Thu Nov 21 19:04:59 2013 (r258433) +++ vendor/NetBSD/mtree/dist/spec.c Thu Nov 21 19:16:52 2013 (r258434) @@ -1,4 +1,4 @@ -/* $NetBSD: spec.c,v 1.87 2013/10/16 17:26:14 christos Exp $ */ +/* $NetBSD: spec.c,v 1.88 2013/10/17 17:22:59 christos Exp $ */ /*- * Copyright (c) 1989, 1993 @@ -67,7 +67,7 @@ #if 0 static char sccsid[] = "@(#)spec.c 8.2 (Berkeley) 4/28/95"; #else -__RCSID("$NetBSD: spec.c,v 1.87 2013/10/16 17:26:14 christos Exp $"); +__RCSID("$NetBSD: spec.c,v 1.88 2013/10/17 17:22:59 christos Exp $"); #endif #endif /* not lint */ @@ -81,6 +81,7 @@ __RCSID("$NetBSD: spec.c,v 1.87 2013/10/ #include #include #include +#include #include #include #include From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 21 19:17:40 2013 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5BE78941; Thu, 21 Nov 2013 19:17:40 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 31EBE2E47; Thu, 21 Nov 2013 19:17:40 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rALJHe8E020760; Thu, 21 Nov 2013 19:17:40 GMT (envelope-from brooks@svn.freebsd.org) Received: (from brooks@localhost) by svn.freebsd.org (8.14.7/8.14.5/Submit) id rALJHelb020759; Thu, 21 Nov 2013 19:17:40 GMT (envelope-from brooks@svn.freebsd.org) Message-Id: <201311211917.rALJHelb020759@svn.freebsd.org> From: Brooks Davis Date: Thu, 21 Nov 2013 19:17:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r258435 - vendor/NetBSD/mtree/20131121 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2013 19:17:40 -0000 Author: brooks Date: Thu Nov 21 19:17:39 2013 New Revision: 258435 URL: http://svnweb.freebsd.org/changeset/base/258435 Log: Tag 2013-11-21 import of NetBSD's mtree Added: vendor/NetBSD/mtree/20131121/ - copied from r258434, vendor/NetBSD/mtree/dist/