From owner-svn-src-vendor@FreeBSD.ORG Sun Nov 4 20:47:40 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 64BDEE5; Sun, 4 Nov 2012 20:47:40 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 487648FC0A; Sun, 4 Nov 2012 20:47:40 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA4Kleuj049447; Sun, 4 Nov 2012 20:47:40 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA4KleKc049435; Sun, 4 Nov 2012 20:47:40 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201211042047.qA4KleKc049435@svn.freebsd.org> From: Xin LI Date: Sun, 4 Nov 2012 20:47:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242581 - vendor/less/dist X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 20:47:40 -0000 Author: delphij Date: Sun Nov 4 20:47:39 2012 New Revision: 242581 URL: http://svn.freebsd.org/changeset/base/242581 Log: Vendor import of less v453 (beta). Modified: vendor/less/dist/NEWS vendor/less/dist/README vendor/less/dist/configure vendor/less/dist/configure.ac vendor/less/dist/defines.ds vendor/less/dist/defines.h.in vendor/less/dist/defines.o2 vendor/less/dist/defines.o9 vendor/less/dist/defines.wn vendor/less/dist/less.man vendor/less/dist/less.nro vendor/less/dist/lessecho.man vendor/less/dist/lessecho.nro vendor/less/dist/lesskey.man vendor/less/dist/lesskey.nro vendor/less/dist/option.c vendor/less/dist/output.c vendor/less/dist/position.c vendor/less/dist/screen.c vendor/less/dist/version.c Modified: vendor/less/dist/NEWS ============================================================================== --- vendor/less/dist/NEWS Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/NEWS Sun Nov 4 20:47:39 2012 (r242581) @@ -11,6 +11,22 @@ ====================================================================== + Major changes between "less" versions 451 and 453 + +* Allow backslash escaping of metacharacters in LESS environment variable. + +* Don't quit if syntax errors are found in command line options. + +* Increase sizes of some internal buffers. + +* Fix configure bug with --with-regex=none. + +* Fix crash with "stty rows 0". + +* Fix Win32 attribute display bug. + +====================================================================== + Major changes between "less" versions 444 and 451 * Add ESC-F command to keep reading data until a pattern is found. Modified: vendor/less/dist/README ============================================================================== --- vendor/less/dist/README Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/README Sun Nov 4 20:47:39 2012 (r242581) @@ -1,7 +1,7 @@ - Less, version 451 + Less, version 453 - This is the distribution of less, version 451, released 21 Jul 2012. + This is the distribution of less, version 453, released 27 Oct 2012. This program is part of the GNU project (http://www.gnu.org). This program is free software. You may redistribute it and/or Modified: vendor/less/dist/configure ============================================================================== --- vendor/less/dist/configure Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/configure Sun Nov 4 20:47:39 2012 (r242581) @@ -5188,8 +5188,7 @@ fi # Checks for regular expression functions. have_regex=no have_posix_regex=unknown -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for regcomp" >&5 -$as_echo_n "checking for regcomp... " >&6; } +supported_regex="" # Select a regular expression library. WANT_REGEX=auto @@ -5204,6 +5203,8 @@ if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = posix; then # Some versions of Solaris have a regcomp() function, but it doesn't work! # So we run a test program. If we're cross-compiling, do it the old way. +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for POSIX regcomp" >&5 +$as_echo_n "checking for POSIX regcomp... " >&6; } if test "$cross_compiling" = yes; then : have_posix_regex=unknown else @@ -5232,10 +5233,10 @@ rm -f core *.core core.conftest.* gmon.o fi if test $have_posix_regex = yes; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: using POSIX regcomp" >&5 -$as_echo "using POSIX regcomp" >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } $as_echo "#define HAVE_POSIX_REGCOMP 1" >>confdefs.h - + supported_regex="$supported_regex posix" have_regex=yes elif test $have_posix_regex = unknown; then cat confdefs.h - <<_ACEOF >conftest.$ac_ext @@ -5252,10 +5253,10 @@ regex_t *r; regfree(r); } _ACEOF if ac_fn_c_try_link "$LINENO"; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: result: using POSIX regcomp" >&5 -$as_echo "using POSIX regcomp" >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } $as_echo "#define HAVE_POSIX_REGCOMP 1" >>confdefs.h - have_regex=yes + have_regex=yes; supported_regex="$supported_regex posix" fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext @@ -5267,14 +5268,14 @@ fi fi if test $have_regex = no; then -if test $WANT_REGEX = auto -o $WANT_REGEX = gnu; then -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for re_compile_pattern in -lc" >&5 -$as_echo_n "checking for re_compile_pattern in -lc... " >&6; } -if ${ac_cv_lib_c_re_compile_pattern+:} false; then : +if test $WANT_REGEX = auto -o $WANT_REGEX = pcre; then +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pcre_compile in -lpcre" >&5 +$as_echo_n "checking for pcre_compile in -lpcre... " >&6; } +if ${ac_cv_lib_pcre_pcre_compile+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS -LIBS="-lc $LIBS" +LIBS="-lpcre $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ @@ -5284,44 +5285,43 @@ cat confdefs.h - <<_ACEOF >conftest.$ac_ #ifdef __cplusplus extern "C" #endif -char re_compile_pattern (); +char pcre_compile (); int main () { -return re_compile_pattern (); +return pcre_compile (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_c_re_compile_pattern=yes + ac_cv_lib_pcre_pcre_compile=yes else - ac_cv_lib_c_re_compile_pattern=no + ac_cv_lib_pcre_pcre_compile=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_c_re_compile_pattern" >&5 -$as_echo "$ac_cv_lib_c_re_compile_pattern" >&6; } -if test "x$ac_cv_lib_c_re_compile_pattern" = xyes; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: result: using gnu" >&5 -$as_echo "using gnu" >&6; }; $as_echo "#define HAVE_GNU_REGEX 1" >>confdefs.h - have_regex=yes +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_pcre_pcre_compile" >&5 +$as_echo "$ac_cv_lib_pcre_pcre_compile" >&6; } +if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then : + $as_echo "#define HAVE_PCRE 1" >>confdefs.h + LIBS="$LIBS -lpcre" have_regex=yes; supported_regex="$supported_regex pcre" fi fi fi if test $have_regex = no; then -if test $WANT_REGEX = auto -o $WANT_REGEX = pcre; then -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pcre_compile in -lpcre" >&5 -$as_echo_n "checking for pcre_compile in -lpcre... " >&6; } -if ${ac_cv_lib_pcre_pcre_compile+:} false; then : +if test $WANT_REGEX = auto -o $WANT_REGEX = gnu; then +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for re_compile_pattern in -lc" >&5 +$as_echo_n "checking for re_compile_pattern in -lc... " >&6; } +if ${ac_cv_lib_c_re_compile_pattern+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS -LIBS="-lpcre $LIBS" +LIBS="-lc $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ @@ -5331,30 +5331,29 @@ cat confdefs.h - <<_ACEOF >conftest.$ac_ #ifdef __cplusplus extern "C" #endif -char pcre_compile (); +char re_compile_pattern (); int main () { -return pcre_compile (); +return re_compile_pattern (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_pcre_pcre_compile=yes + ac_cv_lib_c_re_compile_pattern=yes else - ac_cv_lib_pcre_pcre_compile=no + ac_cv_lib_c_re_compile_pattern=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_pcre_pcre_compile" >&5 -$as_echo "$ac_cv_lib_pcre_pcre_compile" >&6; } -if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: result: using pcre" >&5 -$as_echo "using pcre" >&6; }; $as_echo "#define HAVE_PCRE 1" >>confdefs.h - LIBS="$LIBS -lpcre" have_regex=yes +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_c_re_compile_pattern" >&5 +$as_echo "$ac_cv_lib_c_re_compile_pattern" >&6; } +if test "x$ac_cv_lib_c_re_compile_pattern" = xyes; then : + $as_echo "#define HAVE_GNU_REGEX 1" >>confdefs.h + have_regex=yes; supported_regex="$supported_regex gnu" fi fi @@ -5364,9 +5363,8 @@ if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = regcmp; then ac_fn_c_check_func "$LINENO" "regcmp" "ac_cv_func_regcmp" if test "x$ac_cv_func_regcmp" = xyes; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: result: using regcmp" >&5 -$as_echo "using regcmp" >&6; }; $as_echo "#define HAVE_REGCMP 1" >>confdefs.h - have_regex=yes + $as_echo "#define HAVE_REGCMP 1" >>confdefs.h + have_regex=yes; supported_regex="$supported_regex regcmp" fi fi @@ -5374,6 +5372,8 @@ fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = regcomp; then +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for V8 regcomp" >&5 +$as_echo_n "checking for V8 regcomp... " >&6; } cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ @@ -5387,9 +5387,12 @@ regcomp(""); } _ACEOF if ac_fn_c_try_link "$LINENO"; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: result: using V8 regcomp" >&5 -$as_echo "using V8 regcomp" >&6; }; $as_echo "#define HAVE_V8_REGCOMP 1" >>confdefs.h - have_regex=yes + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; }; $as_echo "#define HAVE_V8_REGCOMP 1" >>confdefs.h + have_regex=yes; supported_regex="$supported_regex regcomp" +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext @@ -5400,18 +5403,21 @@ if test $have_regex = no && test -f ${sr if test $WANT_REGEX = auto -o $WANT_REGEX = regcomp-local; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: using V8 regcomp -- local source" >&5 $as_echo "using V8 regcomp -- local source" >&6; }; $as_echo "#define HAVE_V8_REGCOMP 1" >>confdefs.h - $as_echo "#define HAVE_REGEXEC2 1" >>confdefs.h + +supported_regex="$supported_regex regcomp-local" +$as_echo "#define HAVE_REGEXEC2 1" >>confdefs.h REGEX_O='regexp.$(O)' have_regex=yes fi fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = re_comp; then -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: using re_comp" >&5 -$as_echo "using re_comp" >&6; }; ac_fn_c_check_func "$LINENO" "re_comp" "ac_cv_func_re_comp" +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for re_comp" >&5 +$as_echo_n "checking for re_comp... " >&6; } +ac_fn_c_check_func "$LINENO" "re_comp" "ac_cv_func_re_comp" if test "x$ac_cv_func_re_comp" = xyes; then : $as_echo "#define HAVE_RE_COMP 1" >>confdefs.h - have_regex=yes + have_regex=yes; supported_regex="$supported_regex re_comp" fi fi @@ -5420,15 +5426,17 @@ fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = none; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: using no regex" >&5 -$as_echo "using no regex" >&6; }; have_regex=yes; +$as_echo "using no regex" >&6; } +else +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cannot find regular expression library" >&5 +$as_echo "$as_me: WARNING: cannot find regular expression library" >&2;} fi +$as_echo "#define NO_REGEX 1" >>confdefs.h + supported_regex="$supported_regex none" fi -if test $have_regex = no; then -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: cannot find regular expression library" >&5 -$as_echo "cannot find regular expression library" >&6; }; $as_echo "#define NO_REGEX 1" >>confdefs.h - -fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: regular expression library: $supported_regex" >&5 +$as_echo "regular expression library: $supported_regex" >&6; } # Check whether --with-editor was given. Modified: vendor/less/dist/configure.ac ============================================================================== --- vendor/less/dist/configure.ac Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/configure.ac Sun Nov 4 20:47:39 2012 (r242581) @@ -383,7 +383,7 @@ fi # Checks for regular expression functions. have_regex=no have_posix_regex=unknown -AC_MSG_CHECKING(for regcomp) +supported_regex="" # Select a regular expression library. WANT_REGEX=auto @@ -395,6 +395,7 @@ if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = posix; then # Some versions of Solaris have a regcomp() function, but it doesn't work! # So we run a test program. If we're cross-compiling, do it the old way. +AC_MSG_CHECKING(for POSIX regcomp) AC_TRY_RUN([ #include #include @@ -409,16 +410,16 @@ if (rm.rm_sp != text + 1) exit(1); /* ch exit(0); }], have_posix_regex=yes, have_posix_regex=no, have_posix_regex=unknown) if test $have_posix_regex = yes; then - AC_MSG_RESULT(using POSIX regcomp) - AC_DEFINE(HAVE_POSIX_REGCOMP) + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_POSIX_REGCOMP) supported_regex="$supported_regex posix" have_regex=yes elif test $have_posix_regex = unknown; then AC_TRY_LINK([ #include #include ], [regex_t *r; regfree(r);], - AC_MSG_RESULT(using POSIX regcomp) - AC_DEFINE(HAVE_POSIX_REGCOMP) have_regex=yes) + AC_MSG_RESULT(yes) + AC_DEFINE(HAVE_POSIX_REGCOMP) have_regex=yes; supported_regex="$supported_regex posix") else AC_MSG_RESULT(no) fi @@ -426,55 +427,61 @@ fi fi if test $have_regex = no; then -if test $WANT_REGEX = auto -o $WANT_REGEX = gnu; then -AC_CHECK_LIB(c, re_compile_pattern, -[AC_MSG_RESULT(using gnu); AC_DEFINE(HAVE_GNU_REGEX) have_regex=yes], []) +if test $WANT_REGEX = auto -o $WANT_REGEX = pcre; then +AC_CHECK_LIB(pcre, pcre_compile, +[AC_DEFINE(HAVE_PCRE) LIBS="$LIBS -lpcre" have_regex=yes; supported_regex="$supported_regex pcre"], []) fi fi if test $have_regex = no; then -if test $WANT_REGEX = auto -o $WANT_REGEX = pcre; then -AC_CHECK_LIB(pcre, pcre_compile, -[AC_MSG_RESULT(using pcre); AC_DEFINE(HAVE_PCRE) LIBS="$LIBS -lpcre" have_regex=yes], []) +if test $WANT_REGEX = auto -o $WANT_REGEX = gnu; then +AC_CHECK_LIB(c, re_compile_pattern, +[AC_DEFINE(HAVE_GNU_REGEX) have_regex=yes; supported_regex="$supported_regex gnu"], []) fi fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = regcmp; then AC_CHECK_FUNC(regcmp, -AC_MSG_RESULT(using regcmp); AC_DEFINE(HAVE_REGCMP) have_regex=yes) +[AC_DEFINE(HAVE_REGCMP) have_regex=yes; supported_regex="$supported_regex regcmp"],[]) fi fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = regcomp; then +AC_MSG_CHECKING(for V8 regcomp) AC_TRY_LINK([ #include "regexp.h"], [regcomp("");], -AC_MSG_RESULT(using V8 regcomp); AC_DEFINE(HAVE_V8_REGCOMP) have_regex=yes) +[AC_MSG_RESULT(yes); AC_DEFINE(HAVE_V8_REGCOMP) have_regex=yes; supported_regex="$supported_regex regcomp"],[AC_MSG_RESULT(no)]) fi fi if test $have_regex = no && test -f ${srcdir}/regexp.c; then if test $WANT_REGEX = auto -o $WANT_REGEX = regcomp-local; then -AC_MSG_RESULT(using V8 regcomp -- local source); AC_DEFINE(HAVE_V8_REGCOMP) AC_DEFINE(HAVE_REGEXEC2) REGEX_O='regexp.$(O)' AC_SUBST(REGEX_O) have_regex=yes +AC_MSG_RESULT(using V8 regcomp -- local source); AC_DEFINE(HAVE_V8_REGCOMP) +supported_regex="$supported_regex regcomp-local" +AC_DEFINE(HAVE_REGEXEC2) REGEX_O='regexp.$(O)' AC_SUBST(REGEX_O) have_regex=yes fi fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = re_comp; then -AC_MSG_RESULT(using re_comp); AC_CHECK_FUNC(re_comp, AC_DEFINE(HAVE_RE_COMP) have_regex=yes) +AC_MSG_CHECKING(for re_comp) +AC_CHECK_FUNC(re_comp, +[AC_DEFINE(HAVE_RE_COMP) have_regex=yes; supported_regex="$supported_regex re_comp"],[]) fi fi if test $have_regex = no; then if test $WANT_REGEX = auto -o $WANT_REGEX = none; then -AC_MSG_RESULT(using no regex); have_regex=yes; +AC_MSG_RESULT(using no regex) +else +AC_MSG_WARN(cannot find regular expression library) fi +AC_DEFINE(NO_REGEX) supported_regex="$supported_regex none" fi -if test $have_regex = no; then -AC_MSG_RESULT(cannot find regular expression library); AC_DEFINE(NO_REGEX) -fi +AC_MSG_RESULT(regular expression library: $supported_regex) AC_ARG_WITH(editor, [ --with-editor=PROGRAM use PROGRAM as the default editor [vi]], @@ -662,6 +669,7 @@ AH_TOP([ /* * Sizes of various buffers. */ +#if 0 /* old sizes for small memory machines #define CMDBUF_SIZE 512 /* Buffer for multichar commands */ #define UNGOT_SIZE 100 /* Max chars to unget() */ #define LINEBUF_SIZE 1024 /* Max size of line in input file */ @@ -671,6 +679,17 @@ AH_TOP([ #define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ #define TAGLINE_SIZE 512 /* Max size of line in tags file */ #define TABSTOP_MAX 32 /* Max number of custom tab stops */ +#else /* more reasonable sizes for modern machines */ +#define CMDBUF_SIZE 2048 /* Buffer for multichar commands */ +#define UNGOT_SIZE 200 /* Max chars to unget() */ +#define LINEBUF_SIZE 1024 /* Initial max size of line in input file */ +#define OUTBUF_SIZE 1024 /* Output buffer */ +#define PROMPT_SIZE 2048 /* Max size of prompt string */ +#define TERMBUF_SIZE 2048 /* Termcap buffer for tgetent */ +#define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ +#define TAGLINE_SIZE 1024 /* Max size of line in tags file */ +#define TABSTOP_MAX 128 /* Max number of custom tab stops */ +#endif /* Settings automatically determined by configure. */ ]) Modified: vendor/less/dist/defines.ds ============================================================================== --- vendor/less/dist/defines.ds Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/defines.ds Sun Nov 4 20:47:39 2012 (r242581) @@ -185,6 +185,7 @@ /* * Sizes of various buffers. */ +#if 0 /* old sizes for small memory machines #define CMDBUF_SIZE 512 /* Buffer for multichar commands */ #define UNGOT_SIZE 100 /* Max chars to unget() */ #define LINEBUF_SIZE 1024 /* Max size of line in input file */ @@ -194,6 +195,17 @@ #define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ #define TAGLINE_SIZE 512 /* Max size of line in tags file */ #define TABSTOP_MAX 32 /* Max number of custom tab stops */ +#else /* more reasonable sizes for modern machines */ +#define CMDBUF_SIZE 2048 /* Buffer for multichar commands */ +#define UNGOT_SIZE 200 /* Max chars to unget() */ +#define LINEBUF_SIZE 1024 /* Initial max size of line in input file */ +#define OUTBUF_SIZE 1024 /* Output buffer */ +#define PROMPT_SIZE 2048 /* Max size of prompt string */ +#define TERMBUF_SIZE 2048 /* Termcap buffer for tgetent */ +#define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ +#define TAGLINE_SIZE 1024 /* Max size of line in tags file */ +#define TABSTOP_MAX 128 /* Max number of custom tab stops */ +#endif /* Define to `long' if doesn't define. */ #if MSDOS_COMPILER==BORLANDC Modified: vendor/less/dist/defines.h.in ============================================================================== --- vendor/less/dist/defines.h.in Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/defines.h.in Sun Nov 4 20:47:39 2012 (r242581) @@ -182,6 +182,7 @@ /* * Sizes of various buffers. */ +#if 0 /* old sizes for small memory machines #define CMDBUF_SIZE 512 /* Buffer for multichar commands */ #define UNGOT_SIZE 100 /* Max chars to unget() */ #define LINEBUF_SIZE 1024 /* Max size of line in input file */ @@ -191,6 +192,17 @@ #define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ #define TAGLINE_SIZE 512 /* Max size of line in tags file */ #define TABSTOP_MAX 32 /* Max number of custom tab stops */ +#else /* more reasonable sizes for modern machines */ +#define CMDBUF_SIZE 2048 /* Buffer for multichar commands */ +#define UNGOT_SIZE 200 /* Max chars to unget() */ +#define LINEBUF_SIZE 1024 /* Initial max size of line in input file */ +#define OUTBUF_SIZE 1024 /* Output buffer */ +#define PROMPT_SIZE 2048 /* Max size of prompt string */ +#define TERMBUF_SIZE 2048 /* Termcap buffer for tgetent */ +#define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ +#define TAGLINE_SIZE 1024 /* Max size of line in tags file */ +#define TABSTOP_MAX 128 /* Max number of custom tab stops */ +#endif /* Settings automatically determined by configure. */ Modified: vendor/less/dist/defines.o2 ============================================================================== --- vendor/less/dist/defines.o2 Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/defines.o2 Sun Nov 4 20:47:39 2012 (r242581) @@ -166,6 +166,7 @@ /* * Sizes of various buffers. */ +#if 0 /* old sizes for small memory machines #define CMDBUF_SIZE 512 /* Buffer for multichar commands */ #define UNGOT_SIZE 100 /* Max chars to unget() */ #define LINEBUF_SIZE 1024 /* Max size of line in input file */ @@ -175,6 +176,17 @@ #define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ #define TAGLINE_SIZE 512 /* Max size of line in tags file */ #define TABSTOP_MAX 32 /* Max number of custom tab stops */ +#else /* more reasonable sizes for modern machines */ +#define CMDBUF_SIZE 2048 /* Buffer for multichar commands */ +#define UNGOT_SIZE 200 /* Max chars to unget() */ +#define LINEBUF_SIZE 1024 /* Initial max size of line in input file */ +#define OUTBUF_SIZE 1024 /* Output buffer */ +#define PROMPT_SIZE 2048 /* Max size of prompt string */ +#define TERMBUF_SIZE 2048 /* Termcap buffer for tgetent */ +#define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ +#define TAGLINE_SIZE 1024 /* Max size of line in tags file */ +#define TABSTOP_MAX 128 /* Max number of custom tab stops */ +#endif /* Define to `long' if doesn't define. */ /* #define off_t long */ Modified: vendor/less/dist/defines.o9 ============================================================================== --- vendor/less/dist/defines.o9 Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/defines.o9 Sun Nov 4 20:47:39 2012 (r242581) @@ -173,6 +173,7 @@ /* * Sizes of various buffers. */ +#if 0 /* old sizes for small memory machines #define CMDBUF_SIZE 512 /* Buffer for multichar commands */ #define UNGOT_SIZE 100 /* Max chars to unget() */ #define LINEBUF_SIZE 1024 /* Max size of line in input file */ @@ -182,6 +183,17 @@ #define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ #define TAGLINE_SIZE 512 /* Max size of line in tags file */ #define TABSTOP_MAX 32 /* Max number of custom tab stops */ +#else /* more reasonable sizes for modern machines */ +#define CMDBUF_SIZE 2048 /* Buffer for multichar commands */ +#define UNGOT_SIZE 200 /* Max chars to unget() */ +#define LINEBUF_SIZE 1024 /* Initial max size of line in input file */ +#define OUTBUF_SIZE 1024 /* Output buffer */ +#define PROMPT_SIZE 2048 /* Max size of prompt string */ +#define TERMBUF_SIZE 2048 /* Termcap buffer for tgetent */ +#define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ +#define TAGLINE_SIZE 1024 /* Max size of line in tags file */ +#define TABSTOP_MAX 128 /* Max number of custom tab stops */ +#endif /* Define to `long' if doesn't define. */ #define off_t long Modified: vendor/less/dist/defines.wn ============================================================================== --- vendor/less/dist/defines.wn Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/defines.wn Sun Nov 4 20:47:39 2012 (r242581) @@ -167,6 +167,7 @@ /* * Sizes of various buffers. */ +#if 0 /* old sizes for small memory machines #define CMDBUF_SIZE 512 /* Buffer for multichar commands */ #define UNGOT_SIZE 100 /* Max chars to unget() */ #define LINEBUF_SIZE 1024 /* Max size of line in input file */ @@ -176,6 +177,17 @@ #define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ #define TAGLINE_SIZE 512 /* Max size of line in tags file */ #define TABSTOP_MAX 32 /* Max number of custom tab stops */ +#else /* more reasonable sizes for modern machines */ +#define CMDBUF_SIZE 2048 /* Buffer for multichar commands */ +#define UNGOT_SIZE 200 /* Max chars to unget() */ +#define LINEBUF_SIZE 1024 /* Initial max size of line in input file */ +#define OUTBUF_SIZE 1024 /* Output buffer */ +#define PROMPT_SIZE 2048 /* Max size of prompt string */ +#define TERMBUF_SIZE 2048 /* Termcap buffer for tgetent */ +#define TERMSBUF_SIZE 1024 /* Buffer to hold termcap strings */ +#define TAGLINE_SIZE 1024 /* Max size of line in tags file */ +#define TABSTOP_MAX 128 /* Max number of custom tab stops */ +#endif /* Define to `long' if doesn't define. */ /* #define off_t long */ Modified: vendor/less/dist/less.man ============================================================================== --- vendor/less/dist/less.man Sun Nov 4 20:45:31 2012 (r242580) +++ vendor/less/dist/less.man Sun Nov 4 20:47:39 2012 (r242581) @@ -437,63 +437,69 @@ LESS(1) appears in the LESS variable, it can be reset to its default value on the command line by beginning the command line option with "-+". - For options like -P or -D which take a following string, a dollar sign - ($) must be used to signal the end of the string. For example, to set - two -D options on MS-DOS, you must have a dollar sign between them, - like this: + Some options like -k or -D require a string to follow the option let- + ter. The string for that option is considered to end when a space, + tab, dash or dollar sign ($) is found. For example, to set two -D + options on MS-DOS, you can separate them with a dollar sign, like this: - LESS="-Dn9.1$-Ds4.1" + LESS="Dn9.1$Ds4.1" + or a space like this: + + LESS="Dn9.1 Ds4.1" + + Any character may be included literally in an option string by preced- + ing it with a backslash. -? or --help - This option displays a summary of the commands accepted by less - (the same as the h command). (Depending on how your shell - interprets the question mark, it may be necessary to quote the + This option displays a summary of the commands accepted by less + (the same as the h command). (Depending on how your shell + interprets the question mark, it may be necessary to quote the question mark, thus: "-\?".) -a or --search-skip-screen - By default, forward searches start at the top of the displayed - screen and backwards searches start at the bottom of the dis- - played screen (except for repeated searches invoked by the n or - N commands, which start after or before the "target" line + By default, forward searches start at the top of the displayed + screen and backwards searches start at the bottom of the dis- + played screen (except for repeated searches invoked by the n or + N commands, which start after or before the "target" line respectively; see the -j option for more about the target line). - The -a option causes forward searches to instead start at the - bottom of the screen and backward searches to start at the top + The -a option causes forward searches to instead start at the + bottom of the screen and backward searches to start at the top of the screen, thus skipping all lines displayed on the screen. -A or --SEARCH-SKIP-SCREEN - Causes all forward searches (not just non-repeated searches) to - start just after the target line, and all backward searches to - start just before the target line. Thus, forward searches will + Causes all forward searches (not just non-repeated searches) to + start just after the target line, and all backward searches to + start just before the target line. Thus, forward searches will skip part of the displayed screen (from the first line up to and - including the target line). Similarly backwards searches will + including the target line). Similarly backwards searches will skip the displayed screen from the last line up to and including the target line. This was the default behavior in less versions prior to 441. -bn or --buffers=n - Specifies the amount of buffer space less will use for each - file, in units of kilobytes (1024 bytes). By default 64K of - buffer space is used for each file (unless the file is a pipe; - see the -B option). The -b option specifies instead that n + Specifies the amount of buffer space less will use for each + file, in units of kilobytes (1024 bytes). By default 64K of + buffer space is used for each file (unless the file is a pipe; + see the -B option). The -b option specifies instead that n kilobytes of buffer space should be used for each file. If n is - -1, buffer space is unlimited; that is, the entire file can be + -1, buffer space is unlimited; that is, the entire file can be read into memory. -B or --auto-buffers By default, when data is read from a pipe, buffers are allocated automatically as needed. If a large amount of data is read from - the pipe, this can cause a large amount of memory to be allo- + the pipe, this can cause a large amount of memory to be allo- cated. The -B option disables this automatic allocation of buf- - fers for pipes, so that only 64K (or the amount of space speci- + fers for pipes, so that only 64K (or the amount of space speci- fied by the -b option) is used for the pipe. Warning: use of -B - can result in erroneous display, since only the most recently - viewed part of the piped data is kept in memory; any earlier + can result in erroneous display, since only the most recently + viewed part of the piped data is kept in memory; any earlier data is lost. -c or --clear-screen - Causes full screen repaints to be painted from the top line - down. By default, full screen repaints are done by scrolling + Causes full screen repaints to be painted from the top line + down. By default, full screen repaints are done by scrolling from the bottom of the screen. -C or --CLEAR-SCREEN @@ -501,24 +507,24 @@ LESS(1) -d or --dumb The -d option suppresses the error message normally displayed if - the terminal is dumb; that is, lacks some important capability, + the terminal is dumb; that is, lacks some important capability, such as the ability to clear the screen or scroll backward. The - -d option does not otherwise change the behavior of less on a + -d option does not otherwise change the behavior of less on a dumb terminal. -Dxcolor or --color=xcolor [MS-DOS only] Sets the color of the text displayed. x is a sin- - gle character which selects the type of text whose color is - being set: n=normal, s=standout, d=bold, u=underlined, k=blink. - color is a pair of numbers separated by a period. The first - number selects the foreground color and the second selects the - background color of the text. A single number N is the same as + gle character which selects the type of text whose color is + being set: n=normal, s=standout, d=bold, u=underlined, k=blink. + color is a pair of numbers separated by a period. The first + number selects the foreground color and the second selects the + background color of the text. A single number N is the same as N.M, where M is the normal background color. -e or --quit-at-eof - Causes less to automatically exit the second time it reaches - end-of-file. By default, the only way to exit less is via the + Causes less to automatically exit the second time it reaches + end-of-file. By default, the only way to exit less is via the "q" command. -E or --QUIT-AT-EOF @@ -527,7 +533,7 @@ LESS(1) -f or --force Forces non-regular files to be opened. (A non-regular file is a - directory or a device special file.) Also suppresses the warn- + directory or a device special file.) Also suppresses the warn- ing message when a binary file is opened. By default, less will refuse to open non-regular files. Note that some operating sys- tems will not allow directories to be read, even if -f is set. @@ -537,236 +543,236 @@ LESS(1) played on the first screen. -g or --hilite-search - Normally, less will highlight ALL strings which match the last - search command. The -g option changes this behavior to high- - light only the particular string which was found by the last + Normally, less will highlight ALL strings which match the last + search command. The -g option changes this behavior to high- + light only the particular string which was found by the last search command. This can cause less to run somewhat faster than the default. -G or --HILITE-SEARCH - The -G option suppresses all highlighting of strings found by + The -G option suppresses all highlighting of strings found by search commands. -hn or --max-back-scroll=n - Specifies a maximum number of lines to scroll backward. If it + Specifies a maximum number of lines to scroll backward. If it is necessary to scroll backward more than n lines, the screen is repainted in a forward direction instead. (If the terminal does not have the ability to scroll backward, -h0 is implied.) -i or --ignore-case Causes searches to ignore case; that is, uppercase and lowercase - are considered identical. This option is ignored if any upper- - case letters appear in the search pattern; in other words, if a - pattern contains uppercase letters, then that search does not + are considered identical. This option is ignored if any upper- + case letters appear in the search pattern; in other words, if a + pattern contains uppercase letters, then that search does not ignore case. -I or --IGNORE-CASE - Like -i, but searches ignore case even if the pattern contains + Like -i, but searches ignore case even if the pattern contains uppercase letters. -jn or --jump-target=n - Specifies a line on the screen where the "target" line is to be - positioned. The target line is the line specified by any com- - mand to search for a pattern, jump to a line number, jump to a + Specifies a line on the screen where the "target" line is to be + positioned. The target line is the line specified by any com- + mand to search for a pattern, jump to a line number, jump to a file percentage or jump to a tag. The screen line may be speci- - fied by a number: the top line on the screen is 1, the next is + fied by a number: the top line on the screen is 1, the next is 2, and so on. The number may be negative to specify a line rel- ative to the bottom of the screen: the bottom line on the screen - is -1, the second to the bottom is -2, and so on. Alternately, - the screen line may be specified as a fraction of the height of - the screen, starting with a decimal point: .5 is in the middle - of the screen, .3 is three tenths down from the first line, and - so on. If the line is specified as a fraction, the actual line - number is recalculated if the terminal window is resized, so - that the target line remains at the specified fraction of the - screen height. If any form of the -j option is used, forward - searches begin at the line immediately after the target line, - and backward searches begin at the target line, unless changed - by -a or -A. For example, if "-j4" is used, the target line is - the fourth line on the screen, so forward searches begin at the + is -1, the second to the bottom is -2, and so on. Alternately, + the screen line may be specified as a fraction of the height of + the screen, starting with a decimal point: .5 is in the middle + of the screen, .3 is three tenths down from the first line, and + so on. If the line is specified as a fraction, the actual line + number is recalculated if the terminal window is resized, so + that the target line remains at the specified fraction of the + screen height. If any form of the -j option is used, forward + searches begin at the line immediately after the target line, + and backward searches begin at the target line, unless changed + by -a or -A. For example, if "-j4" is used, the target line is + the fourth line on the screen, so forward searches begin at the fifth line on the screen. -J or --status-column - Displays a status column at the left edge of the screen. The - status column shows the lines that matched the current search. - The status column is also used if the -w or -W option is in + Displays a status column at the left edge of the screen. The + status column shows the lines that matched the current search. + The status column is also used if the -w or -W option is in effect. -kfilename or --lesskey-file=filename - Causes less to open and interpret the named file as a lesskey + Causes less to open and interpret the named file as a lesskey (1) file. Multiple -k options may be specified. If the LESSKEY - or LESSKEY_SYSTEM environment variable is set, or if a lesskey + or LESSKEY_SYSTEM environment variable is set, or if a lesskey file is found in a standard place (see KEY BINDINGS), it is also used as a lesskey file. -K or --quit-on-intr - Causes less to exit immediately (with status 2) when an inter- - rupt character (usually ^C) is typed. Normally, an interrupt + Causes less to exit immediately (with status 2) when an inter- + rupt character (usually ^C) is typed. Normally, an interrupt character causes less to stop whatever it is doing and return to - its command prompt. Note that use of this option makes it + its command prompt. Note that use of this option makes it impossible to return to the command prompt from the "F" command. -L or --no-lessopen - Ignore the LESSOPEN environment variable (see the INPUT PRE- - PROCESSOR section below). This option can be set from within - less, but it will apply only to files opened subsequently, not + Ignore the LESSOPEN environment variable (see the INPUT PRE- + PROCESSOR section below). This option can be set from within + less, but it will apply only to files opened subsequently, not to the file which is currently open. -m or --long-prompt - Causes less to prompt verbosely (like more), with the percent + Causes less to prompt verbosely (like more), with the percent into the file. By default, less prompts with a colon. -M or --LONG-PROMPT Causes less to prompt even more verbosely than more. -n or --line-numbers - Suppresses line numbers. The default (to use line numbers) may - cause less to run more slowly in some cases, especially with a - very large input file. Suppressing line numbers with the -n - option will avoid this problem. Using line numbers means: the + Suppresses line numbers. The default (to use line numbers) may + cause less to run more slowly in some cases, especially with a + very large input file. Suppressing line numbers with the -n + option will avoid this problem. Using line numbers means: the line number will be displayed in the verbose prompt and in the = - command, and the v command will pass the current line number to - the editor (see also the discussion of LESSEDIT in PROMPTS + command, and the v command will pass the current line number to + the editor (see also the discussion of LESSEDIT in PROMPTS below). -N or --LINE-NUMBERS - Causes a line number to be displayed at the beginning of each + Causes a line number to be displayed at the beginning of each line in the display. -ofilename or --log-file=filename - Causes less to copy its input to the named file as it is being + Causes less to copy its input to the named file as it is being viewed. This applies only when the input file is a pipe, not an - ordinary file. If the file already exists, less will ask for + ordinary file. If the file already exists, less will ask for confirmation before overwriting it. -Ofilename or --LOG-FILE=filename The -O option is like -o, but it will overwrite an existing file without asking for confirmation. - If no log file has been specified, the -o and -O options can be - used from within less to specify a log file. Without a file + If no log file has been specified, the -o and -O options can be + used from within less to specify a log file. Without a file name, they will simply report the name of the log file. The "s" command is equivalent to specifying -o from within less. -ppattern or --pattern=pattern - The -p option on the command line is equivalent to specifying - +/pattern; that is, it tells less to start at the first occur- + The -p option on the command line is equivalent to specifying + +/pattern; that is, it tells less to start at the first occur- rence of pattern in the file. -Pprompt or --prompt=prompt - Provides a way to tailor the three prompt styles to your own + Provides a way to tailor the three prompt styles to your own preference. This option would normally be put in the LESS envi- ronment variable, rather than being typed in with each less com- mand. Such an option must either be the last option in the LESS - variable, or be terminated by a dollar sign. -Ps followed by a - string changes the default (short) prompt to that string. -Pm - changes the medium (-m) prompt. -PM changes the long (-M) - prompt. -Ph changes the prompt for the help screen. -P= - changes the message printed by the = command. -Pw changes the - message printed while waiting for data (in the F command). All - prompt strings consist of a sequence of letters and special + variable, or be terminated by a dollar sign. -Ps followed by a + string changes the default (short) prompt to that string. -Pm + changes the medium (-m) prompt. -PM changes the long (-M) + prompt. -Ph changes the prompt for the help screen. -P= + changes the message printed by the = command. -Pw changes the + message printed while waiting for data (in the F command). All + prompt strings consist of a sequence of letters and special escape sequences. See the section on PROMPTS for more details. -q or --quiet or --silent - Causes moderately "quiet" operation: the terminal bell is not + Causes moderately "quiet" operation: the terminal bell is not rung if an attempt is made to scroll past the end of the file or before the beginning of the file. If the terminal has a "visual - bell", it is used instead. The bell will be rung on certain - other errors, such as typing an invalid character. The default + bell", it is used instead. The bell will be rung on certain + other errors, such as typing an invalid character. The default is to ring the terminal bell in all such cases. -Q or --QUIET or --SILENT - Causes totally "quiet" operation: the terminal bell is never + Causes totally "quiet" operation: the terminal bell is never rung. -r or --raw-control-chars Causes "raw" control characters to be displayed. The default is - to display control characters using the caret notation; for + to display control characters using the caret notation; for example, a control-A (octal 001) is displayed as "^A". Warning: when the -r option is used, less cannot keep track of the actual - appearance of the screen (since this depends on how the screen + appearance of the screen (since this depends on how the screen responds to each type of control character). Thus, various dis- - play problems may result, such as long lines being split in the + play problems may result, such as long lines being split in the wrong place. -R or --RAW-CONTROL-CHARS - Like -r, but only ANSI "color" escape sequences are output in + Like -r, but only ANSI "color" escape sequences are output in "raw" form. Unlike -r, the screen appearance is maintained cor- rectly in most cases. ANSI "color" escape sequences are sequences of the form: ESC [ ... m - where the "..." is zero or more color specification characters - For the purpose of keeping track of screen appearance, ANSI - color escape sequences are assumed to not move the cursor. You - can make less think that characters other than "m" can end ANSI - color escape sequences by setting the environment variable + where the "..." is zero or more color specification characters + For the purpose of keeping track of screen appearance, ANSI + color escape sequences are assumed to not move the cursor. You + can make less think that characters other than "m" can end ANSI + color escape sequences by setting the environment variable LESSANSIENDCHARS to the list of characters which can end a color - escape sequence. And you can make less think that characters - other than the standard ones may appear between the ESC and the - m by setting the environment variable LESSANSIMIDCHARS to the + escape sequence. And you can make less think that characters + other than the standard ones may appear between the ESC and the + m by setting the environment variable LESSANSIMIDCHARS to the list of characters which can appear. -s or --squeeze-blank-lines - Causes consecutive blank lines to be squeezed into a single + Causes consecutive blank lines to be squeezed into a single blank line. This is useful when viewing nroff output. -S or --chop-long-lines - Causes lines longer than the screen width to be chopped (trun- + Causes lines longer than the screen width to be chopped (trun- cated) rather than wrapped. That is, the portion of a long line that does not fit in the screen width is not shown. The default - is to wrap long lines; that is, display the remainder on the + is to wrap long lines; that is, display the remainder on the next line. -ttag or --tag=tag The -t option, followed immediately by a TAG, will edit the file - containing that tag. For this to work, tag information must be - available; for example, there may be a file in the current + containing that tag. For this to work, tag information must be + available; for example, there may be a file in the current directory called "tags", which was previously built by ctags (1) or an equivalent command. If the environment variable LESSGLOB- - ALTAGS is set, it is taken to be the name of a command compati- - ble with global (1), and that command is executed to find the + ALTAGS is set, it is taken to be the name of a command compati- + ble with global (1), and that command is executed to find the tag. (See http://www.gnu.org/software/global/global.html). The - -t option may also be specified from within less (using the - - command) as a way of examining a new file. The command ":t" is + -t option may also be specified from within less (using the - + command) as a way of examining a new file. The command ":t" is equivalent to specifying -t from within less. *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-vendor@FreeBSD.ORG Sun Nov 4 20:48:26 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E123D245; Sun, 4 Nov 2012 20:48:26 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id AD1518FC12; Sun, 4 Nov 2012 20:48:26 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA4KmQX4049692; Sun, 4 Nov 2012 20:48:26 GMT (envelope-from delphij@svn.freebsd.org) Received: (from delphij@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA4KmQhI049691; Sun, 4 Nov 2012 20:48:26 GMT (envelope-from delphij@svn.freebsd.org) Message-Id: <201211042048.qA4KmQhI049691@svn.freebsd.org> From: Xin LI Date: Sun, 4 Nov 2012 20:48:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242582 - vendor/less/v453 X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 20:48:27 -0000 Author: delphij Date: Sun Nov 4 20:48:26 2012 New Revision: 242582 URL: http://svn.freebsd.org/changeset/base/242582 Log: Tag less v453. Added: vendor/less/v453/ - copied from r242581, vendor/less/dist/ From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:24:07 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0F3D1793; Thu, 8 Nov 2012 01:24:07 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id E5A748FC12; Thu, 8 Nov 2012 01:24:06 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81O6Sp047708; Thu, 8 Nov 2012 01:24:06 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81O6Co047704; Thu, 8 Nov 2012 01:24:06 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080124.qA81O6Co047704@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:24:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242729 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:24:07 -0000 Author: mm Date: Thu Nov 8 01:24:06 2012 New Revision: 242729 URL: http://svnweb.freebsd.org/changeset/base/242729 Log: Update vendor/illumos/dist and vendor-sys/illumos/dist to illumos-gate 13849:3468a95b27cd (illumos ZFS issues #3145, #3212, #3258) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/ztest/ztest.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Thu Nov 8 00:54:43 2012 (r242728) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Thu Nov 8 01:24:06 2012 (r242729) @@ -188,6 +188,7 @@ uint64_t zfs_arc_meta_limit = 0; int zfs_arc_grow_retry = 0; int zfs_arc_shrink_shift = 0; int zfs_arc_p_min_shift = 0; +int zfs_disable_dup_eviction = 0; /* * Note that buffers can be in one of 6 states: @@ -290,6 +291,9 @@ typedef struct arc_stats { kstat_named_t arcstat_l2_size; kstat_named_t arcstat_l2_hdr_size; kstat_named_t arcstat_memory_throttle_count; + kstat_named_t arcstat_duplicate_buffers; + kstat_named_t arcstat_duplicate_buffers_size; + kstat_named_t arcstat_duplicate_reads; } arc_stats_t; static arc_stats_t arc_stats = { @@ -345,7 +349,10 @@ static arc_stats_t arc_stats = { { "l2_io_error", KSTAT_DATA_UINT64 }, { "l2_size", KSTAT_DATA_UINT64 }, { "l2_hdr_size", KSTAT_DATA_UINT64 }, - { "memory_throttle_count", KSTAT_DATA_UINT64 } + { "memory_throttle_count", KSTAT_DATA_UINT64 }, + { "duplicate_buffers", KSTAT_DATA_UINT64 }, + { "duplicate_buffers_size", KSTAT_DATA_UINT64 }, + { "duplicate_reads", KSTAT_DATA_UINT64 } }; #define ARCSTAT(stat) (arc_stats.stat.value.ui64) @@ -1360,6 +1367,17 @@ arc_buf_clone(arc_buf_t *from) hdr->b_buf = buf; arc_get_data_buf(buf); bcopy(from->b_data, buf->b_data, size); + + /* + * This buffer already exists in the arc so create a duplicate + * copy for the caller. If the buffer is associated with user data + * then track the size and number of duplicates. These stats will be + * updated as duplicate buffers are created and destroyed. + */ + if (hdr->b_type == ARC_BUFC_DATA) { + ARCSTAT_BUMP(arcstat_duplicate_buffers); + ARCSTAT_INCR(arcstat_duplicate_buffers_size, size); + } hdr->b_datacnt += 1; return (buf); } @@ -1458,6 +1476,16 @@ arc_buf_destroy(arc_buf_t *buf, boolean_ ASSERT3U(state->arcs_size, >=, size); atomic_add_64(&state->arcs_size, -size); buf->b_data = NULL; + + /* + * If we're destroying a duplicate buffer make sure + * that the appropriate statistics are updated. + */ + if (buf->b_hdr->b_datacnt > 1 && + buf->b_hdr->b_type == ARC_BUFC_DATA) { + ARCSTAT_BUMPDOWN(arcstat_duplicate_buffers); + ARCSTAT_INCR(arcstat_duplicate_buffers_size, -size); + } ASSERT(buf->b_hdr->b_datacnt > 0); buf->b_hdr->b_datacnt -= 1; } @@ -1642,6 +1670,48 @@ arc_buf_size(arc_buf_t *buf) } /* + * Called from the DMU to determine if the current buffer should be + * evicted. In order to ensure proper locking, the eviction must be initiated + * from the DMU. Return true if the buffer is associated with user data and + * duplicate buffers still exist. + */ +boolean_t +arc_buf_eviction_needed(arc_buf_t *buf) +{ + arc_buf_hdr_t *hdr; + boolean_t evict_needed = B_FALSE; + + if (zfs_disable_dup_eviction) + return (B_FALSE); + + mutex_enter(&buf->b_evict_lock); + hdr = buf->b_hdr; + if (hdr == NULL) { + /* + * We are in arc_do_user_evicts(); let that function + * perform the eviction. + */ + ASSERT(buf->b_data == NULL); + mutex_exit(&buf->b_evict_lock); + return (B_FALSE); + } else if (buf->b_data == NULL) { + /* + * We have already been added to the arc eviction list; + * recommend eviction. + */ + ASSERT3P(hdr, ==, &arc_eviction_hdr); + mutex_exit(&buf->b_evict_lock); + return (B_TRUE); + } + + if (hdr->b_datacnt > 1 && hdr->b_type == ARC_BUFC_DATA) + evict_needed = B_TRUE; + + mutex_exit(&buf->b_evict_lock); + return (evict_needed); +} + +/* * Evict buffers from list until we've removed the specified number of * bytes. Move the removed buffers to the appropriate evict state. * If the recycle flag is set, then attempt to "recycle" a buffer: @@ -2626,8 +2696,10 @@ arc_read_done(zio_t *zio) abuf = buf; for (acb = callback_list; acb; acb = acb->acb_next) { if (acb->acb_done) { - if (abuf == NULL) + if (abuf == NULL) { + ARCSTAT_BUMP(arcstat_duplicate_reads); abuf = arc_buf_clone(buf); + } acb->acb_buf = abuf; abuf = NULL; } @@ -3166,6 +3238,16 @@ arc_release(arc_buf_t *buf, void *tag) ASSERT3U(*size, >=, hdr->b_size); atomic_add_64(size, -hdr->b_size); } + + /* + * We're releasing a duplicate user data buffer, update + * our statistics accordingly. + */ + if (hdr->b_type == ARC_BUFC_DATA) { + ARCSTAT_BUMPDOWN(arcstat_duplicate_buffers); + ARCSTAT_INCR(arcstat_duplicate_buffers_size, + -hdr->b_size); + } hdr->b_datacnt -= 1; arc_cksum_verify(buf); arc_buf_unwatch(buf); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Thu Nov 8 00:54:43 2012 (r242728) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Thu Nov 8 01:24:06 2012 (r242729) @@ -2089,7 +2089,24 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, dbuf_evict(db); } else { VERIFY(arc_buf_remove_ref(db->db_buf, db) == 0); - if (!DBUF_IS_CACHEABLE(db)) + + /* + * A dbuf will be eligible for eviction if either the + * 'primarycache' property is set or a duplicate + * copy of this buffer is already cached in the arc. + * + * In the case of the 'primarycache' a buffer + * is considered for eviction if it matches the + * criteria set in the property. + * + * To decide if our buffer is considered a + * duplicate, we must call into the arc to determine + * if multiple buffers are referencing the same + * block on-disk. If so, then we simply evict + * ourselves. + */ + if (!DBUF_IS_CACHEABLE(db) || + arc_buf_eviction_needed(db->db_buf)) dbuf_clear(db); else mutex_exit(&db->db_mtx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Thu Nov 8 00:54:43 2012 (r242728) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Thu Nov 8 01:24:06 2012 (r242729) @@ -99,6 +99,7 @@ int arc_released(arc_buf_t *buf); int arc_has_callback(arc_buf_t *buf); void arc_buf_freeze(arc_buf_t *buf); void arc_buf_thaw(arc_buf_t *buf); +boolean_t arc_buf_eviction_needed(arc_buf_t *buf); #ifdef ZFS_DEBUG int arc_referenced(arc_buf_t *buf); #endif From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:24:07 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2B83A794; Thu, 8 Nov 2012 01:24:07 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 0E0EB8FC15; Thu, 8 Nov 2012 01:24:07 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81O6n6047714; Thu, 8 Nov 2012 01:24:06 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81O6OC047712; Thu, 8 Nov 2012 01:24:06 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080124.qA81O6OC047712@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:24:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242729 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:24:07 -0000 Author: mm Date: Thu Nov 8 01:24:06 2012 New Revision: 242729 URL: http://svnweb.freebsd.org/changeset/base/242729 Log: Update vendor/illumos/dist and vendor-sys/illumos/dist to illumos-gate 13849:3468a95b27cd (illumos ZFS issues #3145, #3212, #3258) Modified: vendor/illumos/dist/cmd/ztest/ztest.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Thu Nov 8 00:54:43 2012 (r242728) +++ vendor/illumos/dist/cmd/ztest/ztest.c Thu Nov 8 01:24:06 2012 (r242729) @@ -119,8 +119,8 @@ #include #include -#define ZTEST_FD_DATA 3 -#define ZTEST_FD_RAND 4 +static int ztest_fd_data = -1; +static int ztest_fd_rand = -1; typedef struct ztest_shared_hdr { uint64_t zh_hdr_size; @@ -708,14 +708,17 @@ process_options(int argc, char **argv) UINT64_MAX >> 2); if (strlen(altdir) > 0) { - char cmd[MAXNAMELEN]; - char realaltdir[MAXNAMELEN]; + char *cmd; + char *realaltdir; char *bin; char *ztest; char *isa; int isalen; - (void) realpath(getexecname(), cmd); + cmd = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); + realaltdir = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); + + VERIFY(NULL != realpath(getexecname(), cmd)); if (0 != access(altdir, F_OK)) { ztest_dump_core = B_FALSE; fatal(B_TRUE, "invalid alternate ztest path: %s", @@ -746,6 +749,9 @@ process_options(int argc, char **argv) fatal(B_TRUE, "invalid alternate lib directory %s", zo->zo_alt_libpath); } + + umem_free(cmd, MAXPATHLEN); + umem_free(realaltdir, MAXPATHLEN); } } @@ -762,10 +768,12 @@ ztest_random(uint64_t range) { uint64_t r; + ASSERT3S(ztest_fd_rand, >=, 0); + if (range == 0) return (0); - if (read(ZTEST_FD_RAND, &r, sizeof (r)) != sizeof (r)) + if (read(ztest_fd_rand, &r, sizeof (r)) != sizeof (r)) fatal(1, "short read from /dev/urandom"); return (r % range); @@ -4695,7 +4703,18 @@ ztest_fault_inject(ztest_ds_t *zd, uint6 if (islog) (void) rw_unlock(&ztest_name_lock); } else { + /* + * Ideally we would like to be able to randomly + * call vdev_[on|off]line without holding locks + * to force unpredictable failures but the side + * effects of vdev_[on|off]line prevent us from + * doing so. We grab the ztest_vdev_lock here to + * prevent a race between injection testing and + * aux_vdev removal. + */ + VERIFY(mutex_lock(&ztest_vdev_lock) == 0); (void) vdev_online(spa, guid0, 0, NULL); + VERIFY(mutex_unlock(&ztest_vdev_lock) == 0); } } @@ -5650,20 +5669,16 @@ ztest_init(ztest_shared_t *zs) } static void -setup_fds(void) +setup_data_fd(void) { - int fd; - - char *tmp = tempnam(NULL, NULL); - fd = open(tmp, O_RDWR | O_CREAT, 0700); - ASSERT3U(fd, ==, ZTEST_FD_DATA); - (void) unlink(tmp); - free(tmp); + static char ztest_name_data[] = "/tmp/ztest.data.XXXXXX"; - fd = open("/dev/urandom", O_RDONLY); - ASSERT3U(fd, ==, ZTEST_FD_RAND); + ztest_fd_data = mkstemp(ztest_name_data); + ASSERT3S(ztest_fd_data, >=, 0); + (void) unlink(ztest_name_data); } + static int shared_data_size(ztest_shared_hdr_t *hdr) { @@ -5685,10 +5700,10 @@ setup_hdr(void) ztest_shared_hdr_t *hdr; hdr = (void *)mmap(0, P2ROUNDUP(sizeof (*hdr), getpagesize()), - PROT_READ | PROT_WRITE, MAP_SHARED, ZTEST_FD_DATA, 0); + PROT_READ | PROT_WRITE, MAP_SHARED, ztest_fd_data, 0); ASSERT(hdr != MAP_FAILED); - VERIFY3U(0, ==, ftruncate(ZTEST_FD_DATA, sizeof (ztest_shared_hdr_t))); + VERIFY3U(0, ==, ftruncate(ztest_fd_data, sizeof (ztest_shared_hdr_t))); hdr->zh_hdr_size = sizeof (ztest_shared_hdr_t); hdr->zh_opts_size = sizeof (ztest_shared_opts_t); @@ -5699,7 +5714,7 @@ setup_hdr(void) hdr->zh_ds_count = ztest_opts.zo_datasets; size = shared_data_size(hdr); - VERIFY3U(0, ==, ftruncate(ZTEST_FD_DATA, size)); + VERIFY3U(0, ==, ftruncate(ztest_fd_data, size)); (void) munmap((caddr_t)hdr, P2ROUNDUP(sizeof (*hdr), getpagesize())); } @@ -5712,14 +5727,14 @@ setup_data(void) uint8_t *buf; hdr = (void *)mmap(0, P2ROUNDUP(sizeof (*hdr), getpagesize()), - PROT_READ, MAP_SHARED, ZTEST_FD_DATA, 0); + PROT_READ, MAP_SHARED, ztest_fd_data, 0); ASSERT(hdr != MAP_FAILED); size = shared_data_size(hdr); (void) munmap((caddr_t)hdr, P2ROUNDUP(sizeof (*hdr), getpagesize())); hdr = ztest_shared_hdr = (void *)mmap(0, P2ROUNDUP(size, getpagesize()), - PROT_READ | PROT_WRITE, MAP_SHARED, ZTEST_FD_DATA, 0); + PROT_READ | PROT_WRITE, MAP_SHARED, ztest_fd_data, 0); ASSERT(hdr != MAP_FAILED); buf = (uint8_t *)hdr; @@ -5738,12 +5753,13 @@ exec_child(char *cmd, char *libpath, boo { pid_t pid; int status; - char cmdbuf[MAXPATHLEN]; + char *cmdbuf = NULL; pid = fork(); if (cmd == NULL) { - (void) strlcpy(cmdbuf, getexecname(), sizeof (cmdbuf)); + cmdbuf = umem_alloc(MAXPATHLEN, UMEM_NOFAIL); + (void) strlcpy(cmdbuf, getexecname(), MAXPATHLEN); cmd = cmdbuf; } @@ -5752,9 +5768,16 @@ exec_child(char *cmd, char *libpath, boo if (pid == 0) { /* child */ char *emptyargv[2] = { cmd, NULL }; + char fd_data_str[12]; struct rlimit rl = { 1024, 1024 }; (void) setrlimit(RLIMIT_NOFILE, &rl); + + (void) close(ztest_fd_rand); + VERIFY3U(11, >=, + snprintf(fd_data_str, 12, "%d", ztest_fd_data)); + VERIFY0(setenv("ZTEST_FD_DATA", fd_data_str, 1)); + (void) enable_extended_FILE_stdio(-1, -1); if (libpath != NULL) VERIFY(0 == setenv("LD_LIBRARY_PATH", libpath, 1)); @@ -5763,6 +5786,11 @@ exec_child(char *cmd, char *libpath, boo fatal(B_TRUE, "exec failed: %s", cmd); } + if (cmdbuf != NULL) { + umem_free(cmdbuf, MAXPATHLEN); + cmd = NULL; + } + while (waitpid(pid, &status, 0) != pid) continue; if (statusp != NULL) @@ -5827,39 +5855,41 @@ main(int argc, char **argv) char timebuf[100]; char numbuf[6]; spa_t *spa; - char cmd[MAXNAMELEN]; + char *cmd; boolean_t hasalt; - - boolean_t ischild = (0 == lseek(ZTEST_FD_DATA, 0, SEEK_CUR)); - ASSERT(ischild || errno == EBADF); + char *fd_data_str = getenv("ZTEST_FD_DATA"); (void) setvbuf(stdout, NULL, _IOLBF, 0); dprintf_setup(&argc, argv); - if (!ischild) { + ztest_fd_rand = open("/dev/urandom", O_RDONLY); + ASSERT3S(ztest_fd_rand, >=, 0); + + if (!fd_data_str) { process_options(argc, argv); - setup_fds(); + setup_data_fd(); setup_hdr(); setup_data(); bcopy(&ztest_opts, ztest_shared_opts, sizeof (*ztest_shared_opts)); } else { + ztest_fd_data = atoi(fd_data_str); setup_data(); bcopy(ztest_shared_opts, &ztest_opts, sizeof (ztest_opts)); } ASSERT3U(ztest_opts.zo_datasets, ==, ztest_shared_hdr->zh_ds_count); /* Override location of zpool.cache */ - (void) asprintf((char **)&spa_config_path, "%s/zpool.cache", - ztest_opts.zo_dir); + VERIFY3U(asprintf((char **)&spa_config_path, "%s/zpool.cache", + ztest_opts.zo_dir), !=, -1); ztest_ds = umem_alloc(ztest_opts.zo_datasets * sizeof (ztest_ds_t), UMEM_NOFAIL); zs = ztest_shared; - if (ischild) { + if (fd_data_str) { metaslab_gang_bang = ztest_opts.zo_metaslab_gang_bang; metaslab_df_alloc_threshold = zs->zs_metaslab_df_alloc_threshold; @@ -5882,7 +5912,8 @@ main(int argc, char **argv) (u_longlong_t)ztest_opts.zo_time); } - (void) strlcpy(cmd, getexecname(), sizeof (cmd)); + cmd = umem_alloc(MAXNAMELEN, UMEM_NOFAIL); + (void) strlcpy(cmd, getexecname(), MAXNAMELEN); zs->zs_do_init = B_TRUE; if (strlen(ztest_opts.zo_alt_ztest) != 0) { @@ -6023,5 +6054,7 @@ main(int argc, char **argv) kills, iters - kills, (100.0 * kills) / MAX(1, iters)); } + umem_free(cmd, MAXNAMELEN); + return (0); } From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:28:40 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5A1F1D41; Thu, 8 Nov 2012 01:28:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 252BD8FC15; Thu, 8 Nov 2012 01:28:40 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81Sexv048415; Thu, 8 Nov 2012 01:28:40 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81Seoh048412; Thu, 8 Nov 2012 01:28:40 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080128.qA81Seoh048412@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:28:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242730 - in vendor-sys/illumos/dist/uts/common: . fs/zfs X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:28:40 -0000 Author: mm Date: Thu Nov 8 01:28:39 2012 New Revision: 242730 URL: http://svnweb.freebsd.org/changeset/base/242730 Log: Update vendor-sys/illumos/dist to illumos-gate 13866:040f4b0699ee (no FreeBSD-relevant changes - illumos issue #3301) Modified: vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Thu Nov 8 01:24:06 2012 (r242729) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Thu Nov 8 01:28:39 2012 (r242730) @@ -21,7 +21,7 @@ # # Copyright (c) 1991, 2010, Oracle and/or its affiliates. All rights reserved. -# Copyright 2011 Nexenta Systems, Inc. All rights reserved. +# Copyright (c) 2012 Nexenta Systems, Inc. All rights reserved. # Copyright (c) 2012 by Delphix. All rights reserved. # @@ -1944,6 +1944,16 @@ NXGE_HCALL_OBJS = \ nxge_hcall.o # +# Virtio modules +# + +# Virtio core +VIRTIO_OBJS = virtio.o + +# Virtio block driver +VIOBLK_OBJS = vioblk.o + +# # kiconv modules # KICONV_EMEA_OBJS += kiconv_emea.o Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Thu Nov 8 01:24:06 2012 (r242729) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Thu Nov 8 01:28:39 2012 (r242730) @@ -168,7 +168,7 @@ vdev_disk_open(vdev_t *vd, uint64_t *psi /* * When opening a disk device, we want to preserve the user's original * intent. We always want to open the device by the path the user gave - * us, even if it is one of multiple paths to the save device. But we + * us, even if it is one of multiple paths to the same device. But we * also want to be able to survive disks being removed/recabled. * Therefore the sequence of opening devices is: * From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:36:07 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0E5F0DE; Thu, 8 Nov 2012 01:36:07 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id E29288FC0A; Thu, 8 Nov 2012 01:36:06 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81a6YY049618; Thu, 8 Nov 2012 01:36:06 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81a6lA049604; Thu, 8 Nov 2012 01:36:06 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080136.qA81a6lA049604@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:36:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242732 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zinject vendor/illumos/dist/lib/libzpool/common vendor/illumos/... X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:36:07 -0000 Author: mm Date: Thu Nov 8 01:36:06 2012 New Revision: 242732 URL: http://svnweb.freebsd.org/changeset/base/242732 Log: Update vendor/illumos/dist and vendor-sys/illumos/dist to illumos-gate 13869:921a99998bb4 (illumos ZFS issue #3246 ZFS I/O deadman thread) Note: illumos disables deadman logic by default on vmware and sparc Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_boot.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_context.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio_inject.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zinject/translate.c vendor/illumos/dist/cmd/zinject/zinject.c vendor/illumos/dist/lib/libzpool/common/kernel.c vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Thu Nov 8 01:36:06 2012 (r242732) @@ -5983,6 +5983,10 @@ spa_sync(spa_t *spa, uint64_t txg) tx = dmu_tx_create_assigned(dp, txg); + spa->spa_sync_starttime = gethrtime(); + VERIFY(cyclic_reprogram(spa->spa_deadman_cycid, + spa->spa_sync_starttime + spa->spa_deadman_synctime)); + /* * If we are upgrading to SPA_VERSION_RAIDZ_DEFLATE this txg, * set spa_deflate if we have no raid-z vdevs. @@ -6111,6 +6115,8 @@ spa_sync(spa_t *spa, uint64_t txg) } dmu_tx_commit(tx); + VERIFY(cyclic_reprogram(spa->spa_deadman_cycid, CY_INFINITY)); + /* * Clear the dirty config list. */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Thu Nov 8 01:36:06 2012 (r242732) @@ -26,6 +26,7 @@ #include #include +#include #include #include #include @@ -249,6 +250,26 @@ int zfs_flags = 0; */ int zfs_recover = 0; +extern int zfs_txg_synctime_ms; + +/* + * Expiration time in units of zfs_txg_synctime_ms. This value has two + * meanings. First it is used to determine when the spa_deadman logic + * should fire. By default the spa_deadman will fire if spa_sync has + * not completed in 1000 * zfs_txg_synctime_ms (i.e. 1000 seconds). + * Secondly, the value determines if an I/O is considered "hung". + * Any I/O that has not completed in zfs_deadman_synctime is considered + * "hung" resulting in a system panic. + * 1000 zfs_txg_synctime_ms (i.e. 1000 seconds). + */ +uint64_t zfs_deadman_synctime = 1000ULL; + +/* + * Override the zfs deadman behavior via /etc/system. By default the + * deadman is enabled except on VMware and sparc deployments. + */ +int zfs_deadman_enabled = -1; + /* * ========================================================================== @@ -418,6 +439,23 @@ spa_lookup(const char *name) } /* + * Fires when spa_sync has not completed within zfs_deadman_synctime_ms. + * If the zfs_deadman_enabled flag is set then it inspects all vdev queues + * looking for potentially hung I/Os. + */ +void +spa_deadman(void *arg) +{ + spa_t *spa = arg; + + zfs_dbgmsg("slow spa_sync: started %llu seconds ago, calls %llu", + (gethrtime() - spa->spa_sync_starttime) / NANOSEC, + ++spa->spa_deadman_calls); + if (zfs_deadman_enabled) + vdev_deadman(spa->spa_root_vdev); +} + +/* * Create an uninitialized spa_t with the given name. Requires * spa_namespace_lock. The caller must ensure that the spa_t doesn't already * exist by calling spa_lookup() first. @@ -427,6 +465,8 @@ spa_add(const char *name, nvlist_t *conf { spa_t *spa; spa_config_dirent_t *dp; + cyc_handler_t hdlr; + cyc_time_t when; ASSERT(MUTEX_HELD(&spa_namespace_lock)); @@ -458,6 +498,25 @@ spa_add(const char *name, nvlist_t *conf spa->spa_proc = &p0; spa->spa_proc_state = SPA_PROC_NONE; + hdlr.cyh_func = spa_deadman; + hdlr.cyh_arg = spa; + hdlr.cyh_level = CY_LOW_LEVEL; + + spa->spa_deadman_synctime = zfs_deadman_synctime * + zfs_txg_synctime_ms * MICROSEC; + + /* + * This determines how often we need to check for hung I/Os after + * the cyclic has already fired. Since checking for hung I/Os is + * an expensive operation we don't want to check too frequently. + * Instead wait for 5 synctimes before checking again. + */ + when.cyt_interval = 5ULL * zfs_txg_synctime_ms * MICROSEC; + when.cyt_when = CY_INFINITY; + mutex_enter(&cpu_lock); + spa->spa_deadman_cycid = cyclic_add(&hdlr, &when); + mutex_exit(&cpu_lock); + refcount_create(&spa->spa_refcount); spa_config_lock_init(spa); @@ -540,6 +599,12 @@ spa_remove(spa_t *spa) nvlist_free(spa->spa_load_info); spa_config_set(spa, NULL); + mutex_enter(&cpu_lock); + if (spa->spa_deadman_cycid != CYCLIC_NONE) + cyclic_remove(spa->spa_deadman_cycid); + mutex_exit(&cpu_lock); + spa->spa_deadman_cycid = CYCLIC_NONE; + refcount_destroy(&spa->spa_refcount); spa_config_lock_destroy(spa); @@ -1507,6 +1572,12 @@ spa_prev_software_version(spa_t *spa) } uint64_t +spa_deadman_synctime(spa_t *spa) +{ + return (spa->spa_deadman_synctime); +} + +uint64_t dva_get_dsize_sync(spa_t *spa, const dva_t *dva) { uint64_t asize = DVA_GET_ASIZE(dva); @@ -1600,7 +1671,9 @@ spa_init(int mode) spa_mode_global = mode; -#ifndef _KERNEL +#ifdef _KERNEL + spa_arch_init(); +#else if (spa_mode_global != FREAD && dprintf_find_string("watch")) { arc_procfd = open("/proc/self/ctl", O_WRONLY); if (arc_procfd == -1) { Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Thu Nov 8 01:36:06 2012 (r242732) @@ -604,6 +604,7 @@ extern boolean_t spa_suspended(spa_t *sp extern uint64_t spa_bootfs(spa_t *spa); extern uint64_t spa_delegation(spa_t *spa); extern objset_t *spa_meta_objset(spa_t *spa); +extern uint64_t spa_deadman_synctime(spa_t *spa); /* Miscellaneous support routines */ extern void spa_activate_mos_feature(spa_t *spa, const char *feature); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_boot.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_boot.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_boot.h Thu Nov 8 01:36:06 2012 (r242732) @@ -23,6 +23,10 @@ * Use is subject to license terms. */ +/* + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + #ifndef _SYS_SPA_BOOT_H #define _SYS_SPA_BOOT_H @@ -35,6 +39,8 @@ extern "C" { extern char *spa_get_bootprop(char *prop); extern void spa_free_bootprop(char *prop); +extern void spa_arch_init(void); + #ifdef __cplusplus } #endif Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Thu Nov 8 01:36:06 2012 (r242732) @@ -227,6 +227,10 @@ struct spa { uint64_t spa_feat_for_write_obj; /* required to write to pool */ uint64_t spa_feat_for_read_obj; /* required to read from pool */ uint64_t spa_feat_desc_obj; /* Feature descriptions */ + cyclic_id_t spa_deadman_cycid; /* cyclic id */ + uint64_t spa_deadman_calls; /* number of deadman calls */ + uint64_t spa_sync_starttime; /* starting time fo spa_sync */ + uint64_t spa_deadman_synctime; /* deadman expiration timer */ /* * spa_refcnt & spa_config_lock must be the last elements * because refcount_t changes size based on compilation options. Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h Thu Nov 8 01:36:06 2012 (r242732) @@ -79,6 +79,7 @@ extern void vdev_metaslab_fini(vdev_t *v extern void vdev_metaslab_set_size(vdev_t *); extern void vdev_expand(vdev_t *vd, uint64_t txg); extern void vdev_split(vdev_t *vd); +extern void vdev_deadman(vdev_t *vd); extern void vdev_get_stats(vdev_t *vd, vdev_stat_t *vs); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Thu Nov 8 01:36:06 2012 (r242732) @@ -104,6 +104,8 @@ struct vdev_queue { avl_tree_t vq_read_tree; avl_tree_t vq_write_tree; avl_tree_t vq_pending_tree; + uint64_t vq_io_complete_ts; + uint64_t vq_io_delta_ts; kmutex_t vq_lock; }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_context.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_context.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_context.h Thu Nov 8 01:36:06 2012 (r242732) @@ -22,8 +22,10 @@ * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. */ + /* * Copyright 2011 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2012 by Delphix. All rights reserved. */ #ifndef _SYS_ZFS_CONTEXT_H @@ -67,6 +69,7 @@ extern "C" { #include #include #include +#include #define CPU_SEQID (CPU->cpu_seqid) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h Thu Nov 8 01:36:06 2012 (r242732) @@ -240,12 +240,24 @@ typedef struct zinject_record { uint32_t zi_iotype; int32_t zi_duration; uint64_t zi_timer; + uint32_t zi_cmd; + uint32_t zi_pad; } zinject_record_t; #define ZINJECT_NULL 0x1 #define ZINJECT_FLUSH_ARC 0x2 #define ZINJECT_UNLOAD_SPA 0x4 +typedef enum zinject_type { + ZINJECT_UNINITIALIZED, + ZINJECT_DATA_FAULT, + ZINJECT_DEVICE_FAULT, + ZINJECT_LABEL_FAULT, + ZINJECT_IGNORED_WRITES, + ZINJECT_PANIC, + ZINJECT_DELAY_IO, +} zinject_type_t; + typedef struct zfs_share { uint64_t z_exportdata; uint64_t z_sharedata; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Thu Nov 8 01:36:06 2012 (r242732) @@ -21,8 +21,6 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - */ -/* * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2012 by Delphix. All rights reserved. */ @@ -403,6 +401,7 @@ struct zio { uint64_t io_offset; uint64_t io_deadline; + uint64_t io_timestamp; avl_node_t io_offset_node; avl_node_t io_deadline_node; avl_tree_t *io_vdev_tree; @@ -548,6 +547,7 @@ extern int zio_handle_fault_injection(zi extern int zio_handle_device_injection(vdev_t *vd, zio_t *zio, int error); extern int zio_handle_label_injection(zio_t *zio, int error); extern void zio_handle_ignored_writes(zio_t *zio); +extern uint64_t zio_handle_io_delay(zio_t *zio); /* * Checksum ereport functions Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Thu Nov 8 01:36:06 2012 (r242732) @@ -3153,3 +3153,41 @@ vdev_split(vdev_t *vd) } vdev_propagate_state(cvd); } + +void +vdev_deadman(vdev_t *vd) +{ + for (int c = 0; c < vd->vdev_children; c++) { + vdev_t *cvd = vd->vdev_child[c]; + + vdev_deadman(cvd); + } + + if (vd->vdev_ops->vdev_op_leaf) { + vdev_queue_t *vq = &vd->vdev_queue; + + mutex_enter(&vq->vq_lock); + if (avl_numnodes(&vq->vq_pending_tree) > 0) { + spa_t *spa = vd->vdev_spa; + zio_t *fio; + uint64_t delta; + + /* + * Look at the head of all the pending queues, + * if any I/O has been outstanding for longer than + * the spa_deadman_synctime we panic the system. + */ + fio = avl_first(&vq->vq_pending_tree); + delta = ddi_get_lbolt64() - fio->io_timestamp; + if (delta > NSEC_TO_TICK(spa_deadman_synctime(spa))) { + zfs_dbgmsg("SLOW IO: zio timestamp %llu, " + "delta %llu, last io %llu", + fio->io_timestamp, delta, + vq->vq_io_complete_ts); + fm_panic("I/O to pool '%s' appears to be " + "hung.", spa_name(spa)); + } + } + mutex_exit(&vq->vq_lock); + } +} Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Thu Nov 8 01:36:06 2012 (r242732) @@ -23,6 +23,10 @@ * Use is subject to license terms. */ +/* + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + #include #include #include @@ -288,6 +292,7 @@ again: zio_buf_alloc(size), size, fio->io_type, ZIO_PRIORITY_AGG, flags | ZIO_FLAG_DONT_CACHE | ZIO_FLAG_DONT_QUEUE, vdev_queue_agg_io_done, NULL); + aio->io_timestamp = fio->io_timestamp; nio = fio; do { @@ -359,7 +364,8 @@ vdev_queue_io(zio_t *zio) mutex_enter(&vq->vq_lock); - zio->io_deadline = (ddi_get_lbolt64() >> zfs_vdev_time_shift) + + zio->io_timestamp = ddi_get_lbolt64(); + zio->io_deadline = (zio->io_timestamp >> zfs_vdev_time_shift) + zio->io_priority; vdev_queue_io_add(vq, zio); @@ -384,10 +390,16 @@ vdev_queue_io_done(zio_t *zio) { vdev_queue_t *vq = &zio->io_vd->vdev_queue; + if (zio_injection_enabled) + delay(SEC_TO_TICK(zio_handle_io_delay(zio))); + mutex_enter(&vq->vq_lock); avl_remove(&vq->vq_pending_tree, zio); + vq->vq_io_complete_ts = ddi_get_lbolt64(); + vq->vq_io_delta_ts = vq->vq_io_complete_ts - zio->io_timestamp; + for (int i = 0; i < zfs_vdev_ramp_rate; i++) { zio_t *nio = vdev_queue_io_to_issue(vq, zfs_vdev_max_pending); if (nio == NULL) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio_inject.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio_inject.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio_inject.c Thu Nov 8 01:36:06 2012 (r242732) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012 by Delphix. All rights reserved. */ /* @@ -147,14 +148,8 @@ zio_handle_fault_injection(zio_t *zio, i for (handler = list_head(&inject_handlers); handler != NULL; handler = list_next(&inject_handlers, handler)) { - /* Ignore errors not destined for this pool */ - if (zio->io_spa != handler->zi_spa) - continue; - - /* Ignore device errors and panic injection */ - if (handler->zi_record.zi_guid != 0 || - handler->zi_record.zi_func[0] != '\0' || - handler->zi_record.zi_duration != 0) + if (zio->io_spa != handler->zi_spa || + handler->zi_record.zi_cmd != ZINJECT_DATA_FAULT) continue; /* If this handler matches, return EIO */ @@ -197,10 +192,7 @@ zio_handle_label_injection(zio_t *zio, i uint64_t start = handler->zi_record.zi_start; uint64_t end = handler->zi_record.zi_end; - /* Ignore device only faults or panic injection */ - if (handler->zi_record.zi_start == 0 || - handler->zi_record.zi_func[0] != '\0' || - handler->zi_record.zi_duration != 0) + if (handler->zi_record.zi_cmd != ZINJECT_LABEL_FAULT) continue; /* @@ -246,13 +238,7 @@ zio_handle_device_injection(vdev_t *vd, for (handler = list_head(&inject_handlers); handler != NULL; handler = list_next(&inject_handlers, handler)) { - /* - * Ignore label specific faults, panic injection - * or fake writes - */ - if (handler->zi_record.zi_start != 0 || - handler->zi_record.zi_func[0] != '\0' || - handler->zi_record.zi_duration != 0) + if (handler->zi_record.zi_cmd != ZINJECT_DEVICE_FAULT) continue; if (vd->vdev_guid == handler->zi_record.zi_guid) { @@ -316,10 +302,8 @@ zio_handle_ignored_writes(zio_t *zio) handler = list_next(&inject_handlers, handler)) { /* Ignore errors not destined for this pool */ - if (zio->io_spa != handler->zi_spa) - continue; - - if (handler->zi_record.zi_duration == 0) + if (zio->io_spa != handler->zi_spa || + handler->zi_record.zi_cmd != ZINJECT_IGNORED_WRITES) continue; /* @@ -355,11 +339,8 @@ spa_handle_ignored_writes(spa_t *spa) for (handler = list_head(&inject_handlers); handler != NULL; handler = list_next(&inject_handlers, handler)) { - /* Ignore errors not destined for this pool */ - if (spa != handler->zi_spa) - continue; - - if (handler->zi_record.zi_duration == 0) + if (spa != handler->zi_spa || + handler->zi_record.zi_cmd != ZINJECT_IGNORED_WRITES) continue; if (handler->zi_record.zi_duration > 0) { @@ -379,6 +360,34 @@ spa_handle_ignored_writes(spa_t *spa) rw_exit(&inject_lock); } +uint64_t +zio_handle_io_delay(zio_t *zio) +{ + vdev_t *vd = zio->io_vd; + inject_handler_t *handler; + uint64_t seconds = 0; + + if (zio_injection_enabled == 0) + return (0); + + rw_enter(&inject_lock, RW_READER); + + for (handler = list_head(&inject_handlers); handler != NULL; + handler = list_next(&inject_handlers, handler)) { + + if (handler->zi_record.zi_cmd != ZINJECT_DELAY_IO) + continue; + + if (vd->vdev_guid == handler->zi_record.zi_guid) { + seconds = handler->zi_record.zi_timer; + break; + } + + } + rw_exit(&inject_lock); + return (seconds); +} + /* * Create a new handler for the given record. We add it to the list, adding * a reference to the spa_t in the process. We increment zio_injection_enabled, From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:36:07 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 304D6DF; Thu, 8 Nov 2012 01:36:07 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 11D908FC0C; Thu, 8 Nov 2012 01:36:07 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81a6Uk049627; Thu, 8 Nov 2012 01:36:06 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81a6Bs049623; Thu, 8 Nov 2012 01:36:06 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080136.qA81a6Bs049623@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:36:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242732 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zinject vendor/illumos/dist/lib/libzpool/common vendor/illumos/... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:36:07 -0000 Author: mm Date: Thu Nov 8 01:36:06 2012 New Revision: 242732 URL: http://svnweb.freebsd.org/changeset/base/242732 Log: Update vendor/illumos/dist and vendor-sys/illumos/dist to illumos-gate 13869:921a99998bb4 (illumos ZFS issue #3246 ZFS I/O deadman thread) Note: illumos disables deadman logic by default on vmware and sparc Modified: vendor/illumos/dist/cmd/zinject/translate.c vendor/illumos/dist/cmd/zinject/zinject.c vendor/illumos/dist/lib/libzpool/common/kernel.c vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_boot.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_context.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_ioctl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio_inject.c Modified: vendor/illumos/dist/cmd/zinject/translate.c ============================================================================== --- vendor/illumos/dist/cmd/zinject/translate.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor/illumos/dist/cmd/zinject/translate.c Thu Nov 8 01:36:06 2012 (r242732) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2006, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012 by Delphix. All rights reserved. */ #include @@ -472,6 +473,20 @@ translate_device(const char *pool, const &record->zi_guid) == 0); } + /* + * Device faults can take on three different forms: + * 1). delayed or hanging I/O + * 2). zfs label faults + * 3). generic disk faults + */ + if (record->zi_timer != 0) { + record->zi_cmd = ZINJECT_DELAY_IO; + } else if (label_type != TYPE_INVAL) { + record->zi_cmd = ZINJECT_LABEL_FAULT; + } else { + record->zi_cmd = ZINJECT_DEVICE_FAULT; + } + switch (label_type) { case TYPE_LABEL_UBERBLOCK: record->zi_start = offsetof(vdev_label_t, vl_uberblock[0]); Modified: vendor/illumos/dist/cmd/zinject/zinject.c ============================================================================== --- vendor/illumos/dist/cmd/zinject/zinject.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor/illumos/dist/cmd/zinject/zinject.c Thu Nov 8 01:36:06 2012 (r242732) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012 by Delphix. All rights reserved. */ /* @@ -600,7 +601,7 @@ main(int argc, char **argv) } while ((c = getopt(argc, argv, - ":aA:b:d:f:Fg:qhIc:t:T:l:mr:s:e:uL:p:")) != -1) { + ":aA:b:d:D:f:Fg:qhIc:t:T:l:mr:s:e:uL:p:")) != -1) { switch (c) { case 'a': flags |= ZINJECT_FLUSH_ARC; @@ -626,6 +627,15 @@ main(int argc, char **argv) case 'd': device = optarg; break; + case 'D': + record.zi_timer = strtoull(optarg, &end, 10); + if (errno != 0 || *end != '\0') { + (void) fprintf(stderr, "invalid i/o delay " + "value: '%s'\n", optarg); + usage(); + return (1); + } + break; case 'e': if (strcasecmp(optarg, "io") == 0) { error = EIO; @@ -690,6 +700,7 @@ main(int argc, char **argv) case 'p': (void) strlcpy(record.zi_func, optarg, sizeof (record.zi_func)); + record.zi_cmd = ZINJECT_PANIC; break; case 'q': quiet = 1; @@ -763,13 +774,15 @@ main(int argc, char **argv) argc -= optind; argv += optind; + if (record.zi_duration != 0) + record.zi_cmd = ZINJECT_IGNORED_WRITES; + if (cancel != NULL) { /* * '-c' is invalid with any other options. */ if (raw != NULL || range != NULL || type != TYPE_INVAL || - level != 0 || record.zi_func[0] != '\0' || - record.zi_duration != 0) { + level != 0 || record.zi_cmd != ZINJECT_UNINITIALIZED) { (void) fprintf(stderr, "cancel (-c) incompatible with " "any other options\n"); usage(); @@ -801,8 +814,7 @@ main(int argc, char **argv) * for doing injection, so handle it separately here. */ if (raw != NULL || range != NULL || type != TYPE_INVAL || - level != 0 || record.zi_func[0] != '\0' || - record.zi_duration != 0) { + level != 0 || record.zi_cmd != ZINJECT_UNINITIALIZED) { (void) fprintf(stderr, "device (-d) incompatible with " "data error injection\n"); usage(); @@ -836,7 +848,7 @@ main(int argc, char **argv) } else if (raw != NULL) { if (range != NULL || type != TYPE_INVAL || level != 0 || - record.zi_func[0] != '\0' || record.zi_duration != 0) { + record.zi_cmd != ZINJECT_UNINITIALIZED) { (void) fprintf(stderr, "raw (-b) format with " "any other options\n"); usage(); @@ -859,13 +871,14 @@ main(int argc, char **argv) return (1); } + record.zi_cmd = ZINJECT_DATA_FAULT; if (translate_raw(raw, &record) != 0) return (1); if (!error) error = EIO; - } else if (record.zi_func[0] != '\0') { + } else if (record.zi_cmd == ZINJECT_PANIC) { if (raw != NULL || range != NULL || type != TYPE_INVAL || - level != 0 || device != NULL || record.zi_duration != 0) { + level != 0 || device != NULL) { (void) fprintf(stderr, "panic (-p) incompatible with " "other options\n"); usage(); @@ -883,7 +896,7 @@ main(int argc, char **argv) if (argv[1] != NULL) record.zi_type = atoi(argv[1]); dataset[0] = '\0'; - } else if (record.zi_duration != 0) { + } else if (record.zi_cmd == ZINJECT_IGNORED_WRITES) { if (nowrites == 0) { (void) fprintf(stderr, "-s or -g meaningless " "without -I (ignore writes)\n"); @@ -937,6 +950,7 @@ main(int argc, char **argv) return (1); } + record.zi_cmd = ZINJECT_DATA_FAULT; if (translate_record(type, argv[0], range, level, &record, pool, dataset) != 0) return (1); Modified: vendor/illumos/dist/lib/libzpool/common/kernel.c ============================================================================== --- vendor/illumos/dist/lib/libzpool/common/kernel.c Thu Nov 8 01:35:02 2012 (r242731) +++ vendor/illumos/dist/lib/libzpool/common/kernel.c Thu Nov 8 01:36:06 2012 (r242732) @@ -46,6 +46,7 @@ int aok; uint64_t physmem; vnode_t *rootdir = (vnode_t *)0xabcd1234; char hw_serial[HW_HOSTID_LEN]; +kmutex_t cpu_lock; vmem_t *zio_arena = NULL; struct utsname utsname = { @@ -790,6 +791,26 @@ ddi_strtoull(const char *str, char **npt return (0); } +/* ARGSUSED */ +cyclic_id_t +cyclic_add(cyc_handler_t *hdlr, cyc_time_t *when) +{ + return (1); +} + +/* ARGSUSED */ +void +cyclic_remove(cyclic_id_t id) +{ +} + +/* ARGSUSED */ +int +cyclic_reprogram(cyclic_id_t id, hrtime_t expiration) +{ + return (1); +} + /* * ========================================================================= * kernel emulation setup & teardown @@ -823,6 +844,8 @@ kernel_init(int mode) system_taskq_init(); + mutex_init(&cpu_lock, NULL, MUTEX_DEFAULT, NULL); + spa_init(mode); } Modified: vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h ============================================================================== --- vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Thu Nov 8 01:35:02 2012 (r242731) +++ vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Thu Nov 8 01:36:06 2012 (r242732) @@ -453,6 +453,9 @@ extern vnode_t *rootdir; extern void delay(clock_t ticks); +#define SEC_TO_TICK(sec) ((sec) * hz) +#define NSEC_TO_TICK(usec) ((usec) / (NANOSEC / hz)) + #define gethrestime_sec() time(NULL) #define gethrestime(t) \ do {\ @@ -572,6 +575,34 @@ void ksiddomain_rele(ksiddomain_t *); #define ddi_log_sysevent(_a, _b, _c, _d, _e, _f, _g) \ sysevent_post_event(_c, _d, _b, "libzpool", _e, _f) +/* + * Cyclic information + */ +extern kmutex_t cpu_lock; + +typedef uintptr_t cyclic_id_t; +typedef uint16_t cyc_level_t; +typedef void (*cyc_func_t)(void *); + +#define CY_LOW_LEVEL 0 +#define CY_INFINITY INT64_MAX +#define CYCLIC_NONE ((cyclic_id_t)0) + +typedef struct cyc_time { + hrtime_t cyt_when; + hrtime_t cyt_interval; +} cyc_time_t; + +typedef struct cyc_handler { + cyc_func_t cyh_func; + void *cyh_arg; + cyc_level_t cyh_level; +} cyc_handler_t; + +extern cyclic_id_t cyclic_add(cyc_handler_t *, cyc_time_t *); +extern void cyclic_remove(cyclic_id_t); +extern int cyclic_reprogram(cyclic_id_t, hrtime_t); + #ifdef __cplusplus } #endif From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:38:31 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 142793C2; Thu, 8 Nov 2012 01:38:31 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id E9F2C8FC08; Thu, 8 Nov 2012 01:38:30 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81cU55050013; Thu, 8 Nov 2012 01:38:30 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81cU2Q050008; Thu, 8 Nov 2012 01:38:30 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080138.qA81cU2Q050008@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:38:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242733 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzp... X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:38:31 -0000 Author: mm Date: Thu Nov 8 01:38:30 2012 New Revision: 242733 URL: http://svnweb.freebsd.org/changeset/base/242733 Log: Update vendor/illumos/dist and vendor/illumos-sys/dist to illumos-gate 13871:a9c12c2c1647 (zfs changes, illumos issues #3306, #3321) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzpool/common/kernel.c vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h vendor/illumos/dist/man/man1m/zdb.1m vendor/illumos/dist/man/man1m/zpool.1m Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Thu Nov 8 01:36:06 2012 (r242732) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Thu Nov 8 01:38:30 2012 (r242733) @@ -322,6 +322,14 @@ extern void vdev_set_min_asize(vdev_t *v */ extern int zfs_vdev_cache_size; +/* + * The vdev_buf_t is used to translate between zio_t and buf_t, and back again. + */ +typedef struct vdev_buf { + buf_t vb_buf; /* buffer that describes the io */ + zio_t *vb_io; /* pointer back to the original zio_t */ +} vdev_buf_t; + #ifdef __cplusplus } #endif Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Thu Nov 8 01:36:06 2012 (r242732) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Thu Nov 8 01:38:30 2012 (r242733) @@ -40,11 +40,6 @@ extern ldi_ident_t zfs_li; -typedef struct vdev_disk_buf { - buf_t vdb_buf; - zio_t *vdb_io; -} vdev_disk_buf_t; - static void vdev_disk_hold(vdev_t *vd) { @@ -397,8 +392,8 @@ vdev_disk_physio(ldi_handle_t vd_lh, cad static void vdev_disk_io_intr(buf_t *bp) { - vdev_disk_buf_t *vdb = (vdev_disk_buf_t *)bp; - zio_t *zio = vdb->vdb_io; + vdev_buf_t *vb = (vdev_buf_t *)bp; + zio_t *zio = vb->vb_io; /* * The rest of the zio stack only deals with EIO, ECKSUM, and ENXIO. @@ -410,7 +405,7 @@ vdev_disk_io_intr(buf_t *bp) if (zio->io_error == 0 && bp->b_resid != 0) zio->io_error = EIO; - kmem_free(vdb, sizeof (vdev_disk_buf_t)); + kmem_free(vb, sizeof (vdev_buf_t)); zio_interrupt(zio); } @@ -441,7 +436,7 @@ vdev_disk_io_start(zio_t *zio) { vdev_t *vd = zio->io_vd; vdev_disk_t *dvd = vd->vdev_tsd; - vdev_disk_buf_t *vdb; + vdev_buf_t *vb; struct dk_callback *dkc; buf_t *bp; int error; @@ -505,10 +500,10 @@ vdev_disk_io_start(zio_t *zio) return (ZIO_PIPELINE_CONTINUE); } - vdb = kmem_alloc(sizeof (vdev_disk_buf_t), KM_SLEEP); + vb = kmem_alloc(sizeof (vdev_buf_t), KM_SLEEP); - vdb->vdb_io = zio; - bp = &vdb->vdb_buf; + vb->vb_io = zio; + bp = &vb->vb_buf; bioinit(bp); bp->b_flags = B_BUSY | B_NOCACHE | Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c Thu Nov 8 01:36:06 2012 (r242732) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c Thu Nov 8 01:38:30 2012 (r242733) @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -140,12 +141,55 @@ vdev_file_close(vdev_t *vd) vd->vdev_tsd = NULL; } +/* + * Implements the interrupt side for file vdev types. This routine will be + * called when the I/O completes allowing us to transfer the I/O to the + * interrupt taskqs. For consistency, the code structure mimics disk vdev + * types. + */ +static void +vdev_file_io_intr(buf_t *bp) +{ + vdev_buf_t *vb = (vdev_buf_t *)bp; + zio_t *zio = vb->vb_io; + + zio->io_error = (geterror(bp) != 0 ? EIO : 0); + if (zio->io_error == 0 && bp->b_resid != 0) + zio->io_error = ENOSPC; + + kmem_free(vb, sizeof (vdev_buf_t)); + zio_interrupt(zio); +} + +static void +vdev_file_io_strategy(void *arg) +{ + buf_t *bp = arg; + vnode_t *vp = bp->b_private; + ssize_t resid; + int error; + + error = vn_rdwr((bp->b_flags & B_READ) ? UIO_READ : UIO_WRITE, + vp, bp->b_un.b_addr, bp->b_bcount, ldbtob(bp->b_lblkno), + UIO_SYSSPACE, 0, RLIM64_INFINITY, kcred, &resid); + + if (error == 0) { + bp->b_resid = resid; + biodone(bp); + } else { + bioerror(bp, error); + biodone(bp); + } +} + static int vdev_file_io_start(zio_t *zio) { + spa_t *spa = zio->io_spa; vdev_t *vd = zio->io_vd; vdev_file_t *vf = vd->vdev_tsd; - ssize_t resid; + vdev_buf_t *vb; + buf_t *bp; if (zio->io_type == ZIO_TYPE_IOCTL) { /* XXPOLICY */ @@ -166,15 +210,22 @@ vdev_file_io_start(zio_t *zio) return (ZIO_PIPELINE_CONTINUE); } - zio->io_error = vn_rdwr(zio->io_type == ZIO_TYPE_READ ? - UIO_READ : UIO_WRITE, vf->vf_vnode, zio->io_data, - zio->io_size, zio->io_offset, UIO_SYSSPACE, - 0, RLIM64_INFINITY, kcred, &resid); + vb = kmem_alloc(sizeof (vdev_buf_t), KM_SLEEP); - if (resid != 0 && zio->io_error == 0) - zio->io_error = ENOSPC; + vb->vb_io = zio; + bp = &vb->vb_buf; - zio_interrupt(zio); + bioinit(bp); + bp->b_flags = (zio->io_type == ZIO_TYPE_READ ? B_READ : B_WRITE); + bp->b_bcount = zio->io_size; + bp->b_un.b_addr = zio->io_data; + bp->b_lblkno = lbtodb(zio->io_offset); + bp->b_bufsize = zio->io_size; + bp->b_private = vf->vf_vnode; + bp->b_iodone = (int (*)())vdev_file_io_intr; + + taskq_dispatch_ent(spa->spa_zio_taskq[ZIO_TYPE_FREE][ZIO_TASKQ_ISSUE], + vdev_file_io_strategy, bp, 0, &zio->io_tqent); return (ZIO_PIPELINE_STOP); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Thu Nov 8 01:36:06 2012 (r242732) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Thu Nov 8 01:38:30 2012 (r242733) @@ -2917,7 +2917,7 @@ zio_done(zio_t *zio) * Hand it off to the otherwise-unused claim taskq. */ ASSERT(zio->io_tqent.tqent_next == NULL); - (void) taskq_dispatch_ent( + taskq_dispatch_ent( spa->spa_zio_taskq[ZIO_TYPE_CLAIM][ZIO_TASKQ_ISSUE], (task_func_t *)zio_reexecute, zio, 0, &zio->io_tqent); From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:38:31 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3E2473C3; Thu, 8 Nov 2012 01:38:31 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 1F7E78FC0A; Thu, 8 Nov 2012 01:38:31 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81cUGF050024; Thu, 8 Nov 2012 01:38:31 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81cU8s050017; Thu, 8 Nov 2012 01:38:30 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080138.qA81cU8s050017@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:38:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242733 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/zpool vendor/illumos/dist/lib/libzp... X-SVN-Group: vendor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:38:31 -0000 Author: mm Date: Thu Nov 8 01:38:30 2012 New Revision: 242733 URL: http://svnweb.freebsd.org/changeset/base/242733 Log: Update vendor/illumos/dist and vendor/illumos-sys/dist to illumos-gate 13871:a9c12c2c1647 (zfs changes, illumos issues #3306, #3321) Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/lib/libzpool/common/kernel.c vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h vendor/illumos/dist/man/man1m/zdb.1m vendor/illumos/dist/man/man1m/zpool.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Thu Nov 8 01:36:06 2012 (r242732) +++ vendor/illumos/dist/cmd/zdb/zdb.c Thu Nov 8 01:38:30 2012 (r242733) @@ -86,6 +86,7 @@ extern void dump_intent_log(zilog_t *); uint64_t *zopt_object = NULL; int zopt_objects = 0; libzfs_handle_t *g_zfs; +uint64_t max_inflight = 200; /* * These libumem hooks provide a reasonable set of defaults for the allocator's @@ -108,13 +109,14 @@ usage(void) { (void) fprintf(stderr, "Usage: %s [-CumdibcsDvhLXFPA] [-t txg] [-e [-p path...]] " - "poolname [object...]\n" - " %s [-divPA] [-e -p path...] dataset [object...]\n" - " %s -m [-LXFPA] [-t txg] [-e [-p path...]] " + "[-U config] [-M inflight I/Os] poolname [object...]\n" + " %s [-divPA] [-e -p path...] [-U config] dataset " + "[object...]\n" + " %s -m [-LXFPA] [-t txg] [-e [-p path...]] [-U config] " "poolname [vdev [metaslab...]]\n" " %s -R [-A] [-e [-p path...]] poolname " "vdev:offset:size[:flags]\n" - " %s -S [-PA] [-e [-p path...]] poolname\n" + " %s -S [-PA] [-e [-p path...]] [-U config] poolname\n" " %s -l [-uA] device\n" " %s -C [-A] [-U config]\n\n", cmdname, cmdname, cmdname, cmdname, cmdname, cmdname, cmdname); @@ -161,6 +163,8 @@ usage(void) (void) fprintf(stderr, " -P print numbers in parseable form\n"); (void) fprintf(stderr, " -t -- highest txg to use when " "searching for uberblocks\n"); + (void) fprintf(stderr, " -M -- " + "specify the maximum number of checksumming I/Os [default is 200]"); (void) fprintf(stderr, "Specify an option more than once (e.g. -bb) " "to make only that option verbose\n"); (void) fprintf(stderr, "Default is to dump everything non-verbosely\n"); @@ -2028,6 +2032,45 @@ zdb_count_block(zdb_cb_t *zcb, zilog_t * bp, NULL, NULL, ZIO_FLAG_CANFAIL)), ==, 0); } +static void +zdb_blkptr_done(zio_t *zio) +{ + spa_t *spa = zio->io_spa; + blkptr_t *bp = zio->io_bp; + int ioerr = zio->io_error; + zdb_cb_t *zcb = zio->io_private; + zbookmark_t *zb = &zio->io_bookmark; + + zio_data_buf_free(zio->io_data, zio->io_size); + + mutex_enter(&spa->spa_scrub_lock); + spa->spa_scrub_inflight--; + cv_broadcast(&spa->spa_scrub_io_cv); + + if (ioerr && !(zio->io_flags & ZIO_FLAG_SPECULATIVE)) { + char blkbuf[BP_SPRINTF_LEN]; + + zcb->zcb_haderrors = 1; + zcb->zcb_errors[ioerr]++; + + if (dump_opt['b'] >= 2) + sprintf_blkptr(blkbuf, bp); + else + blkbuf[0] = '\0'; + + (void) printf("zdb_blkptr_cb: " + "Got error %d reading " + "<%llu, %llu, %lld, %llx> %s -- skipping\n", + ioerr, + (u_longlong_t)zb->zb_objset, + (u_longlong_t)zb->zb_object, + (u_longlong_t)zb->zb_level, + (u_longlong_t)zb->zb_blkid, + blkbuf); + } + mutex_exit(&spa->spa_scrub_lock); +} + /* ARGSUSED */ static int zdb_blkptr_cb(spa_t *spa, zilog_t *zilog, const blkptr_t *bp, arc_buf_t *pbuf, @@ -2049,39 +2092,23 @@ zdb_blkptr_cb(spa_t *spa, zilog_t *zilog is_metadata = (BP_GET_LEVEL(bp) != 0 || DMU_OT_IS_METADATA(type)); if (dump_opt['c'] > 1 || (dump_opt['c'] && is_metadata)) { - int ioerr; size_t size = BP_GET_PSIZE(bp); - void *data = malloc(size); + void *data = zio_data_buf_alloc(size); int flags = ZIO_FLAG_CANFAIL | ZIO_FLAG_SCRUB | ZIO_FLAG_RAW; /* If it's an intent log block, failure is expected. */ if (zb->zb_level == ZB_ZIL_LEVEL) flags |= ZIO_FLAG_SPECULATIVE; - ioerr = zio_wait(zio_read(NULL, spa, bp, data, size, - NULL, NULL, ZIO_PRIORITY_ASYNC_READ, flags, zb)); + mutex_enter(&spa->spa_scrub_lock); + while (spa->spa_scrub_inflight > max_inflight) + cv_wait(&spa->spa_scrub_io_cv, &spa->spa_scrub_lock); + spa->spa_scrub_inflight++; + mutex_exit(&spa->spa_scrub_lock); - free(data); + zio_nowait(zio_read(NULL, spa, bp, data, size, + zdb_blkptr_done, zcb, ZIO_PRIORITY_ASYNC_READ, flags, zb)); - if (ioerr && !(flags & ZIO_FLAG_SPECULATIVE)) { - zcb->zcb_haderrors = 1; - zcb->zcb_errors[ioerr]++; - - if (dump_opt['b'] >= 2) - sprintf_blkptr(blkbuf, bp); - else - blkbuf[0] = '\0'; - - (void) printf("zdb_blkptr_cb: " - "Got error %d reading " - "<%llu, %llu, %lld, %llx> %s -- skipping\n", - ioerr, - (u_longlong_t)zb->zb_objset, - (u_longlong_t)zb->zb_object, - (u_longlong_t)zb->zb_level, - (u_longlong_t)zb->zb_blkid, - blkbuf); - } } zcb->zcb_readfails = 0; @@ -2283,6 +2310,18 @@ dump_block_stats(spa_t *spa) zcb.zcb_haderrors |= traverse_pool(spa, 0, flags, zdb_blkptr_cb, &zcb); + /* + * If we've traversed the data blocks then we need to wait for those + * I/Os to complete. We leverage "The Godfather" zio to wait on + * all async I/Os to complete. + */ + if (dump_opt['c']) { + (void) zio_wait(spa->spa_async_zio_root); + spa->spa_async_zio_root = zio_root(spa, NULL, NULL, + ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE | + ZIO_FLAG_GODFATHER); + } + if (zcb.zcb_haderrors) { (void) printf("\nError counts:\n\n"); (void) printf("\t%5s %s\n", "errno", "count"); @@ -3040,7 +3079,7 @@ main(int argc, char **argv) dprintf_setup(&argc, argv); - while ((c = getopt(argc, argv, "bcdhilmsuCDRSAFLXevp:t:U:P")) != -1) { + while ((c = getopt(argc, argv, "bcdhilmM:suCDRSAFLXevp:t:U:P")) != -1) { switch (c) { case 'b': case 'c': @@ -3069,6 +3108,15 @@ main(int argc, char **argv) case 'v': verbose++; break; + case 'M': + max_inflight = strtoull(optarg, NULL, 0); + if (max_inflight == 0) { + (void) fprintf(stderr, "maximum number " + "of inflight I/Os must be greater " + "than 0\n"); + usage(); + } + break; case 'p': if (searchdirs == NULL) { searchdirs = umem_alloc(sizeof (char *), Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Thu Nov 8 01:36:06 2012 (r242732) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Thu Nov 8 01:38:30 2012 (r242733) @@ -238,7 +238,7 @@ get_usage(zpool_help_t idx) { case HELP_REMOVE: return (gettext("\tremove ...\n")); case HELP_REOPEN: - return (""); /* Undocumented command */ + return (gettext("\treopen \n")); case HELP_SCRUB: return (gettext("\tscrub [-s] ...\n")); case HELP_STATUS: @@ -3550,22 +3550,37 @@ zpool_do_reguid(int argc, char **argv) * zpool reopen * * Reopen the pool so that the kernel can update the sizes of all vdevs. - * - * NOTE: This command is currently undocumented. If the command is ever - * exposed then the appropriate usage() messages will need to be made. */ int zpool_do_reopen(int argc, char **argv) { + int c; int ret = 0; zpool_handle_t *zhp; char *pool; + /* check options */ + while ((c = getopt(argc, argv, "")) != -1) { + switch (c) { + case '?': + (void) fprintf(stderr, gettext("invalid option '%c'\n"), + optopt); + usage(B_FALSE); + } + } + argc--; argv++; - if (argc != 1) - return (2); + if (argc < 1) { + (void) fprintf(stderr, gettext("missing pool name\n")); + usage(B_FALSE); + } + + if (argc > 1) { + (void) fprintf(stderr, gettext("too many arguments\n")); + usage(B_FALSE); + } pool = argv[0]; if ((zhp = zpool_open_canfail(g_zfs, pool)) == NULL) Modified: vendor/illumos/dist/lib/libzpool/common/kernel.c ============================================================================== --- vendor/illumos/dist/lib/libzpool/common/kernel.c Thu Nov 8 01:36:06 2012 (r242732) +++ vendor/illumos/dist/lib/libzpool/common/kernel.c Thu Nov 8 01:38:30 2012 (r242733) @@ -1012,3 +1012,48 @@ zfs_onexit_cb_data(minor_t minor, uint64 { return (0); } + +void +bioinit(buf_t *bp) +{ + bzero(bp, sizeof (buf_t)); +} + +void +biodone(buf_t *bp) +{ + if (bp->b_iodone != NULL) { + (*(bp->b_iodone))(bp); + return; + } + ASSERT((bp->b_flags & B_DONE) == 0); + bp->b_flags |= B_DONE; +} + +void +bioerror(buf_t *bp, int error) +{ + ASSERT(bp != NULL); + ASSERT(error >= 0); + + if (error != 0) { + bp->b_flags |= B_ERROR; + } else { + bp->b_flags &= ~B_ERROR; + } + bp->b_error = error; +} + + +int +geterror(struct buf *bp) +{ + int error = 0; + + if (bp->b_flags & B_ERROR) { + error = bp->b_error; + if (!error) + error = EIO; + } + return (error); +} Modified: vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h ============================================================================== --- vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Thu Nov 8 01:36:06 2012 (r242732) +++ vendor/illumos/dist/lib/libzpool/common/sys/zfs_context.h Thu Nov 8 01:38:30 2012 (r242733) @@ -603,6 +603,36 @@ extern cyclic_id_t cyclic_add(cyc_handle extern void cyclic_remove(cyclic_id_t); extern int cyclic_reprogram(cyclic_id_t, hrtime_t); +/* + * Buf structure + */ +#define B_BUSY 0x0001 +#define B_DONE 0x0002 +#define B_ERROR 0x0004 +#define B_READ 0x0040 /* read when I/O occurs */ +#define B_WRITE 0x0100 /* non-read pseudo-flag */ + +typedef struct buf { + int b_flags; + size_t b_bcount; + union { + caddr_t b_addr; + } b_un; + + lldaddr_t _b_blkno; +#define b_lblkno _b_blkno._f + size_t b_resid; + size_t b_bufsize; + int (*b_iodone)(struct buf *); + int b_error; + void *b_private; +} buf_t; + +extern void bioinit(buf_t *); +extern void biodone(buf_t *); +extern void bioerror(buf_t *, int); +extern int geterror(buf_t *); + #ifdef __cplusplus } #endif Modified: vendor/illumos/dist/man/man1m/zdb.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zdb.1m Thu Nov 8 01:36:06 2012 (r242732) +++ vendor/illumos/dist/man/man1m/zdb.1m Thu Nov 8 01:38:30 2012 (r242733) @@ -11,6 +11,7 @@ .\" .\" .\" Copyright 2012, Richard Lowe. +.\" Copyright (c) 2012 by Delphix. All rights reserved. .\" .TH "ZDB" "1M" "February 15, 2012" "" "" @@ -19,21 +20,23 @@ .SH "SYNOPSIS" \fBzdb\fR [-CumdibcsDvhLXFPA] [-e [-p \fIpath\fR...]] [-t \fItxg\fR] - \fIpoolname\fR [\fIobject\fR ...] + [-U \fIcache\fR] [-M \fIinflight I/Os\fR] [\fIpoolname\fR + [\fIobject\fR ...]] .P -\fBzdb\fR [-divPA] [-e [-p \fIpath\fR...]] \fIdataset\fR [\fIobject\fR ...] +\fBzdb\fR [-divPA] [-e [-p \fIpath\fR...]] [-U \fIcache\fR] + \fIdataset\fR [\fIobject\fR ...] .P -\fBzdb\fR -m [-LXFPA] [-t \fItxg\fR] [-e [-p \fIpath\fR...]] \fIpoolname\fR - [\fIvdev\fR [\fImetaslab\fR ...]] +\fBzdb\fR -m [-LXFPA] [-t \fItxg\fR] [-e [-p \fIpath\fR...]] [-U \fIcache\fR] + \fIpoolname\fR [\fIvdev\fR [\fImetaslab\fR ...]] .P -\fBzdb\fR -R [-A] [-e [-p \fIpath\fR...]] \fIpoolname\fR +\fBzdb\fR -R [-A] [-e [-p \fIpath\fR...]] [-U \fIcache\fR] \fIpoolname\fR \fIvdev\fR:\fIoffset\fR:\fIsize\fR[:\fIflags\fR] .P -\fBzdb\fR -S [-AP] [-e [-p \fIpath\fR...]] \fIpoolname\fR +\fBzdb\fR -S [-AP] [-e [-p \fIpath\fR...]] [-U \fIcache\fR] \fIpoolname\fR .P \fBzdb\fR -l [-uA] \fIdevice\fR @@ -357,6 +360,18 @@ transactions. .sp .ne 2 .na +\fB-M \fIinflight I/Os\fR \fR +.ad +.sp .6 +.RS 4n +Limit the number of outstanding checksum I/Os to the specified value. The +default value is 200. This option affects the performance of the \fB-c\fR +option. +.RE + +.sp +.ne 2 +.na \fB-P\fR .ad .sp .6 @@ -384,8 +399,7 @@ and their associated transaction numbers .ad .sp .6 .RS 4n -Use a cache file other than \fB/etc/zfs/zpool.cache\fR. This option is only -valid with \fB-C\fR +Use a cache file other than \fB/etc/zfs/zpool.cache\fR. .RE .sp Modified: vendor/illumos/dist/man/man1m/zpool.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zpool.1m Thu Nov 8 01:36:06 2012 (r242732) +++ vendor/illumos/dist/man/man1m/zpool.1m Thu Nov 8 01:38:30 2012 (r242733) @@ -113,6 +113,11 @@ zpool \- configures ZFS storage pools .LP .nf +\fBzpool reopen\fR \fIpool\fR +.fi + +.LP +.nf \fBzpool remove\fR \fIpool\fR \fIdevice\fR ... .fi @@ -1550,8 +1555,18 @@ become available to the pool. .ad .sp .6 .RS 4n -Generates a new unique identifier for the pool. You must ensure that all devices in this pool are online and -healthy before performing this action. +Generates a new unique identifier for the pool. You must ensure that all +devices in this pool are online and healthy before performing this action. +.RE + +.sp +.ne 2 +.na +\fB\fBzpool reopen\fR \fIpool\fR +.ad +.sp .6 +.RS 4n +Reopen all the vdevs associated with the pool. .RE .sp From owner-svn-src-vendor@FreeBSD.ORG Thu Nov 8 01:42:55 2012 Return-Path: Delivered-To: svn-src-vendor@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 41271853; Thu, 8 Nov 2012 01:42:55 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 256818FC08; Thu, 8 Nov 2012 01:42:55 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA81gt0s050832; Thu, 8 Nov 2012 01:42:55 GMT (envelope-from mm@svn.freebsd.org) Received: (from mm@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA81gsIj050822; Thu, 8 Nov 2012 01:42:54 GMT (envelope-from mm@svn.freebsd.org) Message-Id: <201211080142.qA81gsIj050822@svn.freebsd.org> From: Martin Matuska Date: Thu, 8 Nov 2012 01:42:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r242735 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-vendor@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the vendor work area tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 01:42:55 -0000 Author: mm Date: Thu Nov 8 01:42:54 2012 New Revision: 242735 URL: http://svnweb.freebsd.org/changeset/base/242735 Log: Update vendor-sys/illumos/dist to illumos-gate 13879:4eac7a87eff2 (illumos ZFS issues #3329, #3330, #3331, #3335) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Thu Nov 8 01:42:54 2012 (r242735) @@ -875,8 +875,9 @@ metaslab_activate(metaslab_t *msp, uint6 if ((msp->ms_weight & METASLAB_ACTIVE_MASK) == 0) { space_map_load_wait(sm); if (!sm->sm_loaded) { - int error = space_map_load(sm, sm_ops, SM_FREE, - &msp->ms_smo, + space_map_obj_t *smo = &msp->ms_smo; + + int error = space_map_load(sm, sm_ops, SM_FREE, smo, spa_meta_objset(msp->ms_group->mg_vd->vdev_spa)); if (error) { metaslab_group_sort(msp->ms_group, msp, 0); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Thu Nov 8 01:42:54 2012 (r242735) @@ -130,6 +130,7 @@ boolean_t zio_taskq_sysdc = B_TRUE; /* u uint_t zio_taskq_basedc = 80; /* base duty cycle */ boolean_t spa_create_process = B_TRUE; /* no process ==> no sysdc */ +extern int zfs_sync_pass_deferred_free; /* * This (illegal) pool name is used when temporarily importing a spa_t in order @@ -6040,7 +6041,7 @@ spa_sync(spa_t *spa, uint64_t txg) spa_errlog_sync(spa, txg); dsl_pool_sync(dp, txg); - if (pass <= SYNC_PASS_DEFERRED_FREE) { + if (pass < zfs_sync_pass_deferred_free) { zio_t *zio = zio_root(spa, NULL, NULL, 0); bplist_iterate(free_bpl, spa_free_sync_cb, zio, tx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Thu Nov 8 01:42:54 2012 (r242735) @@ -1687,6 +1687,7 @@ spa_init(int mode) refcount_init(); unique_init(); + space_map_init(); zio_init(); dmu_init(); zil_init(); @@ -1709,6 +1710,7 @@ spa_fini(void) zil_fini(); dmu_fini(); zio_fini(); + space_map_fini(); unique_fini(); refcount_fini(); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Thu Nov 8 01:42:54 2012 (r242735) @@ -32,6 +32,23 @@ #include #include +static kmem_cache_t *space_seg_cache; + +void +space_map_init(void) +{ + ASSERT(space_seg_cache == NULL); + space_seg_cache = kmem_cache_create("space_seg_cache", + sizeof (space_seg_t), 0, NULL, NULL, NULL, NULL, NULL, 0); +} + +void +space_map_fini(void) +{ + kmem_cache_destroy(space_seg_cache); + space_seg_cache = NULL; +} + /* * Space map routines. * NOTE: caller is responsible for all locking. @@ -124,7 +141,7 @@ space_map_add(space_map_t *sm, uint64_t avl_remove(sm->sm_pp_root, ss_after); } ss_after->ss_start = ss_before->ss_start; - kmem_free(ss_before, sizeof (*ss_before)); + kmem_cache_free(space_seg_cache, ss_before); ss = ss_after; } else if (merge_before) { ss_before->ss_end = end; @@ -137,7 +154,7 @@ space_map_add(space_map_t *sm, uint64_t avl_remove(sm->sm_pp_root, ss_after); ss = ss_after; } else { - ss = kmem_alloc(sizeof (*ss), KM_SLEEP); + ss = kmem_cache_alloc(space_seg_cache, KM_SLEEP); ss->ss_start = start; ss->ss_end = end; avl_insert(&sm->sm_root, ss, where); @@ -184,7 +201,7 @@ space_map_remove(space_map_t *sm, uint64 avl_remove(sm->sm_pp_root, ss); if (left_over && right_over) { - newseg = kmem_alloc(sizeof (*newseg), KM_SLEEP); + newseg = kmem_cache_alloc(space_seg_cache, KM_SLEEP); newseg->ss_start = end; newseg->ss_end = ss->ss_end; ss->ss_end = start; @@ -197,7 +214,7 @@ space_map_remove(space_map_t *sm, uint64 ss->ss_start = end; } else { avl_remove(&sm->sm_root, ss); - kmem_free(ss, sizeof (*ss)); + kmem_cache_free(space_seg_cache, ss); ss = NULL; } @@ -237,7 +254,7 @@ space_map_vacate(space_map_t *sm, space_ while ((ss = avl_destroy_nodes(&sm->sm_root, &cookie)) != NULL) { if (func != NULL) func(mdest, ss->ss_start, ss->ss_end - ss->ss_start); - kmem_free(ss, sizeof (*ss)); + kmem_cache_free(space_seg_cache, ss); } sm->sm_space = 0; } @@ -409,7 +426,7 @@ space_map_sync(space_map_t *sm, uint8_t spa_t *spa = dmu_objset_spa(os); void *cookie = NULL; space_seg_t *ss; - uint64_t bufsize, start, size, run_len; + uint64_t bufsize, start, size, run_len, delta, sm_space; uint64_t *entry, *entry_map, *entry_map_end; ASSERT(MUTEX_HELD(sm->sm_lock)); @@ -438,11 +455,13 @@ space_map_sync(space_map_t *sm, uint8_t SM_DEBUG_SYNCPASS_ENCODE(spa_sync_pass(spa)) | SM_DEBUG_TXG_ENCODE(dmu_tx_get_txg(tx)); + delta = 0; + sm_space = sm->sm_space; while ((ss = avl_destroy_nodes(&sm->sm_root, &cookie)) != NULL) { size = ss->ss_end - ss->ss_start; start = (ss->ss_start - sm->sm_start) >> sm->sm_shift; - sm->sm_space -= size; + delta += size; size >>= sm->sm_shift; while (size) { @@ -464,7 +483,7 @@ space_map_sync(space_map_t *sm, uint8_t start += run_len; size -= run_len; } - kmem_free(ss, sizeof (*ss)); + kmem_cache_free(space_seg_cache, ss); } if (entry != entry_map) { @@ -476,8 +495,15 @@ space_map_sync(space_map_t *sm, uint8_t smo->smo_objsize += size; } + /* + * Ensure that the space_map's accounting wasn't changed + * while we were in the middle of writing it out. + */ + VERIFY3U(sm->sm_space, ==, sm_space); + zio_buf_free(entry_map, bufsize); + sm->sm_space -= delta; VERIFY0(sm->sm_space); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h Thu Nov 8 01:42:54 2012 (r242735) @@ -21,7 +21,10 @@ /* * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. - * Copyright (c) 2011 by Delphix. All rights reserved. + */ + +/* + * Copyright (c) 2012 by Delphix. All rights reserved. */ #ifndef _SYS_METASLAB_IMPL_H Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Thu Nov 8 01:42:54 2012 (r242735) @@ -486,14 +486,6 @@ extern int spa_scan_stop(spa_t *spa); extern void spa_sync(spa_t *spa, uint64_t txg); /* only for DMU use */ extern void spa_sync_allpools(void); -/* - * DEFERRED_FREE must be large enough that regular blocks are not - * deferred. XXX so can't we change it back to 1? - */ -#define SYNC_PASS_DEFERRED_FREE 2 /* defer frees after this pass */ -#define SYNC_PASS_DONT_COMPRESS 4 /* don't compress after this pass */ -#define SYNC_PASS_REWRITE 1 /* rewrite new bps after this pass */ - /* spa namespace global mutex */ extern kmutex_t spa_namespace_lock; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h Thu Nov 8 01:42:54 2012 (r242735) @@ -23,6 +23,10 @@ * Use is subject to license terms. */ +/* + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + #ifndef _SYS_SPACE_MAP_H #define _SYS_SPACE_MAP_H @@ -136,6 +140,8 @@ struct space_map_ops { typedef void space_map_func_t(space_map_t *sm, uint64_t start, uint64_t size); +extern void space_map_init(void); +extern void space_map_fini(void); extern void space_map_create(space_map_t *sm, uint64_t start, uint64_t size, uint8_t shift, kmutex_t *lp); extern void space_map_destroy(space_map_t *sm); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_impl.h Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_impl.h Thu Nov 8 01:42:54 2012 (r242735) @@ -23,6 +23,10 @@ * Use is subject to license terms. */ +/* + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + #ifndef _ZIO_IMPL_H #define _ZIO_IMPL_H @@ -143,6 +147,7 @@ enum zio_stage { #define ZIO_FREE_PIPELINE \ (ZIO_INTERLOCK_STAGES | \ ZIO_STAGE_FREE_BP_INIT | \ + ZIO_STAGE_ISSUE_ASYNC | \ ZIO_STAGE_DVA_FREE) #define ZIO_DDT_FREE_PIPELINE \ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Thu Nov 8 01:41:04 2012 (r242734) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Thu Nov 8 01:42:54 2012 (r242735) @@ -83,6 +83,22 @@ extern vmem_t *zio_alloc_arena; extern int zfs_mg_alloc_failures; /* + * The following actions directly effect the spa's sync-to-convergence logic. + * The values below define the sync pass when we start performing the action. + * Care should be taken when changing these values as they directly impact + * spa_sync() performance. Tuning these values may introduce subtle performance + * pathologies and should only be done in the context of performance analysis. + * These tunables will eventually be removed and replaced with #defines once + * enough analysis has been done to determine optimal values. + * + * The 'zfs_sync_pass_deferred_free' pass must be greater than 1 to ensure that + * regular blocks are not deferred. + */ +int zfs_sync_pass_deferred_free = 2; /* defer frees starting in this pass */ +int zfs_sync_pass_dont_compress = 5; /* don't compress starting in this pass */ +int zfs_sync_pass_rewrite = 2; /* rewrite new bps starting in this pass */ + +/* * An allocating zio is one that either currently has the DVA allocate * stage set or will have it later in its lifetime. */ @@ -690,7 +706,7 @@ zio_free_sync(zio_t *pio, spa_t *spa, ui ASSERT(!BP_IS_HOLE(bp)); ASSERT(spa_syncing_txg(spa) == txg); - ASSERT(spa_sync_pass(spa) <= SYNC_PASS_DEFERRED_FREE); + ASSERT(spa_sync_pass(spa) < zfs_sync_pass_deferred_free); zio = zio_create(pio, spa, txg, bp, NULL, BP_GET_PSIZE(bp), NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_FREE, flags, @@ -987,7 +1003,7 @@ zio_write_bp_init(zio_t *zio) ASSERT(zio->io_child_type == ZIO_CHILD_LOGICAL); ASSERT(!BP_GET_DEDUP(bp)); - if (pass > SYNC_PASS_DONT_COMPRESS) + if (pass >= zfs_sync_pass_dont_compress) compress = ZIO_COMPRESS_OFF; /* Make sure someone doesn't change their mind on overwrites */ @@ -1016,7 +1032,7 @@ zio_write_bp_init(zio_t *zio) * There should only be a handful of blocks after pass 1 in any case. */ if (bp->blk_birth == zio->io_txg && BP_GET_PSIZE(bp) == psize && - pass > SYNC_PASS_REWRITE) { + pass >= zfs_sync_pass_rewrite) { ASSERT(psize != 0); enum zio_stage gang_stages = zio->io_pipeline & ZIO_GANG_STAGES; zio->io_pipeline = ZIO_REWRITE_PIPELINE | gang_stages;