From owner-svn-src-all@FreeBSD.ORG Fri Mar 25 20:51:20 2011 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C858A106566C; Fri, 25 Mar 2011 20:51:20 +0000 (UTC) (envelope-from mdf@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id B48058FC16; Fri, 25 Mar 2011 20:51:20 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id p2PKpKaJ093170; Fri, 25 Mar 2011 20:51:20 GMT (envelope-from mdf@svn.freebsd.org) Received: (from mdf@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id p2PKpKmX093162; Fri, 25 Mar 2011 20:51:20 GMT (envelope-from mdf@svn.freebsd.org) Message-Id: <201103252051.p2PKpKmX093162@svn.freebsd.org> From: Matthew D Fleming Date: Fri, 25 Mar 2011 20:51:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-7@freebsd.org X-SVN-Group: stable-7 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r220008 - in stable/7: share/man/man9 sys/conf sys/kern sys/sys X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Mar 2011 20:51:20 -0000 Author: mdf Date: Fri Mar 25 20:51:20 2011 New Revision: 220008 URL: http://svn.freebsd.org/changeset/base/220008 Log: MFC r192908 (introduced before the stable/8 split from CURRENT): fail(9) support: Add support for kernel fault injection using KFAIL_POINT_* macros and fail_point_* infrastructure. Add example fail point in vfs_bio.c to simulate VM buf pressure. MFC r192978: Fix style/grammar issues in fail(9) man page. MFC r216616: Move the fail_point_entry definition from fail.h to kern_fail.c, which allows putting the enumeration constants of fail point types with the text string that matches them. MFC r216620: Initialize fp_location for explicitly managed fail points, and push the parentheses around the location for simple fail points into the location string. This makes the print on fail point set more consistent between the two versions. Also fix up fail.h a little for style(9): only use one of sys/param.h and sys/types.h, and use the existing __XSTRING() macro instead of rolling our own. Also fix up a few tabs on changed and nearby lines. Lastly, since KFAIL_POINT_{BEGIN,END} are not meant for use outside this file, just eliminate the macros entirely. Added: stable/7/share/man/man9/fail.9 - copied, changed from r192908, head/share/man/man9/fail.9 stable/7/sys/kern/kern_fail.c - copied, changed from r192908, head/sys/kern/kern_fail.c stable/7/sys/sys/fail.h - copied, changed from r192908, head/sys/sys/fail.h Modified: stable/7/share/man/man9/Makefile stable/7/sys/conf/files stable/7/sys/kern/vfs_bio.c stable/7/sys/sys/queue.h Directory Properties: stable/7/share/man/man9/ (props changed) stable/7/sys/ (props changed) stable/7/sys/cddl/contrib/opensolaris/ (props changed) stable/7/sys/contrib/dev/acpica/ (props changed) stable/7/sys/contrib/pf/ (props changed) Modified: stable/7/share/man/man9/Makefile ============================================================================== --- stable/7/share/man/man9/Makefile Fri Mar 25 20:19:15 2011 (r220007) +++ stable/7/share/man/man9/Makefile Fri Mar 25 20:51:20 2011 (r220008) @@ -97,6 +97,7 @@ MAN= accept_filter.9 \ DRIVER_MODULE.9 \ EVENTHANDLER.9 \ extattr.9 \ + fail.9 \ fetch.9 \ firmware.9 \ g_access.9 \ Copied and modified: stable/7/share/man/man9/fail.9 (from r192908, head/share/man/man9/fail.9) ============================================================================== --- head/share/man/man9/fail.9 Wed May 27 16:36:54 2009 (r192908, copy source) +++ stable/7/share/man/man9/fail.9 Fri Mar 25 20:51:20 2011 (r220008) @@ -48,9 +48,11 @@ .Fn KFAIL_POINT_GOTO "parent" "name" "error_var" "label" .Sh DESCRIPTION Fail points are used to add code points where errors may be injected -in a user controlled fashion. Fail points provide a convenient wrapper -around user provided error injection code, providing a -.Xr sysctl 9 MIB , and a parser for that MIB that describes how the error +in a user controlled fashion. +Fail points provide a convenient wrapper around user-provided error +injection code, providing a +.Xr sysctl 9 +MIB, and a parser for that MIB that describes how the error injection code should fire. .Pp The base fail point macro is @@ -64,16 +66,19 @@ their own fail point trees), and .Fa name is the name of the MIB in that tree, and .Fa code -is the error injection code. The +is the error injection code. +The .Fa code argument does not require braces, but it is considered good style to -use braces for any multi-line code arguments. Inside the +use braces for any multi-line code arguments. +Inside the .Fa code argument, the evaluation of .Sy RETURN_VALUE is derived from the .Fn return -value set in the sysctl MIB. See +value set in the sysctl MIB. +See .Sx SYSCTL SETTINGS below. .Pp @@ -99,14 +104,14 @@ is the equivalent of .Sh SYSCTL VARIABLES The .Fn KFAIL_POINT_* -macros add sysctl MIBs where specified. Many base kernel MIBs can be -found in the +macros add sysctl MIBs where specified. +Many base kernel MIBs can be found in the .Sy debug.fail_point tree (referenced in code by .Sy DEBUG_FP ). .Pp -The sysctl setting recognizes the following grammar: +The sysctl variable may be set using the following grammar: .Pp :: ( "->" )* @@ -135,27 +140,30 @@ Sleep the specified number of millisecon .It Sy panic Panic .It Sy break -Break into the debugger. +Break into the debugger, or trap if there is no debugger support .It Sy print Print that the fail point executed .El .Pp The % and * modifiers prior to control when - is executed. The % form (e.g. "1.2%") can be used to -specify a probability that will execute. The * form -(e.g. "5*") can be used to specify the number of times should -be executed before this is disabled. Only the last probability -and the last count are used if multiple are specified, i.e. "1.2%2%" -is the same as "2%". When both a probability and a count are -specified, the probability is evaluated before the count, i.e. "2%5*" -means "2% of the time, but only execute it 5 times total". -.Pp -The operator -> can be used to express cascading terms. If you specify -->, it means that if doesn't 'execute', -is evaluated. For the purpose of this operator, the return() and -print() operators are the only types that cascade. A return() term -only cascades if the code executes, and a print() term only cascades -when passed a non-zero argument. + is executed. +The % form (e.g. "1.2%") can be used to specify a +probability that will execute. +The * form (e.g. "5*") can be used to specify the number of +times should be executed before this is disabled. +Only the last probability and the last count are used if multiple +are specified, i.e. "1.2%2%" is the same as "2%". +When both a probability and a count are specified, the probability +is evaluated before the count, i.e. "2%5*" means "2% of the time, +but only 5 times total". +.Pp +The operator -> can be used to express cascading terms. +If you specify ->, it means that if doesn't +'execute', is evaluated. +For the purpose of this operator, the return() and print() operators +are the only types that cascade. +A return() term only cascades if the code executes, and a print() +term only cascades when passed a non-zero argument. .Pp .Sh EXAMPLES .Bl -tag @@ -164,29 +172,32 @@ when passed a non-zero argument. .Fa code with RETURN_VALUE set to 5. .It Sy sysctl debug.fail_point.foobar="2%return(5)->5%return(22)" -2/100th of the time, execute +2/100ths of the time, execute .Fa code -with RETURN_VALUE set to 5. If that doesn't happen, 5% of the time -execute +with RETURN_VALUE set to 5. +If that doesn't happen, 5% of the time execute .Fa code with RETURN_VALUE set to 22. .It Sy sysctl debug.fail_point.foobar="5*return(5)->0.1%return(22)" -For 5 times, return 5. After that, 1/1000ths of the time, return 22. +For 5 times, return 5. +After that, 1/1000th of the time, return 22. .It Sy sysctl debug.fail_point.foobar="0.1%5*return(5)" -Return 5 for 1 in 1000 executions, but only execute 5 times total. +Return 5 for 1 in 1000 executions, but only 5 times total. .It Sy sysctl debug.fail_point.foobar="1%*sleep(50)" -1/100ths of the time, sleep 50ms. +1/100th of the time, sleep 50ms. .El .Pp .Sh CAVEATS It's easy to shoot yourself in the foot by setting fail points too -aggressively or setting too many in combination. For example, forcing +aggressively or setting too many in combination. +For example, forcing .Fn malloc to fail consistently is potentially harmful to uptime. .Pp The .Fn sleep -sysctl setting may not be appropriate in all situations. Currently, +sysctl setting may not be appropriate in all situations. +Currently, .Fn fail_point_eval does not verify whether the context is appropriate for calling .Fn msleep . Modified: stable/7/sys/conf/files ============================================================================== --- stable/7/sys/conf/files Fri Mar 25 20:19:15 2011 (r220007) +++ stable/7/sys/conf/files Fri Mar 25 20:51:20 2011 (r220008) @@ -1618,6 +1618,7 @@ kern/kern_environment.c standard kern/kern_event.c standard kern/kern_exec.c standard kern/kern_exit.c standard +kern/kern_fail.c standard kern/kern_fork.c standard kern/kern_idle.c standard kern/kern_intr.c standard Copied and modified: stable/7/sys/kern/kern_fail.c (from r192908, head/sys/kern/kern_fail.c) ============================================================================== --- head/sys/kern/kern_fail.c Wed May 27 16:36:54 2009 (r192908, copy source) +++ stable/7/sys/kern/kern_fail.c Fri Mar 25 20:51:20 2011 (r220008) @@ -76,6 +76,43 @@ MTX_SYSINIT(g_fp_mtx, &g_fp_mtx, "fail p #define FP_LOCK() mtx_lock(&g_fp_mtx) #define FP_UNLOCK() mtx_unlock(&g_fp_mtx) +/** + * Failpoint types. + * Don't change these without changing fail_type_strings in fail.c. + * @ingroup failpoint_private + */ +enum fail_point_t { + FAIL_POINT_OFF, /**< don't fail */ + FAIL_POINT_PANIC, /**< panic */ + FAIL_POINT_RETURN, /**< return an errorcode */ + FAIL_POINT_BREAK, /**< break into the debugger */ + FAIL_POINT_PRINT, /**< print a message */ + FAIL_POINT_SLEEP, /**< sleep for some msecs */ + FAIL_POINT_INVALID, /**< placeholder */ +}; + +static const char *fail_type_strings[] = { + "off", + "panic", + "return", + "break", + "print", + "sleep", +}; + +/** + * Internal structure tracking a single term of a complete failpoint. + * @ingroup failpoint_private + */ +struct fail_point_entry { + enum fail_point_t fe_type; /**< type of entry */ + int fe_arg; /**< argument to type (e.g. return value) */ + int fe_prob; /**< likelihood of firing in millionths */ + int fe_count; /**< number of times to fire, 0 means always */ + + TAILQ_ENTRY(fail_point_entry) fe_entries; /**< next entry in fail point */ +}; + static inline void fail_point_sleep(struct fail_point *fp, struct fail_point_entry *ent, int msecs, enum fail_point_return_code *pret) @@ -102,15 +139,6 @@ enum { PROB_DIGITS = 6, /* number of zero's in above number */ }; -static const char *fail_type_strings[] = { - "off", - "panic", - "return", - "break", - "print", - "sleep", -}; - static char *parse_fail_point(struct fail_point_entries *, char *); static char *parse_term(struct fail_point_entries *, char *); static char *parse_number(int *out_units, int *out_decimal, char *); @@ -149,6 +177,7 @@ fail_point_init(struct fail_point *fp, c va_end(ap); } fp->fp_name = name; + fp->fp_location = ""; fp->fp_flags |= FAIL_POINT_DYNAMIC_NAME; fp->fp_sleep_fn = NULL; fp->fp_sleep_arg = NULL; @@ -344,10 +373,10 @@ fail_point_set(struct fail_point *fp, ch end: #ifdef IWARNING if (error) - IWARNING("Failed to set %s (%s) to %s", + IWARNING("Failed to set %s %s to %s", fp->fp_name, fp->fp_location, buf); else - INOTICE("Set %s (%s) to %s", + INOTICE("Set %s %s to %s", fp->fp_name, fp->fp_location, buf); #endif /* IWARNING */ Modified: stable/7/sys/kern/vfs_bio.c ============================================================================== --- stable/7/sys/kern/vfs_bio.c Fri Mar 25 20:19:15 2011 (r220007) +++ stable/7/sys/kern/vfs_bio.c Fri Mar 25 20:51:20 2011 (r220008) @@ -48,6 +48,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -1180,6 +1181,15 @@ buf_dirty_count_severe(void) return(numdirtybuffers >= hidirtybuffers); } +static __noinline int +buf_vm_page_count_severe(void) +{ + + KFAIL_POINT_CODE(DEBUG_FP, buf_pressure, return 1); + + return vm_page_count_severe(); +} + /* * brelse: * @@ -1252,7 +1262,7 @@ brelse(struct buf *bp) */ if (bp->b_flags & B_DELWRI) bp->b_flags &= ~B_RELBUF; - else if (vm_page_count_severe()) { + else if (buf_vm_page_count_severe()) { /* * XXX This lock may not be necessary since BKGRDINPROG * cannot be set while we hold the buf lock, it can only be @@ -1522,7 +1532,7 @@ bqrelse(struct buf *bp) * cleared if it is already pending. */ BO_LOCK(bp->b_bufobj); - if (!vm_page_count_severe() || bp->b_vflags & BV_BKGRDINPROG) { + if (!buf_vm_page_count_severe() || bp->b_vflags & BV_BKGRDINPROG) { BO_UNLOCK(bp->b_bufobj); bp->b_qindex = QUEUE_CLEAN; TAILQ_INSERT_TAIL(&bufqueues[QUEUE_CLEAN], bp, @@ -1593,7 +1603,7 @@ vfs_vmio_release(struct buf *bp) vm_page_free(m); } else if (bp->b_flags & B_DIRECT) { vm_page_try_to_free(m); - } else if (vm_page_count_severe()) { + } else if (buf_vm_page_count_severe()) { vm_page_try_to_cache(m); } } Copied and modified: stable/7/sys/sys/fail.h (from r192908, head/sys/sys/fail.h) ============================================================================== --- head/sys/sys/fail.h Wed May 27 16:36:54 2009 (r192908, copy source) +++ stable/7/sys/sys/fail.h Fri Mar 25 20:51:20 2011 (r220008) @@ -32,29 +32,12 @@ #ifndef _SYS_FAIL_H_ #define _SYS_FAIL_H_ -#include - -#include #include +#include +#include #include #include - -/** - * Failpoint types. - * Don't change these without changing fail_type_strings in fail.c. - * @ingroup failpoint_private - */ -enum fail_point_t { - FAIL_POINT_OFF, /**< don't fail */ - FAIL_POINT_PANIC, /**< panic */ - FAIL_POINT_RETURN, /**< return an errorcode */ - FAIL_POINT_BREAK, /**< break into the debugger */ - FAIL_POINT_PRINT, /**< print a message */ - FAIL_POINT_SLEEP, /**< sleep for some msecs */ - FAIL_POINT_INVALID, /**< placeholder */ -}; - /** * Failpoint return codes, used internally. * @ingroup failpoint_private @@ -65,6 +48,7 @@ enum fail_point_return_code { FAIL_POINT_RC_QUEUED, /**< sleep_fn will be called */ }; +struct fail_point_entry; TAILQ_HEAD(fail_point_entries, fail_point_entry); /** * Internal failpoint structure, tracking all the current details of the @@ -84,18 +68,7 @@ struct fail_point { #define FAIL_POINT_DYNAMIC_NAME 0x01 /**< Must free name on destroy */ -/** - * Internal structure tracking a single term of a complete failpoint. - * @ingroup failpoint_private - */ -struct fail_point_entry { - enum fail_point_t fe_type; /**< type of entry */ - int fe_arg; /**< argument to type (e.g. return value) */ - int fe_prob; /**< likelihood of firing in millionths */ - int fe_count; /**< number of times to fire, 0 means always */ - - TAILQ_ENTRY(fail_point_entry) fe_entries; /**< next entry in fail point */ -}; +__BEGIN_DECLS /* Private failpoint eval function -- use fail_point_eval() instead. */ enum fail_point_return_code fail_point_eval_nontrivial(struct fail_point *, @@ -152,11 +125,11 @@ fail_point_eval(struct fail_point *fp, i return (fail_point_eval_nontrivial(fp, ret)); } +__END_DECLS + /* Declare a fail_point and its sysctl in a function. */ -#define _FAIL_POINT_NAME(name) _fail_point_##name -#define _STRINGIFY_HELPER(x) #x -#define _STRINGIFY(x) _STRINGIFY_HELPER(x) -#define _FAIL_POINT_LOCATION() __FILE__ ":" _STRINGIFY(__LINE__) +#define _FAIL_POINT_NAME(name) _fail_point_##name +#define _FAIL_POINT_LOCATION() "(" __FILE__ ":" __XSTRING(__LINE__) ")" /** * Instantiate a failpoint which returns "value" from the function when triggered. @@ -202,59 +175,49 @@ fail_point_eval(struct fail_point *fp, i /** * Instantiate a failpoint which runs arbitrary code when triggered. * @param parent The parent sysctl under which to locate the sysctl - * @param name The name of the failpoint in the sysctl tree (and printouts) + * @param name The name of the failpoint in the sysctl tree + * (and printouts) * @param code The arbitrary code to run when triggered. Can reference - * "RETURN_VALUE" if desired to extract the specified user - * return-value when triggered + * "RETURN_VALUE" if desired to extract the specified + * user return-value when triggered. Note that this is + * implemented with a do-while loop so be careful of + * break and continue statements. */ #define KFAIL_POINT_CODE(parent, name, code) \ - KFAIL_POINT_START(parent, name) { \ +do { \ + int RETURN_VALUE; \ + static struct fail_point _FAIL_POINT_NAME(name) = { \ + #name, \ + _FAIL_POINT_LOCATION(), \ + TAILQ_HEAD_INITIALIZER(_FAIL_POINT_NAME(name).fp_entries), \ + 0, \ + NULL, NULL, \ + }; \ + SYSCTL_OID(parent, OID_AUTO, name, \ + CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_MPSAFE, \ + &_FAIL_POINT_NAME(name), 0, fail_point_sysctl, \ + "A", ""); \ + \ + if (__predict_false( \ + fail_point_eval(&_FAIL_POINT_NAME(name), &RETURN_VALUE))) { \ + \ code; \ - } FAIL_POINT_END + \ + } \ +} while (0) + /** * @} * (end group failpoint) */ -/** - * Internal macro to implement above #defines -- should not be used directly. - * @ingroup failpoint_private - */ -#define KFAIL_POINT_START(parent, name) \ - do { \ - int RETURN_VALUE; \ - static struct fail_point _FAIL_POINT_NAME(name) = { \ - #name, \ - _FAIL_POINT_LOCATION(), \ - TAILQ_HEAD_INITIALIZER( \ - _FAIL_POINT_NAME(name).fp_entries), \ - 0, \ - NULL, NULL, \ - }; \ - SYSCTL_OID(parent, OID_AUTO, name, \ - CTLTYPE_STRING | CTLFLAG_RW, \ - &_FAIL_POINT_NAME(name), 0, fail_point_sysctl, \ - "A", ""); \ - \ - if (__predict_false( \ - fail_point_eval(&_FAIL_POINT_NAME(name), \ - &RETURN_VALUE))) { - -/** - * Internal macro to implement above #defines -- should not be used directly. - * @ingroup failpoint_private - */ -#define FAIL_POINT_END \ - } \ - } while (0) - #ifdef _KERNEL int fail_point_sysctl(SYSCTL_HANDLER_ARGS); /* The fail point sysctl tree. */ SYSCTL_DECL(_debug_fail_point); +#define DEBUG_FP _debug_fail_point #endif -#define DEBUG_FP _debug_fail_point #endif /* _SYS_FAIL_H_ */ Modified: stable/7/sys/sys/queue.h ============================================================================== --- stable/7/sys/sys/queue.h Fri Mar 25 20:19:15 2011 (r220007) +++ stable/7/sys/sys/queue.h Fri Mar 25 20:51:20 2011 (r220008) @@ -313,6 +313,20 @@ struct { \ (head)->stqh_last = &STAILQ_NEXT((elm), field); \ } while (0) +#define STAILQ_SWAP(head1, head2, type) do { \ + struct type *swap_first = STAILQ_FIRST(head1); \ + struct type **swap_last = (head1)->stqh_last; \ + STAILQ_FIRST(head1) = STAILQ_FIRST(head2); \ + (head1)->stqh_last = (head2)->stqh_last; \ + STAILQ_FIRST(head2) = swap_first; \ + (head2)->stqh_last = swap_last; \ + if (STAILQ_EMPTY(head1)) \ + (head1)->stqh_last = &STAILQ_FIRST(head1); \ + if (STAILQ_EMPTY(head2)) \ + (head2)->stqh_last = &STAILQ_FIRST(head2); \ +} while (0) + + /* * List declarations. */ @@ -417,6 +431,16 @@ struct { \ TRASHIT(*oldprev); \ } while (0) +#define LIST_SWAP(head1, head2, type, field) do { \ + struct type *swap_tmp = LIST_FIRST((head1)); \ + LIST_FIRST((head1)) = LIST_FIRST((head2)); \ + LIST_FIRST((head2)) = swap_tmp; \ + if ((swap_tmp = LIST_FIRST((head1))) != NULL) \ + swap_tmp->field.le_prev = &LIST_FIRST((head1)); \ + if ((swap_tmp = LIST_FIRST((head2))) != NULL) \ + swap_tmp->field.le_prev = &LIST_FIRST((head2)); \ +} while (0) + /* * Tail queue declarations. */ @@ -586,6 +610,22 @@ struct { \ QMD_TRACE_ELEM(&(elm)->field); \ } while (0) +#define TAILQ_SWAP(head1, head2, type, field) do { \ + struct type *swap_first = (head1)->tqh_first; \ + struct type **swap_last = (head1)->tqh_last; \ + (head1)->tqh_first = (head2)->tqh_first; \ + (head1)->tqh_last = (head2)->tqh_last; \ + (head2)->tqh_first = swap_first; \ + (head2)->tqh_last = swap_last; \ + if ((swap_first = (head1)->tqh_first) != NULL) \ + swap_first->field.tqe_prev = &(head1)->tqh_first; \ + else \ + (head1)->tqh_last = &(head1)->tqh_first; \ + if ((swap_first = (head2)->tqh_first) != NULL) \ + swap_first->field.tqe_prev = &(head2)->tqh_first; \ + else \ + (head2)->tqh_last = &(head2)->tqh_first; \ +} while (0) #ifdef _KERNEL