Date: Wed, 26 Mar 2003 14:48:31 -0500 (EST) From: Robert Watson <rwatson@FreeBSD.org> To: arch@FreeBSD.org Subject: M_NOWAIT failure handling -- not so very rosy picture Message-ID: <Pine.NEB.3.96L.1030326144501.18064X-100000@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
I'm running a diskless system with the attached patch; the results have
not been so very pleasing. I'm collecting a set of panics and traces to
mail out to relevant developers, but it does give one pause. A related
patch for the mbuf allocator would probably also give some interesting
results. The patch is far from perfect, but has been enough to result in
some interesting scenarios.
# sysctl debug.malloc_failure_rate=10 # Fail one in ten
Some things to try that I've bumped into so far:
# sysctl -a > /dev/null
# mdconfig -a -s 5m -t malloc
# dd if=/dev/zero of=/dev/md0
# newfs /dev/md0
Both of these seem to be storage-related and I've e-mailed phk about them,
but I suspect there are a lot of others hanging around.
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org Network Associates Laboratories
Index: kern_malloc.c
===================================================================
RCS file: /home/ncvs/src/sys/kern/kern_malloc.c,v
retrieving revision 1.119
diff -u -r1.119 kern_malloc.c
--- kern_malloc.c 10 Mar 2003 20:24:54 -0000 1.119
+++ kern_malloc.c 26 Mar 2003 19:27:28 -0000
@@ -138,6 +138,20 @@
/* time_uptime of last malloc(9) failure */
static time_t t_malloc_fail;
+#ifdef MALLOC_MAKE_FAILURES
+/*
+ * Cause malloc failures ever (n) mallocs with M_NOWAIT. If set to 0,
+ * don't cause failures.
+ */
+static int malloc_failure_rate;
+static int malloc_nowait_count;
+static int malloc_failure_count;
+SYSCTL_INT(_debug, OID_AUTO, malloc_failure_rate, CTLFLAG_RW,
+ &malloc_failure_rate, 0, "Every (n) mallocs with M_NOWAIT will fail");
+SYSCTL_INT(_debug, OID_AUTO, malloc_failure_count, CTLFLAG_RD,
+ &malloc_failure_count, 0, "Number of imposed malloc failures");
+#endif
+
int
malloc_last_fail(void)
{
@@ -187,6 +201,15 @@
#if 0
if (size == 0)
Debugger("zero size malloc");
+#endif
+#ifdef MALLOC_MAKE_FAILURES
+ if ((flags & M_NOWAIT) && (malloc_failure_rate != 0)) {
+ atomic_add_int(&malloc_nowait_count, 1);
+ if ((malloc_nowait_count % malloc_failure_rate) == 0) {
+ atomic_add_int(&malloc_failure_count, 1);
+ return (NULL);
+ }
+ }
#endif
if (flags & M_WAITOK)
KASSERT(curthread->td_intr_nesting_level == 0,
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1030326144501.18064X-100000>