Date: Thu, 16 Nov 2006 13:24:36 +0300 From: Gleb Smirnoff <glebius@FreeBSD.org> To: stable@FreeBSD.org Subject: Re: RELENG_6 panic under heavy load Message-ID: <20061116102436.GN32700@FreeBSD.org> In-Reply-To: <20061113084430.GE59604@dimma.mow.oilspace.com> References: <20061113084430.GE59604@dimma.mow.oilspace.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I wonder why UMA was suspected to be the problem. Dima gave
me access to the core. Here are more details from the trace:
Unread portion of the kernel message buffer:
panic: thread 100147(nagios):1 holds process lock but isn't blocked on a lock
#9 0xd060038e in panic (fmt=0xd08094d9 "thread %d(%s):%d holds %s but isn't blocked on a lock\n") at /usr/src/sys/kern/kern_shutdown.c:549
#10 0xd0629228 in propagate_priority (td=0xd745c900) at /usr/src/sys/kern/subr_turnstile.c:239
#11 0xd0629f32 in turnstile_wait (lock=0xd5dd5498, owner=0xd745c900) at /usr/src/sys/kern/subr_turnstile.c:643
#12 0xd05f4fc1 in _mtx_lock_sleep (m=0xd5dd5498, tid=3583683968, opts=0, file=0x12 <Address 0x12 out of bounds>, line=18) at /usr/src/sys/kern/kern_mutex.c:579
#13 0xd05f4992 in _mtx_lock_flags (m=0xd5dd5498, opts=0, file=0xd0806c3d "/usr/src/sys/kern/kern_thread.c", line=824) at /usr/src/sys/kern/kern_mutex.c:288
#14 0xd060d340 in thread_single (mode=0) at /usr/src/sys/kern/kern_thread.c:824
#15 0xd05e38b9 in fork1 (td=0xd59aad80, flags=20, pages=0, procp=0xf5cacccc) at /usr/src/sys/kern/kern_fork.c:274
#16 0xd05e3509 in fork (td=0xd59aad80, uap=0xf5cacd04) at /usr/src/sys/kern/kern_fork.c:98
#17 0xd07a6d10 in syscall (frame=
{tf_fs = 134938683, tf_es = 59, tf_ds = -809566149, tf_edi = 134953856, tf_esi = 673312612, tf_ebp = -809526568, tf_isp = -171258524, tf_ebx = 672261300, tf_edx = 0, tf_ecx = 134963456, tf_eax = 2, tf_trapno = 12, tf_err = 2, tf_eip = 672684403, tf_cs = 51, tf_eflags = 642, tf_esp = -809526660, tf_ss = 59})
at /usr/src/sys/i386/i386/trap.c:983
#18 0xd078f38f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200
#19 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) frame 10
#10 0xd0629228 in propagate_priority (td=0xd745c900) at /usr/src/sys/kern/subr_turnstile.c:239
239 KASSERT(TD_ON_LOCK(td), (
(kgdb) list
234 #endif
235
236 /*
237 * If we aren't blocked on a lock, we should be.
238 */
239 KASSERT(TD_ON_LOCK(td), (
240 "thread %d(%s):%d holds %s but isn't blocked on a lock\n",
241 td->td_tid, td->td_proc->p_comm, td->td_state,
242 ts->ts_lockobj->lo_name));
243
(kgdb) frame 14
#14 0xd060d340 in thread_single (mode=0) at /usr/src/sys/kern/kern_thread.c:824
824 PROC_LOCK(p);
(kgdb) list
819 thread_stopped(p);
820 thread_suspend_one(td);
821 PROC_UNLOCK(p);
822 mi_switch(SW_VOL, NULL);
823 mtx_unlock_spin(&sched_lock);
824 PROC_LOCK(p);
825 mtx_lock_spin(&sched_lock);
826 if (mode == SINGLE_EXIT)
827 remaining = p->p_numthreads;
828 else if (mode == SINGLE_BOUNDARY)
--
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061116102436.GN32700>