Date: Wed, 20 Nov 2002 17:56:47 -0800 From: "Joel M. Baldwin" <qumqats@outel.org> Cc: current@FreeBSD.org Subject: more info - Re: panic: sleeping thread owns a mutex - with debug traceback Message-ID: <17798412.1037815007@[192.168.1.20]> In-Reply-To: <XFMail.20021120142746.jhb@FreeBSD.org> References: <XFMail.20021120142746.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--On Wednesday, November 20, 2002 2:27 PM -0500 John Baldwin <jhb@FreeBSD.org> wrote: > On 20-Nov-2002 Joel M. Baldwin wrote: >> --On Wednesday, November 20, 2002 12:01 PM -0500 Robert Watson >> <rwatson@FreeBSD.ORG> wrote: >> >>> On Wed, 20 Nov 2002, Robert Watson wrote: >>> >>>> Hmm. Another thread has decided to sleep while holding an inpcb >>>> mutex. Any chance this can be reproduced while running WITNESS? >>>> If so, you should get a panic earlier when the other thread sleeps >>>> in the first place. The easiest way to do that is if you can >>>> reproduce the panic with WITNESS. If you can't reproduce the >>>> panic, you may be able to extract this from your system core using >>>> gdb -- you want to figure out what the thread owner of the mutex >>>> is doing -- in the context of the kassert() below, td is the >>>> pointer to the thread that owns the mutex. I'm not sure how to >>>> extract a stack trace from that information, unfortunately, >>>> perhaps someone can give us pointers. (note: td from the >>>> priority_propagate() argument is shadowed, which is annoying). >>> >>> Ack. I mis-read. You want the stack from thread td1 (the mutex >>> owner), not thread td. >> >> The kernel that produced the core dump ALREADY HAS >> WITNESS and WITNESS_SKIPSPIN! :( >> >> I'll try to get more info from kgdb, but I doubt that I'll have >> much luck since I've never tried using gdb before. > > Erm. Did you manage to look at dmesg then? If so, you would have > seen warnings from WITNESS earlier about the locks messing up. If > you can reproduce this and are letting it sit unattended, a better > plan might be to turn on witness_ddb (it's a kernel option, loader > tunable, and sysctl (debug.witness_ddb)) and then when the original > error occurs it will drop into the debugger with a very useful error > message. You can also get a useful trace at that point from ddb. > > -- > > John Baldwin <jhb@FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ ok, I'vd had 3 events now. The first event Hard Locked the system and 'pa' printed out on the serial console. Like it was trying to panic and got hosed in the middle of printing the message. The second event rebooted the system with nothing printed out on the console. The third event paniced with a live console. Note that the 'panic' command didn't produce a core dump. Slab at 0xc2c39fd4, freei 3 = 0. panic: Duplicate free of item 0xc2c39264 from zone 0xc0e90c00(VMSPACE) cpuid = 0; lapic.id = 00000000 Debugger("panic") Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0 db> where t Debugger(c0392fc9,0,c03a9554,cd2d9c4c,1) at Debugger+0x55 panic(c03a9554,c2c39264,c0e90c00,c03a7695,6a8) at panic+0x11f uma_dbg_free(c0e90c00,0,c2c39264,6a8,0) at uma_dbg_free+0x122 uma_zfree_arg(c0e90c00,c2c39264,0,12e,c2fc6000) at uma_zfree_arg+0x124 vmspace_free(c2c39264,c03a7307,31d,31c,186a0) at vmspace_free+0xbe swapout_procs(1,0,68,c03a8f12,0) at swapout_procs+0x387 vm_daemon(0,cd2d9d48,c0390767,355,0) at vm_daemon+0x6e fork_exit(c031ab40,0,cd2d9d48) at fork_exit+0xa5 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xcd2d9d7c, ebp = 0 --- db> show locks exclusive sleep mutex VMSPACE (UMA zone) r = 0 (0xc0e90c24) locked @ ../../../vm/uma_core.c:1704 exclusive sleep mutex PCPU VMSPACE (UMA cpu) r = 0 (0xc0e90ce8) locked @ ../../../vm/uma_core.c:1686 shared sx allproc r = 0 (0xc03d0600) locked @ ../../../vm/vm_glue.c:657 exclusive sleep mutex Giant r = 0 (0xc03d0000) locked @ ../../../vm/vm_pageout.c:1501 db> panic panic: from debugger cpuid = 0; lapic.id = 00000000 boot() called on cpu#0 Uptime: 22m57s To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?17798412.1037815007>