Date: Wed, 20 Nov 2002 12:00:17 -0500 (EST) From: Robert Watson <rwatson@freebsd.org> To: "Joel M. Baldwin" <qumqats@outel.org> Cc: current@freebsd.org Subject: Re: panic: sleeping thread owns a mutex - with debug traceback Message-ID: <Pine.NEB.3.96L.1021120115452.40737Y-100000@fledge.watson.org> In-Reply-To: <167309338.1037758638@[192.168.1.20]>
next in thread | previous in thread | raw e-mail | index | archive | help
Hmm. Another thread has decided to sleep while holding an inpcb mutex.
Any chance this can be reproduced while running WITNESS? If so, you
should get a panic earlier when the other thread sleeps in the first
place. The easiest way to do that is if you can reproduce the panic with
WITNESS. If you can't reproduce the panic, you may be able to extract
this from your system core using gdb -- you want to figure out what the
thread owner of the mutex is doing -- in the context of the kassert()
below, td is the pointer to the thread that owns the mutex. I'm not sure
how to extract a stack trace from that information, unfortunately, perhaps
someone can give us pointers. (note: td from the priority_propagate()
argument is shadowed, which is annoying).
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org Network Associates Laboratories
On Wed, 20 Nov 2002, Joel M. Baldwin wrote:
>
> Under heavy system load and heavy swapping I had the following
> panic occur.
>
> -------------------------------------------------------------
>
> Panic message from the serial console:
>
> panic: sleeping thread owns a mutex
> cpuid = 1; lapic.id = 01000000
> Debugger("panic")
> Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0
> db> panic t
> Debugger(c0392fc9,1000000,c0392171,cd29ac18,1) at Debugger+0x55
> panic(c0392171,1,c03920e8,6b,c039614a) at panic+0x11f
> propagate_priority(c0eadc40,2,c03920e8,23b,c03cffa0) at
> propagate_priority+0x104
> _mtx_lock_sleep(c3520a20,0,c039f4dd,182,c040af98) at
> _mtx_lock_sleep+0x219
> _mtx_lock_flags(c3520a20,0,c039f4dd,182,c0393d1d) at
> _mtx_lock_flags+0x98
> syncache_timer(1,0,c0393d1d,bf,220649) at syncache_timer+0xaf
> softclock(0,0,c0390a04,230,c0eac700) at softclock+0x19c
> ithread_loop(c0ea0600,cd29ad48,c0390767,355,0) at ithread_loop+0x182
> fork_exit(c01fef40,c0ea0600,cd29ad48) at fork_exit+0xa5
> fork_trampoline() at fork_trampoline+0x1a
> --- trap 0x1, eip = 0, esp = 0xcd29ad7c, ebp = 0 ---
> db> panic
> panic: from debugger
> cpuid = 1; lapic.id = 01000000
> boot() called on cpu#1
> Uptime: 6h11m40s
> pfs_vncache_unload(): 1 entries remaining
> Dumping 255 MB
> 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
> Dump complete
> Automatic reboot in 15 seconds - press a key on the console to abort
> Rebooting...
> cpu_reset called on cpu#1
> cpu_reset: Restarting BSP
> cpu_reset_proxy: Stopped CPU 1
>
> --------------------------------------------------------------------
>
> kgdb traceback information:
>
> su-2.05b# gdb -k
> GNU gdb 5.2.1 (FreeBSD)
> Copyright 2002 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and
> you are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for
> details.
> This GDB was configured as "i386-undermydesk-freebsd".
> (kgdb) symbol-file
> /usr/src/sys/i386/compile/testGeneric.smp/kernel.debug
> Reading symbols from
> /usr/src/sys/i386/compile/testGeneric.smp/kernel.debug...done.
> (kgdb) exec-file /boot/kernel.testgeneric.smp/kernel
> (kgdb) core-file /var/crash/vmcore.30
> panic: from debugger
> panic messages:
> ---
> panic: sleeping thread owns a mutex
> cpuid = 1; lapic.id = 01000000
> panic: from debugger
> cpuid = 1; lapic.id = 01000000
> boot() called on cpu#1
> Uptime: 6h11m40s
> pfs_vncache_unload(): 1 entries remaining
> Dumping 255 MB
> 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240
> ---
> #0 doadump () at ../../../kern/kern_shutdown.c:232
> 232 dumping++;
> (kgdb) where
> #0 doadump () at ../../../kern/kern_shutdown.c:232
> #1 0xc02118bd in boot (howto=260) at ../../../kern/kern_shutdown.c:364
> #2 0xc0211b77 in panic () at ../../../kern/kern_shutdown.c:517
> #3 0xc014d252 in db_panic () at ../../../ddb/db_command.c:450
> #4 0xc014d1d2 in db_command (last_cmdp=0xc03bf2a0, cmd_table=0x0,
> aux_cmd_tablep=0xc03b5ec8,
> aux_cmd_tablep_end=0xc03b5ee0) at ../../../ddb/db_command.c:346
> #5 0xc014d2e6 in db_command_loop () at ../../../ddb/db_command.c:472
> #6 0xc014ff7a in db_trap (type=3, code=0) at ../../../ddb/db_trap.c:72
> #7 0xc033e2a0 in kdb_trap (type=3, code=0, regs=0xcd29ab90) at
> ../../../i386/i386/db_interface.c:166
> #8 0xc03572cf in trap (frame=
> {tf_fs = -1069547496, tf_es = 16, tf_ds = -1069744112, tf_edi =
> -1058350016, tf_esi = 256, tf_ebp = -852907044, tf_isp = -852907076,
> tf_ebx = 0, tf_edx = 0, tf_ecx = 1, tf_eax = 18, tf_trapno = 3, tf_err
> = 0, tf_eip = -1070340715, tf_cs = 8, tf_eflags = 662, tf_esp =
> -1069880826, tf_ss = -1069994039}) at ../../../i386/i386/trap.c:603
> #9 0xc033fac8 in calltrap () at {standard input}:99
> #10 0xc0211b5f in panic (fmt=0x0) at ../../../kern/kern_shutdown.c:503
> #11 0xc0207b54 in propagate_priority (td=0x0) at
> ../../../kern/kern_mutex.c:125
> #12 0xc0208329 in _mtx_lock_sleep (m=0xc3520a20, opts=0,
> file=0xc039f4dd "../../../netinet/tcp_syncache.c", line=386) at
> ../../../kern/kern_mutex.c:624
> #13 0xc0207db8 in _mtx_lock_flags (m=0xc3520a20, opts=0,
> file=0xc039f4dd "../../../netinet/tcp_syncache.c", line=386) at
> ../../../kern/kern_mutex.c:325
> #14 0xc029b9ef in syncache_timer (xslot=0x1) at
> ../../../netinet/tcp_syncache.c:386
> #15 0xc021f51c in softclock (dummy=0x0) at
> ../../../kern/kern_timeout.c:195
> #16 0xc01ff0c2 in ithread_loop (arg=0xc0ea0600) at
> ../../../kern/kern_intr.c:535
>
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-current" in the body of the message
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1021120115452.40737Y-100000>