From owner-freebsd-current Wed Nov 20 7: 4:50 2002 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F73E37B406 for ; Wed, 20 Nov 2002 07:04:44 -0800 (PST) Received: from mail.speakeasy.net (mail13.speakeasy.net [216.254.0.213]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1AA4443E77 for ; Wed, 20 Nov 2002 07:04:44 -0800 (PST) (envelope-from jhb@FreeBSD.org) Received: (qmail 18174 invoked from network); 20 Nov 2002 15:04:45 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail13.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 20 Nov 2002 15:04:45 -0000 Received: from laptop.baldwin.cx (laptop.baldwin.cx [192.168.0.4]) by server.baldwin.cx (8.12.6/8.12.6) with ESMTP id gAKF4g2D034582; Wed, 20 Nov 2002 10:04:42 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <167309338.1037758638@[192.168.1.20]> Date: Wed, 20 Nov 2002 10:04:47 -0500 (EST) From: John Baldwin To: "Joel M. Baldwin" Subject: RE: panic: sleeping thread owns a mutex - with debug traceback Cc: current@freebsd.org Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 20-Nov-2002 Joel M. Baldwin wrote: > > Under heavy system load and heavy swapping I had the following > panic occur. Unfortunately with this panic by the time you get to it it is too late to see what the original cause was. If you use WITNESS then WITNESS will catch the original problem and report it. However, using WITNESS results in a significant performance degradation. Using WITNESS with WITNESS_SKIPSPIN can help with that some, but it is still very slow. > ------------------------------------------------------------- > > Panic message from the serial console: > > panic: sleeping thread owns a mutex > cpuid = 1; lapic.id = 01000000 > Debugger("panic") > Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0 > db> panic_ __ __ __ __ _t > Debugger(c0392fc9,1000000,c0392171,cd29ac18,1) at Debugger+0x55 > panic(c0392171,1,c03920e8,6b,c039614a) at panic+0x11f > propagate_priority(c0eadc40,2,c03920e8,23b,c03cffa0) at > propagate_priority+0x104 > _mtx_lock_sleep(c3520a20,0,c039f4dd,182,c040af98) at > _mtx_lock_sleep+0x219 > _mtx_lock_flags(c3520a20,0,c039f4dd,182,c0393d1d) at > _mtx_lock_flags+0x98 > syncache_timer(1,0,c0393d1d,bf,220649) at syncache_timer+0xaf > softclock(0,0,c0390a04,230,c0eac700) at softclock+0x19c > ithread_loop(c0ea0600,cd29ad48,c0390767,355,0) at ithread_loop+0x182 > fork_exit(c01fef40,c0ea0600,cd29ad48) at fork_exit+0xa5 > fork_trampoline() at fork_trampoline+0x1a > --- trap 0x1, eip = 0, esp = 0xcd29ad7c, ebp = 0 --- > db> panic > panic: from debugger > cpuid = 1; lapic.id = 01000000 > boot() called on cpu#1 > Uptime: 6h11m40s > pfs_vncache_unload(): 1 entries remaining > Dumping 255 MB > 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 > Dump complete > Automatic reboot in 15 seconds - press a key on the console to abort > Rebooting... > cpu_reset called on cpu#1 > cpu_reset: Restarting BSP > cpu_reset_proxy: Stopped CPU 1 > > -------------------------------------------------------------------- > > kgdb traceback information: > > su-2.05b# gdb -k > GNU gdb 5.2.1 (FreeBSD) > Copyright 2002 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "i386-undermydesk-freebsd". > (kgdb) symbol-file > /usr/src/sys/i386/compile/testGeneric.smp/kernel.debug > Reading symbols from > /usr/src/sys/i386/compile/testGeneric.smp/kernel.debug...done. > (kgdb) exec-file /boot/kernel.testgeneric.smp/kernel > (kgdb) core-file /var/crash/vmcore.30 > panic: from debugger > panic messages: > --- > panic: sleeping thread owns a mutex > cpuid = 1; lapic.id = 01000000 > panic: from debugger > cpuid = 1; lapic.id = 01000000 > boot() called on cpu#1 > Uptime: 6h11m40s > pfs_vncache_unload(): 1 entries remaining > Dumping 255 MB > 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 > --- >#0 doadump () at ../../../kern/kern_shutdown.c:232 > 232 dumping++; > (kgdb) where >#0 doadump () at ../../../kern/kern_shutdown.c:232 >#1 0xc02118bd in boot (howto=260) at ../../../kern/kern_shutdown.c:364 >#2 0xc0211b77 in panic () at ../../../kern/kern_shutdown.c:517 >#3 0xc014d252 in db_panic () at ../../../ddb/db_command.c:450 >#4 0xc014d1d2 in db_command (last_cmdp=0xc03bf2a0, cmd_table=0x0, > aux_cmd_tablep=0xc03b5ec8, > aux_cmd_tablep_end=0xc03b5ee0) at ../../../ddb/db_command.c:346 >#5 0xc014d2e6 in db_command_loop () at ../../../ddb/db_command.c:472 >#6 0xc014ff7a in db_trap (type=3, code=0) at ../../../ddb/db_trap.c:72 >#7 0xc033e2a0 in kdb_trap (type=3, code=0, regs=0xcd29ab90) at > ../../../i386/i386/db_interface.c:166 >#8 0xc03572cf in trap (frame= > {tf_fs = -1069547496, tf_es = 16, tf_ds = -1069744112, tf_edi = > -1058350016, tf_esi = 256, tf_ebp = -852907044, tf_isp = -852907076, > tf_ebx = 0, tf_edx = 0, tf_ecx = 1, tf_eax = 18, tf_trapno = 3, tf_err > = 0, tf_eip = -1070340715, tf_cs = 8, tf_eflags = 662, tf_esp = > -1069880826, tf_ss = -1069994039}) at ../../../i386/i386/trap.c:603 >#9 0xc033fac8 in calltrap () at {standard input}:99 >#10 0xc0211b5f in panic (fmt=0x0) at ../../../kern/kern_shutdown.c:503 >#11 0xc0207b54 in propagate_priority (td=0x0) at > ../../../kern/kern_mutex.c:125 >#12 0xc0208329 in _mtx_lock_sleep (m=0xc3520a20, opts=0, > file=0xc039f4dd "../../../netinet/tcp_syncache.c", line=386) at > ../../../kern/kern_mutex.c:624 >#13 0xc0207db8 in _mtx_lock_flags (m=0xc3520a20, opts=0, > file=0xc039f4dd "../../../netinet/tcp_syncache.c", line=386) at > ../../../kern/kern_mutex.c:325 >#14 0xc029b9ef in syncache_timer (xslot=0x1) at > ../../../netinet/tcp_syncache.c:386 >#15 0xc021f51c in softclock (dummy=0x0) at > ../../../kern/kern_timeout.c:195 >#16 0xc01ff0c2 in ithread_loop (arg=0xc0ea0600) at > ../../../kern/kern_intr.c:535 > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-current" in the body of the message -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message