From owner-freebsd-current@FreeBSD.ORG Fri Aug 17 16:54:33 2007 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E55716A419 for ; Fri, 17 Aug 2007 16:54:33 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.freebsd.org (Postfix) with ESMTP id 4D4FB13C45D for ; Fri, 17 Aug 2007 16:54:33 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.1/8.14.1) id l7HGsWjS082088; Fri, 17 Aug 2007 11:54:32 -0500 (CDT) (envelope-from dan) Date: Fri, 17 Aug 2007 11:54:32 -0500 From: Dan Nelson To: Alexander Kabaev Message-ID: <20070817165431.GA49455@dan.emsphone.com> References: <20070817134526.GA27365@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20070817134526.GA27365@freefall.freebsd.org> X-OS: FreeBSD 7.0-CURRENT User-Agent: Mutt/1.5.16 (2007-06-09) Cc: current@freebsd.org Subject: Re: Double mutex destruction X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2007 16:54:33 -0000 In the last episode (Aug 17), Alexander Kabaev said: > I left development box running 7.0 sitting idle at work overnight and > was greeted with the following panic this morning: > > panic: mtx_lock() of destroyed mutex @ ../../../net/route.c:1303 > cpuid = 1 > KDB: enter: panic > [thread pid 4088 tid 100128 ] > Stopped at kdb_enter+0x32: leave > db> bt > Tracing pid 4088 tid 100128 td 0xc5539cc0 > kdb_enter(c07c56bd,1,c07c4619,f199f9bc,1,...) at kdb_enter+0x32 > panic(c07c4619,c07d2ce5,517,507,c5386a7c,...) at panic+0x124 > _mtx_lock_flags(c5aeb510,0,c07d2ce5,517,f199fa28,...) at _mtx_lock_flags+0x65 > rt_check(f199fa20,f199fa3c,c541ec30,c080a6c4,c07d2cee,...) at rt_check+0x111 > arpresolve(c51e4000,c5584bb8,c5587d00,c541ec30,f199fa56,...) at arpresolve+0xb0 [...] I get this about once a week or so, and Andre Guibert de Bruet has also reported it. Occasionally instead of an assertion panic, I get a trap 12: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 03 fault virtual address = 0x188 fault code = supervisor read, page not present instruction pointer = 0x20:0xc05dc855 stack pointer = 0x28:0xe74b0920 frame pointer = 0x28:0xe74b0938 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = resume, IOPL = 0 current process = 2623 (epic-EPIC4-2.6) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper(c082afcc,e74b07fc,c05e9541,c08435d9,0,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08435d9,0,c080f97f,e74b0808,0,...) at kdb_backtrace+0x29 panic(c080f97f,c084484e,c83e377c,1,1,...) at panic+0x111 trap_fatal(c0844750,c,c082707b,efa498f6,c83e3558,...) at trap_fatal+0x383 trap(e74b08e0) at trap+0x11b calltrap() at calltrap+0x6 --- trap 0xc, eip = 0xc05dc855, esp = 0xe74b0920, ebp = 0xe74b0938 --- _mtx_lock_sleep(c62d3d80,c83e6000,0,0,0,...) at _mtx_lock_sleep+0x85 rt_check(e74b0980,e74b099c,c5a64e10,0,0,...) at rt_check+0x120 arpresolve(c3bd0c00,c62d4d20,c3fff400,c5a64e10,e74b09b6,...) at arpresolve+0xb4 ether_output(c3bd0c00,c3fff400,c5a64e10,c62d4d20,c807d7e0,...) at ether_output+0x8e ip_output(c3fff400,0,e74b0a28,0,0,...) at ip_output+0xb45 tcp_output(c744a000,c437a100,e74b0c60,1,0,...) at tcp_output+0x11de tcp_usr_send(c7fc3630,0,c437a100,0,0,...) at tcp_usr_send+0x262 sosend_generic(c7fc3630,0,e74b0c60,c437a100,0,...) at sosend_generic+0x6a5 sosend(c7fc3630,0,e74b0c60,0,0,...) at sosend+0x3f soo_write(c9869948,e74b0c60,c9871000,0,c83e6000,...) at soo_write+0x4b dofilewrite(e74b0c60,ffffffff,ffffffff,0,c9869948,...) at dofilewrite+0x97 kern_writev(c83e6000,3,e74b0c60,bfbf8d02,0,...) at kern_writev+0x58 write(c83e6000,e74b0cfc,c,16,e74b0d2c,...) at write+0x4f syscall(e74b0d38) at syscall+0x365 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (4, FreeBSD ELF32, write), eip = 0x28448577, esp = 0xbfbf8c8c, ebp = 0xbfbf8cb8 --- Uptime: 6h40m30s Physical memory: 1015 MB Dumping 246 MB: 231 215 199 183 167 151 135 119 103 87 71 55 39 23 7 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) (kgdb) #0 doadump () at pcpu.h:195 #1 0xc05e9287 in boot (howto=260) at ../../../kern/kern_shutdown.c:409 #2 0xc05e9570 in panic (fmt=Variable "fmt" is not available.) at ../../../kern/kern_shutdown.c:563 #3 0xc07cabd3 in trap_fatal (frame=0xe74b08e0, eva=392) at ../../../i386/i386/trap.c:872 #4 0xc07cb5cb in trap (frame=0xe74b08e0) at ../../../i386/i386/trap.c:277 #5 0xc07b070b in calltrap () at ../../../i386/i386/exception.s:139 #6 0xc05dc855 in _mtx_lock_sleep (m=0xc62d3d80, tid=3359531008, opts=0, file=0x0, line=0) at ../../../kern/kern_mutex.c:360 #7 0xc0695a60 in rt_check (lrt=0xe74b0980, lrt0=0xe74b099c, dst=0xc5a64e10) at ../../../net/route.c:1303 #8 0xc069f644 in arpresolve (ifp=0xc3bd0c00, rt0=0xc62d4d20, m=0xc3fff400, dst=0xc5a64e10, desten=0xe74b09b6 " ├D⌠ ├") at ../../../netinet/if_ether.c:373 #9 0xc068964e in ether_output (ifp=0xc3bd0c00, m=0xc3fff400, dst=0xc5a64e10, rt0=0xc62d4d20) at ../../../net/if_ethersubr.c:175 #10 0xc06ba9a5 in ip_output (m=0xc3fff400, opt=0x0, ro=0xe74b0a28, flags=Variable "flags" is not available.) at ../../../netinet/ip_output.c:547 #11 0xc06c2c7e in tcp_output (tp=0xc744a000) at ../../../netinet/tcp_output.c:1125 #12 0xc06cba12 in tcp_usr_send (so=0xc7fc3630, flags=Variable "flags" is not available.) at ../../../netinet/tcp_usrreq.c:839 #13 0xc06406a5 in sosend_generic (so=0xc7fc3630, addr=0x0, uio=0xe74b0c60, top=0xc437a100, control=0x0, flags=0, td=0xc83e6000) at ../../../kern/uipc_socket.c:1241 #14 0xc063c34f in sosend (so=0xc7fc3630, addr=0x0, uio=0xe74b0c60, top=0x0, control=0x0, flags=0, td=0xc83e6000) at ../../../kern/uipc_socket.c:1287 #15 0xc0624a3b in soo_write (fp=0xc9869948, uio=0xe74b0c60, active_cred=0xc9871000, flags=0, td=0xc83e6000) at ../../../kern/sys_socket.c:104 #16 0xc061e0c7 in dofilewrite (td=0xc83e6000, fd=3, fp=0xc9869948, auio=0xe74b0c60, offset=-1, flags=0) at file.h:254 #17 0xc061e3b8 in kern_writev (td=0xc83e6000, fd=3, auio=0xe74b0c60) at ../../../kern/sys_generic.c:404 #18 0xc061e42f in write (td=0xc83e6000, uap=0xe74b0cfc) at ../../../kern/sys_generic.c:320 #19 0xc07cb1e5 in syscall (frame=0xe74b0d38) at ../../../i386/i386/trap.c:1008 #20 0xc07b0770 in Xint0x80_syscall () at ../../../i386/i386/exception.s:196 #21 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) -- Dan Nelson dnelson@allantgroup.com