From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 15 07:22:35 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AC27A5D9 for ; Mon, 15 Jul 2013 07:22:35 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell0.rawbw.com (shell0.rawbw.com [198.144.192.45]) by mx1.freebsd.org (Postfix) with ESMTP id 9AD9CD05 for ; Mon, 15 Jul 2013 07:22:35 +0000 (UTC) Received: from eagle.yuri.org (stunnel@localhost [127.0.0.1]) (authenticated bits=0) by shell0.rawbw.com (8.14.4/8.14.4) with ESMTP id r6F7MS6P005288 for ; Mon, 15 Jul 2013 00:22:28 -0700 (PDT) (envelope-from yuri@rawbw.com) Message-ID: <51E3A334.8020203@rawbw.com> Date: Mon, 15 Jul 2013 00:22:28 -0700 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130628 Thunderbird/17.0.7 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: Kernel crashes after sleep: how to debug? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2013 07:22:35 -0000 After sleep/wakeup cycle my 9.1-STABLE r253105 amd64 system has a tendency to sometimes randomly crash after a while. It doesn't happen every time. See kgdb log below. I am not sure there is enough information to lead to the cause of the issue. It looks like it crashes near the line: #7 0xffffffff8091a181 in _mtx_trylock (m=0x100000000, opts=0, file=, line=0) at /usr/src/sys/kern/kern_mutex.c:295 295 if (SCHEDULER_STOPPED()) Current language: auto; currently c (kgdb) l 290 uint64_t waittime = 0; 291 int contested = 0; 292 #endif 293 int rval; 294 295 if (SCHEDULER_STOPPED()) 296 return (1); 297 298 KASSERT(m->mtx_lock != MTX_DESTROYED, 299 ("mtx_trylock() of destroyed mutex @ %s:%d", file, line)); Current thread was: * 67 Thread 100064 (PID=5: pagedaemon) doadump (textdump=) at pcpu.h:234 How to find the cause of the crash? Yuri --- kgdb log --- # kgdb /boot/kernel/kernel vmcore.0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x100000018 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8091a181 stack pointer = 0x28:0xffffff80d51c6ab0 frame pointer = 0x28:0xffffff80d51c6ad0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 5 (pagedaemon) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff80968416 at kdb_backtrace+0x66 #1 0xffffffff8092e43e at panic+0x1ce #2 0xffffffff80d12940 at trap_fatal+0x290 #3 0xffffffff80d12ca1 at trap_pfault+0x211 #4 0xffffffff80d13254 at trap+0x344 #5 0xffffffff80cfc583 at calltrap+0x8 #6 0xffffffff80baea78 at vm_pageout+0x998 #7 0xffffffff808fc10f at fork_exit+0x11f #8 0xffffffff80cfcaae at fork_trampoline+0xe Uptime: 2h21m27s Dumping 407 out of 2919 MB:..4%..12%..24%..32%..44%..52%..63%..71%..83%..91% Reading symbols from /boot/modules/cuse4bsd.ko...done. Loaded symbols for /boot/modules/cuse4bsd.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /usr/local/libexec/linux_adobe/linux_adobe.ko...done. Loaded symbols for /usr/local/libexec/linux_adobe/linux_adobe.ko Reading symbols from /boot/kernel/radeon.ko...Reading symbols from /boot/kernel/radeon.ko.symbols...done. done. Loaded symbols for /boot/kernel/radeon.ko Reading symbols from /boot/kernel/drm.ko...Reading symbols from /boot/kernel/drm.ko.symbols...done. done. Loaded symbols for /boot/kernel/drm.ko #0 doadump (textdump=) at pcpu.h:234 234 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump (textdump=) at pcpu.h:234 #1 0xffffffff8092df16 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 #2 0xffffffff8092e417 in panic (fmt=0x1
) at /usr/src/sys/kern/kern_shutdown.c:637 #3 0xffffffff80d12940 in trap_fatal (frame=0xc, eva=) at /usr/src/sys/amd64/amd64/trap.c:879 #4 0xffffffff80d12ca1 in trap_pfault (frame=0xffffff80d51c6a00, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 #5 0xffffffff80d13254 in trap (frame=0xffffff80d51c6a00) at /usr/src/sys/amd64/amd64/trap.c:463 #6 0xffffffff80cfc583 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0xffffffff8091a181 in _mtx_trylock (m=0x100000000, opts=0, file=, line=0) at /usr/src/sys/kern/kern_mutex.c:295 #8 0xffffffff80baea78 in vm_pageout () at /usr/src/sys/vm/vm_pageout.c:829 #9 0xffffffff808fc10f in fork_exit (callout=0xffffffff80bae0e0 , arg=0x0, frame=0xffffff80d51c6c40) at /usr/src/sys/kern/kern_fork.c:988 #10 0xffffffff80cfcaae in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 #11 0x0000000000000000 in ?? ()