From owner-freebsd-hackers@freebsd.org Thu Jan 21 18:48:38 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6A36A8B0DB for ; Thu, 21 Jan 2016 18:48:38 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1A081CEF; Thu, 21 Jan 2016 18:48:38 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id B468BB97D; Thu, 21 Jan 2016 13:48:37 -0500 (EST) From: John Baldwin To: Andriy Gapon Cc: Ryan Stone , "freebsd-hackers@freebsd.org" Subject: Re: How to get anything useful out of kgdb? Date: Thu, 21 Jan 2016 09:18:54 -0800 Message-ID: <1668788.bFLpJ6aV8e@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <56A0CA17.7030204@FreeBSD.org> References: <554E41EE.2010202@ignoranthack.me> <16207260.7oBjvc4tcM@ralph.baldwin.cx> <56A0CA17.7030204@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 21 Jan 2016 13:48:37 -0500 (EST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jan 2016 18:48:39 -0000 On Thursday, January 21, 2016 02:07:51 PM Andriy Gapon wrote: > On 08/01/2016 22:05, John Baldwin wrote: > > So I figured out why newer kgdb wasn't unwinding through NULL function pointer > > traps yesterday. I'm not sure if the fix will help with your case as well, > > but it might be worth trying. The changes are in > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206044 > > Thank you very much! > It does help in my case: > > (kgdb) bt > #0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:291 > #1 0xffffffff8063453f in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:359 > #2 0xffffffff80634ba4 in vpanic (fmt=, ap=) at > /usr/src/sys/kern/kern_shutdown.c:635 > #3 0xffffffff806348a3 in panic (fmt=) at > /usr/src/sys/kern/kern_shutdown.c:568 > #4 0xffffffff8041bba7 in db_panic (addr=, > have_addr=, count=, modif=) at > /usr/src/sys/ddb/db_command.c:473 > #5 0xffffffff8041b67b in db_command (last_cmdp=, cmd_table=0x0, > dopager=) at /usr/src/sys/ddb/db_command.c:440 > #6 0xffffffff8041b524 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 > #7 0xffffffff8041de0b in db_trap (type=, code=) at > /usr/src/sys/ddb/db_main.c:251 > #8 0xffffffff80669de8 in kdb_trap (type=19, code=0, tf=0xffffffff80f976d0 > ) at /usr/src/sys/kern/subr_kdb.c:653 > #9 0xffffffff80820d26 in trap (frame=0xffffffff80f976d0 ) at > /usr/src/sys/amd64/amd64/trap.c:381 > #10 > #11 0xffffffff80619e1f in __mtx_assert (c=, what=, > file=, line=) at /usr/src/sys/kern/kern_mutex.c:842 > #12 0xffffffff807fef86 in vm_reserv_free_page (m=0xfffff80229e07cc0) at > /usr/src/sys/vm/vm_reserv.c:832 > #13 0xffffffff807f2b96 in vm_page_free_toq (m=0xfffff80229e07cc0) at > /usr/src/sys/vm/vm_page.c:2432 > #14 0xffffffff807f2e4d in vm_page_free (m=0xffffffff81129198 > ) at /usr/src/sys/vm/vm_page.c:962 > #15 0xffffffff821c28e2 in ttm_bo_vm_fault (vm_obj=0xfffff8021fcdeb00, > offset=5304320, prot=, mres=0xfffffe02b8357050) at > /usr/src/sys/modules/drm2/drm2/../../../dev/drm2/ttm/ttm_bo_vm.c:269 > #16 0xffffffff807d4fd3 in dev_pager_getpages (object=0xfffff8021fcdeb00, > ma=0xfffffe02b8357050, count=1, reqpage=0) at /usr/src/sys/vm/device_pager.c:321 > #17 0xffffffff807f9d67 in vm_pager_get_pages (object=0xfffff8021fcdeb00, > m=0xfffffe02b8357050, count=1, reqpage=0) at /usr/src/sys/vm/vm_pager.c:291 > #18 0xffffffff807e0d84 in vm_fault_hold (map=0xfffff8001dd57000, > vaddr=34729947136, fault_type=2 '\002', fault_flags=0, m_hold=0x0) at > /usr/src/sys/vm/vm_fault.c:672 > #19 0xffffffff807e05ee in vm_fault (map=0xfffff8001dd57000, vaddr= out>, fault_type=2 '\002', fault_flags=0) at /usr/src/sys/vm/vm_fault.c:277 > #20 0xffffffff80821342 in trap_pfault (frame=0xfffffe02b8357c00, usermode=1) at > /usr/src/sys/amd64/amd64/trap.c:734 > #21 0xffffffff80820bda in trap (frame=0xfffffe02b8357c00) at > /usr/src/sys/amd64/amd64/trap.c:326 > #22 0xffffffff8082154a in trap_check (frame=0xfffffe02b8357c00) at > /usr/src/sys/amd64/amd64/trap.c:628 > #23 > #24 0x00000008022b4046 in ?? () > Backtrace stopped: Cannot access memory at address 0x7fffffffe7c8 > > See http://article.gmane.org/gmane.os.freebsd.devel.hackers/56410 for comparison. > One small regression is that previously there was nmi_calltrap in the trace, now > it's just "". Yeah, listing trapframes as that is fallout from the fix. I've thought about adding a custom frame type so I can install a custom frame printer and output things like the trap number the way ddb does, but that would be a far more invasive change that might be harder to maintain going forward. Glad that this helped though! -- John Baldwin