Date: Fri, 2 Oct 2015 09:26:23 +0300 From: Andriy Gapon <avg@FreeBSD.org> To: Ryan Stone <rysto32@gmail.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@FreeBSD.org>, John Baldwin <jhb@FreeBSD.org> Subject: Re: How to get anything useful out of kgdb? Message-ID: <560E238F.9050609@FreeBSD.org> In-Reply-To: <CAFMmRNyM6Tc7P8rLJmMSVXOFkK4Tc0OCOtc=E9dLEtzKrEtjLg@mail.gmail.com> References: <554E41EE.2010202@ignoranthack.me> <2063489.pgabuk9nPJ@ralph.baldwin.cx> <55561803.9050102@ignoranthack.me> <19618854.y3EeXVtCGX@ralph.baldwin.cx> <55561D9A.30309@ignoranthack.me> <CAFMmRNydJdVm%2BDpptQY_LnRvMfUMmoLkLXZFd1zP3OxaMFHgxw@mail.gmail.com> <555627EC.2020007@ignoranthack.me> <CAFMmRNyM6Tc7P8rLJmMSVXOFkK4Tc0OCOtc=E9dLEtzKrEtjLg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 15/05/2015 20:57, Ryan Stone wrote:
> *Sigh*, kgdb isn't unwinding the trap frame properly. You can try this to
> figure out where it was running:
I wonder, what is a reason for this?
Can that be fixed in kgdb itself?
It seems that usually kgdb handles trap frames just fine, but not always?
> That gives you the top of the callstack at the time that the core was
> taken. To get the rest of it, try:
>
> define trace_stack
> set $frame_ptr=$arg0
> set $iters=0
> while $frame_ptr != 0 && $iters < $arg1
> set $ret_addr=((char*)$frame_ptr) + sizeof(void*)
> printf "frameptr=%p, ret_addr=%p\n", (void*)$frame_ptr, *(void**)$ret_addr
> printf " "
> info line **(void***)$ret_addr
> set $frame_ptr=*(void**)$frame_ptr
> set $iters=$iters+1
> end
> end
>
> trace_stack frame->tf_rbp 20
Thank you for this script.
Here is an example from my practice.
(kgdb) bt
#0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:291
#1 0xffffffff8063453f in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:359
#2 0xffffffff80634ba4 in vpanic (fmt=<value optimized out>, ap=<value optimized
out>) at /usr/src/sys/kern/kern_shutdown.c:635
#3 0xffffffff806348a3 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:568
#4 0xffffffff8041bba7 in db_panic (addr=<value optimized out>, have_addr=false,
count=0, modif=0x0) at /usr/src/sys/ddb/db_command.c:473
#5 0xffffffff8041b67b in db_command (cmd_table=0x0) at
/usr/src/sys/ddb/db_command.c:440
#6 0xffffffff8041b524 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493
#7 0xffffffff8041de0b in db_trap (type=<value optimized out>, code=0) at
/usr/src/sys/ddb/db_main.c:251
#8 0xffffffff80669de8 in kdb_trap (type=19, code=0, tf=0xffffffff80f976d0) at
/usr/src/sys/kern/subr_kdb.c:653
#9 0xffffffff80820d26 in trap (frame=0xffffffff80f976d0) at
/usr/src/sys/amd64/amd64/trap.c:381
#10 0xffffffff80809623 in nmi_calltrap () at
/usr/src/sys/libkern/explicit_bzero.c:28
#11 0xffffffff80619e1f in __mtx_assert (c=<value optimized out>, what=<value
optimized out>, file=<value optimized out>, line=<value optimized out>) at
/usr/src/sys/kern/kern_mutex.c:842
Previous frame inner to this frame (corrupt stack?)
Current language: auto; currently minimal
(kgdb) fr 9
#9 0xffffffff80820d26 in trap (frame=0xffffffff80f976d0) at
/usr/src/sys/amd64/amd64/trap.c:381
381 kdb_trap(type, 0, frame);
(kgdb) trace_stack frame->tf_rbp 20
frameptr=0xfffffe02b8356e90, ret_addr=0xffffffff807fef86
Line 833 of "/usr/src/sys/vm/vm_reserv.c" starts at address
0xffffffff807fef86 <vm_reserv_free_page+38> and ends at 0xffffffff807fef90
<vm_reserv_free_page+48>.
frameptr=0xfffffe02b8356eb0, ret_addr=0xffffffff807f2b96
Line 2432 of "/usr/src/sys/vm/vm_page.c" starts at address
0xffffffff807f2b96 <vm_page_free_toq+262> and ends at 0xffffffff807f2b9c
<vm_page_free_toq+268>.
frameptr=0xfffffe02b8356ed0, ret_addr=0xffffffff807f2e4d
Line 963 of "/usr/src/sys/vm/vm_page.c" starts at address 0xffffffff807f2e4d
<vm_page_free+13> and ends at 0xffffffff807f2e50 <vm_page_free_zero>.
frameptr=0xfffffe02b8356ee0, ret_addr=0xffffffff821c28e2
Line 268 of
"/usr/src/sys/modules/drm2/drm2/../../../dev/drm2/ttm/ttm_bo_vm.c" starts at
address 0xffffffff821c28e2 <ttm_bo_vm_fault+1010> and ends at 0xffffffff821c28ee
<ttm_bo_vm_fault+1022>.
frameptr=0xfffffe02b8356f50, ret_addr=0xffffffff807d4fd3
Line 321 of "/usr/src/sys/vm/device_pager.c" starts at address
0xffffffff807d4fce <dev_pager_getpages+94> and ends at 0xffffffff807d4fdb
<dev_pager_getpages+107>.
frameptr=0xfffffe02b8356fa0, ret_addr=0xffffffff807f9d67
Line 291 of "/usr/src/sys/vm/vm_pager.c" starts at address
0xffffffff807f9d58 <vm_pager_get_pages+40> and ends at 0xffffffff807f9d6a
<vm_pager_get_pages+58>.
frameptr=0xfffffe02b8356fd0, ret_addr=0xffffffff807e0d84
Line 675 of "/usr/src/sys/vm/vm_fault.c" starts at address
0xffffffff807e0d84 <vm_fault_hold+1860> and ends at 0xffffffff807e0d8d
<vm_fault_hold+1869>.
frameptr=0xfffffe02b83578f0, ret_addr=0xffffffff807e05ee
Line 277 of "/usr/src/sys/vm/vm_fault.c" starts at address
0xffffffff807e05d9 <vm_fault+121> and ends at 0xffffffff807e05f1 <vm_fault+145>.
frameptr=0xfffffe02b8357930, ret_addr=0xffffffff80821342
Line 735 of "/usr/src/sys/amd64/amd64/trap.c" starts at address
0xffffffff80821342 <trap_pfault+290> and ends at 0xffffffff80821346
<trap_pfault+294>.
frameptr=0xfffffe02b83579c0, ret_addr=0xffffffff80820bda
Line 326 of "/usr/src/sys/amd64/amd64/trap.c" starts at address
0xffffffff80820bc6 <trap+1366> and ends at 0xffffffff80820bdf <trap+1391>.
frameptr=0xfffffe02b8357bd0, ret_addr=0xffffffff8082154a
Line 629 of "/usr/src/sys/amd64/amd64/trap.c" starts at address
0xffffffff8082154a <trap_check+42> and ends at 0xffffffff80821560
<dblfault_handler>.
frameptr=0xfffffe02b8357bf0, ret_addr=0xffffffff808091e3
Line 28 of "/usr/src/sys/libkern/explicit_bzero.c" starts at address
0xffffffff806e74dd <explicit_bzero+29> and ends at 0xffffffff8088a2d0
<__explicit_bzero_hook>.
frameptr=0x7fffffffe8f0Cannot access memory at address 0x7fffffffe8f8
Output of trace_stack looks perfectly sane for me up to the next trap frame.
--
Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?560E238F.9050609>