Date: Fri, 15 May 2015 09:23:54 -0700 From: Sean Bruno <sbruno@ignoranthack.me> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-hackers@freebsd.org Subject: Re: How to get anything useful out of kgdb? Message-ID: <55561D9A.30309@ignoranthack.me> In-Reply-To: <19618854.y3EeXVtCGX@ralph.baldwin.cx> References: <554E41EE.2010202@ignoranthack.me> <2063489.pgabuk9nPJ@ralph.baldwin.cx> <55561803.9050102@ignoranthack.me> <19618854.y3EeXVtCGX@ralph.baldwin.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 05/15/15 09:08, John Baldwin wrote: > On Friday, May 15, 2015 09:00:03 AM Sean Bruno wrote: >> On 05/15/15 08:50, John Baldwin wrote: >>> On Saturday, May 09, 2015 10:20:46 AM Sean Bruno wrote: >>>> tl;dr What are the kernel config options to get good output >>>> of kgdb? >>>> >>>> >>>> I'm trying to get the ability to debug and display internal >>>> variables, as one does, with kgdb. I'm *must* be doing this >>>> wrong as I cannot get any useful output from accessing >>>> variables that were JUST accessed in order to invoke a >>>> panic() that I have inserted. This is a GENERIC kernel >>>> without INVARIANTS and without WITNESS: >>>> >>>> https://people.freebsd.org/~sbruno/wtf_kgdb.txt >>>> >>>> I seem to have debug enabled and am able to browse source, >>>> but I obviously haven't compiled correctly as things are >>>> optimized out. >>> >>> 1) gdb7 does a better job. I hope to get the kgdb patches >>> into the port soon. If you are feeling brave: >>> >>> # add texinfo for HEAD % pkg install gmake bison >>> >>> % git clone https://github.com/bsdjhb/gdb.git % cd gdb % git >>> checkout freebsd-7.9.0-kgdb % fetch >>> http://people.freebsd.org/~jhb/gdb/build % sh ./build % cd obj >>> # or obj.i386 for i386 % gmake >>> >>> Then use /path/to/git/gdb/obj/gdb/kgdb >>> >>> 2) Even with gdb7 it can't figure out variables that it should >>> figure out sometimes. Other options are either to find the >>> variable in a higher frame (e.g. if it is something like 'td' >>> or a driver softc) or to start poking around in the >>> dissassembly to work out which register it is in (or which >>> register points to a structure that contains it) and go from >>> there. >>> >> >> So one non-obvious thing that I'd like an explanation about: If >> I manually break to the debbugger and cause a dump (doadump), how >> do I poke around in the crash dump later on to find a thread that >> I'm looking for, e.g. I want to poke at various bits inside >> em(4). > > The current thread in a crashdump is the one that calls dump or > panic. If you want to find a different thread you have a couple of > options. First, kgdb exposes each kernel thread as a thread to gdb. > You can switch to individual kernel threads by using 'tid N' (where > N is td_tid) or 'proc N' (where N is a pid). You can also use > 'thread N', but that requires using gdb's internal thread number > which has no relation to tids or pids. There are several ways to > enumerate the list of threads in a core: > > 1) You can use 'info threads' in kgdb to get a list of threads. > > 2) You can run 'ps' against the crashdump using -N and -M to get > the tid and pid values (at least until someone like glebius@ goes > on more of an anti-libkvm tirade and removes crashdump support from > ps). crashinfo runs ps by default, so if you have that enabled you > can look in core.txt.N to get the list of tids / pids. > > 3) You can grab my gdb scripts at ~jhb/gdb/ and source 'gdb6'. > That gives you a 'ps' command kind of like DDB's ps. > OMG ... thank you. :-) I'm guessing, that from the list here, no useful information is going to be available. It looks like all my interrupt threads are idle and all taskqueue threads are asleep: 59 Thread 100049 (PID=12: intr/irq257: em0:rx0) sched_switch (td=0xfffff80002c47940, newtd=<value optimized out>, flags=<value optimized out>) at /home/sbruno/bsd/em_mq/sys/kern/sched_ule.c:1956 58 Thread 100051 (PID=12: intr/irq258: em0:rx1) sched_switch (td=0xfffff80002c47000, newtd=<value optimized out>, flags=<value optimized out>) at /home/sbruno/bsd/em_mq/sys/kern/sched_ule.c:1956 57 Thread 100053 (PID=12: intr/irq259: em0:tx0) sched_switch (td=0xfffff80002c424a0, newtd=<value optimized out>, flags=<value optimized out>) at /home/sbruno/bsd/em_mq/sys/kern/sched_ule.c:1956 56 Thread 100055 (PID=12: intr/irq260: em0:tx1) sched_switch (td=0xfffff80002c5e940, newtd=<value optimized out>, flags=<value optimized out>) at /home/sbruno/bsd/em_mq/sys/kern/sched_ule.c:1956 55 Thread 100057 (PID=12: intr/irq261: em0:link) sched_switch (td=0xfffff80002c5e000, newtd=<value optimized out>, flags=<value optimized out>) at /home/sbruno/bsd/em_mq/sys/kern/sched_ule.c:1956 54 Thread 100058 (PID=12: intr/irq262: em1:rx0) cpustop_handler () at /home/sbruno/bsd/em_mq/sys/x86/x86/mp_x86.c:987 53 Thread 100060 (PID=12: intr/irq263: em1:rx1) cpustop_handler () at /home/sbruno/bsd/em_mq/sys/x86/x86/mp_x86.c:987 52 Thread 100062 (PID=12: intr/irq264: em1:tx0) cpustop_handler () at /home/sbruno/bsd/em_mq/sys/x86/x86/mp_x86.c:987 51 Thread 100064 (PID=12: intr/irq265: em1:tx1) cpustop_handler () at /home/sbruno/bsd/em_mq/sys/x86/x86/mp_x86.c:987 50 Thread 100066 (PID=12: intr/irq266: em1:link) cpustop_handler () at /home/sbruno/bsd/em_mq/sys/x86/x86/mp_x86.c:987 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJVVh2YXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5kUlMH/2ZHs/G/Ql9a+ah3oyyFXosh /KVZ17K6CXywAVPhL2+twXUuzu5zDj4spu4zdlbSIN5BqzpbPyiJgtAc1iT3Elco U7J5KQSaGOopd0ldwkTShormsvUlhDEy3Ed529qN5ppf1YI/yZ6HGYpYQ2gdvRJ1 5148j+q/aLLDpFFbuffT2NXgeMgahW79SEOl/kiI+Qn5KIT+Kp417+1ZriTz2Xx8 TRIayQi2IItU4AmowSToBt5hFqWVywAYLLRur3z12OgAbgodcxLg4IN3muHzfpPY lJh57lb4pSIL2Ic2a7tBcXyaT4lsUKI4uG7Sxa0pq+WUW6zY+XRcc1y+K6k5tCg= =ZIqy -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55561D9A.30309>