Date: Fri, 28 Aug 2015 11:27:05 -0700 From: John-Mark Gurney <jmg@funkthat.com> To: John Baldwin <jhb@freebsd.org> Cc: "'freebsd-arch'" <freebsd-arch@freebsd.org> Subject: Re: Supporting cross-debugging vmcores in libkvm Message-ID: <20150828182705.GD33167@funkthat.com> In-Reply-To: <3121152.ujdxFEovO3@ralph.baldwin.cx> References: <3121152.ujdxFEovO3@ralph.baldwin.cx>
next in thread | previous in thread | raw e-mail | index | archive | help
John Baldwin wrote this message on Tue, Aug 04, 2015 at 10:56 -0700:
> Many debuggers (recent gdb and lldb) support cross-architecture debugging
> just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for
> remote targets already, but I wanted it to also support cross-debugging for
> vmcores.
>
> The existing libkvm/kgdb code in the tree has some limited support for
> cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a)
> and custom kgdb for each target platform. However, gdb (and lldb) both
> support multiple targets in a single binary, so I'd like to have a single
> kgdb binary that can cross-debug anything.
>
> I started hacking on libkvm last weekend and have a prototype that I've used
> (along with some patches to my kgdb port) to debug an amd64 vmcore on an
> i386 machine and vice versa.
>
> To do this I've made some additions to the libkvm API:
>
> 1) A new 'kvaddr_t' type represents a kernel virtual address. This is
> similar to the psaddr_t type used for MI process addresses in userland
> debugging. I almost reused psaddr_t directly, but that would have made
> <libkvm.h> depend on <sys/procfs.h>. Instead, I opted for a separate
> type. It is currently a uint64_t.
I like this.. W/ the work Lovasko has been working on, having this
type is good and makes the most sense...
> 2) A new 'struct kvm_nlist'. This is a stripped-down version of
> 'struct nlist' that uses kvadd_t for n_value instead of an unsigned
> long.
>
> 3) kvm_native() returns true if an open kvm descriptor is for a native
> kernel and memory image.
>
> 4) kvm_nlist2() is like kvm_nlist() but it uses 'struct kvm_nlist'
> instead of 'struct nlist'. Internally symbol names are always
> resolved to kvaddr_t addresses rather than u_long addresses.
> Native kernels still use _fdnlist() from libc to resolve symbols.
> Cross kernels use a caller supplied function to resolve symbols
> (the older cross code for libkvm required the caller to provide
> a global ps_pglobal_lookup symbol typically provided for
> <proc_service.h>).
>
> 5) kvm_open2() is like kvm_openfiles() except that it drops the unused
> 'swapfile' argument and adds a new function pointer argument to a
> symbol resolving function. The function pointer can be NULL in
> which case only native kernels can be opened. Kernels used with
> /dev/mem or /dev/kmem must be native.
>
> 6) kvm_read2() is like kvm_read() except that it uses kvaddr_t
> instead of unsigned long for the kernel virtual address.
All the above looks good...
How will we prevent native only aware apps from getting confused when
accessing non-native cores? Will kvm_openfiles fail for non-native
cores? or will kvm_read fail for non-native cores?
> Adding new symbols (specifically kvm_nlist2 and kvm_read2) preserves
> ABI and API compatibility. Note that most libkvm functions such as
> kvm_getprocs(), etc. only work with native kernels. I have not yet
> done a full sweep to force them to fail for non-native kernels.
For things like getprocs, we could move to using Lovasko's project
to use ctf data to read the proc structures for non-native cores...
Then the core parts of getprocs would not have to change, and it'd
just work in all places..
> Also, the vnet and dpcpu stuff only works for native kernels currently
> though that can be fixed at some point in the future.
>
> For the MD backends, I've added a new kvm_arch switch:
>
> struct kvm_arch {
> int (*ka_probe)(kvm_t *);
> int (*ka_initvtop)(kvm_t *);
> void (*ka_freevtop)(kvm_t *);
> int (*ka_kvatop)(kvm_t *, kvaddr_t, off_t *);
> int (*ka_uvatop)(kvm_t *, const struct proc *, kvaddr_t, off_t *);
> int ka_native;
> };
>
> Each backend implements the necessary callbacks (uvatop is optional)
> and is added to a global linker set that kvm_open2() walks to find the
> appropriate kvm_arch for a given kernel + vmcore. On x86 I've used
> separate kvm_arch structures for "plain" vs minidumps.
>
> The backends now have to avoid using native headers. For ELF handling
> this means using libelf instead of <machine/elf.h> and raw mmap(). For
> the x86 backends it meant defining some duplicate constants for certain
> page table fields since <machine/pmap.h> can't be relied on (e.g.
> I386_PG_V instead of PG_V). I added static assertions in the "native"
> case (e.g. building kvm_i386.c on i386) to ensure the duplicate constants
> match the originals.
>
> You can see the current WIP patches here:
>
> https://github.com/freebsd/freebsd/compare/master...bsdjhb:kgdb_enhancements
>
> What I'm mostly after is comments on the API, etc. Once that is settled I
> will move forward on converting and/or stubbing the other backends (the
> stub route would be to only support other backends on native systems for
> now).
You're API looks good... I can't see anything wrong w/ it...
> Oh, and I do hope to have a 'KGDB' option for the devel/gdb port in the
> near future.
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150828182705.GD33167>