From owner-freebsd-arch@freebsd.org Tue Aug 4 18:14:50 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24D4A9B2993 for ; Tue, 4 Aug 2015 18:14:50 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F40C91EB9 for ; Tue, 4 Aug 2015 18:14:49 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (75-48-78-19.lightspeed.cncrca.sbcglobal.net [75.48.78.19]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 145B4B93A for ; Tue, 4 Aug 2015 14:14:48 -0400 (EDT) From: John Baldwin To: 'freebsd-arch' Subject: Supporting cross-debugging vmcores in libkvm Date: Tue, 04 Aug 2015 10:56:09 -0700 Message-ID: <3121152.ujdxFEovO3@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 04 Aug 2015 14:14:48 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2015 18:14:50 -0000 Many debuggers (recent gdb and lldb) support cross-architecture debugging just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for remote targets already, but I wanted it to also support cross-debugging for vmcores. The existing libkvm/kgdb code in the tree has some limited support for cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a) and custom kgdb for each target platform. However, gdb (and lldb) both support multiple targets in a single binary, so I'd like to have a single kgdb binary that can cross-debug anything. I started hacking on libkvm last weekend and have a prototype that I've used (along with some patches to my kgdb port) to debug an amd64 vmcore on an i386 machine and vice versa. To do this I've made some additions to the libkvm API: 1) A new 'kvaddr_t' type represents a kernel virtual address. This is similar to the psaddr_t type used for MI process addresses in userland debugging. I almost reused psaddr_t directly, but that would have made depend on . Instead, I opted for a separate type. It is currently a uint64_t. 2) A new 'struct kvm_nlist'. This is a stripped-down version of 'struct nlist' that uses kvadd_t for n_value instead of an unsigned long. 3) kvm_native() returns true if an open kvm descriptor is for a native kernel and memory image. 4) kvm_nlist2() is like kvm_nlist() but it uses 'struct kvm_nlist' instead of 'struct nlist'. Internally symbol names are always resolved to kvaddr_t addresses rather than u_long addresses. Native kernels still use _fdnlist() from libc to resolve symbols. Cross kernels use a caller supplied function to resolve symbols (the older cross code for libkvm required the caller to provide a global ps_pglobal_lookup symbol typically provided for ). 5) kvm_open2() is like kvm_openfiles() except that it drops the unused 'swapfile' argument and adds a new function pointer argument to a symbol resolving function. The function pointer can be NULL in which case only native kernels can be opened. Kernels used with /dev/mem or /dev/kmem must be native. 6) kvm_read2() is like kvm_read() except that it uses kvaddr_t instead of unsigned long for the kernel virtual address. Adding new symbols (specifically kvm_nlist2 and kvm_read2) preserves ABI and API compatibility. Note that most libkvm functions such as kvm_getprocs(), etc. only work with native kernels. I have not yet done a full sweep to force them to fail for non-native kernels. Also, the vnet and dpcpu stuff only works for native kernels currently though that can be fixed at some point in the future. For the MD backends, I've added a new kvm_arch switch: struct kvm_arch { int (*ka_probe)(kvm_t *); int (*ka_initvtop)(kvm_t *); void (*ka_freevtop)(kvm_t *); int (*ka_kvatop)(kvm_t *, kvaddr_t, off_t *); int (*ka_uvatop)(kvm_t *, const struct proc *, kvaddr_t, off_t *); int ka_native; }; Each backend implements the necessary callbacks (uvatop is optional) and is added to a global linker set that kvm_open2() walks to find the appropriate kvm_arch for a given kernel + vmcore. On x86 I've used separate kvm_arch structures for "plain" vs minidumps. The backends now have to avoid using native headers. For ELF handling this means using libelf instead of and raw mmap(). For the x86 backends it meant defining some duplicate constants for certain page table fields since can't be relied on (e.g. I386_PG_V instead of PG_V). I added static assertions in the "native" case (e.g. building kvm_i386.c on i386) to ensure the duplicate constants match the originals. You can see the current WIP patches here: https://github.com/freebsd/freebsd/compare/master...bsdjhb:kgdb_enhancements What I'm mostly after is comments on the API, etc. Once that is settled I will move forward on converting and/or stubbing the other backends (the stub route would be to only support other backends on native systems for now). Oh, and I do hope to have a 'KGDB' option for the devel/gdb port in the near future. -- John Baldwin