From owner-freebsd-arch@freebsd.org Fri Aug 28 18:27:12 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D96509C5648 for ; Fri, 28 Aug 2015 18:27:12 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AD4641125; Fri, 28 Aug 2015 18:27:12 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t7SIR5mq075778 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 28 Aug 2015 11:27:05 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t7SIR5WR075777; Fri, 28 Aug 2015 11:27:05 -0700 (PDT) (envelope-from jmg) Date: Fri, 28 Aug 2015 11:27:05 -0700 From: John-Mark Gurney To: John Baldwin Cc: "'freebsd-arch'" Subject: Re: Supporting cross-debugging vmcores in libkvm Message-ID: <20150828182705.GD33167@funkthat.com> References: <3121152.ujdxFEovO3@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3121152.ujdxFEovO3@ralph.baldwin.cx> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Fri, 28 Aug 2015 11:27:05 -0700 (PDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Aug 2015 18:27:13 -0000 John Baldwin wrote this message on Tue, Aug 04, 2015 at 10:56 -0700: > Many debuggers (recent gdb and lldb) support cross-architecture debugging > just fine. My current WIP port of kgdb to gdb7 supports cross-debugging for > remote targets already, but I wanted it to also support cross-debugging for > vmcores. > > The existing libkvm/kgdb code in the tree has some limited support for > cross-debugging. It requires building a custom libkvm (e.g. libkvm-i386.a) > and custom kgdb for each target platform. However, gdb (and lldb) both > support multiple targets in a single binary, so I'd like to have a single > kgdb binary that can cross-debug anything. > > I started hacking on libkvm last weekend and have a prototype that I've used > (along with some patches to my kgdb port) to debug an amd64 vmcore on an > i386 machine and vice versa. > > To do this I've made some additions to the libkvm API: > > 1) A new 'kvaddr_t' type represents a kernel virtual address. This is > similar to the psaddr_t type used for MI process addresses in userland > debugging. I almost reused psaddr_t directly, but that would have made > depend on . Instead, I opted for a separate > type. It is currently a uint64_t. I like this.. W/ the work Lovasko has been working on, having this type is good and makes the most sense... > 2) A new 'struct kvm_nlist'. This is a stripped-down version of > 'struct nlist' that uses kvadd_t for n_value instead of an unsigned > long. > > 3) kvm_native() returns true if an open kvm descriptor is for a native > kernel and memory image. > > 4) kvm_nlist2() is like kvm_nlist() but it uses 'struct kvm_nlist' > instead of 'struct nlist'. Internally symbol names are always > resolved to kvaddr_t addresses rather than u_long addresses. > Native kernels still use _fdnlist() from libc to resolve symbols. > Cross kernels use a caller supplied function to resolve symbols > (the older cross code for libkvm required the caller to provide > a global ps_pglobal_lookup symbol typically provided for > ). > > 5) kvm_open2() is like kvm_openfiles() except that it drops the unused > 'swapfile' argument and adds a new function pointer argument to a > symbol resolving function. The function pointer can be NULL in > which case only native kernels can be opened. Kernels used with > /dev/mem or /dev/kmem must be native. > > 6) kvm_read2() is like kvm_read() except that it uses kvaddr_t > instead of unsigned long for the kernel virtual address. All the above looks good... How will we prevent native only aware apps from getting confused when accessing non-native cores? Will kvm_openfiles fail for non-native cores? or will kvm_read fail for non-native cores? > Adding new symbols (specifically kvm_nlist2 and kvm_read2) preserves > ABI and API compatibility. Note that most libkvm functions such as > kvm_getprocs(), etc. only work with native kernels. I have not yet > done a full sweep to force them to fail for non-native kernels. For things like getprocs, we could move to using Lovasko's project to use ctf data to read the proc structures for non-native cores... Then the core parts of getprocs would not have to change, and it'd just work in all places.. > Also, the vnet and dpcpu stuff only works for native kernels currently > though that can be fixed at some point in the future. > > For the MD backends, I've added a new kvm_arch switch: > > struct kvm_arch { > int (*ka_probe)(kvm_t *); > int (*ka_initvtop)(kvm_t *); > void (*ka_freevtop)(kvm_t *); > int (*ka_kvatop)(kvm_t *, kvaddr_t, off_t *); > int (*ka_uvatop)(kvm_t *, const struct proc *, kvaddr_t, off_t *); > int ka_native; > }; > > Each backend implements the necessary callbacks (uvatop is optional) > and is added to a global linker set that kvm_open2() walks to find the > appropriate kvm_arch for a given kernel + vmcore. On x86 I've used > separate kvm_arch structures for "plain" vs minidumps. > > The backends now have to avoid using native headers. For ELF handling > this means using libelf instead of and raw mmap(). For > the x86 backends it meant defining some duplicate constants for certain > page table fields since can't be relied on (e.g. > I386_PG_V instead of PG_V). I added static assertions in the "native" > case (e.g. building kvm_i386.c on i386) to ensure the duplicate constants > match the originals. > > You can see the current WIP patches here: > > https://github.com/freebsd/freebsd/compare/master...bsdjhb:kgdb_enhancements > > What I'm mostly after is comments on the API, etc. Once that is settled I > will move forward on converting and/or stubbing the other backends (the > stub route would be to only support other backends on native systems for > now). You're API looks good... I can't see anything wrong w/ it... > Oh, and I do hope to have a 'KGDB' option for the devel/gdb port in the > near future. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."