Date: Mon, 22 Nov 2021 10:26:34 -0500 From: Ed Maste <emaste@freebsd.org> To: =?UTF-8?B?TWljaGHFgiBHw7Nybnk=?= <mgorny@moritz.systems> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org>, John Baldwin <jhb@freebsd.org>, Peter Wemm <peter@freebsd.org> Subject: Re: Looking for rationale for the minidump format Message-ID: <CAPyFy2Bhpj5MQK72b1A1X4=ez2okuuDo3rrx8Hn4BHk1OPtRKQ@mail.gmail.com> In-Reply-To: <305082d9f216bb8382773c074eaf7a5c3101cc13.camel@moritz.systems> References: <305082d9f216bb8382773c074eaf7a5c3101cc13.camel@moritz.systems>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 21 Nov 2021 at 09:42, Micha=C5=82 G=C3=B3rny <mgorny@moritz.systems= > wrote: > > Hi, everyone. > > As part of the work contracted by the FreeBSD Foundation, I'm working > on adding explicit minidump support to LLDB. When discussing > the options with upstream, I've been asked why FreeBSD created their own > minidump format. > > I did a bit of digging but TBH all the rationale I could get was to > create partial memory dumps. However, unless I'm mistaken the ELF > format is perfectly capable of that -- e.g. via creating an explicit > segment for every continuous active region. ELF can, indeed. One of my previous co-op students wrote a little tool to convert a minidump to ELF, see https://reviews.freebsd.org/D19253. That's what it does - creates a segment for each contiguous region. If we ended up going this way we could integrate D19253 into savecore. One issue is that originally ELF supported only a 16-bit segment count, the extension to support greater came later on. I think minidumps could also store userland pages, but without any tooling to access them it's not that useful IMO. Oh, I just found an old (2014!) thread where I discussed some of this with Greg Clayton: On Thu, 22 May 2014 at 13:43, Ed Maste <emaste@freebsd.org> wrote: > > Thanks for the insight Greg - a few comments: > > On 22 May 2014 12:39, Greg Clayton <gclayton@apple.com> wrote: > > Core file kernel debugging: > > - You shouldn't have to do too much to the current ProcessELFCore excep= t help it to locate hint address for the list of shared libraries and the m= ain kernel file in memory. This is done in: > > Note that kernel core files on FreeBSD aren't (currently) ELF files, > and may be a different format on different architectures, so there's > some extra work here. Dumps are written in a contiguous blob > (typically to swap) at the time of a crash, and then copied to the > filesystem after the next reboot. > > The dump function for amd64/x86_64 is minidumpsys in > minidump_machdep.c[1], and consists of: > > 1. Dump header (stripped during copy to filesystem) > 2. Minidump header > 3. Copy of kernel message buffer > 4. Bitmap of pages present in the dump > 5. Page directory pages > 6. Pages themselves > 7. Trailer > > All of this is abstracted by the libkvm library, which provide > kvm_read and kvm_write functions which a VA address parameter. I > think this is the right way to introduce support for FreeBSD kernel > cores at first, as I'll explain below. > > > Live kernel debugging: > > - You will make a new process subclass like you were saying unless the = GDB remote protocol is used to talk to the kernel? > > We currently have separate ways of interacting with the kernel for > remote and local debugging. For remote debugging the kernel has a > built-in GDB protocol stub which is available over a serial or > firewire interface, and this should mostly work with the existing LLDB > support. We do local debugging (examining threads, global objects, > etc.) via /dev/mem, abstracted by the same kvm_read and kvm_write > functions in libkvm. Since local FreeBSD debugging will be done on a > FreeBSD host (by definition), using libkvm makes sense. And since > we'll have that, using it also for cores makes sense to get started. > > Of course we'll want to allow debugging a FreeBSD kernel core on OS X > or Linux too. I think the way forward for us there is to create a > utility in the FreeBSD base system that converts the minidump format > to ELF, rather than teaching LLDB about our core formats in a > cross-platform way. This utility could be built-in to savecore(8), > the tool which reads the dump out of a raw partition and writes to the > filesystem. There is also ongoing discussion about bringing a KDP > target to the kernel for live debugging. > > > Extra credit: > > - Get threads that are context switched in memory to show up. In the Ma= cOSX kernel debugger we have the notion of 1 core we can talk to through th= e KDP interface which is the code that runs the debugger. > > We have code[2] for this already in kgdb, our GDB 6.1 port with kernel > support, and we should be able to re-use most of it as is. The kernel > threads just appear as regular threads in kgdb: > > (kgdb) info threads > 827 Thread 102126 (PID=3D28738: kgdb) sched_switch > (td=3D0xfffffe021882b000, newtd=3D0xfffffe0008392920, flags=3D<value > optimized out>) > at /tank/emaste/src/git-stable-9/sys/kern/sched_ule.c:1904 > 826 Thread 102334 (PID=3D28737: sudo) sched_switch > (td=3D0xfffffe02d2669920, newtd=3D0xfffffe0008392920, flags=3D<value > optimized out>) > at /tank/emaste/src/git-stable-9/sys/kern/sched_ule.c:1904 > 825 Thread 104699 (PID=3D28715: more) sched_switch > (td=3D0xfffffe049cec7920, newtd=3D0xfffffe0008390000, flags=3D<value > optimized out>) > at /tank/emaste/src/git-stable-9/sys/kern/sched_ule.c:1904 > ... > > -Ed > > [1] http://svnweb.freebsd.org/base/head/sys/amd64/amd64/minidump_machdep.= c?annotate=3D257216#l219 > [2] http://svnweb.freebsd.org/base/head/gnu/usr.bin/gdb/kgdb/kthr.c?annot= ate=3D246893
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPyFy2Bhpj5MQK72b1A1X4=ez2okuuDo3rrx8Hn4BHk1OPtRKQ>