From owner-dev-commits-doc-all@freebsd.org Thu Sep 23 20:29:52 2021 Return-Path: Delivered-To: dev-commits-doc-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D699D6AD66A for ; Thu, 23 Sep 2021 20:29:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HFmxD5D83z4ZZT; Thu, 23 Sep 2021 20:29:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 94D341916; Thu, 23 Sep 2021 20:29:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 18NKTqAA018054; Thu, 23 Sep 2021 20:29:52 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 18NKTqI1018053; Thu, 23 Sep 2021 20:29:52 GMT (envelope-from git) Date: Thu, 23 Sep 2021 20:29:52 GMT Message-Id: <202109232029.18NKTqI1018053@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Mitchell Horne Subject: git: f978ca7e8e - main - developers-handbook: update the section on Remote GDB MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mhorne X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f978ca7e8e79a9050f0473e9f6dfa49d498b65b1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-doc-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the doc repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2021 20:29:52 -0000 The branch main has been updated by mhorne (src committer): URL: https://cgit.FreeBSD.org/doc/commit/?id=f978ca7e8e79a9050f0473e9f6dfa49d498b65b1 commit f978ca7e8e79a9050f0473e9f6dfa49d498b65b1 Author: Mitchell Horne AuthorDate: 2021-08-10 22:57:45 +0000 Commit: Mitchell Horne CommitDate: 2021-09-23 20:28:18 +0000 developers-handbook: update the section on Remote GDB It is quite out of date, and the existing text doesn't work if followed exactly. This should at least provide enough advice to make use of the feature on amd64. - Update list of required kernel options - Mention that the GDB backend is disabled on stable and release branches - Improve the flow of the example + remove outdated steps - Update the example console output - Give a tip about COPTFLAGS and locals - Some minor additions/improvements to wording Reviewed by: jhb, ygy, domagoj.stolfa_gmail.com Approved by: ygy (docs) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D31494 --- .../developers-handbook/kerneldebug/_index.adoc | 84 ++++++++++++++-------- 1 file changed, 53 insertions(+), 31 deletions(-) diff --git a/documentation/content/en/books/developers-handbook/kerneldebug/_index.adoc b/documentation/content/en/books/developers-handbook/kerneldebug/_index.adoc index 5385ac1426..87c4da03bd 100644 --- a/documentation/content/en/books/developers-handbook/kerneldebug/_index.adoc +++ b/documentation/content/en/books/developers-handbook/kerneldebug/_index.adoc @@ -533,69 +533,91 @@ Remember that it is hard to read the on-line manual while single-stepping the ke [[kerneldebug-online-gdb]] == On-Line Kernel Debugging Using Remote GDB +The FreeBSD kernel provides a second KDB backend for on-line debugging: man:gdb[4]. This feature has been supported since FreeBSD 2.2, and it is actually a very neat one. -GDB has already supported _remote debugging_ for a long time. +GDB has supported _remote debugging_ for a long time. This is done using a very simple protocol along a serial line. -Unlike the other methods described above, you will need two machines for doing this. -One is the host providing the debugging environment, including all the sources, and a copy of the kernel binary with all the symbols in it, -and the other one is the target machine that simply runs a similar copy of the very same kernel (but stripped of the debugging information). - -You should configure the kernel in question with `config -g` if building the "traditional" way. -If building the "new" way, make sure that `makeoptions DEBUG=-g` is in the configuration. -In both cases, include `DDB` in the configuration, and compile it as usual. -This gives a large binary, due to the debugging information. -Copy this kernel to the target machine, strip the debugging symbols off with `strip -x`, and boot it using the `-d` boot option. +Unlike the other debugging methods described above, you will need two machines for doing this. +One is the host providing the debugging environment, including all the sources, and a copy of the kernel binary with all the symbols in it. +The other is the target machine that runs a copy of the very same kernel (optionally stripped of the debugging information). + +In order to use remote GDB, ensure that the following options are present in your kernel configuration: +[.programlisting] +.... +makeoptions DEBUG=-g +options KDB +options GDB +.... + +Note that the `GDB` option is turned off by default in `GENERIC` kernels on -STABLE and -RELEASE branches, but enabled on -CURRENT. + +Once built, copy the kernel to the target machine, and boot it. Connect the serial line of the target machine that has "flags 080" set on its uart device to any serial line of the debugging host. -See man:uart[4] for information on how to set the flags on an uart device. -Now, on the debugging machine, go to the compile directory of the target kernel, and start `gdb`: +See man:uart[4] for information on how to set the flags on a uart device. +The target machine must be made to enter the GDB backend, either due to a panic or by taking a purposeful trap into the debugger. +Before doing this, select the GDB debugger backend: [source,bash] .... -% kgdb kernel -GDB is free software and you are welcome to distribute copies of it - under certain conditions; type "show copying" to see the conditions. -There is absolutely no warranty for GDB; type "show warranty" for details. -GDB 4.16 (i386-unknown-freebsd), -Copyright 1996 Free Software Foundation, Inc... -(kgdb) +# sysctl debug.kdb.current=gdb +debug.kdb.current: ddb -> gdb .... -Initialize the remote debugging session (assuming the first serial port is being used) by: +[NOTE] +==== +The supported backends can be listed by the `debug.kdb.available` sysctl. +If the kernel configuration includes `options DDB`, then man:ddb[4] will be selected by default. +If `gdb` does not appear in the list of available backends, then the debug serial port may not have been configured correctly. +==== +Then, force entry to the debugger: [source,bash] .... -(kgdb) target remote /dev/cuau0 +# sysctl debug.kdb.enter=1 +debug.kdb.enter: 0KDB: enter: sysctl debug.kdb.enter .... -Now, on the target host (the one that entered DDB right before even starting the device probe), type: +The target machine now awaits connection from a remote GDB client. +On the debugging machine, go to the compile directory of the target kernel, and start `gdb`: [source,bash] .... -Debugger("Boot flags requested debugger") -Stopped at Debugger+0x35: movb $0, edata+0x51bc -db> gdb +# cd /usr/obj/usr/src/amd64.amd64/sys/GENERIC/ +# kgdb kernel +GNU gdb (GDB) 10.2 [GDB v10.2 for FreeBSD] +Copyright (C) 2021 Free Software Foundation, Inc. +... +Reading symbols from kernel... +Reading symbols from /usr/obj/usr/src/amd64.amd64/sys/GENERIC/kernel.debug... +(kgdb) .... -DDB will respond with: +Initialize the remote debugging session (assuming the first serial port is being used) by: [source,bash] .... -Next trap will enter GDB remote protocol mode +(kgdb) target remote /dev/cuau0 .... -Every time you type `gdb`, the mode will be toggled between remote GDB and local DDB -In order to force a next trap immediately, simply type `s` (step). Your hosting GDB will now gain control over the target kernel: [source,bash] .... Remote debugging using /dev/cuau0 -Debugger (msg=0xf01b0383 "Boot flags requested debugger") - at ../../i386/i386/db_interface.c:257 +kdb_enter (why=, msg=) at /usr/src/sys/kern/subr_kdb.c:506 +506 kdb_why = KDB_WHY_UNSET; (kgdb) .... +[TIP] +==== +Depending on the compiler used, some local variables may appear as ``, preventing them from being inspected directly by `gdb`. +If this causes problems while debugging, it is possible to build the kernel at a decreased optimization level, which may improve the visibility of some variables. +This can be done by passing `COPTFLAGS=-O1` to man:make[1]. +However, certain classes of kernel bugs may manifest differently (or not at all) when the optimization level is changed. +==== + You can use this session almost as any other GDB session, including full access to the source, running it in gud-mode inside an Emacs window (which gives you an automatic source code display in another Emacs window), etc.