From owner-freebsd-arch@FreeBSD.ORG Wed Jun 8 20:18:32 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA7EE16A41C for ; Wed, 8 Jun 2005 20:18:32 +0000 (GMT) (envelope-from antoine@madhouse.dreadbsd.org) Received: from barton.dreadbsd.org (madhouse.dreadbsd.org [82.67.196.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4275443D4C for ; Wed, 8 Jun 2005 20:18:31 +0000 (GMT) (envelope-from antoine@madhouse.dreadbsd.org) Received: from barton.dreadbsd.org (localhost [127.0.0.1]) by barton.dreadbsd.org (8.13.4/8.13.1) with ESMTP id j58KIUdg030918 for ; Wed, 8 Jun 2005 22:18:30 +0200 (CEST) (envelope-from antoine@madhouse.dreadbsd.org) Received: (from antoine@localhost) by barton.dreadbsd.org (8.13.4/8.13.1/Submit) id j58KITDC030917; Wed, 8 Jun 2005 22:18:29 +0200 (CEST) (envelope-from antoine) Date: Wed, 8 Jun 2005 22:18:29 +0200 From: Antoine Brodin To: freebsd-arch@freebsd.org Message-Id: <20050608221829.75c2de12.antoine.brodin@laposte.net> X-Mailer: Sylpheed version 1.9.12 (GTK+ 2.6.7; i386-portbld-freebsd6.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: RFC: Stack saving/tracing functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jun 2005 20:18:33 -0000 Hi, With Jeff@'s help, I implemented stack saving/tracing functionality. It allows you to save a stack (the program counters) and to manipulate it: print it to the console, print it to a sbuf, or print it to KTR. This can be used in lockmgr to simplify the API when DEBUG_LOCKS is defined. It can be useful for debugging. The code has only been tested on i386. It is available at http://bsd.miki.eu.org/~antoine/stack/stack-06-08b.diff To test it, patch your sources and compile a kernel with ddb. Then compile and kldload the test module: http://bsd.miki.eu.org/~antoine/stack/Makefile + http://bsd.miki.eu.org/~antoine/stack/testmod.c you can sysctl kern.teststack after you have kldloaded it too. It should print something like: %%% #0 0xc353287b at testmod_modevent+75 #1 0xc04c4841 at module_register_init+129 #2 0xc04be7d0 at linker_file_sysinit+144 #3 0xc04beb0a at linker_load_file+250 #4 0xc04c13c9 at linker_load_module+201 #5 0xc04bf74a at kldload+282 #6 0xc0621bb2 at syscall+658 #7 0xc061015f at Xint0x80_syscall+31 %%% You can also compile your kernel with ddb, KTR, KTR_COMPILE=(KTR_LOCK) and DEBUG_LOCKS and sysctl debug.do_stack_trace=1, sysctl debug.ktr.cpumask=255, sysctl debug.ktr.mask=8 and use ktrdump. It should produce traces like: %%% 316 687503258397 lockmgr(): lkp == 0xc424e1b4 (lk_wmesg == "ufs"), owner == 0xc32a6640, exclusivecount == 1, flags == 0x6, td == 0xc32a6640 317 687503258733 #0 0xc04c3ad3 0xc052eba5 0xc0635d78 0xc0538936 0xc053f153 0xc053f06f 318 687503258817 #1 0xc06274b2 0xc06159ef 0x0 0x0 0x0 0x0 319 687503259671 LOCK (sleep mutex) vnode_free_list r = 0 at /usr/src/sys/kern/vfs_subr.c:2752 %%% and you can do post processing with addr2line: %%% $ addr2line -f -e kernel.debug 0xc04c3ad3 0xc052eba5 0xc0635d78 \ 0xc0538936 0xc053f153 0xc053f06f lockmgr /usr/src/sys/kern/kern_lock.c:177 vop_stdunlock /usr/src/sys/kern/vfs_default.c:271 VOP_UNLOCK_APV /usr/obj/usr/src/sys/BARTONDBG/vnode_if.c:1692 vput /usr/src/sys/kern/vfs_subr.c:1974 kern_lstat /usr/src/sys/kern/vfs_syscalls.c:2126 lstat /usr/src/sys/kern/vfs_syscalls.c:2104 %%% Known problems: . On alpha a stack_save should be expensive since it uses a db_search_symbol/db_symbol_values for each function in the stack. . On ia64 the code hasn't been written yet, I don't know if libuwx is reentrant and if it can sleep. Another question: Since the stack saving/tracing functionality depends on ddb, should kern/subr_stack.c be moved to ddb/stack.c and sys/stack.h to ddb/stack.h? Could you review/comment/test it? Cheers, Antoine