From owner-freebsd-questions@FreeBSD.ORG Tue Feb 28 06:50:33 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 667801065670 for ; Tue, 28 Feb 2012 06:50:33 +0000 (UTC) (envelope-from msabramo@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 26F808FC08 for ; Tue, 28 Feb 2012 06:50:32 +0000 (UTC) Received: by obbwc7 with SMTP id wc7so1629797obb.13 for ; Mon, 27 Feb 2012 22:50:32 -0800 (PST) Received-SPF: pass (google.com: domain of msabramo@gmail.com designates 10.60.13.1 as permitted sender) client-ip=10.60.13.1; Authentication-Results: mr.google.com; spf=pass (google.com: domain of msabramo@gmail.com designates 10.60.13.1 as permitted sender) smtp.mail=msabramo@gmail.com; dkim=pass header.i=msabramo@gmail.com Received: from mr.google.com ([10.60.13.1]) by 10.60.13.1 with SMTP id d1mr5947424oec.41.1330411832653 (num_hops = 1); Mon, 27 Feb 2012 22:50:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=mpH2wo2J5LhFoSzltxH+gfLUft4tAmZm5UESEjCBTlY=; b=kAW+wgfBM6GMIA3KFJJyNxrFBT0MRbBGn+iE+YINgBAI8nfTTHTrKjLBue6uzbRtXo PFSvEZpZlqYqSfJI2+DwoPtMeGONP7manOSG8KF7U/+0X5IotHvoZU4IIBZHqThfOA/Z LE6yPrlY0x64Y6834FDsPk4+t1CsFSjposSAg= MIME-Version: 1.0 Received: by 10.60.13.1 with SMTP id d1mr5244407oec.41.1330410137739; Mon, 27 Feb 2012 22:22:17 -0800 (PST) Received: by 10.60.24.7 with HTTP; Mon, 27 Feb 2012 22:22:17 -0800 (PST) Date: Mon, 27 Feb 2012 22:22:17 -0800 Message-ID: From: Marc Abramowitz To: freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: DTrace userland X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Feb 2012 06:50:33 -0000 I'm using FreeBSD 9.0 on amd64 in VMware Fusion and trying to DTrace userland programs. I think I must be doing something wrong. I recompiled my kernel and world, following the instructions at http://wiki.freebsd.org/DTrace and I've read http://wiki.freebsd.org/DTrace/userland: The test.c pid provider example worked fine for me: $ sudo dtrace -s pid.d -c ./test dtrace: script 'pid.d' matched 2 probes dtrace: buffer size lowered to 1m CPU ID FUNCTION:NAME 0 43030 main:entry 0 43031 sleep:entry 0 43031 sleep:entry 0 43031 sleep:entry As does a simple probe of test.c specified with the -n option: [marca@freebsd9-0 ~]$ sudo dtrace -n 'pid$target:test:main:entry' -c ./test dtrace: description 'pid$target:test:main:entry' matched 1 probe dtrace: buffer size lowered to 1m CPU ID FUNCTION:NAME 0 43030 main:entry When I start trying to dtrace other programs, things don't go so well... $ sudo dtrace -n ":::entry" -c /usr/local/bin/python Python 2.4.5 (#2, Dec 5 2011, 15:19:09) [GCC 4.2.1 20070831 patched [FreeBSD]] on freebsd9 Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.getpid() 1603 >>> dtrace: failed to control pid 1603: process exited with status 0 $ sudo dtrace -n 'pid$target:::entry' -c '/bin/cat hello_world.txt' dtrace: description 'pid$target:::entry' matched 3315 probes dtrace: buffer size lowered to 1m CPU ID FUNCTION:NAME 0 43448 _rtld_bind:entry 0 43903 rlock_acquire:entry 0 43125 def_thread_set_flag:entry (Had to hit Ctrl-C to exit; it never displayed hello_world.txt to stdout) [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo make install ... [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo dtrace -n 'pid$target:::entry' -c '/usr/local/bin/gcat config.log' dtrace: description 'pid$target:::entry' matched 3823 probes dtrace: buffer size lowered to 1m CPU ID FUNCTION:NAME 0 43524 _rtld_bind:entry 0 43979 rlock_acquire:entry 0 43201 def_thread_set_flag:entry ^C $ sudo dtrace -n 'pid$target:cat:main:entry' -c '/bin/cat hello_world.txt' causes a kernel panic. According to the core.txt file, it was a "Fatal trap 10: trace trap while in kernel mode" and here's the KDB backtrace: KDB: stack backtrace: #0 0xffffffff8089025e at kdb_backtrace+0x5e #1 0xffffffff80858ce7 at panic+0x187 #2 0xffffffff80b4bf20 at trap_fatal+0x290 #3 0xffffffff80b4c540 at trap+0x180 #4 0xffffffff80b36963 at calltrap+0x8 #5 0xffffffff8162583d at dtrace_assfail+0x2d #6 0xffffffff8188aa2e at fasttrap_provider_free+0x1de #7 0xffffffff8188ad13 at fasttrap_pid_cleanup_cb+0x1c3 #8 0xffffffff8086dfa1 at softclock+0x3a1 #9 0xffffffff8082d724 at intr_event_execute_handlers+0x104 #10 0xffffffff8082eee4 at ithread_loop+0xa4 #11 0xffffffff8082a34f at fork_exit+0x11f #12 0xffffffff80b36e8e at fork_trampoline+0xe [marca@freebsd9-0 /usr/ports/sysutils/coreutils]$ sudo dtrace -n 'pid$target:gcat::entry' -c '/usr/local/bin/gcat config.log' (Another kernel panic) I can provide full crash dumps if necessary. Any idea what's going on here? Cheers, Marc