From owner-freebsd-dtrace@FreeBSD.ORG Sun Oct 27 19:53:14 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 86FFAEC4 for ; Sun, 27 Oct 2013 19:53:14 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 56C3C2938 for ; Sun, 27 Oct 2013 19:53:14 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id e14so9919538iej.11 for ; Sun, 27 Oct 2013 12:53:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=FMbwu3goFNc9Hkh0D74yQOEbUxA61xTlGGlhrHwbARU=; b=g1aE1DQpSH2ZtIhT19V+Lh/5P8WXEIDoNv+mITMfimDLCza2P25Zm1GrLe/Xi2kLDF 6zGAg1lJ+C4PE+HYpEsYdMrUDHn4wpMXTIGUNeS16fMO8JtroIdvKXsnd1u4ke5mBU7d mLcjsDTPdlabgqpTQdFJ933AgpYHjv5FSmzvzOwkMAQ5YE+EN3de2ZyiIfAfP5ZKj5yR YPMAKavpKxQwoWAH9MubuGM1gKaTE+fhNuTDkjfdRtED6AokQVGhuJvZGjBETTdZd4L/ vH1vsBdN2e7NuKuVdMbS/sx3f9c8wNgqoe4WujZymWRTim9EQQUnNHQK7YWE4JCtkAiW jTCg== X-Received: by 10.42.208.211 with SMTP id gd19mr11433061icb.15.1382903593634; Sun, 27 Oct 2013 12:53:13 -0700 (PDT) Received: from charmander.uwaterloo.ca (rn-nat2-uw-129-97-124-209.net.uwaterloo.ca. [129.97.124.209]) by mx.google.com with ESMTPSA id i11sm15895127igh.0.2013.10.27.12.53.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 27 Oct 2013 12:53:12 -0700 (PDT) Sender: Mark Johnston Date: Sun, 27 Oct 2013 15:53:07 -0400 From: Mark Johnston To: symbolics@gmx.com Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131027195307.GA3206@charmander.uwaterloo.ca> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> <20131025145956.GA26814@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131025145956.GA26814@lemon> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Oct 2013 19:53:14 -0000 On Fri, Oct 25, 2013 at 03:59:56PM +0100, symbolics@gmx.com wrote: > On Fri, Oct 25, 2013 at 11:47:06AM +0100, symbolics@gmx.com wrote: > > On Wed, Oct 23, 2013 at 10:59:02PM -0400, Mark Johnston wrote: > > > On Wed, Oct 23, 2013 at 09:30:09PM +0100, symbolics@gmx.com wrote: > > > > Hi, > > > > > > > > http://dtrace.org/blogs/brendan/2011/02/11/dtrace-pid-provider-arguments/ > > > > > > > > I tried to follow some of the examples but I crash the Firefox process > > > > each time. Sometimes DTrace manages to collect a little data before the > > > > death. > > > > > > > > [...] > > > > > > > > Is this a known problem or should I send a PR? > > > > > > Thanks for reporting this: I was able to reproduce the crash and managed > > > to find a nasty pair of bugs. Could you test the patch below and let me > > > know if it fixes the problem for you as well? If you see more crashes, > > > please include the backtrace and signo from gdb again; it would likely > > > be a different problem that needs to be debugged and fixed separately. > > > > Hi Mark, > > > > This helps but there still may be some issues. First time I used this > > I found that when I killed the DTrace process Firefox went down too > > with a SIGTRAP. I have a possibly unhelpful core from this: > > > > Another data point. I attached to mutt and reviewed some of the calls it > was making. Subsequently I killed DTrace, went to to look at other > things and a while later when back to check my mail. On attempting to > change into a different mail folder mutt died with a SIGTRAP. It seems > like DTrace isn't tidying up after itself? > > (gdb) bt > #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff6c28) > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > #1 0x0000000000000000 in ?? () Ok, I think I've figured out this one too. As you note, dtrace(1) isn't cleaning up some of its breakpoints properly when it detaches. In particular, it's not stopping the victim process before it tries to remove breakpoints using ptrace(2); however, ptrace requires the target process to be stopped, else it will return EBUSY. So the breakpoint in the rtld gets left behind, and it turns out that r_debug_state() is called every time a process tries to dlopen() a shared object. mutt was a good example since it seems to dlopen() iconv-related stuff as I scan through my inbox; one can inspect this with DTrace. :) i.e. with something like 'pid$target::dlopen:entry {trace(copyinstr(arg0));}' With this observation it becomes easy to reproduce the problem using a test program that does something like while (1) { dlopen("/lib/libnonexistent.so.100", RTLD_LAZY); sleep(1); } A somewhat crude patch which fixes this for me is below; it just adds code to send SIGSTOP to the target process before trying to remove breakpoints. Does anyone see any problems with this? Perhaps it should be libproc's responsibility to ensure that the victim process is stopped before trying a ptrace(PT_IO, ...) to add/remove breakpoints? Thanks, -Mark diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c index d40a0ae..6ed78e4 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c @@ -505,7 +505,7 @@ dt_proc_control(void *arg) dt_proc_t *dpr = datap->dpcd_proc; dt_proc_hash_t *dph = dpr->dpr_hdl->dt_procs; struct ps_prochandle *P = dpr->dpr_proc; - int pid = dpr->dpr_pid; + int pid = dpr->dpr_pid, status; #if defined(sun) int pfd = Pctlfd(P); @@ -702,7 +702,22 @@ pwait_locked: */ (void) pthread_mutex_lock(&dpr->dpr_lock); +#if defined(__FreeBSD__) + /* + * On FreeBSD, the victim process must be stopped before ptrace(2) can + * be used to remove breakpoints. + */ + if (kill(dpr->dpr_pid, SIGSTOP) == 0 && + wait4(dpr->dpr_pid, &status, WSTOPPED | WEXITED, NULL) != -1 && + WIFSTOPPED(status)) { + dt_proc_bpdestroy(dpr, B_TRUE); + kill(dpr->dpr_pid, SIGCONT); + } else + dt_dprintf("pid %d: failed to remove breakpoints\n", + dpr->dpr_pid); +#else dt_proc_bpdestroy(dpr, B_TRUE); +#endif dpr->dpr_done = B_TRUE; dpr->dpr_tid = 0; From owner-freebsd-dtrace@FreeBSD.ORG Sun Oct 27 19:59:30 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D277DF05 for ; Sun, 27 Oct 2013 19:59:30 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A098C294E for ; Sun, 27 Oct 2013 19:59:30 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id at1so10191488iec.29 for ; Sun, 27 Oct 2013 12:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=jHtOZ+e1IOVL9xYEWPbZvl7JpvV0Q9O3b7C89baOJAo=; b=aiZNJ+QasmjZyhkZlPYO3auSIue4uGycxLVHWmWTNmBg5CKdHcnhagt/S+gBYT4tOz FKjNJSpHq+nDR4JpWuU52Qu39lpv/+KGV6fpJXtd+1sdYmjZeUEo3YIDh5qvnwvHMDpP foT0e/QdqZpSD5/oZkRHhh5KdFKJtgG/kf3finAr8kPLRUQ3wfUWKlr34gkFUqMmT0aj P/PRaKED0sbod+8UbMKwpNLHBP7rsPF3WHqmtpPwcKWQBPafVGfRkCc93uYxWhckehl/ F3ATJ6TE+y1Zv1TuES3bUnKSRIem6nEPGkeLCveXRo3M8baCOaeQ0AwMPNLNoZSHEnSN vlGQ== X-Received: by 10.50.164.165 with SMTP id yr5mr6138414igb.38.1382903969931; Sun, 27 Oct 2013 12:59:29 -0700 (PDT) Received: from charmander.uwaterloo.ca (rn-nat2-uw-129-97-124-209.net.uwaterloo.ca. [129.97.124.209]) by mx.google.com with ESMTPSA id o15sm15905301igx.6.2013.10.27.12.59.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 27 Oct 2013 12:59:29 -0700 (PDT) Sender: Mark Johnston Date: Sun, 27 Oct 2013 15:59:24 -0400 From: Mark Johnston To: symbolics@gmx.com Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131027195924.GA7526@charmander.uwaterloo.ca> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131025104706.GB1705@lemon> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Oct 2013 19:59:31 -0000 On Fri, Oct 25, 2013 at 11:47:06AM +0100, symbolics@gmx.com wrote: > On Wed, Oct 23, 2013 at 10:59:02PM -0400, Mark Johnston wrote: > > On Wed, Oct 23, 2013 at 09:30:09PM +0100, symbolics@gmx.com wrote: > > > Hi, > > > > > > http://dtrace.org/blogs/brendan/2011/02/11/dtrace-pid-provider-arguments/ > > > > > > I tried to follow some of the examples but I crash the Firefox process > > > each time. Sometimes DTrace manages to collect a little data before the > > > death. > > > > > > [...] > > > > > > Is this a known problem or should I send a PR? > > > > Thanks for reporting this: I was able to reproduce the crash and managed > > to find a nasty pair of bugs. Could you test the patch below and let me > > know if it fixes the problem for you as well? If you see more crashes, > > please include the backtrace and signo from gdb again; it would likely > > be a different problem that needs to be debugged and fixed separately. > > Hi Mark, > > This helps but there still may be some issues. First time I used this > I found that when I killed the DTrace process Firefox went down too > with a SIGTRAP. I have a possibly unhelpful core from this: Just FYI, I've committed the push %rbp emulation fix as r257143: http://svnweb.freebsd.org/base?view=revision&revision=257143 > > Loaded symbols for /libexec/ld-elf.so.1 > #0 0x000000080061d541 in r_debug_state (rd=0x801354480, m=0x7ffffb394f98) > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > 3491 { > [New Thread 81b2da400 (LWP 100202/StreamTrans #12)] > [New Thread 81c4fb000 (LWP 100200/DOM Worker)] > [New Thread 81392b400 (LWP 100198/SSL Cert #2)] > [New Thread 816c62400 (LWP 100197/DNS Resolver #5)] > [New Thread 8160ccc00 (LWP 100196/Analysis Helper)] > [New Thread 8160cd400 (LWP 100195/Analysis Helper)] > [New Thread 8160cc400 (LWP 100194/Analysis Helper)] > [New Thread 8176f2800 (LWP 100187/DNS Resolver #4)] > [New Thread 8176f2400 (LWP 100186/DNS Resolver #3)] > [New Thread 819db1000 (LWP 100185/mozStorage #7)] > [New Thread 819dab000 (LWP 100183/mozStorage #6)] > [New Thread 8199bf000 (LWP 100182/mozStorage #5)] > [New Thread 8176e7c00 (LWP 100181/DNS Resolver #2)] > [New Thread 8176e8000 (LWP 100180/URL Classifier)] > [New Thread 8176ef000 (LWP 100179/mozStorage #4)] > [New Thread 81392a800 (LWP 100178/SSL Cert #1)] > [New Thread 818672400 (LWP 100177/Proxy Resolution)] > [New Thread 818671000 (LWP 100176/Cert Verify)] > [New Thread 8176e9c00 (LWP 100175/DNS Resolver #1)] > [New Thread 818671c00 (LWP 100174/Image Scaler)] > [New Thread 8176ed800 (LWP 100173/mozStorage #3)] > [New Thread 815ccd800 (LWP 100172/mozStorage #2)] > [New Thread 817051c00 (LWP 100171/mozStorage #1)] > [New Thread 815ee4800 (LWP 100170/localStorage DB)] > [New Thread 816c62000 (LWP 100167/ImageDecoder #1)] > [New Thread 801a04000 (LWP 100166/HTML5 Parser)] > [New Thread 8160c5c00 (LWP 100165/Cache I/O)] > [New Thread 815133800 (LWP 100163/DOM Worker)] > [New Thread 814eccc00 (LWP 100162/Timer)] > [New Thread 813845c00 (LWP 100161/Hang Monitor)] > [New Thread 813843400 (LWP 100160/JS Watchdog)] > [New Thread 813842c00 (LWP 100159/firefox)] > [New Thread 813842800 (LWP 100158/JS GC Helper)] > [New Thread 813842400 (LWP 100157/Socket Thread)] > [New Thread 801a09400 (LWP 100156/Gecko_IOThread)] > [New Thread 801a02400 (LWP 100079/firefox)] > (gdb) bt full > #0 0x000000080061d541 in r_debug_state (rd=0x801354480, m=0x7ffffb394f98) > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > No locals. > #1 0x000000000000000a in ?? () > No symbol table info available. > #2 0x00000001013f2260 in ?? () > No symbol table info available. > #3 0x00000008016b4a01 in _gmonparam () from /lib/libc.so.7 > No symbol table info available. > #4 0x000000021a067540 in ?? () > No symbol table info available. > #5 0x0000000000000020 in ?? () > No symbol table info available. > #6 0x0000000800000002 in ?? () > No symbol table info available. > #7 0x00007ffffb395000 in ?? () > No symbol table info available. > #8 0x00000008013e2cb4 in __jemalloc_tcache_alloc_small (tcache=0x800636000, > size=, zero=133) at tcache.h:323 > tbin = (tcache_bin_t *) 0x1a058b48197520a8 > ret = > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > > Another time I saw the DTrace process exit by itself. The Firefox > process was still present but stuck in uwait with the UI completely > unresponsive. I tried reattaching to it but that didn't go anywhere. > > Having said that, I can't seem to reproduce the crash consistently. I'll > keep playing with this and see if I can break it. Finally, I still see > the gelf_getehdr() error messages. If these aren't really errors, should > they be announced? > > By the way, the DTrace command I used for testing was: > > pid$target::malloc:entry { @ = quantize(arg0); } > profile-5sec { printa(@); } > > Thanks for your work! > > --sym > _______________________________________________ > freebsd-dtrace@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace > To unsubscribe, send any mail to "freebsd-dtrace-unsubscribe@freebsd.org" From owner-freebsd-dtrace@FreeBSD.ORG Mon Oct 28 05:04:07 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 68455DC1 for ; Mon, 28 Oct 2013 05:04:07 +0000 (UTC) (envelope-from adam.leventhal@delphix.com) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 317862198 for ; Mon, 28 Oct 2013 05:04:06 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wn1so3149972obc.13 for ; Sun, 27 Oct 2013 22:04:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JwuMFjOV3deMWreGeTXl+2oMm9/OioS5WaX9HO04PIs=; b=WBdUd/W3PKJK/xU617CermQpMo5AfwNLB5xK8Xfb40VYrlat7J+Iri/L4Ez0vFM4Xr 85MSgd8iYA/NAdvYB8vVreJypR5JQXGHDIMcfBx6qAS5rPkfHEBpETZbE/OwTVXH+xqB 7jlOYz28b1CItE1VKYkxrCjTXPBrhgoAKaw1U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=JwuMFjOV3deMWreGeTXl+2oMm9/OioS5WaX9HO04PIs=; b=aWESX12F6avpaLWKvyP3prP3V8T9XrAgCw8D4fZY11D7l/pOtq/jm35GCxjdBuVWCE aFgwMywVcjFwnU4EJOLYIh/ld8W8VxRxwoSz6oQh+fPXByMdARyHZZ6MJqhz7WFNk1b/ Gnsw3MvRm1bTMnbZmcETtsWXIyjfcdddzJ5bPBtbp++YNQAjIkiOY4YaAMVsoQuyLHmI /tBFysJRbrva4aKoqGSlUZVAVwGuCl+d0dcAeIlasdjxdc1BnJSyc6cVMAaAxT4WLIV3 6/thouRHOtf6szigKwO3EtoHb5tt9VKB9lWqPy6UzOBJ/xqURRus4M+WxjV8788gRFBn V/tQ== X-Gm-Message-State: ALoCoQkM+bXDAt+iabDgh15O7lOVbmG034tL0PeWhHdOlaf/etfwLlLvEfQt2ypbyvQa9PaVZT9Y MIME-Version: 1.0 X-Received: by 10.182.121.137 with SMTP id lk9mr13574290obb.32.1382936646257; Sun, 27 Oct 2013 22:04:06 -0700 (PDT) Received: by 10.76.92.67 with HTTP; Sun, 27 Oct 2013 22:04:06 -0700 (PDT) In-Reply-To: <20131024025902.GA2286@charmander> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> Date: Sun, 27 Oct 2013 22:04:06 -0700 Message-ID: Subject: Re: Firefox crash during dtrace attach under -CURRENT From: Adam Leventhal To: Mark Johnston Content-Type: text/plain; charset=ISO-8859-1 Cc: dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Oct 2013 05:04:07 -0000 Hey Mark, For reference, here's the relevant code from illumos: #ifdef __amd64 if (p->p_model == DATAMODEL_NATIVE) { #endif addr = rp->r_sp - sizeof (uintptr_t); ret = fasttrap_sulword((void *)addr, rp->r_fp); #ifdef __amd64 } else { addr = rp->r_sp - sizeof (uint32_t); ret = fasttrap_suword32((void *)addr, (uint32_t)rp->r_fp); } #endif > For anyone interested, the bug is that fasttrap's ebp push instruction > emulation code is just wrong: it's supposed to save %rbp at %rsp - 8. > But instead it tries to save %rsp at %rsp - 8, and also reverses the > uaddr/kaddr arguments to copyout(), resulting in strange crashes. I > managed to narrow in on the problem with a test program that prints %rbp > immediately before and after a tracepoint. Perhaps rp->r_fp was mistakenly swapped for &rp->r_rsp. > Can anyone review this diff? I'd like to check it in soon, assuming > that I haven't also made a mistake somewhere. :) > diff --git a/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c b/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c > index 8b5ce9f..bb5c9af 100644 > --- a/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c > +++ b/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c > @@ -1399,12 +1399,12 @@ fasttrap_pid_probe(struct reg *rp) > #ifdef __amd64 > if (p->p_model == DATAMODEL_NATIVE) { > addr = rp->r_rsp - sizeof (uintptr_t); > - ret = fasttrap_sulword((void *)addr, &rp->r_rsp); > + ret = fasttrap_sulword(&rp->r_rbp, (void *)addr); > } else { > #endif > #ifdef __i386__ > addr = rp->r_rsp - sizeof (uint32_t); > - ret = fasttrap_suword32((void *)addr, &rp->r_rsp); > + ret = fasttrap_suword32(&rp->r_rbp, (void *)addr); > #endif > #ifdef __amd64 > } This looks correct, but I'd suggest you might want to instead change the code in fasttrap_isa.c to look more like the illumos version, and change the macro definition: -#define fasttrap_suword64(_k, _u) copyout((_k), (_u), sizeof(uint64_t)) +#define fasttrap_suword64(_u, _k) copyout((_k), &(_u), sizeof(uint64_t)) Adam -- Adam Leventhal CTO, Delphix http://blog.delphix.com/ahl From owner-freebsd-dtrace@FreeBSD.ORG Mon Oct 28 09:36:17 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F1F06AFB; Mon, 28 Oct 2013 09:36:16 +0000 (UTC) (envelope-from vg.freebsd@gmail.com) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1FF042EC2; Mon, 28 Oct 2013 09:36:15 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so2609775lbj.17 for ; Mon, 28 Oct 2013 02:36:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=va+r2sBbXYLfAUJrd5Lo2L3aynhe6AQJrmDMJdH9Qxs=; b=pNtz1aA3OWQNfBHfwVY8pqrq+tcdmMGdaSKQwbC+rdXU0wYk4dBmLx37XawKncVScQ ywvZloaSBLDdp1s7PTN61Ik903L7/g4mjMXzw90/1vXQu4xVi6O5edVC8toC3KxezKhJ 4qU9u9LWsoRIvLpIKdfz1ZWQihafzSLs3GyjkIJ9F66FLcto1zfkM63WxjvvdKapcDJT co6px8wxU6jxTFEP1MPVS7IrLdWrz8+5rFILVWQrQS9FJLhrYahmNEmJ69+dWEKKDfzr jWSSD+rvpHFsWpN1StvgZM6+7D7hBvGm7xszoEhZ0PueD0jV7zTu344/6hUn6SjexRF4 Rnew== X-Received: by 10.152.26.201 with SMTP id n9mr243380lag.52.1382952973333; Mon, 28 Oct 2013 02:36:13 -0700 (PDT) Received: from gmail.com ([91.192.150.18]) by mx.google.com with ESMTPSA id mt8sm12390256lbb.12.2013.10.28.02.36.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 28 Oct 2013 02:36:12 -0700 (PDT) Sender: Veniamin Gvozdikov Date: Mon, 28 Oct 2013 09:36:09 +0000 From: Veniamin Gvozdikov To: George Neville-Neil Subject: Re: Integration DTrace problems Message-ID: <20131028093608.GA13153@gmail.com> References: <6A174747-D855-481D-A191-67A2805BC9AE@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Oct 2013 09:36:17 -0000 Hello, On Sat, Oct 12, 2013 at 05:33:25PM -0400, George Neville-Neil wrote: > > For more details go to link http://zlonet.ru/page/dtrace-integration-features/ . > I moved one before testing on FreeBSD 10. https://github.com/zloidemon/zloidemon.github.com/blob/source/_posts/2013-09-29-dtrace-integration-features.md > Can you submit bugs for these? Yes, but I want reproduce that on FreeBSD 10. I've already upgraded and installed my env on PC the FreeBSD 10.0 BETA1. I've just tried to build application with DTrace support on the FreeBSD and I had troubles again. My box version: FreeBSD jdevel 10.0-BETA1 FreeBSD 10.0-BETA1 #0 r256420: Sun Oct 13 01:43:07 UTC 2013 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 Today I had issues with generation haeader files: This is output from dtrace -h file.d: dtrace: failed to compile script /home/zloidemon/Repos/tarantool/include/tarantool_provider.d: "/usr/lib/dtrace/io.d", line 43: operator -> cannot be applied to a forward declaration: no struct devstat definition is available I have same issue with another files *.d files. But in the FreeBSD 9.1-p7 jail it works fine. I always have good header files from dtrace application. Thank you. From owner-freebsd-dtrace@FreeBSD.ORG Tue Oct 29 21:11:46 2013 Return-Path: Delivered-To: dtrace@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0DCA9D84; Tue, 29 Oct 2013 21:11:46 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2B30A2803; Tue, 29 Oct 2013 21:11:44 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA15681; Tue, 29 Oct 2013 23:11:43 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VbGZj-0005x6-9I; Tue, 29 Oct 2013 23:11:43 +0200 Message-ID: <5270246B.6070105@FreeBSD.org> Date: Tue, 29 Oct 2013 23:11:07 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: dtrace@FreeBSD.org Subject: sdt "sname" removal X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Oct 2013 21:11:46 -0000 I never understood why FreeBSD SDT as opposed to upstream SDT requires the same or almost the same probe name to be specified twice. This seems to be silly and a little bit error-prone. In other words, I do not see any reason not to re-use the original upstream trick where double underscore in a providers name in the C code gets converted to a single dash in a DTrace provider name. [*] So here is my take at that: http://people.freebsd.org/~avg/sdt-sname-removal.diff An inline preview of the change: -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_ok, priv-ok, "int"); -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_err, priv-err, "int"); +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__ok, "int"); +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__err, "int"); It's possible that I missed some places where old style SDT_PROBE_DEFINE macros are used or where an old probe name is used with SDT_PROBE_ARGTYPE or SDT_PROBE. Please test, review, comment, etc. Thank you! -- Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Tue Oct 29 21:22:14 2013 Return-Path: Delivered-To: dtrace@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 80FE84E1; Tue, 29 Oct 2013 21:22:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A506028ED; Tue, 29 Oct 2013 21:22:13 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA15866; Tue, 29 Oct 2013 23:22:12 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VbGjr-0005xh-Vy; Tue, 29 Oct 2013 23:22:12 +0200 Message-ID: <527026B3.2070309@FreeBSD.org> Date: Tue, 29 Oct 2013 23:20:51 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Mark Johnston Subject: Re: "unstable" sdt probes References: <5268F461.7080504@FreeBSD.org> <20131024161620.GA1710@charmander> <526A9CB5.2050207@FreeBSD.org> <20131026180643.GA98676@raichu> In-Reply-To: <20131026180643.GA98676@raichu> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Oct 2013 21:22:14 -0000 on 26/10/2013 21:06 Mark Johnston said the following: > The patch here is what I had in mind: > http://people.freebsd.org/~markj/patches/zfs_probes.diff > > I've only compile-tested it, but it should create illumos-compatible ZFS > probes without changing any ZFS code, assuming I understand exactly how > they're creating/naming probes. :) The simplicity and straightforwardness of your patch is very seducing! :-) I think that you missed sys/cddl/compat/opensolaris/sys/sdt.h, but that's a very a minor issue that is trivial to fix. I had something a little bit more elaborate on my mind. Something that would allow DTRACE_PROBE macros to work without providing any additional code per each probe. And also something that would make DTRACE_PROBE macros appealing to use in the FreeBSD code proper. So, I got some time to hack on this and here is a result: http://people.freebsd.org/~avg/dtrace-probe-macros.diff This change depends upon another change that I've just posted. So, no surprise that I feel preference for my change, but I think that your change has certain advantage as well (esp. brevity and clarity). What do you think? Thank you! -- Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 08:15:19 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D3057D1B for ; Wed, 30 Oct 2013 08:15:19 +0000 (UTC) (envelope-from symbolics@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7CAE92C8B for ; Wed, 30 Oct 2013 08:15:19 +0000 (UTC) Received: from lemon ([80.7.17.14]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0MMGWH-1VadiT2Eaz-0082eH for ; Wed, 30 Oct 2013 09:15:08 +0100 Received: by lemon (Postfix, from userid 1001) id 000CFEB36F; Wed, 30 Oct 2013 08:15:07 +0000 (GMT) Date: Wed, 30 Oct 2013 08:15:07 +0000 From: symbolics@gmx.com To: dtrace@freebsd.org Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131030081507.GA1674@lemon> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> <20131025145956.GA26814@lemon> <20131027195307.GA3206@charmander.uwaterloo.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131027195307.GA3206@charmander.uwaterloo.ca> X-Provags-ID: V03:K0:Z3TryzU/EWoG6TKG6K3Owp4Bz5N7490jNHwrVG1FTnQKs6S+j4z VZHL9TQaQmSCxC+eADBePoV9McFAXTJR3lm8/FDuYZwsxXMXOo7q2AciGcj/vc0aaPMBtxO Klv42N29HchRkeHlQKrgJ2RV0TO3Uvo45vBDHsa4FlHV9ybRWtyX9si/ie8Yk3ZDt8TTeX9 Hhel0HDZXIGhHnMrOjoog== X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 08:15:19 -0000 On Sun, Oct 27, 2013 at 03:53:07PM -0400, Mark Johnston wrote: > On Fri, Oct 25, 2013 at 03:59:56PM +0100, symbolics@gmx.com wrote: > > On Fri, Oct 25, 2013 at 11:47:06AM +0100, symbolics@gmx.com wrote: > > > On Wed, Oct 23, 2013 at 10:59:02PM -0400, Mark Johnston wrote: > > > > On Wed, Oct 23, 2013 at 09:30:09PM +0100, symbolics@gmx.com wrote: > > > > > Hi, > > > > > > > > > > http://dtrace.org/blogs/brendan/2011/02/11/dtrace-pid-provider-arguments/ > > > > > > > > > > I tried to follow some of the examples but I crash the Firefox process > > > > > each time. Sometimes DTrace manages to collect a little data before the > > > > > death. > > > > > > > > > > [...] > > > > > > > > > > Is this a known problem or should I send a PR? > > > > > > > > Thanks for reporting this: I was able to reproduce the crash and managed > > > > to find a nasty pair of bugs. Could you test the patch below and let me > > > > know if it fixes the problem for you as well? If you see more crashes, > > > > please include the backtrace and signo from gdb again; it would likely > > > > be a different problem that needs to be debugged and fixed separately. > > > > > > Hi Mark, > > > > > > This helps but there still may be some issues. First time I used this > > > I found that when I killed the DTrace process Firefox went down too > > > with a SIGTRAP. I have a possibly unhelpful core from this: > > > > > > > Another data point. I attached to mutt and reviewed some of the calls it > > was making. Subsequently I killed DTrace, went to to look at other > > things and a while later when back to check my mail. On attempting to > > change into a different mail folder mutt died with a SIGTRAP. It seems > > like DTrace isn't tidying up after itself? > > > > (gdb) bt > > #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff6c28) > > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > > #1 0x0000000000000000 in ?? () > > Ok, I think I've figured out this one too. As you note, dtrace(1) isn't > cleaning up some of its breakpoints properly when it detaches. In > particular, it's not stopping the victim process before it tries to > remove breakpoints using ptrace(2); however, ptrace requires the target > process to be stopped, else it will return EBUSY. So the breakpoint in > the rtld gets left behind, and it turns out that r_debug_state() is called > every time a process tries to dlopen() a shared object. > > mutt was a good example since it seems to dlopen() iconv-related stuff > as I scan through my inbox; one can inspect this with DTrace. :) > i.e. with something like > > 'pid$target::dlopen:entry {trace(copyinstr(arg0));}' > > With this observation it becomes easy to reproduce the problem using a > test program that does something like > > while (1) { > dlopen("/lib/libnonexistent.so.100", RTLD_LAZY); > sleep(1); > } > > A somewhat crude patch which fixes this for me is below; it just adds > code to send SIGSTOP to the target process before trying to remove > breakpoints. Does anyone see any problems with this? Perhaps it should > be libproc's responsibility to ensure that the victim process is stopped > before trying a ptrace(PT_IO, ...) to add/remove breakpoints? > > Thanks, > -Mark > Hi Mark, I've tried the patch but I can still reproduce the crash using mutt. I attached to the running mutt process with dtruss and watched that work a little bit, killed dtruss and carried on using mutt. I then tried to open a large mail folder and it crashed with a SIGTRAP. Backtrace, FWIW: (gdb) bt #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff84a8) at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 #1 0x0000000000000001 in ?? () #2 0x0000000102c00000 in ?? () #3 0x0000000000000001 in ?? () #4 0x00000002028000c0 in ?? () #5 0x0000000000000001 in ?? () #6 0x0000000000000002 in ?? () #7 0x00007fffffff8d00 in ?? () #8 0x0000000801f25e3c in arena_avail_insert (arena=0xffffffff, chunk=0x7fffffff84a8, pageind=34395310320, npages=, maybe_adjac_pred=, maybe_adjac_succ=false) at /usr/obj/usr/home/dm/git/freebsd/lib/libc/jemalloc_arena.c:274 Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal --sym From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 12:37:18 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 97750210 for ; Wed, 30 Oct 2013 12:37:18 +0000 (UTC) (envelope-from symbolics@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 284132D8E for ; Wed, 30 Oct 2013 12:37:18 +0000 (UTC) Received: from lemon ([80.7.17.14]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0LkgAG-1WBeie2Dp6-00aTLL for ; Wed, 30 Oct 2013 13:37:16 +0100 Received: by lemon (Postfix, from userid 1001) id 0DCCEEB372; Wed, 30 Oct 2013 12:37:16 +0000 (GMT) Date: Wed, 30 Oct 2013 12:37:16 +0000 From: symbolics@gmx.com To: dtrace@freebsd.org Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131030123716.GA2037@lemon> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> <20131025145956.GA26814@lemon> <20131027195307.GA3206@charmander.uwaterloo.ca> <20131030081507.GA1674@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131030081507.GA1674@lemon> X-Provags-ID: V03:K0:Ox5UAYYJzTToDs8qKz7Azzz64GEvuq+Zzgg29A3Ogg/lgr0oEHN QSZTGiyqlXExVaDiolV1dtVsKyo9tXg/DEJNFshdw9vE1Y80HMni5SvbnpAIGTF/GOemS7K j583R2PWxdSc52af2Q44AgiUvz5V6mqk9WYN/526UgATOI5+PW3snYwOMOlT2Dg661gpHoC rS5Fp2n2ySgrMNBtiTFMQ== X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 12:37:18 -0000 On Wed, Oct 30, 2013 at 08:15:07AM +0000, symbolics@gmx.com wrote: > On Sun, Oct 27, 2013 at 03:53:07PM -0400, Mark Johnston wrote: > > On Fri, Oct 25, 2013 at 03:59:56PM +0100, symbolics@gmx.com wrote: > > > On Fri, Oct 25, 2013 at 11:47:06AM +0100, symbolics@gmx.com wrote: > > > > On Wed, Oct 23, 2013 at 10:59:02PM -0400, Mark Johnston wrote: > > > > > On Wed, Oct 23, 2013 at 09:30:09PM +0100, symbolics@gmx.com wrote: > > > > > > Hi, > > > > > > > > > > > > http://dtrace.org/blogs/brendan/2011/02/11/dtrace-pid-provider-arguments/ > > > > > > > > > > > > I tried to follow some of the examples but I crash the Firefox process > > > > > > each time. Sometimes DTrace manages to collect a little data before the > > > > > > death. > > > > > > > > > > > > [...] > > > > > > > > > > > > Is this a known problem or should I send a PR? > > > > > > > > > > Thanks for reporting this: I was able to reproduce the crash and managed > > > > > to find a nasty pair of bugs. Could you test the patch below and let me > > > > > know if it fixes the problem for you as well? If you see more crashes, > > > > > please include the backtrace and signo from gdb again; it would likely > > > > > be a different problem that needs to be debugged and fixed separately. > > > > > > > > Hi Mark, > > > > > > > > This helps but there still may be some issues. First time I used this > > > > I found that when I killed the DTrace process Firefox went down too > > > > with a SIGTRAP. I have a possibly unhelpful core from this: > > > > > > > > > > Another data point. I attached to mutt and reviewed some of the calls it > > > was making. Subsequently I killed DTrace, went to to look at other > > > things and a while later when back to check my mail. On attempting to > > > change into a different mail folder mutt died with a SIGTRAP. It seems > > > like DTrace isn't tidying up after itself? > > > > > > (gdb) bt > > > #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff6c28) > > > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > > > #1 0x0000000000000000 in ?? () > > > > Ok, I think I've figured out this one too. As you note, dtrace(1) isn't > > cleaning up some of its breakpoints properly when it detaches. In > > particular, it's not stopping the victim process before it tries to > > remove breakpoints using ptrace(2); however, ptrace requires the target > > process to be stopped, else it will return EBUSY. So the breakpoint in > > the rtld gets left behind, and it turns out that r_debug_state() is called > > every time a process tries to dlopen() a shared object. > > > > mutt was a good example since it seems to dlopen() iconv-related stuff > > as I scan through my inbox; one can inspect this with DTrace. :) > > i.e. with something like > > > > 'pid$target::dlopen:entry {trace(copyinstr(arg0));}' > > > > With this observation it becomes easy to reproduce the problem using a > > test program that does something like > > > > while (1) { > > dlopen("/lib/libnonexistent.so.100", RTLD_LAZY); > > sleep(1); > > } > > > > A somewhat crude patch which fixes this for me is below; it just adds > > code to send SIGSTOP to the target process before trying to remove > > breakpoints. Does anyone see any problems with this? Perhaps it should > > be libproc's responsibility to ensure that the victim process is stopped > > before trying a ptrace(PT_IO, ...) to add/remove breakpoints? > > > > Thanks, > > -Mark > > > > Hi Mark, > > I've tried the patch but I can still reproduce the crash using mutt. I > attached to the running mutt process with dtruss and watched that work a > little bit, killed dtruss and carried on using mutt. I then tried to > open a large mail folder and it crashed with a SIGTRAP. Backtrace, FWIW: > > (gdb) bt > #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff84a8) > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > #1 0x0000000000000001 in ?? () > #2 0x0000000102c00000 in ?? () > #3 0x0000000000000001 in ?? () > #4 0x00000002028000c0 in ?? () > #5 0x0000000000000001 in ?? () > #6 0x0000000000000002 in ?? () > #7 0x00007fffffff8d00 in ?? () > #8 0x0000000801f25e3c in arena_avail_insert (arena=0xffffffff, > chunk=0x7fffffff84a8, pageind=34395310320, npages=, > maybe_adjac_pred=, maybe_adjac_succ=false) > at /usr/obj/usr/home/dm/git/freebsd/lib/libc/jemalloc_arena.c:274 > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > Just quickly adding, this crash & trace is perfectly reproducible. I did a second round of kernel building and testing to make sure I hadn't messed up. --sym From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 13:19:33 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 48159294 for ; Wed, 30 Oct 2013 13:19:33 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 07771201B for ; Wed, 30 Oct 2013 13:19:32 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id ii20so3711814qab.11 for ; Wed, 30 Oct 2013 06:19:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=NOk3lmJ2qORNgIylujzPefFzCowoq9D83YfMRM/G+ZY=; b=Nc3CZFR8dG/ffW4nvWvIDXgtMkQaTyqTPD/FzXzSSbLOUqiXN9gNtp9HjIvvgwQfqp u9AI1CUpErIqtK5VXv2C3pd/UScWmjRlGxAqVLCavrjwVuPleF9rQbQ6RDZ9tJzgnH93 DoTArRCHgEa/JQ5FAq6Jm+gGXhqK7OezrUrQRhvE+umrqsKd2mIJbFrfJjAwBcjTTCHg WiL98zjX7Aq9ZLuAHd+hcMP/9EG1NtlrYgrU2AEUZaKXHEhodaCgLT6sK8Il2jgNxkoS iKso5q7Zar8BbT6de2LUFmdaKXFvFLcdT3wD59zsqKNk+M0psHfLv6zePAC7X+JqFBbx j1nQ== X-Received: by 10.224.127.68 with SMTP id f4mr8060414qas.62.1383139172161; Wed, 30 Oct 2013 06:19:32 -0700 (PDT) Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13]) by mx.google.com with ESMTPSA id v19sm76231801qaw.0.2013.10.30.06.19.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Oct 2013 06:19:31 -0700 (PDT) Sender: Mark Johnston Date: Wed, 30 Oct 2013 09:19:26 -0400 From: Mark Johnston To: symbolics@gmx.com Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131030131926.GA83433@raichu> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> <20131025145956.GA26814@lemon> <20131027195307.GA3206@charmander.uwaterloo.ca> <20131030081507.GA1674@lemon> <20131030123716.GA2037@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131030123716.GA2037@lemon> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 13:19:33 -0000 On Wed, Oct 30, 2013 at 12:37:16PM +0000, symbolics@gmx.com wrote: > On Wed, Oct 30, 2013 at 08:15:07AM +0000, symbolics@gmx.com wrote: > > On Sun, Oct 27, 2013 at 03:53:07PM -0400, Mark Johnston wrote: > > > On Fri, Oct 25, 2013 at 03:59:56PM +0100, symbolics@gmx.com wrote: > > > > On Fri, Oct 25, 2013 at 11:47:06AM +0100, symbolics@gmx.com wrote: > > > > > On Wed, Oct 23, 2013 at 10:59:02PM -0400, Mark Johnston wrote: > > > > > > On Wed, Oct 23, 2013 at 09:30:09PM +0100, symbolics@gmx.com wrote: > > > > > > > Hi, > > > > > > > > > > > > > > http://dtrace.org/blogs/brendan/2011/02/11/dtrace-pid-provider-arguments/ > > > > > > > > > > > > > > I tried to follow some of the examples but I crash the Firefox process > > > > > > > each time. Sometimes DTrace manages to collect a little data before the > > > > > > > death. > > > > > > > > > > > > > > [...] > > > > > > > > > > > > > > Is this a known problem or should I send a PR? > > > > > > > > > > > > Thanks for reporting this: I was able to reproduce the crash and managed > > > > > > to find a nasty pair of bugs. Could you test the patch below and let me > > > > > > know if it fixes the problem for you as well? If you see more crashes, > > > > > > please include the backtrace and signo from gdb again; it would likely > > > > > > be a different problem that needs to be debugged and fixed separately. > > > > > > > > > > Hi Mark, > > > > > > > > > > This helps but there still may be some issues. First time I used this > > > > > I found that when I killed the DTrace process Firefox went down too > > > > > with a SIGTRAP. I have a possibly unhelpful core from this: > > > > > > > > > > > > > Another data point. I attached to mutt and reviewed some of the calls it > > > > was making. Subsequently I killed DTrace, went to to look at other > > > > things and a while later when back to check my mail. On attempting to > > > > change into a different mail folder mutt died with a SIGTRAP. It seems > > > > like DTrace isn't tidying up after itself? > > > > > > > > (gdb) bt > > > > #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff6c28) > > > > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > > > > #1 0x0000000000000000 in ?? () > > > > > > Ok, I think I've figured out this one too. As you note, dtrace(1) isn't > > > cleaning up some of its breakpoints properly when it detaches. In > > > particular, it's not stopping the victim process before it tries to > > > remove breakpoints using ptrace(2); however, ptrace requires the target > > > process to be stopped, else it will return EBUSY. So the breakpoint in > > > the rtld gets left behind, and it turns out that r_debug_state() is called > > > every time a process tries to dlopen() a shared object. > > > > > > mutt was a good example since it seems to dlopen() iconv-related stuff > > > as I scan through my inbox; one can inspect this with DTrace. :) > > > i.e. with something like > > > > > > 'pid$target::dlopen:entry {trace(copyinstr(arg0));}' > > > > > > With this observation it becomes easy to reproduce the problem using a > > > test program that does something like > > > > > > while (1) { > > > dlopen("/lib/libnonexistent.so.100", RTLD_LAZY); > > > sleep(1); > > > } > > > > > > A somewhat crude patch which fixes this for me is below; it just adds > > > code to send SIGSTOP to the target process before trying to remove > > > breakpoints. Does anyone see any problems with this? Perhaps it should > > > be libproc's responsibility to ensure that the victim process is stopped > > > before trying a ptrace(PT_IO, ...) to add/remove breakpoints? > > > > > > Thanks, > > > -Mark > > > > > > > Hi Mark, > > > > I've tried the patch but I can still reproduce the crash using mutt. I > > attached to the running mutt process with dtruss and watched that work a > > little bit, killed dtruss and carried on using mutt. I then tried to > > open a large mail folder and it crashed with a SIGTRAP. Backtrace, FWIW: > > > > (gdb) bt > > #0 0x0000000800722541 in r_debug_state (rd=0x802425480, m=0x7fffffff84a8) > > at /usr/home/dm/git/freebsd/libexec/rtld-elf/rtld.c:3491 > > #1 0x0000000000000001 in ?? () > > #2 0x0000000102c00000 in ?? () > > #3 0x0000000000000001 in ?? () > > #4 0x00000002028000c0 in ?? () > > #5 0x0000000000000001 in ?? () > > #6 0x0000000000000002 in ?? () > > #7 0x00007fffffff8d00 in ?? () > > #8 0x0000000801f25e3c in arena_avail_insert (arena=0xffffffff, > > chunk=0x7fffffff84a8, pageind=34395310320, npages=, > > maybe_adjac_pred=, maybe_adjac_succ=false) > > at /usr/obj/usr/home/dm/git/freebsd/lib/libc/jemalloc_arena.c:274 > > Previous frame inner to this frame (corrupt stack?) > > Current language: auto; currently minimal > > > > Just quickly adding, this crash & trace is perfectly reproducible. I did > a second round of kernel building and testing to make sure I hadn't > messed up. Ah, my apologies: I should have mentioned that this is not a kernel change but rather a change to libdtrace, though I will probably move this change to libproc instead when I have a chance. The easiest way to rebuild libdtrace, assuming that your sources are in sync with what's on the box, is to run # cd $SRCBASE/cddl/lib/libdtrace # make && make install Then dtrace(1) will be using the patched libdtrace.so. -Mark From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 13:32:09 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E844991E for ; Wed, 30 Oct 2013 13:32:09 +0000 (UTC) (envelope-from symbolics@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6721A2177 for ; Wed, 30 Oct 2013 13:32:09 +0000 (UTC) Received: from lemon ([80.7.17.14]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0M4nt7-1VrxQl06Jh-00yxTI for ; Wed, 30 Oct 2013 14:32:02 +0100 Received: by lemon (Postfix, from userid 1001) id 8232CEB372; Wed, 30 Oct 2013 13:32:01 +0000 (GMT) Date: Wed, 30 Oct 2013 13:32:01 +0000 From: symbolics@gmx.com To: dtrace@freebsd.org Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131030133201.GA2560@lemon> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> <20131025145956.GA26814@lemon> <20131027195307.GA3206@charmander.uwaterloo.ca> <20131030081507.GA1674@lemon> <20131030123716.GA2037@lemon> <20131030131926.GA83433@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131030131926.GA83433@raichu> X-Provags-ID: V03:K0:N9ubg7puJBGkDDkAjzPaV90ps9+H5nwXQ0nd/uFiY8wuMWsFIpf o8EwytWUp4snAy4t6kBLArSy+VIxxiJU370k6x/hIhTUVZ6B67Wlk8CcLNwNsrWXBJNbg6B Kcd3H8gn+Xw6BCGlLenw9sGGy/igi7zRdXlwGU0EJ/d2NE4u3TLhsrgVyusdM4WeGya8JP+ aINAympdiUsc5J8O2ZSbQ== X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 13:32:10 -0000 On Wed, Oct 30, 2013 at 09:19:26AM -0400, Mark Johnston wrote: > On Wed, Oct 30, 2013 at 12:37:16PM +0000, symbolics@gmx.com wrote: > > On Wed, Oct 30, 2013 at 08:15:07AM +0000, symbolics@gmx.com wrote: ... > > > > Just quickly adding, this crash & trace is perfectly reproducible. I did > > a second round of kernel building and testing to make sure I hadn't > > messed up. > > Ah, my apologies: I should have mentioned that this is not a kernel > change but rather a change to libdtrace, though I will probably move > this change to libproc instead when I have a chance. The easiest way to > rebuild libdtrace, assuming that your sources are in sync with what's on > the box, is to run > > # cd $SRCBASE/cddl/lib/libdtrace > # make && make install > Ah, that'll teach me to pay attention! The patch is good so far. --sym From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 14:15:15 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4B3D3F32 for ; Wed, 30 Oct 2013 14:15:15 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0BB5224DF for ; Wed, 30 Oct 2013 14:15:14 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id cm18so3740304qab.17 for ; Wed, 30 Oct 2013 07:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=GNtoo0NR4Lr72VZCJF5JJiqyBj8C1phCTVoJp1I8Mgo=; b=jy401qHWwr/exSNi08I3OCRFR1WhxAh+b79FywHaYQmhKkfJ6J0+OXMunfpsq2i/Hm st+BMRKBir+iKw37GFbtirPN369ZE/RRt4EtTfIIBfcZ/T84h5oV1g8iTii/LK0fcGei 2RKimaV3m20mIQ2nvwkn/KdlYHay1xsih5wfKYn5APXmAUweeBDGz4avTbn6prKHzJxW 9EUmBj3QyI4RDoo5k4EK8g8E4sbfOwdLqMjYcrGexANWSdize0h2lOFL0r5dP9RqUDaa iOId8IaSpki2Od0tL6xdNmIIEV7cK41ipSJSCKRBXKBhoPkhGClvwwC5T9wpqaBHJdCM YdUA== X-Received: by 10.224.4.2 with SMTP id 2mr8258734qap.29.1383142514131; Wed, 30 Oct 2013 07:15:14 -0700 (PDT) Received: from charmander.sandvine.com ([64.7.137.182]) by mx.google.com with ESMTPSA id r5sm77475068qaj.13.2013.10.30.07.15.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Oct 2013 07:15:13 -0700 (PDT) Sender: Mark Johnston Date: Wed, 30 Oct 2013 10:14:59 -0400 From: Mark Johnston To: symbolics@gmx.com Subject: Re: Firefox crash during dtrace attach under -CURRENT Message-ID: <20131030141458.GA1795@charmander.sandvine.com> References: <20131023203009.GA92945@lemon> <20131024025902.GA2286@charmander> <20131025104706.GB1705@lemon> <20131025145956.GA26814@lemon> <20131027195307.GA3206@charmander.uwaterloo.ca> <20131030081507.GA1674@lemon> <20131030123716.GA2037@lemon> <20131030131926.GA83433@raichu> <20131030133201.GA2560@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131030133201.GA2560@lemon> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 14:15:15 -0000 On Wed, Oct 30, 2013 at 01:32:01PM +0000, symbolics@gmx.com wrote: > On Wed, Oct 30, 2013 at 09:19:26AM -0400, Mark Johnston wrote: > > On Wed, Oct 30, 2013 at 12:37:16PM +0000, symbolics@gmx.com wrote: > > > On Wed, Oct 30, 2013 at 08:15:07AM +0000, symbolics@gmx.com wrote: > ... > > > > > > Just quickly adding, this crash & trace is perfectly reproducible. I did > > > a second round of kernel building and testing to make sure I hadn't > > > messed up. > > > > Ah, my apologies: I should have mentioned that this is not a kernel > > change but rather a change to libdtrace, though I will probably move > > this change to libproc instead when I have a chance. The easiest way to > > rebuild libdtrace, assuming that your sources are in sync with what's on > > the box, is to run > > > > # cd $SRCBASE/cddl/lib/libdtrace > > # make && make install > > > > Ah, that'll teach me to pay attention! The patch is good so far. Awesome, thanks. As usual, please let me know about any new issues if you see them. Thanks for your patience! -Mark From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 17:25:37 2013 Return-Path: Delivered-To: dtrace@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7112B8A1 for ; Wed, 30 Oct 2013 17:25:37 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A82612427 for ; Wed, 30 Oct 2013 17:25:36 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA05820 for ; Wed, 30 Oct 2013 19:25:29 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VbZWK-000AH6-Rb for dtrace@freebsd.org; Wed, 30 Oct 2013 19:25:28 +0200 Message-ID: <527140CB.90502@FreeBSD.org> Date: Wed, 30 Oct 2013 19:24:27 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: dtrace@FreeBSD.org Subject: ERROR: ctfconvert: rc = 2 No entry found [dwarf_next_cu_header(57)] X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 17:25:37 -0000 The following happens during kernel build (probably during linking): ERROR: ctfconvert: rc = 2 No entry found [dwarf_next_cu_header(57)] Anyone knows what this error means? Is this something to be concerned about? -- Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Wed Oct 30 18:26:19 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 39CA319E for ; Wed, 30 Oct 2013 18:26:19 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from nm2-vm1.bullet.mail.bf1.yahoo.com (nm2-vm1.bullet.mail.bf1.yahoo.com [98.139.213.158]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DFE0928BC for ; Wed, 30 Oct 2013 18:26:18 +0000 (UTC) Received: from [98.139.212.149] by nm2.bullet.mail.bf1.yahoo.com with NNFMP; 30 Oct 2013 18:26:16 -0000 Received: from [98.139.211.193] by tm6.bullet.mail.bf1.yahoo.com with NNFMP; 30 Oct 2013 18:26:16 -0000 Received: from [127.0.0.1] by smtp202.mail.bf1.yahoo.com with NNFMP; 30 Oct 2013 18:26:16 -0000 X-Yahoo-Newman-Id: 722126.11137.bm@smtp202.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: Xyq2AS0VM1lEvP4zCOZQQCW4KdrKyOoj_jMqI9gO1b0dyZz OY0wlC4CyPdT9sLGxNcXHB9irEzij50BOsWgBHBH3bn3.YaFoaLy.G6eDrnW myGQQOt0QQHP2pXFztL46oijU2sW3hUas2fa6Z9xHD5V0irGYBjLeYIR.nGY oCvSQWg8K5Su4b8J9sferRgwYu7htc0d1GOWL0It08NJSKgUXXHC.KveJjee qdvlcE7b9Xxd_sRuCgA.NoXIRVYSRlOEKNxTHjRXmzIuOuGY9tWfdF9..FVm JwpP8hFOSdLWOycoAtyuI.4_f.8ioECXy0edHnTJBDfwoqmEiedn.ukByaPu hw.QNNgLnkICxHGEi_TbQ7CJLuIfhJg5I9r75C6EmbjN92b6UqgoHTbuqRia lz0DevUmdxTXRdQ044AnKZEid3jI_w1pGOT5tngFxnXHurDxg6X5jQY_P16E h7X_Rgtl5r8zGQieLKuJ2K7bG_d2IesjoH7mN6NY7ZEn8DUHoncoduSL7B7_ qQMfXZYSEiualjT_PmSISJMdDqIIPkg5uQe2AiUo0aMYFBNSIK5mw1XjiFrU 8hn6IgZPtmwMEmnJdqPSAU5HsXWLRQJI4.OiWxXLU._yW.UuLyJw5jUx95DU M2UtXADRFRCueFmnG7.nTwMJqY_VkrmiMlEGAXc5T6hw_lu0q1FukDeG7TL2 GzBPAxrp73eoQIrF43qsE8q9.YpWxyAydkimR1dbifB1ER9tYQskgpSMfuUx X X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with ) by smtp202.mail.bf1.yahoo.com with SMTP; 30 Oct 2013 11:26:16 -0700 PDT Message-ID: <52714F47.1080509@FreeBSD.org> Date: Wed, 30 Oct 2013 13:26:15 -0500 From: Pedro Giffuni Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Andriy Gapon , dtrace@FreeBSD.org Subject: Re: ERROR: ctfconvert: rc = 2 No entry found [dwarf_next_cu_header(57)] References: <527140CB.90502@FreeBSD.org> In-Reply-To: <527140CB.90502@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 18:26:19 -0000 Hello Andriy: El 30/10/2013 12:24 p. m., Andriy Gapon escribió: > > The following happens during kernel build (probably during linking): > ERROR: ctfconvert: rc = 2 No entry found [dwarf_next_cu_header(57)] > > > Anyone knows what this error means? > Is this something to be concerned about? > Ouch ... Can you try reverting r254941 and see if it fixes it? I was trying to make our CTF stuff more similar to illumos but I see it can cause an ENOENT in dw_read(). [1] Pedro. http://svnweb.freebsd.org/base/head/cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c?r1=254941&r2=254940&pathrev=254941 [1] From owner-freebsd-dtrace@FreeBSD.ORG Thu Oct 31 03:36:40 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B255DD84; Thu, 31 Oct 2013 03:36:40 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qe0-x22e.google.com (mail-qe0-x22e.google.com [IPv6:2607:f8b0:400d:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 50E012B41; Thu, 31 Oct 2013 03:36:40 +0000 (UTC) Received: by mail-qe0-f46.google.com with SMTP id s14so1401417qeb.5 for ; Wed, 30 Oct 2013 20:36:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=pKpfJe6Cv6wc9DS+1G+bt3uxUhKsiAbQAyMvXrTRkKQ=; b=K9zXHCJ0/k7PXHfb6UmeraBCz6A7dqhT0+aqcMDV36JIv8oD//E5Ojb93xqMq5j5Hg 6dwpy0IYtUZKtZSzJ/H9oKSpElPOdZX6wUCdhVg9nBR73wGtz0E/qr3pDrE8m+2Qmvaw HrsFXx19Te/23HyYzxFj7gm+FnlCPs8rrS7GBrtbQGCIn9njwiUEWAnETAeHIsi/4NU0 AtAE2//wYNrHOVSh62PFyQfRlBXRFwt/UuY0fABaKqOEiaFjMFqsH4PewmTDKpFberMh /OuCZ4EGOCh6qlhY+DtqjrjWoalV5kNHFaxiWmkw/0StRmWhjlyENmtKQef7by1+PUzP EYog== X-Received: by 10.224.76.10 with SMTP id a10mr2228308qak.9.1383190599419; Wed, 30 Oct 2013 20:36:39 -0700 (PDT) Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13]) by mx.google.com with ESMTPSA id og1sm3084840qeb.3.2013.10.30.20.36.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Oct 2013 20:36:38 -0700 (PDT) Sender: Mark Johnston Date: Wed, 30 Oct 2013 23:36:36 -0400 From: Mark Johnston To: Andriy Gapon Subject: Re: sdt "sname" removal Message-ID: <20131031033636.GC9355@raichu> References: <5270246B.6070105@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5270246B.6070105@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Oct 2013 03:36:40 -0000 On Tue, Oct 29, 2013 at 11:11:07PM +0200, Andriy Gapon wrote: > > I never understood why FreeBSD SDT as opposed to upstream SDT requires the same > or almost the same probe name to be specified twice. This seems to be silly and > a little bit error-prone. > In other words, I do not see any reason not to re-use the original upstream > trick where double underscore in a providers name in the C code gets converted > to a single dash in a DTrace provider name. [*] > > So here is my take at that: > http://people.freebsd.org/~avg/sdt-sname-removal.diff > > An inline preview of the change: > -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_ok, priv-ok, "int"); > -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_err, priv-err, "int"); > +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__ok, "int"); > +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__err, "int"); > > It's possible that I missed some places where old style SDT_PROBE_DEFINE macros > are used or where an old probe name is used with SDT_PROBE_ARGTYPE or SDT_PROBE. A good way to test this is to compare the output of 'dtrace -lv' with and without your change. If nothing changes, I'd be pretty confident that the diff is correct. > > Please test, review, comment, etc. I don't think this diff will apply cleanly to head - I've made some changes that will cause conflicts, and the diff doesn't touch netinet/in_kdtrace.c or kern/subr_devstat.c. Could you also update the SDT(9) man page? Also the "strlcpy(name, ..." immediately before the loop you added to sdt.c becomes redundant. Thanks, -Mark From owner-freebsd-dtrace@FreeBSD.ORG Thu Oct 31 03:55:27 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 82EB557D; Thu, 31 Oct 2013 03:55:27 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qc0-x232.google.com (mail-qc0-x232.google.com [IPv6:2607:f8b0:400d:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 31FB72C35; Thu, 31 Oct 2013 03:55:27 +0000 (UTC) Received: by mail-qc0-f178.google.com with SMTP id x19so1362531qcw.37 for ; Wed, 30 Oct 2013 20:55:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=bTkqD/N0t7kXbZBPxuaH+We5rYrjfpEaztfC6syqeKo=; b=jYkNpju8Tk77ggqJKFY4ftYyK4yLWUFQc7QSOX2SwWDp9WtmYyvHfcz/ucecHh8lM3 5++YlRnfAQYcltCVUcHWkXl7uEQQHr7fhDcNEJ41xhyrw0kfEGJNj8UtZMy0w4Jo9GlE okAunyY+btre/iWkfmPv6yjul9IT0Qj7Q/Iu7KRR4vK8q12ZtEva74Y0e20V+rJRa7Lk u+sKjHb63jQkW5iXaluS5Ih14Xu7yG0Bh7pzqf6ow+syLEIcifxcEYPYFgTra1y64LZR b+itCoitP+sdNlvYaVTdVMJGK99QGFrCrgXpT5WijlafH6MwCmwe1kS8+6AoP8uPtxQ5 R/kA== X-Received: by 10.224.24.131 with SMTP id v3mr2389673qab.48.1383191726334; Wed, 30 Oct 2013 20:55:26 -0700 (PDT) Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13]) by mx.google.com with ESMTPSA id v19sm5028590qaw.0.2013.10.30.20.55.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 30 Oct 2013 20:55:25 -0700 (PDT) Sender: Mark Johnston Date: Wed, 30 Oct 2013 23:55:23 -0400 From: Mark Johnston To: Andriy Gapon Subject: Re: "unstable" sdt probes Message-ID: <20131031035523.GD9355@raichu> References: <5268F461.7080504@FreeBSD.org> <20131024161620.GA1710@charmander> <526A9CB5.2050207@FreeBSD.org> <20131026180643.GA98676@raichu> <527026B3.2070309@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <527026B3.2070309@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Oct 2013 03:55:27 -0000 On Tue, Oct 29, 2013 at 11:20:51PM +0200, Andriy Gapon wrote: > on 26/10/2013 21:06 Mark Johnston said the following: > > The patch here is what I had in mind: > > http://people.freebsd.org/~markj/patches/zfs_probes.diff > > > > I've only compile-tested it, but it should create illumos-compatible ZFS > > probes without changing any ZFS code, assuming I understand exactly how > > they're creating/naming probes. :) > > The simplicity and straightforwardness of your patch is very seducing! :-) > I think that you missed sys/cddl/compat/opensolaris/sys/sdt.h, but that's a very > a minor issue that is trivial to fix. Oops. I didn't even know that file existed. :) > > I had something a little bit more elaborate on my mind. Something that would > allow DTRACE_PROBE macros to work without providing any additional code per each > probe. And also something that would make DTRACE_PROBE macros appealing to use > in the FreeBSD code proper. Those macros don't address the problem of setting argument types though. We could do what illumos does, i.e. have a single giant table in sdt_subr.c, but IMHO the SDT macros are more flexible. They let you create and modify probes in a kernel module without recompiling the kernel and rebooting, which something I've found handy in the past. For the purpose of creating unstable providers or quick debugging, I guess it's ok to not define argument types, so I don't object to the DTRACE_PROBE interface itself, but it'd be nice to avoid having both SDT_* calls and DTRACE_* calls all over the tree. That'd be confusing. > > So, I got some time to hack on this and here is a result: > http://people.freebsd.org/~avg/dtrace-probe-macros.diff > This change depends upon another change that I've just posted. Huh, for some reason I thought that DATA_SET() didn't work properly in a function body. I remember trying something similar when I changed sdt to use linker sets, and giving up because of some related problems, but I don't remember exactly why. If it works, then I guess it's ok, so long as the sdt code handles duplicate entries in the probe set. :) > > So, no surprise that I feel preference for my change, but I think that your > change has certain advantage as well (esp. brevity and clarity). > > What do you think? The patch looks ok to me, but I can see that it won't apply to head/ - I think it'll need at least r257152 and r254468. Thanks, -Mark From owner-freebsd-dtrace@FreeBSD.ORG Thu Oct 31 06:50:13 2013 Return-Path: Delivered-To: dtrace@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DEB776BF; Thu, 31 Oct 2013 06:50:13 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CF3D523B5; Thu, 31 Oct 2013 06:50:12 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id IAA14748; Thu, 31 Oct 2013 08:50:11 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Vbm54-000E0c-SJ; Thu, 31 Oct 2013 08:50:10 +0200 Message-ID: <5271FD6B.1040807@FreeBSD.org> Date: Thu, 31 Oct 2013 08:49:15 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Mark Johnston Subject: Re: sdt "sname" removal References: <5270246B.6070105@FreeBSD.org> <20131031033636.GC9355@raichu> In-Reply-To: <20131031033636.GC9355@raichu> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Oct 2013 06:50:13 -0000 on 31/10/2013 05:36 Mark Johnston said the following: > On Tue, Oct 29, 2013 at 11:11:07PM +0200, Andriy Gapon wrote: >> >> I never understood why FreeBSD SDT as opposed to upstream SDT requires the same >> or almost the same probe name to be specified twice. This seems to be silly and >> a little bit error-prone. >> In other words, I do not see any reason not to re-use the original upstream >> trick where double underscore in a providers name in the C code gets converted >> to a single dash in a DTrace provider name. [*] >> >> So here is my take at that: >> http://people.freebsd.org/~avg/sdt-sname-removal.diff >> >> An inline preview of the change: >> -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_ok, priv-ok, "int"); >> -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_err, priv-err, "int"); >> +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__ok, "int"); >> +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__err, "int"); >> >> It's possible that I missed some places where old style SDT_PROBE_DEFINE macros >> are used or where an old probe name is used with SDT_PROBE_ARGTYPE or SDT_PROBE. > > A good way to test this is to compare the output of 'dtrace -lv' with and > without your change. If nothing changes, I'd be pretty confident that > the diff is correct. Provided that my kernel has all of the SDT probes :-) >> >> Please test, review, comment, etc. > > I don't think this diff will apply cleanly to head - I've made some changes > that will cause conflicts, and the diff doesn't touch netinet/in_kdtrace.c > or kern/subr_devstat.c. Oh, yes, my head is from ~ 2 month ago. Need to update ASAP and will rebase the change then. > Could you also update the SDT(9) man page? Also > the "strlcpy(name, ..." immediately before the loop you added to sdt.c > becomes redundant. Good points. Will fix. -- Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Thu Oct 31 07:01:20 2013 Return-Path: Delivered-To: dtrace@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4E53F931; Thu, 31 Oct 2013 07:01:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4F48F246D; Thu, 31 Oct 2013 07:01:18 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA14879; Thu, 31 Oct 2013 09:01:17 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VbmFp-000E1Q-99; Thu, 31 Oct 2013 09:01:17 +0200 Message-ID: <52720017.3060809@FreeBSD.org> Date: Thu, 31 Oct 2013 09:00:39 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Mark Johnston Subject: Re: "unstable" sdt probes References: <5268F461.7080504@FreeBSD.org> <20131024161620.GA1710@charmander> <526A9CB5.2050207@FreeBSD.org> <20131026180643.GA98676@raichu> <527026B3.2070309@FreeBSD.org> <20131031035523.GD9355@raichu> In-Reply-To: <20131031035523.GD9355@raichu> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Oct 2013 07:01:20 -0000 on 31/10/2013 05:55 Mark Johnston said the following: > On Tue, Oct 29, 2013 at 11:20:51PM +0200, Andriy Gapon wrote: >> on 26/10/2013 21:06 Mark Johnston said the following: >>> The patch here is what I had in mind: >>> http://people.freebsd.org/~markj/patches/zfs_probes.diff >>> >>> I've only compile-tested it, but it should create illumos-compatible ZFS >>> probes without changing any ZFS code, assuming I understand exactly how >>> they're creating/naming probes. :) >> >> The simplicity and straightforwardness of your patch is very seducing! :-) >> I think that you missed sys/cddl/compat/opensolaris/sys/sdt.h, but that's a very >> a minor issue that is trivial to fix. > > Oops. I didn't even know that file existed. :) > >> >> I had something a little bit more elaborate on my mind. Something that would >> allow DTRACE_PROBE macros to work without providing any additional code per each >> probe. And also something that would make DTRACE_PROBE macros appealing to use >> in the FreeBSD code proper. > > Those macros don't address the problem of setting argument types though. I think nothing should prevent just explicitly using SDT_PROBE_ARGTYPE for probes defined via DTRACE_PROBE. If we want the types, that is. > We could do what illumos does, i.e. have a single giant table in > sdt_subr.c, but IMHO the SDT macros are more flexible. They let you > create and modify probes in a kernel module without recompiling the > kernel and rebooting, which something I've found handy in the past. Additionally, I think that what SDT_PROBE_ARGTYPE does can also be done directly in DTRACE_PROBE, because type information is provided to it. > For the purpose of creating unstable providers or quick debugging, I > guess it's ok to not define argument types, so I don't object to the > DTRACE_PROBE interface itself, but it'd be nice to avoid having both > SDT_* calls and DTRACE_* calls all over the tree. That'd be confusing. I agree.. The current status quo would be SDT_* in the FreeBSD proper code and DTRACE_* in the code with Solaris/illumos origins. >> So, I got some time to hack on this and here is a result: >> http://people.freebsd.org/~avg/dtrace-probe-macros.diff >> This change depends upon another change that I've just posted. > > Huh, for some reason I thought that DATA_SET() didn't work properly in a > function body. I remember trying something similar when I changed sdt to > use linker sets, and giving up because of some related problems, but I > don't remember exactly why. If it works, then I guess it's ok Seems to work just fine. DATA_SET just places an address of a global or static object into a linker set as far as I can see. > so long as > the sdt code handles duplicate entries in the probe set. :) Where would those comes from? Do we expect that there could be more than one SDT probe calls with the same name? But even if yes, I think that duplicates should be fine as long as they are exact duplicates. >> So, no surprise that I feel preference for my change, but I think that your >> change has certain advantage as well (esp. brevity and clarity). >> >> What do you think? > > The patch looks ok to me, but I can see that it won't apply to head/ - I > think it'll need at least r257152 and r254468. Yeah, I need to update my tree and rebase the patch. Thank you for the review! -- Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Sat Nov 2 03:56:50 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 70E51D29 for ; Sat, 2 Nov 2013 03:56:50 +0000 (UTC) (envelope-from john.37@gmail.com) Received: from mail-ob0-x233.google.com (mail-ob0-x233.google.com [IPv6:2607:f8b0:4003:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 408362D25 for ; Sat, 2 Nov 2013 03:56:50 +0000 (UTC) Received: by mail-ob0-f179.google.com with SMTP id uy5so5350862obc.10 for ; Fri, 01 Nov 2013 20:56:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=gcO+eD9BEHr2IlcjRenoHvXJScuhLOH1v121VvPYkEA=; b=xuxDdYeJNPL6+TduJX+3RZ6zCWnGHKuY1vhp7z1X+osZLNwAHBSKYBd6qhyHISZEGK BWEpP9OKGN2iUuHJeVMTNGj9r4OBtUc5eo86CVgfS+EgKZBlJUEN/Bom9gVoRiJ28UwX VqUkR2QzJsTEUpcNoc2uHTtv8/Rguy8qrq1m6CQh191f9vn12Untsw2gn2FZoL9TehdC NnpUPe2tqfYbM9Mi+t0rFGc80dgrljMyvpSOVNCnx/Kg+r59pIh2Wcl1HgWdGVKloq76 38jVvmovXuGjhNCZmyxXQDhrBEhcHIIkL1fV5E4zxTwRTdWz1bQHW3doo8RlBiTzksUi TcLA== MIME-Version: 1.0 X-Received: by 10.60.44.193 with SMTP id g1mr5066156oem.47.1383364609519; Fri, 01 Nov 2013 20:56:49 -0700 (PDT) Received: by 10.60.35.74 with HTTP; Fri, 1 Nov 2013 20:56:49 -0700 (PDT) Date: Sat, 2 Nov 2013 11:56:49 +0800 Message-ID: Subject: dtrace showed matched probes but nothing in output on FreeBSD 9.1-RELEASE From: John Luk To: freebsd-dtrace@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Nov 2013 03:56:50 -0000 Hi all, I'm a newbie in dtrace, and I following this tutorial from Oracle: http://docs.oracle.com/cd/E19205-01/820-4221/ to learn dtrace. In the example of test.c and ufunc.d, I expected output like this: % dtrace -s ufunc.d -c ./a.out a.out dtrace: script 'ufunc.d' matched 5 probes dtrace: pid 27210 has exited inet_makeaddr 1 foo1 1 foo 1 main 1 __fsr 1 But I got this instead: # dtrace -s ufunc.d -c ./a.out a.out dtrace: script 'ufunc.d' matched 5 probes dtrace: pid 86498 has exited # My system info: root@home:/home/spin6lock/test # dtrace -V dtrace: Sun D 1.7 root@home:/home/spin6lock/test # uname -a FreeBSD 9.1-RELEASE FreeBSD 9.1-RELEASE #1: Mon Oct 28 20:52:03 CST 2013 root@xiangling:/usr/obj/usr/src/sys/DTRACE amd64 Any clues? Thanks in advanced. Cheers, spin6lock From owner-freebsd-dtrace@FreeBSD.ORG Sat Nov 2 04:12:46 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 836CA24A; Sat, 2 Nov 2013 04:12:46 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4322A2DDE; Sat, 2 Nov 2013 04:12:46 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id ar20so8667965iec.26 for ; Fri, 01 Nov 2013 21:12:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=DkoW/v8U5QWlsnV/3tqXgMXnEp4dXN/FNs0Jw3TR6G4=; b=qLjA1qy6NZHJGmoCMVaNR+qprhSxy7DXHeRNVbHGGw5Svh3OtZoSL0xcxKPGXsXBkV FU63i2bzV96JB9mAMImpzK/6LXqccFReWnE5vIXt4BUi/T19VcB9BFU5dxzbipqJVFj8 ARYgJcIfey4R7plQXcMhl/JeIIWt4lkwRtqzjKdDVzvtGO0ASx9NU0KfSsOHnrYvVXtv TMlO8jmID8JQPkCURiWyTK+XEQwgEeRdAWdlkMWNMJdQIvb4NvESLSP7Loc16QqpbZxr XNcp/CWkLsYesGTP/5drN5qIqXiYAaLx5ZjXQEXRxjHCsnMOwn/wFPZZvr7LtoCBeEoS utbA== X-Received: by 10.50.66.101 with SMTP id e5mr4939082igt.26.1383365565680; Fri, 01 Nov 2013 21:12:45 -0700 (PDT) Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13]) by mx.google.com with ESMTPSA id qi3sm7187998igc.8.2013.11.01.21.12.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 01 Nov 2013 21:12:44 -0700 (PDT) Sender: Mark Johnston Date: Sat, 2 Nov 2013 00:12:40 -0400 From: Mark Johnston To: Andriy Gapon Subject: Re: "unstable" sdt probes Message-ID: <20131102041240.GA23329@raichu> References: <5268F461.7080504@FreeBSD.org> <20131024161620.GA1710@charmander> <526A9CB5.2050207@FreeBSD.org> <20131026180643.GA98676@raichu> <527026B3.2070309@FreeBSD.org> <20131031035523.GD9355@raichu> <52720017.3060809@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52720017.3060809@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Nov 2013 04:12:46 -0000 On Thu, Oct 31, 2013 at 09:00:39AM +0200, Andriy Gapon wrote: > on 31/10/2013 05:55 Mark Johnston said the following: > > On Tue, Oct 29, 2013 at 11:20:51PM +0200, Andriy Gapon wrote: > >> on 26/10/2013 21:06 Mark Johnston said the following: > >>> The patch here is what I had in mind: > >>> http://people.freebsd.org/~markj/patches/zfs_probes.diff > >>> > >>> I've only compile-tested it, but it should create illumos-compatible ZFS > >>> probes without changing any ZFS code, assuming I understand exactly how > >>> they're creating/naming probes. :) > >> > >> The simplicity and straightforwardness of your patch is very seducing! :-) > >> I think that you missed sys/cddl/compat/opensolaris/sys/sdt.h, but that's a very > >> a minor issue that is trivial to fix. > > > > Oops. I didn't even know that file existed. :) > > > >> > >> I had something a little bit more elaborate on my mind. Something that would > >> allow DTRACE_PROBE macros to work without providing any additional code per each > >> probe. And also something that would make DTRACE_PROBE macros appealing to use > >> in the FreeBSD code proper. > > > > Those macros don't address the problem of setting argument types though. > > I think nothing should prevent just explicitly using SDT_PROBE_ARGTYPE for > probes defined via DTRACE_PROBE. If we want the types, that is. > > > We could do what illumos does, i.e. have a single giant table in > > sdt_subr.c, but IMHO the SDT macros are more flexible. They let you > > create and modify probes in a kernel module without recompiling the > > kernel and rebooting, which something I've found handy in the past. > > Additionally, I think that what SDT_PROBE_ARGTYPE does can also be done directly > in DTRACE_PROBE, because type information is provided to it. > > > For the purpose of creating unstable providers or quick debugging, I > > guess it's ok to not define argument types, so I don't object to the > > DTRACE_PROBE interface itself, but it'd be nice to avoid having both > > SDT_* calls and DTRACE_* calls all over the tree. That'd be confusing. > > I agree.. The current status quo would be SDT_* in the FreeBSD proper code and > DTRACE_* in the code with Solaris/illumos origins. > > >> So, I got some time to hack on this and here is a result: > >> http://people.freebsd.org/~avg/dtrace-probe-macros.diff > >> This change depends upon another change that I've just posted. > > > > Huh, for some reason I thought that DATA_SET() didn't work properly in a > > function body. I remember trying something similar when I changed sdt to > > use linker sets, and giving up because of some related problems, but I > > don't remember exactly why. If it works, then I guess it's ok > > Seems to work just fine. > DATA_SET just places an address of a global or static object into a linker set > as far as I can see. > > > so long as > > the sdt code handles duplicate entries in the probe set. :) > > Where would those comes from? > Do we expect that there could be more than one SDT probe calls with the same > name? But even if yes, I think that duplicates should be fine as long as they > are exact duplicates. Sure, there are multiple probe sites for the sdt:::arc-miss probe, for example. For the ip provider, there are multiple probe sites for ip:::receive: at least one for IPv4, and at least one for IPv6. > > >> So, no surprise that I feel preference for my change, but I think that your > >> change has certain advantage as well (esp. brevity and clarity). > >> > >> What do you think? > > > > The patch looks ok to me, but I can see that it won't apply to head/ - I > > think it'll need at least r257152 and r254468. > > Yeah, I need to update my tree and rebase the patch. > Thank you for the review! > > -- > Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Sat Nov 2 10:21:32 2013 Return-Path: Delivered-To: dtrace@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EAA2BEFD; Sat, 2 Nov 2013 10:21:32 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 152EF2CF8; Sat, 2 Nov 2013 10:21:31 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA21075; Sat, 02 Nov 2013 12:19:20 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VcYIa-000NiU-0z; Sat, 02 Nov 2013 12:19:20 +0200 Message-ID: <5274D170.6050505@FreeBSD.org> Date: Sat, 02 Nov 2013 12:18:24 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Mark Johnston Subject: Re: "unstable" sdt probes References: <5268F461.7080504@FreeBSD.org> <20131024161620.GA1710@charmander> <526A9CB5.2050207@FreeBSD.org> <20131026180643.GA98676@raichu> <527026B3.2070309@FreeBSD.org> <20131031035523.GD9355@raichu> <52720017.3060809@FreeBSD.org> <20131102041240.GA23329@raichu> In-Reply-To: <20131102041240.GA23329@raichu> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: dtrace@FreeBSD.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Nov 2013 10:21:33 -0000 on 02/11/2013 06:12 Mark Johnston said the following: > Sure, there are multiple probe sites for the sdt:::arc-miss probe, for > example. For the ip provider, there are multiple probe sites for > ip:::receive: at least one for IPv4, and at least one for IPv6. Okay, I think that the following lines in sdt_create_probe should handle this: if (dtrace_probe_lookup(prov->id, mod, func, name) != DTRACE_IDNONE) return; It's, of course, a little bit wasteful to have a few duplicate sdt_probe and sdt_argtype structs, but should not be too terrible. I wonder if there is some low level trick to collapse this duplicates into a single object during linking. Probably not... -- Andriy Gapon From owner-freebsd-dtrace@FreeBSD.ORG Sat Nov 2 12:54:21 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6B6D8605 for ; Sat, 2 Nov 2013 12:54:21 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3C5642423 for ; Sat, 2 Nov 2013 12:54:21 +0000 (UTC) Received: from pool-71-167-166-95.nycmny.fios.verizon.net ([71.167.166.95]:59476 helo=new-host.home) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80.1) (envelope-from ) id 1VcaiZ-0001gj-RI; Sat, 02 Nov 2013 08:54:20 -0400 Content-Type: multipart/signed; boundary="Apple-Mail=_E85DF1E5-8618-48DE-A6A4-4AA3830ADCE4"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1811\)) Subject: Re: dtrace showed matched probes but nothing in output on FreeBSD 9.1-RELEASE From: George Neville-Neil In-Reply-To: Date: Sat, 2 Nov 2013 08:54:20 -0400 Message-Id: References: To: John Luk X-Mailer: Apple Mail (2.1811) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: freebsd-dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Nov 2013 12:54:21 -0000 --Apple-Mail=_E85DF1E5-8618-48DE-A6A4-4AA3830ADCE4 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On Nov 1, 2013, at 23:56 , John Luk wrote: > Hi all, > I'm a newbie in dtrace, and I following this tutorial from Oracle: > http://docs.oracle.com/cd/E19205-01/820-4221/ to learn dtrace. In the > example of test.c and ufunc.d, I expected output like this: > > % dtrace -s ufunc.d -c ./a.out a.out > > dtrace: script 'ufunc.d' matched 5 probes > dtrace: pid 27210 has exited > > inet_makeaddr 1 > foo1 1 > foo 1 > main 1 > __fsr 1 > > > But I got this instead: > > # dtrace -s ufunc.d -c ./a.out a.out > dtrace: script 'ufunc.d' matched 5 probes > dtrace: pid 86498 has exited > > # > > My system info: > root@home:/home/spin6lock/test # dtrace -V > dtrace: Sun D 1.7 > root@home:/home/spin6lock/test # uname -a > FreeBSD 9.1-RELEASE FreeBSD 9.1-RELEASE #1: Mon Oct 28 20:52:03 CST > 2013 root@xiangling:/usr/obj/usr/src/sys/DTRACE amd64 > > Any clues? Thanks in advanced. > How did you compile the progam to a.out? Best, George --Apple-Mail=_E85DF1E5-8618-48DE-A6A4-4AA3830ADCE4 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlJ09fwACgkQYdh2wUQKM9LIawCg1IF4GGnpuNKbiLRvdGCYYEGB aHUAmgMrRdovDrCxaA9NtNu6JWv7M7nR =H8rI -----END PGP SIGNATURE----- --Apple-Mail=_E85DF1E5-8618-48DE-A6A4-4AA3830ADCE4-- From owner-freebsd-dtrace@FreeBSD.ORG Sat Nov 2 13:48:15 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 27BBFF9 for ; Sat, 2 Nov 2013 13:48:15 +0000 (UTC) (envelope-from john.37@gmail.com) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E9310260C for ; Sat, 2 Nov 2013 13:48:14 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id wp18so5648920obc.15 for ; Sat, 02 Nov 2013 06:48:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+mpQVkwBOTU/WwBIQhj+587tOtqa7QuAqdypJ81ROG8=; b=m7oX1Cckqf3WMlOpKqVWex6hS03nUiQTNmoVt0D+84tUXDlv0xjOrclZk3up7LLIOZ L3L68QwaxBOKDeWSvWvXG5e+gJSYlpQStYvqxyxbNa8aEDTXIoOMbl/y11cC5gxTMmu3 RWw/5BzEOJzbVeJtekvJgOk9V09W1/EepG64jx3vLv/tcm//mOIOdUNuVLh8B+rsaW9l oA+c8gYflBvatZl4J8Gk9AaCAufqH6FacgalQ6pKoIAxXmPhaskCH3VujRT7uuscXbFk Eo/dawTp2a3s6eplIyW/AZxeQXfwzwA85tlwWYQHTn5TelVj+4H4Ape4OeFaew4TwXR0 hOPA== MIME-Version: 1.0 X-Received: by 10.60.40.136 with SMTP id x8mr1360282oek.49.1383400093358; Sat, 02 Nov 2013 06:48:13 -0700 (PDT) Received: by 10.60.35.74 with HTTP; Sat, 2 Nov 2013 06:48:13 -0700 (PDT) In-Reply-To: References: Date: Sat, 2 Nov 2013 21:48:13 +0800 Message-ID: Subject: Re: dtrace showed matched probes but nothing in output on FreeBSD 9.1-RELEASE From: John Luk To: George Neville-Neil Content-Type: text/plain; charset=UTF-8 Cc: freebsd-dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Nov 2013 13:48:15 -0000 gcc -g test.c, and I saw my test function name in dtrace -l|tail. 2013/11/2 George Neville-Neil : > > On Nov 1, 2013, at 23:56 , John Luk wrote: > >> Hi all, >> I'm a newbie in dtrace, and I following this tutorial from Oracle: >> http://docs.oracle.com/cd/E19205-01/820-4221/ to learn dtrace. In the >> example of test.c and ufunc.d, I expected output like this: >> >> % dtrace -s ufunc.d -c ./a.out a.out >> >> dtrace: script 'ufunc.d' matched 5 probes >> dtrace: pid 27210 has exited >> >> inet_makeaddr 1 >> foo1 1 >> foo 1 >> main 1 >> __fsr 1 >> >> >> But I got this instead: >> >> # dtrace -s ufunc.d -c ./a.out a.out >> dtrace: script 'ufunc.d' matched 5 probes >> dtrace: pid 86498 has exited >> >> # >> >> My system info: >> root@home:/home/spin6lock/test # dtrace -V >> dtrace: Sun D 1.7 >> root@home:/home/spin6lock/test # uname -a >> FreeBSD 9.1-RELEASE FreeBSD 9.1-RELEASE #1: Mon Oct 28 20:52:03 CST >> 2013 root@xiangling:/usr/obj/usr/src/sys/DTRACE amd64 >> >> Any clues? Thanks in advanced. >> > > How did you compile the progam to a.out? > > Best, > George > >