From owner-freebsd-dtrace@FreeBSD.ORG Sun Sep 15 19:28:07 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C2AD096B for ; Sun, 15 Sep 2013 19:28:07 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from nm27-vm2.bullet.mail.ne1.yahoo.com (nm27-vm2.bullet.mail.ne1.yahoo.com [98.138.91.215]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 65EF62181 for ; Sun, 15 Sep 2013 19:28:03 +0000 (UTC) Received: from [98.138.90.56] by nm27.bullet.mail.ne1.yahoo.com with NNFMP; 15 Sep 2013 19:27:57 -0000 Received: from [98.138.226.127] by tm9.bullet.mail.ne1.yahoo.com with NNFMP; 15 Sep 2013 19:27:57 -0000 Received: from [127.0.0.1] by smtp206.mail.ne1.yahoo.com with NNFMP; 15 Sep 2013 19:27:57 -0000 X-Yahoo-Newman-Id: 595644.52995.bm@smtp206.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: IMKO9fQVM1nMbVjf8hnjdVmEAlgaMWXtmTataSZwXlH_zyn UsPlMly5FqJ0yQ9LzYag3jDbCwYuAH88ayCOOz4Q0rJRa6OlrlrpvqUFna9j sdqNL3fYtTVYBCo9WexXzqJE2lGDx0tsSLNvJdQFuUgulB2DasSK2pQULPWw jjvlFXAy35GEwkGF7xHHe4nQHSrJ2NcyqmdnDdklyPsE_8inhygkItflCn46 3s8ftWKXFJbttrm.PEC9kgokzp5OMjNtiQqQem0XoPlZ_ksqodzZOgBz4JYo W1h1cVhMdMKRuKwj..Q6wvORqaHyniCjpVSZQrQOr8Stya1ZVynziXxBGfLc O_8oYG4gsYb7wgF5aZysootzNYe1HSPcb5p7tNjauvLhTih1IhhvhP.B3la9 88djMxu4Kc1.aPDlPRLHQo0a1lLZ3JGmaHPNi_NvZWOWVdiBdB5cCyNlPI8M hHgTbiJhINE2pKYeabfkZhxOt9yPam6qcXG.7X0yLHvgGmcwqxByKnLcDzXG JwsJItMQwcRo9y0KV0UQ35jOGOZ6i2peOut6kVs4r8csLDY6PM1mWErFVRsm 1PstLIDgal3gxvrYfqPotkV_KZLAqImdy0meNBfxevyIGrWosGgDknyEFk9A eIvWNeFFq7ZbcxZ9f X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with ) by smtp206.mail.ne1.yahoo.com with SMTP; 15 Sep 2013 12:27:57 -0700 PDT Message-ID: <52360A3E.40804@FreeBSD.org> Date: Sun, 15 Sep 2013 14:27:58 -0500 From: Pedro Giffuni Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: dtrace@FreeBSD.org Subject: vmem(9) use in Dtrace Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Sep 2013 19:28:07 -0000 Hi; Just noticed this, in the old DtraceTODO wiki: https://wiki.freebsd.org/DTraceTODO There is mention of some use that Dtrace makes of Solaris's vmem allocator, specifically vmem_create(), vmem_destroy(), vmem_alloc(), and vmem_free(). These functions (not exact but similar ones) have been brought to FreeBSD 10 (r252330). According to opengrok we may just have to uncomment some Solaris code in sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c This would reduce differences with the Solaris code but it has to be examined carefully as I think John Birrell had already solved the issue by providing the resource id's natively (kmem). Just thought I would point it out ... It may be that it is not worth spending too much time on it since the existing code should just work. Pedro. From owner-freebsd-dtrace@FreeBSD.ORG Sun Sep 15 19:53:49 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 34152AA6; Sun, 15 Sep 2013 19:53:49 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-ve0-x22c.google.com (mail-ve0-x22c.google.com [IPv6:2607:f8b0:400c:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D77B4227A; Sun, 15 Sep 2013 19:53:48 +0000 (UTC) Received: by mail-ve0-f172.google.com with SMTP id oz11so2361925veb.17 for ; Sun, 15 Sep 2013 12:53:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=U6yzXai2yf11hevd1q7vul6IsYgyktzjfPqMVUnZd2g=; b=w0mWYMjvH/pY8hu9HPzb5mg6FVTYP9jgE4Ylds2CXAb4jlMmHSOOscUIIvovBAxvis OFaq4IG8G6N2AqhX1J0yi80cfVcMBibdsD5YYvHLq4cELnhYbDpc7MVvXVBjneq3bafB hErYp4s3/QV80vmqJWJ0vaY5bXNUzcNca1amAFCD5N+geYbfMiFJWlD84FsSmxIOEABq ttLWjFr5wOyd8/5FGwIOMcnqu6fBNX5RQ+JCv2LU9lqv6HcZFf2DfKiYtT/dEwY0gqT0 iBXD1gD+AgQ1C/RxK6v7UkTUA/ytRLcO/pqvKgfH8XsxBldqMbnJVrcbpTq75cIfzCCq e+0w== MIME-Version: 1.0 X-Received: by 10.221.56.194 with SMTP id wd2mr23735964vcb.7.1379274827897; Sun, 15 Sep 2013 12:53:47 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.220.65.132 with HTTP; Sun, 15 Sep 2013 12:53:47 -0700 (PDT) In-Reply-To: <52360A3E.40804@FreeBSD.org> References: <52360A3E.40804@FreeBSD.org> Date: Sun, 15 Sep 2013 21:53:47 +0200 X-Google-Sender-Auth: Vudm9r220tycwK83XbkP9J3Pzzs Message-ID: Subject: Re: vmem(9) use in Dtrace From: Davide Italiano To: Pedro Giffuni Content-Type: text/plain; charset=ISO-8859-1 Cc: dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Sep 2013 19:53:49 -0000 On Sun, Sep 15, 2013 at 9:27 PM, Pedro Giffuni wrote: > Hi; > > Just noticed this, in the old DtraceTODO wiki: > > https://wiki.freebsd.org/DTraceTODO > > There is mention of some use that Dtrace makes of Solaris's vmem > allocator, specifically vmem_create(), vmem_destroy(), vmem_alloc(), and > vmem_free(). These functions (not exact but similar ones) have > been brought to FreeBSD 10 (r252330). > > According to opengrok we may just have to uncomment some Solaris > code in sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c > We need also to provide at least some sort of wrappers for e.g. flags passed to vmem_alloc which takes VM_SLEEP in solaris while in FreeBSD this is M_WAITOK etc... > This would reduce differences with the Solaris code but it has > to be examined carefully as I think John Birrell had already > solved the issue by providing the resource id's natively (kmem). > > Just thought I would point it out ... It may be that it is not > worth spending too much time on it since the existing code should > just work. > vmem is designed to be a generic resource allocator. I still need to analyze how Dtrace uses it but in general it provides better scalability and less fragmentation wrt home-rolled allocators we have in the kernel. FWIW, Illumos uses it for several things, e.g. pid/tid allocation. It might worth a try. > Pedro. > _______________________________________________ > freebsd-dtrace@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace > To unsubscribe, send any mail to "freebsd-dtrace-unsubscribe@freebsd.org" Thanks, -- Davide "There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare From owner-freebsd-dtrace@FreeBSD.ORG Sun Sep 15 22:16:29 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CAD6C629 for ; Sun, 15 Sep 2013 22:16:29 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-yh0-x235.google.com (mail-yh0-x235.google.com [IPv6:2607:f8b0:4002:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8F67228E4 for ; Sun, 15 Sep 2013 22:16:29 +0000 (UTC) Received: by mail-yh0-f53.google.com with SMTP id b20so1668813yha.26 for ; Sun, 15 Sep 2013 15:16:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=EjiLOwkUSJQ8YzhRYW1jg14jzsAZfNlBHCy5/1pCdNo=; b=lK3v7HnYTRYMghF6W0+uNQklwEm1h3EGg4jyETbnljX7WH9D+SON//bx3o3etU576w LMWbRZ9EPrkckuwwvOVYS1/xVW8QSKM/zY0yc0B6z4cKrTvLU8vrv/c5GUMVgbJg7PcA bJXN0/7Hy1M6LxWSrq76d0T4YmqpktFD8Xpn3Dy5yT06IzdQnTiY5og7AQDcPuq4Q+lD y/yhR0IIFidZ6uegeqIYjZsRUMe8MRt/Wnqz3U3YYRBmBx+vWhQrtIc5oAIjPQ8+82YL ruBpgFBhhJ8JCIeGAsjVY45MZd0KvOVDRVYGYU6p7HXUX2bXMyNhzj8DDdDs+RkVXlO6 ix0A== X-Received: by 10.236.83.69 with SMTP id p45mr24664126yhe.40.1379283388647; Sun, 15 Sep 2013 15:16:28 -0700 (PDT) Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13]) by mx.google.com with ESMTPSA id t62sm638340yhd.26.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 15 Sep 2013 15:16:28 -0700 (PDT) Sender: Mark Johnston Date: Sun, 15 Sep 2013 18:16:23 -0400 From: Mark Johnston To: dtrace-discuss@lists.dtrace.org Subject: pr_psargs on FreeBSD Message-ID: <20130915221622.GA2981@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Sep 2013 22:16:29 -0000 Hi! One of the problems with FreeBSD's DTrace implementation is that it doesn't have a good way of getting the arguments of a given process. This is because of the way that they're stored, which is as a single buffer with null-terminator between consecutive arguments. In particular, there is no way within DTrace to convert such a buffer to a single string (say by replacing all but the last terminator with spaces). So scripts like execsnoop will only print the first element of a process' argv on FreeBSD, which isn't particularly helpful for interpreted programs. You can get a fixed number of arguments with something like printf("%s %s %s", cupsinfo->pr_psargs, curpsinfo->pr_psargs + strlen(curpsinfo->pr_psargs) + 1, curpsinfo->pr_psargs + strlen(curpsinfo->pr_psargs) + 1 + strlen(curpsinfo->pr_psargs + strlen(curpsinfo->pr_psargs) + 1) + 1); but this is less than ideal. :) Apparently OS X has the same problem, but I don't have a machine to test with, so I'm not completely sure. It seems to me that there are two ways to solve this problem: change the format that FreeBSD uses to store process arguments, or add a function to DTrace which can convert the argument buffer to a string. I'm not too keen on the former approach, so here's a proposal for the latter. It'd be great to get some feedback on it and find out whether anyone thinks it's a terrible idea. :) It's kind of a strong-armed approach, but the problem's been around for a while and I can't think of any clever solutions - I'd love to see an alternative solution. My patch against FreeBSD (pasted below) adds a function called memstr() to DTrace. memstr() takes three arguments: addr, c, and len, and returns a copy of addr of length len with all null-terminators replaced by c, and with the last byte replaced by a null-terminator. In particular, memstr() always returns a string of length len - 1, unless len == 0, in which case we return the empty string. With this function, the translator for the pr_psargs fields becomes translator psinfo_t < struct proc *T > { ... pr_psargs = memstr(T->p_args->ar_args, ' ', T->p_args->ar_length); ... }; and execsnoop works properly. Any thoughts on this function? Have I missed a better solution? A patch for FreeBSD is below. Thanks, -Mark diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c index 9c9b2a6..83c81e5 100644 --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c @@ -122,8 +122,9 @@ #define DT_VERS_1_8_1 DT_VERSION_NUMBER(1, 8, 1) #define DT_VERS_1_9 DT_VERSION_NUMBER(1, 9, 0) #define DT_VERS_1_9_1 DT_VERSION_NUMBER(1, 9, 1) -#define DT_VERS_LATEST DT_VERS_1_9_1 -#define DT_VERS_STRING "Sun D 1.9.1" +#define DT_VERS_1_9_2 DT_VERSION_NUMBER(1, 9, 2) +#define DT_VERS_LATEST DT_VERS_1_9_2 +#define DT_VERS_STRING "Sun D 1.9.2" const dt_version_t _dtrace_versions[] = { DT_VERS_1_0, /* D API 1.0.0 (PSARC 2001/466) Solaris 10 FCS */ @@ -145,6 +146,7 @@ const dt_version_t _dtrace_versions[] = { DT_VERS_1_8_1, /* D API 1.8.1 */ DT_VERS_1_9, /* D API 1.9 */ DT_VERS_1_9_1, /* D API 1.9.1 */ + DT_VERS_1_9_2, /* D API 1.9.2 */ 0 }; @@ -311,6 +313,8 @@ static const dt_ident_t _dtrace_globals[] = { &dt_idops_func, "void(@)" }, { "memref", DT_IDENT_FUNC, 0, DIF_SUBR_MEMREF, DT_ATTR_STABCMN, DT_VERS_1_1, &dt_idops_func, "uintptr_t *(void *, size_t)" }, +{ "memstr", DT_IDENT_FUNC, 0, DIF_SUBR_MEMSTR, DT_ATTR_STABCMN, DT_VERS_1_0, + &dt_idops_func, "string(void *, char, size_t)" }, { "min", DT_IDENT_AGGFUNC, 0, DTRACEAGG_MIN, DT_ATTR_STABCMN, DT_VERS_1_0, &dt_idops_func, "void(@)" }, { "mod", DT_IDENT_ACTFUNC, 0, DT_ACT_MOD, DT_ATTR_STABCMN, diff --git a/cddl/lib/libdtrace/psinfo.d b/cddl/lib/libdtrace/psinfo.d index 068e72e..b2a009a 100644 --- a/cddl/lib/libdtrace/psinfo.d +++ b/cddl/lib/libdtrace/psinfo.d @@ -57,7 +57,7 @@ translator psinfo_t < struct proc *T > { pr_gid = T->p_ucred->cr_rgid; pr_egid = T->p_ucred->cr_groups[0]; pr_addr = 0; - pr_psargs = stringof(T->p_args->ar_args); + pr_psargs = memstr(T->p_args->ar_args, ' ', T->p_args->ar_length); pr_arglen = T->p_args->ar_length; pr_jailid = T->p_ucred->cr_prison->pr_id; }; diff --git a/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c b/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c index babc42c4..6d991fb 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c +++ b/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c @@ -4920,6 +4920,38 @@ inetout: regs[rd] = (uintptr_t)end + 1; break; } + case DIF_SUBR_MEMSTR: { + char *str = (char *)mstate->dtms_scratch_ptr; + uintptr_t mem = tupregs[0].dttk_value; + char c = tupregs[1].dttk_value; + size_t size = tupregs[2].dttk_value; + uint8_t n; + int i; + + regs[rd] = 0; + + if (size == 0) + break; + + if (!dtrace_canload(mem, size - 1, mstate, vstate)) + break; + + if (!DTRACE_INSCRATCH(mstate, size)) { + DTRACE_CPUFLAG_SET(CPU_DTRACE_NOSCRATCH); + break; + } + + for (i = 0; i < size - 1; i++) { + n = dtrace_load8(mem++); + str[i] = (n == 0) ? c : n; + } + str[size - 1] = 0; + + regs[rd] = (uintptr_t)str; + mstate->dtms_scratch_ptr += size; + break; + } + case DIF_SUBR_TYPEREF: { uintptr_t size = 4 * sizeof(uintptr_t); uintptr_t *typeref = (uintptr_t *) P2ROUNDUP(mstate->dtms_scratch_ptr, sizeof(uintptr_t)); @@ -9102,6 +9134,7 @@ dtrace_difo_validate_helper(dtrace_difo_t *dp) subr == DIF_SUBR_NTOHL || subr == DIF_SUBR_NTOHLL || subr == DIF_SUBR_MEMREF || + subr == DIF_SUBR_MEMSTR || subr == DIF_SUBR_TYPEREF) break; diff --git a/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h b/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h index 8728e30..295457c 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h +++ b/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h @@ -311,8 +311,9 @@ typedef enum dtrace_probespec { #define DIF_SUBR_SX_SHARED_HELD 48 #define DIF_SUBR_SX_EXCLUSIVE_HELD 49 #define DIF_SUBR_SX_ISEXCLUSIVE 50 +#define DIF_SUBR_MEMSTR 51 -#define DIF_SUBR_MAX 50 /* max subroutine value */ +#define DIF_SUBR_MAX 51 /* max subroutine value */ typedef uint32_t dif_instr_t; From owner-freebsd-dtrace@FreeBSD.ORG Mon Sep 16 00:36:32 2013 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7F71D977 for ; Mon, 16 Sep 2013 00:36:32 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from nm27-vm5.bullet.mail.ne1.yahoo.com (nm27-vm5.bullet.mail.ne1.yahoo.com [98.138.91.249]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C4CE2F5B for ; Mon, 16 Sep 2013 00:36:32 +0000 (UTC) Received: from [98.138.101.132] by nm27.bullet.mail.ne1.yahoo.com with NNFMP; 16 Sep 2013 00:30:40 -0000 Received: from [98.138.84.38] by tm20.bullet.mail.ne1.yahoo.com with NNFMP; 16 Sep 2013 00:30:40 -0000 Received: from [127.0.0.1] by smtp106.mail.ne1.yahoo.com with NNFMP; 16 Sep 2013 00:30:40 -0000 X-Yahoo-Newman-Id: 693927.40817.bm@smtp106.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: Rs3JyKgVM1lprkGOwXLD78c8t5EszgU5puLcb5C3Pq539kf v7v_maaSCqH8W9FDjALC.pSmCVNCMPjV7S6tAACvD0LSqapl_bX8HGGIeFwW D0ZI_De52H4wIm7ieT0YRgIBHcYW2y0lLmhY3yMG2rZH55j468JfpInq27CL 28B3MEk7Ket6Mbye17mN0kx1_ykc_2kym47U1X21iEFikl8gNN8AcgjXMU3A W.ChSS.U0aC2Upb1yMuLqqhgxrhyb_Xfc.4_X5oMUh0XHgkCS4nQmqoBtHQu D.iYZAIfPz8XqjN.dgN4AA4qDVMf6k1ZP79p1cwP_0djW605rpwjBdZZAUym s2_yGQlTMGIwkYGQGtL1XHZ2TuiZqQ5EVzYZxJJjuXzZwt41XihhTYPIjxBu xL3VoK81tOnXV._VkRtxgQ_CNr.R3tIrWSesuu1vAtiOt0H42XirACEXiKht G9L1She1NvpcP5sgrMAmpbCC2DrBJliO.772t5CrQExFkj.f_XtyqwrbhqfJ 1ECgx8ysEHHupr1rjvW4ixfzmqPDmBhPhWMRITZJMs3MWNG8PLonfT0BBUyE - X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with ) by smtp106.mail.ne1.yahoo.com with SMTP; 15 Sep 2013 17:30:40 -0700 PDT Message-ID: <52365131.8020709@FreeBSD.org> Date: Sun, 15 Sep 2013 19:30:41 -0500 From: Pedro Giffuni Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: Davide Italiano Subject: Re: vmem(9) use in Dtrace References: <52360A3E.40804@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: jeff@FreeBSD.org, dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Sep 2013 00:36:32 -0000 On 15/09/2013 2:53 p. m., Davide Italiano wrote: > On Sun, Sep 15, 2013 at 9:27 PM, Pedro Giffuni wrote: >> Hi; >> >> Just noticed this, in the old DtraceTODO wiki: >> >> https://wiki.freebsd.org/DTraceTODO >> >> There is mention of some use that Dtrace makes of Solaris's vmem >> allocator, specifically vmem_create(), vmem_destroy(), vmem_alloc(), and >> vmem_free(). These functions (not exact but similar ones) have >> been brought to FreeBSD 10 (r252330). >> >> According to opengrok we may just have to uncomment some Solaris >> code in sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c >> > > We need also to provide at least some sort of wrappers for e.g. flags > passed to vmem_alloc which takes VM_SLEEP in solaris while in FreeBSD > this is M_WAITOK etc... > >> This would reduce differences with the Solaris code but it has >> to be examined carefully as I think John Birrell had already >> solved the issue by providing the resource id's natively (kmem). >> >> Just thought I would point it out ... It may be that it is not >> worth spending too much time on it since the existing code should >> just work. >> > > vmem is designed to be a generic resource allocator. I still need to > analyze how Dtrace uses it but in general it provides better > scalability and less fragmentation wrt home-rolled allocators we have > in the kernel. FWIW, Illumos uses it for several things, e.g. pid/tid > allocation. It might worth a try. > FWIW, the next point in the old DTrace TODO list is that Solaris has vmem_cache_*. Maybe providing a cache is something that may be useful in FreeBSD too. Regards, Pedro. From owner-freebsd-dtrace@FreeBSD.ORG Mon Sep 16 18:20:13 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D3E0FB30; Mon, 16 Sep 2013 18:20:13 +0000 (UTC) (envelope-from nico@cryptonector.com) Received: from homiemail-a32.g.dreamhost.com (caiajhbdccah.dreamhost.com [208.97.132.207]) by mx1.freebsd.org (Postfix) with ESMTP id AAFFB225E; Mon, 16 Sep 2013 18:20:13 +0000 (UTC) Received: from homiemail-a32.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a32.g.dreamhost.com (Postfix) with ESMTP id D516558406A; Mon, 16 Sep 2013 11:20:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cryptonector.com; h= mime-version:in-reply-to:references:date:message-id:subject:from :to:cc:content-type; s=cryptonector.com; bh=JePi91bZ+SeBbhE69Ho7 R6fhYr8=; b=Rl3R4cgLBjb1G/Hb+d1Gi5GTwFXv8ae4Aocjt7igTsgN/0pdh6ou 9OV4L+fSHEPIsXu7WBuJnZS7/vGfEVLTpU2h2ksKdujBnCi+RQvIa09ZlMohVWub xWuRAB7ODsWyDFenpo8tgQQlmwvlkwW/SHlXjjg11zKz12Djl+76Its= Received: from mail-we0-f169.google.com (mail-we0-f169.google.com [74.125.82.169]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: nico@cryptonector.com) by homiemail-a32.g.dreamhost.com (Postfix) with ESMTPSA id BDDDF584064; Mon, 16 Sep 2013 11:19:58 -0700 (PDT) Received: by mail-we0-f169.google.com with SMTP id t60so4116000wes.14 for ; Mon, 16 Sep 2013 11:19:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9Eu9AEkTLHPn7RhabmPnSN+par7+7OFZMREe3lD0HfI=; b=L+vYPnVV4qrmkzogrrnxYdpW9E62YDBO6AZgOYPukbBBRVZoQZiWUOhkhJ/XnHl/F4 qlP9D7gra+gIeGghMVJJhnrVu6cZiPWX+Hi94kHfhkPyjWGv+RAnIGXtj2/2boXpE5MV xQWwahncSVmZwkdOlk/VAcxT2vbaDsfcP8a5uEYD08hnKZcJAri+yEj+F+LhjLIPId3E Q+rWfG1XJhmjTd06CQg/UeJYJNLDbFmcIGYbYfa19b2o3rSxvfUVzm6kHHqaMv0o7Cck JrPcBcIWtOGQW50UYaSbjP4e/W/T/wymhe6G/8BjytTRT+7Il5xWe7LPAfymOaPILz+K rtYA== MIME-Version: 1.0 X-Received: by 10.194.11.67 with SMTP id o3mr24180161wjb.0.1379355597222; Mon, 16 Sep 2013 11:19:57 -0700 (PDT) Received: by 10.216.240.70 with HTTP; Mon, 16 Sep 2013 11:19:57 -0700 (PDT) In-Reply-To: <20130915221622.GA2981@raichu> References: <20130915221622.GA2981@raichu> Date: Mon, 16 Sep 2013 13:19:57 -0500 Message-ID: Subject: Re: [dtrace-discuss] pr_psargs on FreeBSD From: Nico Williams To: Mark Johnston Content-Type: text/plain; charset=UTF-8 Cc: dtrace-discuss@lists.dtrace.org, freebsd-dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Sep 2013 18:20:13 -0000 On Sun, Sep 15, 2013 at 5:16 PM, Mark Johnston wrote: > Hi! > > One of the problems with FreeBSD's DTrace implementation is that it > doesn't have a good way of getting the arguments of a given process. > This is because of the way that they're stored, which is as a single > buffer with null-terminator between consecutive arguments. In > particular, there is no way within DTrace to convert such a buffer to a > single string (say by replacing all but the last terminator with > spaces). So scripts like execsnoop will only print the first element of > a process' argv on FreeBSD, which isn't particularly helpful for > interpreted programs. > > You can get a fixed number of arguments with something like > > printf("%s %s %s", cupsinfo->pr_psargs, > curpsinfo->pr_psargs + strlen(curpsinfo->pr_psargs) + 1, > curpsinfo->pr_psargs + strlen(curpsinfo->pr_psargs) + 1 + > strlen(curpsinfo->pr_psargs + strlen(curpsinfo->pr_psargs) + 1) + 1); > > but this is less than ideal. :) > > Apparently OS X has the same problem, but I don't have a machine to test > with, so I'm not completely sure. > > It seems to me that there are two ways to solve this problem: change the > format that FreeBSD uses to store process arguments, or add a function > to DTrace which can convert the argument buffer to a string. I'm not too > keen on the former approach, so here's a proposal for the latter. It'd > be great to get some feedback on it and find out whether anyone thinks > it's a terrible idea. :) > > It's kind of a strong-armed approach, but the problem's been around for a > while and I can't think of any clever solutions - I'd love to see an > alternative solution. > > My patch against FreeBSD (pasted below) adds a function called memstr() > to DTrace. memstr() takes three arguments: addr, c, and len, and returns > a copy of addr of length len with all null-terminators replaced by c, > and with the last byte replaced by a null-terminator. In particular, > memstr() always returns a string of length len - 1, unless len == 0, in > which case we return the empty string. > > With this function, the translator for the pr_psargs fields becomes > > translator psinfo_t < struct proc *T > { > ... > pr_psargs = memstr(T->p_args->ar_args, ' ', T->p_args->ar_length); > ... > }; > > and execsnoop works properly. Any thoughts on this function? Have I missed > a better solution? A patch for FreeBSD is below. > > Thanks, > -Mark > > diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c > index 9c9b2a6..83c81e5 100644 > --- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c > +++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c > @@ -122,8 +122,9 @@ > #define DT_VERS_1_8_1 DT_VERSION_NUMBER(1, 8, 1) > #define DT_VERS_1_9 DT_VERSION_NUMBER(1, 9, 0) > #define DT_VERS_1_9_1 DT_VERSION_NUMBER(1, 9, 1) > -#define DT_VERS_LATEST DT_VERS_1_9_1 > -#define DT_VERS_STRING "Sun D 1.9.1" > +#define DT_VERS_1_9_2 DT_VERSION_NUMBER(1, 9, 2) > +#define DT_VERS_LATEST DT_VERS_1_9_2 > +#define DT_VERS_STRING "Sun D 1.9.2" > > const dt_version_t _dtrace_versions[] = { > DT_VERS_1_0, /* D API 1.0.0 (PSARC 2001/466) Solaris 10 FCS */ > @@ -145,6 +146,7 @@ const dt_version_t _dtrace_versions[] = { > DT_VERS_1_8_1, /* D API 1.8.1 */ > DT_VERS_1_9, /* D API 1.9 */ > DT_VERS_1_9_1, /* D API 1.9.1 */ > + DT_VERS_1_9_2, /* D API 1.9.2 */ > 0 > }; > > @@ -311,6 +313,8 @@ static const dt_ident_t _dtrace_globals[] = { > &dt_idops_func, "void(@)" }, > { "memref", DT_IDENT_FUNC, 0, DIF_SUBR_MEMREF, DT_ATTR_STABCMN, DT_VERS_1_1, > &dt_idops_func, "uintptr_t *(void *, size_t)" }, > +{ "memstr", DT_IDENT_FUNC, 0, DIF_SUBR_MEMSTR, DT_ATTR_STABCMN, DT_VERS_1_0, > + &dt_idops_func, "string(void *, char, size_t)" }, > { "min", DT_IDENT_AGGFUNC, 0, DTRACEAGG_MIN, DT_ATTR_STABCMN, DT_VERS_1_0, > &dt_idops_func, "void(@)" }, > { "mod", DT_IDENT_ACTFUNC, 0, DT_ACT_MOD, DT_ATTR_STABCMN, > diff --git a/cddl/lib/libdtrace/psinfo.d b/cddl/lib/libdtrace/psinfo.d > index 068e72e..b2a009a 100644 > --- a/cddl/lib/libdtrace/psinfo.d > +++ b/cddl/lib/libdtrace/psinfo.d > @@ -57,7 +57,7 @@ translator psinfo_t < struct proc *T > { > pr_gid = T->p_ucred->cr_rgid; > pr_egid = T->p_ucred->cr_groups[0]; > pr_addr = 0; > - pr_psargs = stringof(T->p_args->ar_args); > + pr_psargs = memstr(T->p_args->ar_args, ' ', T->p_args->ar_length); > pr_arglen = T->p_args->ar_length; > pr_jailid = T->p_ucred->cr_prison->pr_id; > }; > diff --git a/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c b/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c > index babc42c4..6d991fb 100644 > --- a/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c > +++ b/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c > @@ -4920,6 +4920,38 @@ inetout: regs[rd] = (uintptr_t)end + 1; > break; > } > > + case DIF_SUBR_MEMSTR: { > + char *str = (char *)mstate->dtms_scratch_ptr; > + uintptr_t mem = tupregs[0].dttk_value; > + char c = tupregs[1].dttk_value; > + size_t size = tupregs[2].dttk_value; > + uint8_t n; > + int i; > + > + regs[rd] = 0; > + > + if (size == 0) > + break; > + > + if (!dtrace_canload(mem, size - 1, mstate, vstate)) > + break; > + > + if (!DTRACE_INSCRATCH(mstate, size)) { > + DTRACE_CPUFLAG_SET(CPU_DTRACE_NOSCRATCH); > + break; > + } > + > + for (i = 0; i < size - 1; i++) { > + n = dtrace_load8(mem++); > + str[i] = (n == 0) ? c : n; > + } > + str[size - 1] = 0; > + > + regs[rd] = (uintptr_t)str; > + mstate->dtms_scratch_ptr += size; > + break; > + } > + > case DIF_SUBR_TYPEREF: { > uintptr_t size = 4 * sizeof(uintptr_t); > uintptr_t *typeref = (uintptr_t *) P2ROUNDUP(mstate->dtms_scratch_ptr, sizeof(uintptr_t)); > @@ -9102,6 +9134,7 @@ dtrace_difo_validate_helper(dtrace_difo_t *dp) > subr == DIF_SUBR_NTOHL || > subr == DIF_SUBR_NTOHLL || > subr == DIF_SUBR_MEMREF || > + subr == DIF_SUBR_MEMSTR || > subr == DIF_SUBR_TYPEREF) > break; > > diff --git a/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h b/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h > index 8728e30..295457c 100644 > --- a/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h > +++ b/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h > @@ -311,8 +311,9 @@ typedef enum dtrace_probespec { > #define DIF_SUBR_SX_SHARED_HELD 48 > #define DIF_SUBR_SX_EXCLUSIVE_HELD 49 > #define DIF_SUBR_SX_ISEXCLUSIVE 50 > +#define DIF_SUBR_MEMSTR 51 > > -#define DIF_SUBR_MAX 50 /* max subroutine value */ > +#define DIF_SUBR_MAX 51 /* max subroutine value */ > > typedef uint32_t dif_instr_t; > > > > ------------------------------------------- > dtrace-discuss > Archives: https://www.listbox.com/member/archive/184261/=now > RSS Feed: https://www.listbox.com/member/archive/rss/184261/21484527-a123725d > Modify Your Subscription: https://www.listbox.com/member/?member_id=21484527&id_secret=21484527-085b137e > Powered by Listbox: http://www.listbox.com From owner-freebsd-dtrace@FreeBSD.ORG Wed Sep 18 13:34:47 2013 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 589B7A82; Wed, 18 Sep 2013 13:34:47 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay01.ispgateway.de (smtprelay01.ispgateway.de [80.67.18.43]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 17D0B2B09; Wed, 18 Sep 2013 13:34:46 +0000 (UTC) Received: from [78.35.157.165] (helo=fabiankeil.de) by smtprelay01.ispgateway.de with esmtpsa (SSLv3:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1VMHnM-0005eH-7s; Wed, 18 Sep 2013 15:27:52 +0200 Date: Wed, 18 Sep 2013 15:19:12 +0200 From: Fabian Keil To: Mark Johnston Subject: Re: pr_psargs on FreeBSD Message-ID: <5e28d134.49bf1edd@fabiankeil.de> In-Reply-To: <20130915221622.GA2981@raichu> References: <20130915221622.GA2981@raichu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/1=JbcjTFsUTqjTHqYcnOBU1"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Cc: dtrace-discuss@lists.dtrace.org, freebsd-dtrace@freebsd.org X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Sep 2013 13:34:47 -0000 --Sig_/1=JbcjTFsUTqjTHqYcnOBU1 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Mark Johnston wrote: > It's kind of a strong-armed approach, but the problem's been around for a > while and I can't think of any clever solutions - I'd love to see an > alternative solution. >=20 > My patch against FreeBSD (pasted below) adds a function called memstr() > to DTrace. memstr() takes three arguments: addr, c, and len, and returns > a copy of addr of length len with all null-terminators replaced by c, > and with the last byte replaced by a null-terminator. In particular, > memstr() always returns a string of length len - 1, unless len =3D=3D 0, = in > which case we return the empty string. [...] > and execsnoop works properly. Any thoughts on this function? Have I missed > a better solution? A patch for FreeBSD is below. Looks good (and clever) to me and seems to work as expected. Thanks a lot. I briefly looked into this a couple of months ago and was under the impression that it would require a lot more code ... Fabian --Sig_/1=JbcjTFsUTqjTHqYcnOBU1 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iEYEARECAAYFAlI5qE8ACgkQBYqIVf93VJ2gjgCglcTYYrq47+tzbIkHxKb5r1wK n0cAniU/bnpvEzYZZfOks505a39Z/Gfy =x+E/ -----END PGP SIGNATURE----- --Sig_/1=JbcjTFsUTqjTHqYcnOBU1--