From owner-freebsd-dtrace@FreeBSD.ORG Thu Jan 9 18:48:33 2014 Return-Path: Delivered-To: dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5C3486B for ; Thu, 9 Jan 2014 18:48:33 +0000 (UTC) Received: from nm31-vm6.bullet.mail.bf1.yahoo.com (nm31-vm6.bullet.mail.bf1.yahoo.com [72.30.239.14]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4E0E3125D for ; Thu, 9 Jan 2014 18:48:33 +0000 (UTC) Received: from [98.139.212.149] by nm31.bullet.mail.bf1.yahoo.com with NNFMP; 09 Jan 2014 18:45:13 -0000 Received: from [98.139.211.199] by tm6.bullet.mail.bf1.yahoo.com with NNFMP; 09 Jan 2014 18:45:13 -0000 Received: from [127.0.0.1] by smtp208.mail.bf1.yahoo.com with NNFMP; 09 Jan 2014 18:45:13 -0000 X-Yahoo-Newman-Id: 86002.73460.bm@smtp208.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: z5jdTcsVM1kdhFzfaTRZwPLYvErCyWhh4yd_K.4Gef6Fugj wAF90d6b7tS.Td2TYU1N9EaLKWs76UNdgvbbjvIOvHHFyKyMOrejVmNCYjGA t33QUOlYMiKYyAN7BKK.pBGTJvkNEBoJ2MTvSgBqK5gUc0PWg7uepMVpkYd6 1wCK8HLAIuTFNDq6C7oJs7.tToPZHHntXZ1BLDY32CvCTXLV4xFoGa1MPEF2 F_9JGb_kIQWcYLUBqsyi3owNKJjGFsGSB2PPqC4GXxpp.EYTk2CINDMSBJQz FTxSN1.UgcCwdcHuq08bc7B3Yvwaz3nftrDJhe9OeApwMm2v0Cdd5k8ct2xJ AB_rBc8H4gJBdrhbiiiWIB7_dOsUAVN_5e1cTb86pkNGY6WTkYVxiizZSrVY rVUqsRupzPuDRfj0vCA8h53yD8OroJArhq60HJvGHIdDoqF0.s8_KEcWNK6m GKhAe9hIGUd7zDwmDoy4Y0UoH3ECDAZeA5z_yTyg.Fz.._7kHFpNDdKaZYbA zBQQzzaKDhju3HlqgCWrVYvRrNtOqrN3AGwOWbjYJNOXpmna3J1vacrre471 qDOpfUwaib8v.ehaNYq.Hsbaz.42lenuN3CcMfD1nQGcqEwpzICsCi8NzJBo fQDeQV8U2geMgJDOBZVIR68oxvwvH_JsrUZEA8LyFM1Pbi4ailP_A5Xk2IsC J X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with plain [98.139.211.125]) by smtp208.mail.bf1.yahoo.com with SMTP; 09 Jan 2014 10:45:12 -0800 PST Message-ID: <52CEEE3B.7010400@FreeBSD.org> Date: Thu, 09 Jan 2014 13:45:15 -0500 From: Pedro Giffuni Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: dtrace@freebsd.org Subject: CTF vs dwarf4 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2014 18:48:33 -0000 Hello; Just thought I'd point out, for those not following toolchain@, that there are issues with newer compilers using dwarf4 by default[1]. It looks like instead of waiting for the elftoolchain project to catch up with the dwarf standard we could be using dwarf support from LLVM instead though [2]. Apparently the linux dtrace port is having similar issues. Another task for the Dtrace TODO list I guess, but perhaps the Illumos guys following the list may have some input. Regards, Pedro. [1] http://docs.freebsd.org/cgi/mid.cgi?29C2D69E-9EC8-418D-A333-FC1A8DA2133B [2] http://docs.freebsd.org/cgi/mid.cgi?CAPyFy2Bnpha=+49E+rL0Rd=TKtgOybGQAtOWE8m5sPCNiT5_Pw From owner-freebsd-dtrace@FreeBSD.ORG Thu Jan 9 19:38:04 2014 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02D66A54 for ; Thu, 9 Jan 2014 19:38:04 +0000 (UTC) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CA50D16AB for ; Thu, 9 Jan 2014 19:38:03 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id kl14so3742600pab.27 for ; Thu, 09 Jan 2014 11:37:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=+RdIaoheyGR8IdE/9ZvhtjCDSCNdoLQdXupa1ipSpDs=; b=lDziiai8CllwK9TGAngC3trZvz41C8lXWCGyIUmjkeep7h7XdRzLLeYL9sIPOzngIX KF3+OcLp/cVbmvpvakhW0KQ3nv/BfYMRpfg9pvGeOa3hrq0QtMsZ569sOkJ2Z9JuSPWJ 3mZvgT2xj5wwBY5MrSlj5P0FV9Fw6qFiSSiRa3npMxnhwmjgL969eTWDUvDfLinSFICD l34EEEnOa4Q0JC6NDn0fDLz4t6jcBMX/wC1skhEBEF5i1eNjm47FB8MGruG3myKdfwSb W+OoA63i6FCllvv2z8PQMri0390ujR7gBb4co+xgA0pWzcC+LajY1roorBswQ0NCf1RR wiow== X-Gm-Message-State: ALoCoQkh2nHBxjNMMVmZriwpYJyoZkchRWrwgJX7vyPZm3IYeaNqwbz1+5cEQReqDjzOzeGpc/lK X-Received: by 10.68.163.193 with SMTP id yk1mr5845071pbb.138.1389296277257; Thu, 09 Jan 2014 11:37:57 -0800 (PST) Received: from zanarkand.local ([38.104.194.102]) by mx.google.com with ESMTPSA id da3sm11867383pbc.30.2014.01.09.11.37.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 09 Jan 2014 11:37:55 -0800 (PST) Message-ID: <52CEFA91.3090301@joyent.com> Date: Thu, 09 Jan 2014 11:37:53 -0800 From: Robert Mustacchi User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-dtrace@freebsd.org Subject: Re: CTF vs dwarf4 References: <52CEEE3B.7010400@FreeBSD.org> In-Reply-To: <52CEEE3B.7010400@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2014 19:38:04 -0000 On 1/9/14 10:45 , Pedro Giffuni wrote: > Hello; > > Just thought I'd point out, for those not following toolchain@, that > there are issues with newer compilers using dwarf4 by default[1]. > > It looks like instead of waiting for the elftoolchain project to catch > up with the dwarf standard we could be using dwarf support from LLVM > instead though [2]. > > Apparently the linux dtrace port is having similar issues. Another task > for the Dtrace TODO list I guess, but perhaps the Illumos guys following > the list may have some input. As one of the illumos folk following the list, I can kind of describe what at least some of us would like to do with CTF. It's worth mentioning that we're still using gcc4.4 so we're not in a realm where DWARF4 is a concern for us yet. I'm not familiar with all the differences yet, so I'm not sure how useful the rest of this e-mail will be. Currently I've been retooling ctfconvert + ctfmerge to be actually implemented in terms of libctf to make it easier for other things to actually consume, such as more long term illumos ld(1) and ideally the compilers themselves. None of this code is finished or in illumos yet. That really means that I have a slightly more useful merge (and as a result diff) written, and convert is kind of a work in progress. Probably the simplest way forward that we can all leverage is writing a new ctfconvert in terms of DWARF4 that can exist side by side the old one. eg. if you have an older compiler with dwarf2 (or you're still stuck with stabs or some other language has its own exquisite format), that'll still work. illumos last updated libdwarf in 2011. If there's a newer libdwarf that has better DWARF4 support or we need to come up with something else, let's make sure not to duplicate effort. Robert From owner-freebsd-dtrace@FreeBSD.ORG Thu Jan 9 20:50:47 2014 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C6AD483 for ; Thu, 9 Jan 2014 20:50:47 +0000 (UTC) Received: from nm24-vm0.bullet.mail.bf1.yahoo.com (nm24-vm0.bullet.mail.bf1.yahoo.com [98.139.213.161]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B84701DAB for ; Thu, 9 Jan 2014 20:50:40 +0000 (UTC) Received: from [66.196.81.172] by nm24.bullet.mail.bf1.yahoo.com with NNFMP; 09 Jan 2014 20:44:53 -0000 Received: from [98.139.211.203] by tm18.bullet.mail.bf1.yahoo.com with NNFMP; 09 Jan 2014 20:44:53 -0000 Received: from [127.0.0.1] by smtp212.mail.bf1.yahoo.com with NNFMP; 09 Jan 2014 20:44:53 -0000 X-Yahoo-Newman-Id: 72630.22785.bm@smtp212.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: QkGNyRQVM1nNoPD6hVWCcEmejk2raqsVFXSCAy6bB.4_jlF ZHgHBim0qhU02Xvmi4jfC.ZNtohpvE08iZxU7cGo3D3Nxcju.M3UHip02Wgp HdQ8heEf1xHzmFXha0dwjvEfatp0kJPbl55m3TuoJpKAfA_ccmMDerBErlyN C5FtFJp5N.8SGrvFe.HQMYqnydUzZ7Rf_wbLM.MK0k_sWsm_LQ9L98s4AtE4 lNw11taOeloohiz_F6p0cTbRN5_bFQMhmPqe_yBebTqo622sivCsy.ID6aNQ Js4rBwOz1GXK1rGeA8kC8E8WBRKpcSj5noJ0ibN3yC8OKVlKXXY7O1Dzwctt HEnvkQ5ERrV1jPDU4nAEov6BgM..cZW5.TScVRgEASQwzcydan1PyxZd5bxZ d2_4LFZ7TyIdVZbm5L6rElVh8ufxQtmm9XAwxCZltXdCRCODkPz4MqD0q2tD 4Z58labQoBGNybioaB9Kzw_ACfV2zyfJkrCrbFjwXgOfgqq4vjcs.cBPVOIy ARtKEg0iGgoatwl_V_rBhP4ni3l0dkbtDy0WQCv3JFVF1CmzwfWMDILeWhch Ruhmn_4e9Kt9nNHq5BcacZeYmh3Rwi2RXp2qDp8duXYAqqE4zfuX7ePLlxkv FStGW7N9hwbpt.F9YDAFlhkVxZtcMwsxQSaokRgWY25adyA-- X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with plain [63.250.193.228]) by smtp212.mail.bf1.yahoo.com with SMTP; 09 Jan 2014 12:44:53 -0800 PST Message-ID: <52CF0A42.6090601@FreeBSD.org> Date: Thu, 09 Jan 2014 15:44:50 -0500 From: Pedro Giffuni User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Robert Mustacchi , freebsd-dtrace@freebsd.org Subject: Re: CTF vs dwarf4 References: <52CEEE3B.7010400@FreeBSD.org> <52CEFA91.3090301@joyent.com> In-Reply-To: <52CEFA91.3090301@joyent.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2014 20:50:47 -0000 Hi Robert; On 09.01.2014 14:37, Robert Mustacchi wrote: > On 1/9/14 10:45 , Pedro Giffuni wrote: >> Hello; >> >> Just thought I'd point out, for those not following toolchain@, that >> there are issues with newer compilers using dwarf4 by default[1]. >> >> It looks like instead of waiting for the elftoolchain project to catch >> up with the dwarf standard we could be using dwarf support from LLVM >> instead though [2]. >> >> Apparently the linux dtrace port is having similar issues. Another task >> for the Dtrace TODO list I guess, but perhaps the Illumos guys following >> the list may have some input. > As one of the illumos folk following the list, I can kind of describe > what at least some of us would like to do with CTF. It's worth > mentioning that we're still using gcc4.4 so we're not in a realm where > DWARF4 is a concern for us yet. I'm not familiar with all the > differences yet, so I'm not sure how useful the rest of this e-mail will be. Great to have you here! > Currently I've been retooling ctfconvert + ctfmerge to be actually > implemented in terms of libctf to make it easier for other things to > actually consume, such as more long term illumos ld(1) and ideally the > compilers themselves. None of this code is finished or in illumos yet. > That really means that I have a slightly more useful merge (and as a > result diff) written, and convert is kind of a work in progress. As a side note ... completely unrelated to the issue at hand. I don't know much about how DTrace uses dwarf, but I had some contact with the Oracle guys doing the linux port and I think they are doing something drastically different to illumos (and FreeBSD). They are avoiding the issue of getting DTrace in the kernel and they are using dwarf2 instead of the stabs-like format supported by CTF. Their stuff is in a GIT repository somewhere but I would have to dig deep in my private email for the finer details. They were likely to have problems with dwarf4 too. > Probably the simplest way forward that we can all leverage is writing a > new ctfconvert in terms of DWARF4 that can exist side by side the old > one. eg. if you have an older compiler with dwarf2 (or you're still > stuck with stabs or some other language has its own exquisite format), > that'll still work. illumos last updated libdwarf in 2011. If there's a > newer libdwarf that has better DWARF4 support or we need to come up with > something else, let's make sure not to duplicate effort. I recall Illumos is using SGI's libdwarf which we have avoiding using mainly because it's copyleft but I recall there were also some implementation details that made it undesirable. I haven't followed the SGI libdwarf but when I last tried to update the pre-packaged port I noticed it was becoming more difficult to port to non-linux (that may have been solved though). Some FreeBSD developers reimplemented libelf and libdwarf specifically to use it in DTrace: http://sourceforge.net/p/elftoolchain/wiki/libdwarf/ Their implementation is very clean and portable and it's basically what we have been using but we have to update it and I am not sure about their dwarf4 support. If illumos and freebsd can use the same libdwarf than that would be good. OTOH, with llvm having support for dwarf4 it would certainly be tempting to try to integrate what we can with the toolchain. I don't see illumos advancing much in the llvm direction though, so this option would be good for FreeBSD but probably not for illumos. FreeBSD 10+ has moved to llvm/clang already and, as noted in the previous posting, FreeBSD 11 is using the latest clang with dwarf4 so the issue became urgent. Pedro. From owner-freebsd-dtrace@FreeBSD.ORG Sat Jan 11 21:07:18 2014 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3216798C for ; Sat, 11 Jan 2014 21:07:18 +0000 (UTC) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 039571504 for ; Sat, 11 Jan 2014 21:07:17 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id q10so5913059pdj.25 for ; Sat, 11 Jan 2014 13:07:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=eTSMugk9OwYGkN1wlqVafAqyg2dofqEDZXtwu2rthVU=; b=J62yw5qUKzjbeRiMF/6Xo5Smpb7NpJnd+Z6TQ04ttyAlbYDqRt51kspZk3QMBZaXLb fwX0PeT1RIcyr5oaK26vG0bitTxjbCnuarjaG6+qo0YTZOPs3d70M8zBnnD1dC7j/+dD E1S00taIe6ZlyeLgj0+XwPuZWfmKN8hEusqow5U8rsL/dtRVKGgUi1w+aRB+PRrJksRJ KnmBSdeDKnxC6LuCVtuCu7vrvWdcr7eGXtIl2Myebp5R3EqD/jUD/4MHRcqGxVFRx5FU lrDdk3bZ4ekeHeZgehGpP6/GUo4+rIiH1ca3/TtD8/yFtyXaJUhJBI7JmIF/EHXaqQqb P3jQ== X-Gm-Message-State: ALoCoQmOeT94hICvM1vit+QvepSXsw0rns7XuwjIvg/9BhztcdPMHgnk8S8EPqJtCuScO0pNhY6V X-Received: by 10.66.197.164 with SMTP id iv4mr20508019pac.18.1389473989534; Sat, 11 Jan 2014 12:59:49 -0800 (PST) Received: from [10.138.0.3] (c-69-181-44-24.hsd1.ca.comcast.net. [69.181.44.24]) by mx.google.com with ESMTPSA id dq3sm26658609pbc.35.2014.01.11.12.59.47 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 11 Jan 2014 12:59:48 -0800 (PST) Message-ID: <52D1B0C2.7000100@joyent.com> Date: Sat, 11 Jan 2014 12:59:46 -0800 From: Robert Mustacchi User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Pedro Giffuni , freebsd-dtrace@freebsd.org Subject: Re: CTF vs dwarf4 References: <52CEEE3B.7010400@FreeBSD.org> <52CEFA91.3090301@joyent.com> <52CF0A42.6090601@FreeBSD.org> In-Reply-To: <52CF0A42.6090601@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jan 2014 21:07:18 -0000 Sorry for the delay in getting back to you. On 01/09/2014 12:44 PM, Pedro Giffuni wrote: >> Currently I've been retooling ctfconvert + ctfmerge to be actually >> implemented in terms of libctf to make it easier for other things to >> actually consume, such as more long term illumos ld(1) and ideally the >> compilers themselves. None of this code is finished or in illumos yet. >> That really means that I have a slightly more useful merge (and as a >> result diff) written, and convert is kind of a work in progress. > As a side note ... completely unrelated to the issue at hand. > > I don't know much about how DTrace uses dwarf, but I had some contact > with the Oracle guys doing the linux port and I think they are doing > something drastically different to illumos (and FreeBSD). They are > avoiding the issue of getting DTrace in the kernel and they are using > dwarf2 instead of the stabs-like format supported by CTF. Their stuff is > in a GIT repository somewhere but I would have to dig deep in my private > email for the finer details. They were likely to have problems with > dwarf4 too. Today DTrace uses this kind of information so that it knows how to refer to types in both userland (on illumos) and the kernel, and how if you do args[x], it knows the type and allows you to dereference it. >> Probably the simplest way forward that we can all leverage is writing a >> new ctfconvert in terms of DWARF4 that can exist side by side the old >> one. eg. if you have an older compiler with dwarf2 (or you're still >> stuck with stabs or some other language has its own exquisite format), >> that'll still work. illumos last updated libdwarf in 2011. If there's a >> newer libdwarf that has better DWARF4 support or we need to come up with >> something else, let's make sure not to duplicate effort. > > I recall Illumos is using SGI's libdwarf which we have avoiding using > mainly because it's copyleft but I recall there were also some > implementation details that made it undesirable. I haven't followed the > SGI libdwarf but when I last tried to update the pre-packaged port I > noticed it was becoming more difficult to port to non-linux (that may > have been solved though). > > Some FreeBSD developers reimplemented libelf and libdwarf specifically > to use it in DTrace: > http://sourceforge.net/p/elftoolchain/wiki/libdwarf/ > > Their implementation is very clean and portable and it's basically what > we have been using but we have to update it and I am not sure about > their dwarf4 support. > > If illumos and freebsd can use the same libdwarf than that would be > good. OTOH, with llvm having support for dwarf4 it would certainly be > tempting to try to integrate what we can with the toolchain. I don't see > illumos advancing much in the llvm direction though, so this option > would be good for FreeBSD but probably not for illumos. I don't see us moving to the llvm/clang world anytime soon unless someone else picks that up and starts teaching it some of the things we've taught gcc. But as we slowly move compilers, we're also probably going to stick to forcing it to emit dwarf2 for the core gate, unless we have a compelling reason to switch to dwarf4. Though despite that, for at least Joyent's equivalent of ports (netbsd pkgsrc), it's probably more worthwhile to get the dwarf4 support. As for libdwarf, the ctf tools are currently our only consumer of libdwarf. There isn't anything that compels us to stick with it, so we could probably move towards the same libdwarf that FreeBSD works on. I'll take a look at that in my current CTF workspace. So if FBSD is willing to continue down the libdwarf angle, I'd be happy to collaborate where I can with that. Robert From owner-freebsd-dtrace@FreeBSD.ORG Sat Jan 11 23:39:01 2014 Return-Path: Delivered-To: freebsd-dtrace@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 359DA2B9 for ; Sat, 11 Jan 2014 23:39:01 +0000 (UTC) Received: from nm21.bullet.mail.bf1.yahoo.com (nm21.bullet.mail.bf1.yahoo.com [98.139.212.180]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D366F1FD9 for ; Sat, 11 Jan 2014 23:39:00 +0000 (UTC) Received: from [98.139.212.150] by nm21.bullet.mail.bf1.yahoo.com with NNFMP; 11 Jan 2014 23:38:59 -0000 Received: from [68.142.230.76] by tm7.bullet.mail.bf1.yahoo.com with NNFMP; 11 Jan 2014 23:38:59 -0000 Received: from [127.0.0.1] by smtp233.mail.bf1.yahoo.com with NNFMP; 11 Jan 2014 23:38:59 -0000 X-Yahoo-Newman-Id: 332131.42448.bm@smtp233.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: QW6ucTwVM1lvz1P8ddAUc80uxP2r0d8rCJTvLMBaCjDxSrY mx.qMpaJE8UcT1Cz2PbteqQhk7_jRyGUm_53y.tUoibniImwQtRRwvCAG5fm Vh7.uAeCGSadot5x.RRPaRDfevw4QdISv3eVFDG8n4xm1TXHBVmupxlacbf_ 2Su4pvdwB0cdPU9APYn1AMubGqoQ7rflXyIsIJk4b1_O9vN6DEu_1qydvDOU acp7vQA5zGVUR0NBJKqdCLiibw1spc6jhvHRq7zxVcQ2mZdFcdzyqCIoWdMJ Djh4iR5fLVTwoJQ2FCsdzZk16xlziezmESmVhXs8.itgsNAG7.wBaXBO4qm2 z_UvmkAbGv.I9YgdbAPNnirSupVZQJ500po8y0M4QNxA9S6JFbn7Cpg2rXZZ 5i8Fy38AFFMbrMU_KPYGmMYRFCW32JGQ_pbjHrzmUiX.evRm5cwG7zRBnT6W tugvbrsQf8gSPTGbk0cZW_B4dhS_JIGEjhPwTufdVn8PmjSpfRsxBTam.RZ_ Nu2F7Igo..5QyNmecV089O89lju7xQa9PCS8yHcwKRuVAjHs2Llfz6QFBpD9 Y7T9ayad1u0Huu7du54CDyPtW1C2sS1PFn.KcNDcrZnZhwoHOHobaPliA.uT JpKs4xeqsXKJmnNL.jX6bsLZY85FxJtqqUk2JZccxj8yzcg-- X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with plain [63.250.193.228]) by smtp233.mail.bf1.yahoo.com with SMTP; 11 Jan 2014 23:38:59 +0000 UTC Message-ID: <52D1D60F.5000500@FreeBSD.org> Date: Sat, 11 Jan 2014 18:38:55 -0500 From: Pedro Giffuni User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Robert Mustacchi , freebsd-dtrace@freebsd.org Subject: Re: CTF vs dwarf4 References: <52CEEE3B.7010400@FreeBSD.org> <52CEFA91.3090301@joyent.com> <52CF0A42.6090601@FreeBSD.org> <52D1B0C2.7000100@joyent.com> In-Reply-To: <52D1B0C2.7000100@joyent.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jan 2014 23:39:01 -0000 Hi Robert; On 11.01.2014 15:59, Robert Mustacchi wrote: > Sorry for the delay in getting back to you. Don't worry, I am doing other stuff myself. > On 01/09/2014 12:44 PM, Pedro Giffuni wrote: >>> Currently I've been retooling ctfconvert + ctfmerge to be actually >>> implemented in terms of libctf to make it easier for other things to >>> actually consume, such as more long term illumos ld(1) and ideally the >>> compilers themselves. None of this code is finished or in illumos yet. >>> That really means that I have a slightly more useful merge (and as a >>> result diff) written, and convert is kind of a work in progress. >> As a side note ... completely unrelated to the issue at hand. >> >> I don't know much about how DTrace uses dwarf, but I had some contact >> with the Oracle guys doing the linux port and I think they are doing >> something drastically different to illumos (and FreeBSD). They are >> avoiding the issue of getting DTrace in the kernel and they are using >> dwarf2 instead of the stabs-like format supported by CTF. Their stuff is >> in a GIT repository somewhere but I would have to dig deep in my private >> email for the finer details. They were likely to have problems with >> dwarf4 too. > Today DTrace uses this kind of information so that it knows how to refer > to types in both userland (on illumos) and the kernel, and how if you do > args[x], it knows the type and allows you to dereference it. > >>> Probably the simplest way forward that we can all leverage is writing a >>> new ctfconvert in terms of DWARF4 that can exist side by side the old >>> one. eg. if you have an older compiler with dwarf2 (or you're still >>> stuck with stabs or some other language has its own exquisite format), >>> that'll still work. illumos last updated libdwarf in 2011. If there's a >>> newer libdwarf that has better DWARF4 support or we need to come up with >>> something else, let's make sure not to duplicate effort. >> I recall Illumos is using SGI's libdwarf which we have avoiding using >> mainly because it's copyleft but I recall there were also some >> implementation details that made it undesirable. I haven't followed the >> SGI libdwarf but when I last tried to update the pre-packaged port I >> noticed it was becoming more difficult to port to non-linux (that may >> have been solved though). >> >> Some FreeBSD developers reimplemented libelf and libdwarf specifically >> to use it in DTrace: >> http://sourceforge.net/p/elftoolchain/wiki/libdwarf/ >> >> Their implementation is very clean and portable and it's basically what >> we have been using but we have to update it and I am not sure about >> their dwarf4 support. >> >> If illumos and freebsd can use the same libdwarf than that would be >> good. OTOH, with llvm having support for dwarf4 it would certainly be >> tempting to try to integrate what we can with the toolchain. I don't see >> illumos advancing much in the llvm direction though, so this option >> would be good for FreeBSD but probably not for illumos. > I don't see us moving to the llvm/clang world anytime soon unless > someone else picks that up and starts teaching it some of the things > we've taught gcc. I read somewhere that Oracle had been sponsoring someone to work on the Solaris clang support, so I couldn't call off the chances. I looked at it briefly (I am not really working on this atm), and one small inconvenience is that the llvm stuff is C++. > But as we slowly move compilers, we're also probably > going to stick to forcing it to emit dwarf2 for the core gate, unless we > have a compelling reason to switch to dwarf4. Though despite that, for > at least Joyent's equivalent of ports (netbsd pkgsrc), it's probably > more worthwhile to get the dwarf4 support. I think it's mostly unavoidable. You just can't keep using an old unsupported compiler for too long. > As for libdwarf, the ctf tools are currently our only consumer of > libdwarf. There isn't anything that compels us to stick with it, so we > could probably move towards the same libdwarf that FreeBSD works on. > I'll take a look at that in my current CTF workspace. So if FBSD is > willing to continue down the libdwarf angle, I'd be happy to collaborate > where I can with that. Luckily both the SGI and FreeBSD/elftoolchain libdwarf use the same API. I haven't had much luck finding out about the dwarf4 support in it but I think it looks like the best option. Pedro.