From owner-freebsd-current@FreeBSD.ORG Sat Jan 15 19:51:49 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 175C1106566B; Sat, 15 Jan 2011 19:51:49 +0000 (UTC) (envelope-from fbsdcurrent@beasties.demon.nl) Received: from smtp-vbr13.xs4all.nl (smtp-vbr13.xs4all.nl [194.109.24.33]) by mx1.freebsd.org (Postfix) with ESMTP id 730D78FC0A; Sat, 15 Jan 2011 19:51:48 +0000 (UTC) Received: from merom.hotsoft.nl (beasties.demon.nl [82.161.3.114]) by smtp-vbr13.xs4all.nl (8.13.8/8.13.8) with ESMTP id p0FJcaVj070616; Sat, 15 Jan 2011 20:38:36 +0100 (CET) (envelope-from fbsdcurrent@beasties.demon.nl) Message-ID: <4D31F7BC.7000606@beasties.demon.nl> Date: Sat, 15 Jan 2011 20:38:36 +0100 From: Hans Ottevanger User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101212 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <20110113171139.GA32250@troutmask.apl.washington.edu> <20110114040549.GA36430@troutmask.apl.washington.edu> <1D4725BB-0A0A-44FB-9FF1-23969C469AA6@freebsd.org> <20110115061023.GA41547@troutmask.apl.washington.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by XS4ALL Virus Scanner Cc: George Neville-Neil , Ryan Stone , Steve Kargl , Garrett Cooper Subject: Re: Profiling code execution on amd64? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Jan 2011 19:51:49 -0000 On 01/15/11 07:15, Garrett Cooper wrote: > On Fri, Jan 14, 2011 at 10:10 PM, Steve Kargl > wrote: >> On Fri, Jan 14, 2011 at 03:40:46PM -0500, George Neville-Neil wrote: >>> >>> On Jan 13, 2011, at 23:05 , Steve Kargl wrote: >>> >>>> On Thu, Jan 13, 2011 at 10:08:30PM -0500, Ryan Stone wrote: >>>>> I would suggest using hwpmc for profiling: >>>>> >>>>> # kldload hwpmc >>>>> # pmcstat -S unhalted-cycles -O /tmp/samples.out ../penetration >>>>> # pmcstat -R /tmp/samples.out -G /tmp/penetration.txt >>>>> >>>>> >>>>> You can also get pmcstat to generate gprof-compatible output with -g, >>>>> but I never use the mode so I'm really not sure what it gives you. I >>>>> think that you have to run gprof on the output or something, but don't >>>>> hold me to that. >>>> >>>> >>>> Thanks. I'll give it a try, but my initial attempt seems to >>>> indicate that one needs to be root to use hwpmc. >>>> >>>> laptop:kargl[210] pmcstat -S unhalted-cycles -O /tmp/samples.out ../penetration >>>> pmcstat: ERROR: Cannot allocate system-mode pmc with specification >>>> "unhalted-cycles": Operation not permitted >>>> >>> >>> You only need to be root to profile the kernel or someone else's process. >>> >>> This tutorial might help: >>> >>> www.dcbsdcon.org/speakers/slides/neville-neil_dcbsdcon2009.pdf >>> >> >> Thanks. I'll look at the tutorial. Meanwhile, should gprof >> be removed from the base system because it appears broken? > > Instead of just removing things, why not determine why things are > broken and try to fix them? > -Garrett > gprof is not as badly broken as it seems 8-) I have encountered the issue described by the original poster regularly over the last few years, using different versions of FreeBSD. The total time reported by gprof is often way shorter than the time reported by time. I have checked that the problem goes back to at least 7.3-RELEASE, and it occurs on both amd64 and i386. I found that this problem is caused by neither the profile start-up code nor gprof knowing about dynamic linking. The start-up code calls profil() with etext as the upper limit of the address range to be profiled. This implies that all dynamically linked code will not be profiled, which explains the lost ticks in gprof. A solution might be to teach both the profiling start-up code and gprof about dynamic linking, but I guess that is not that trivial ... A good work-around is linking statically (-static). Also be aware that with dynamic linking and specifying -pg, a profiled (and static!) version is used only for the C library. So if e.g. the math library is needed -lm_p has to be specified explicitly instead of just -lm, or ticks will be lost. So specifying standard libraries with _p appended might already solve most of the problems. I also remember (and just verified) that in the "good old days" (FreeBSD 2.2.8) static linking took place automatically when profiling was requested, so the issues at hand might not even be new 8-) Kind regards, Hans Ottevanger PS. Thanks for the link to your tutorial, George. I am very interested in hwpmc, and since documentation seems to be somewhat sparse, this really helps.