From owner-freebsd-current@FreeBSD.ORG Sun Mar 13 20:29:59 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77ABF1065672 for ; Sun, 13 Mar 2011 20:29:59 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 248EF8FC0A for ; Sun, 13 Mar 2011 20:29:58 +0000 (UTC) Received: by vxc34 with SMTP id 34so4394771vxc.13 for ; Sun, 13 Mar 2011 13:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=qzJvqk6KF04OcKiIkexVXUarJVrb/1bwzuXZnJN+7FI=; b=sf+ja0SmkxYtAm43mTBdprpp2xaSeR4JXX4wQmT7w9dYtFeRZ4R6L++eDaTT5sUWWC MTJc8pmSTydXC3MIL9RkXh7zwI301mlzbRG12Dp8LFn4YHWf7zUdI7cLq/5gXYe1OBUA 61+JNjSithllYB/P7+RylaSBI1csU2t8T9/yg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=JqPvKUgpzTME+CSbFsohNikHwPXSc6398TGzRuKqnCDS8zktVO8zT2nvYpkObPdsj7 k+8pU+Va74BCjD8+7NLXf+g+2MswGholpFSen9AAh4M7nyo97ygi0Z22kn5JS6PCipR3 hdUo5SWKASWagPuenXgcAZ7kXIy5OAKmugpw0= MIME-Version: 1.0 Received: by 10.52.94.167 with SMTP id dd7mr6720409vdb.206.1300048197817; Sun, 13 Mar 2011 13:29:57 -0700 (PDT) Received: by 10.52.169.165 with HTTP; Sun, 13 Mar 2011 13:29:57 -0700 (PDT) In-Reply-To: <31138978.post@talk.nabble.com> References: <4D7943B1.1030604@FreeBSD.org> <90325.1299852096@critter.freebsd.dk> <4D7A42CC.8020807@FreeBSD.org> <98496.1299861978@critter.freebsd.dk> <4D7B44AF.7040406@FreeBSD.org> <60071.1299936937@critter.freebsd.dk> <31138978.post@talk.nabble.com> Date: Sun, 13 Mar 2011 16:29:57 -0400 Message-ID: From: Mehmet Erol Sanliturk To: Jakub Lach Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-current@freebsd.org Subject: Re: FreeBSD Compiler Benchmark: gcc-base vs. gcc-ports vs. clang X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Mar 2011 20:29:59 -0000 On Sun, Mar 13, 2011 at 3:19 PM, Jakub Lach wrote: > > > Vin=C3=ADcius Zavam wrote: > > > > > > i'm still curious about things like CPUTYPE=3D and -march=3D configured= as > > native, gentlemen. > > is it the "golden egg" to use with our system or not? why "natives" > > aren't in the benchs? > > > > /me feels confused. > > > > > > -- > > Vin=C3=ADcius Zavam > > profiles.google.com/egypcio > > > > Apparently -march=3Dnative would equal -march=3Dcore2 > with 65nm generation Core2s, this is not the case with > Penryns.. But there are none in the test? > > However, I agree that testing with -march=3Dnative > would be simpler and more straightforward. > > regards, > - Jakub Lach > -- > The compilers Clang and GCC may also be compared with the following design because ( on the same computer , multiple parameters are measured ) : Clang Version x ------------------------------------ Repeated Measures ---> p(1) p(2) p(3) p(4) and other parameter= s up to p(m) ----- ----- ----- ----- Computer 1 value value value value ..... Computer 2 value value value value ..... . . . Computer n value value value value ..... GCC Version x ------------------------------------ Repeated Measures ---> p(1) p(2) p(3) p(4) and other parameters up to p(m) ----- ----- ----- ----- Computer n+1 value value value value ..... Computer n+2 value value value value ..... . . . Computer n+n value value value value ..... For each compiler the same number of computers are used ( This is called balanced design ) . Evaluation of unbalanced designs may not be available in used statistical packages , and theoretically , they are NOT very good . Here factors are : (1) Compilers ( CLang , GCC ) (2) Measured parameters as Compilation Parameters such as ( No optimization ) ( Optimization Level 1 ) ( Optimization Level 2 ) ( Processor Kind 1 ) ( Processor Kind 2 ) and ( Code Generation Kind 1 ) ( Code Generation Kind 2 ) and/or any number of other parameters Number of computers as n should be greater than Number of parameters as m . Subjects are the computers which no one of them is equal to the other . Measured parameters are also called treatments . In statistical analysis packages and books on experimental designs this design is called two-factor experiment with repeated measures on one factor . Also the other names may be used : Repeated measures design , or within-subjects factorial design , or multifactor experiments having repeated measures on the same elements Inclusion of two GCC versions into the above table as another compiler may NOT be very well , because GCC compilers are likely VERY CORRELATED with each other ( because they are using the same code generator perhaps some patches applied to distinguish the versions ). To obtain an idea about correlation strength of the GCC compilers , CANONICAL correlation analysis may be used when there are multiple parameters ( do NOT use two-by-two correlation coefficients when there are more than two parameters ) , or , simple correlation when there are only two parameters ( one for each compiler ) . Design is as follows : GCC Version x GCC version y --------------------- --------------------- p(1) P(2) ... p(k) p(1) (p2) ... p(k) Computer 1 v v v v v v computer 2 v v v v v v . . . Computer n v v v v v v where p(1) , p(2) , ... , p(k) are the measured parameters , k : Number of parameters for each block individually ( There may be different parameter sets , but for our purposes equivalent parameter sets are required ) n : Number of observations , where each computer should be different from the others . v : Value measured for a parameter When there is a significant CANONICAL correlation between two compiler related values blocks : (i) it is NOT possible to include the two compilers in the above repeated measures design because of high collinearity . (2) Selection of BEST compiler is possible because two compilers are very similar ( there are no difference between them other than performance level ) = . When the CANONICAL correlation is NOT significant , the other GCC compiler may be included as a third compiler into the repeated measures design . http://en.wikipedia.org/wiki/Repeated_measures_design http://en.wikipedia.org/wiki/Category:Experimental_design http://en.wikipedia.org/wiki/Canonical_correlation http://en.wikipedia.org/wiki/Category:Multivariate_statistics Thank you very much . Mehmet Erol Sanliturk