Date: Sun, 13 Mar 2011 16:29:57 -0400 From: Mehmet Erol Sanliturk <m.e.sanliturk@gmail.com> To: Jakub Lach <jakub_lach@mailplus.pl> Cc: freebsd-current@freebsd.org Subject: Re: FreeBSD Compiler Benchmark: gcc-base vs. gcc-ports vs. clang Message-ID: <AANLkTiniz2zNjZrtCBxZkPhqpG7gdQYYn3LoWAraWyvU@mail.gmail.com> In-Reply-To: <31138978.post@talk.nabble.com> References: <4D7943B1.1030604@FreeBSD.org> <90325.1299852096@critter.freebsd.dk> <4D7A42CC.8020807@FreeBSD.org> <98496.1299861978@critter.freebsd.dk> <4D7B44AF.7040406@FreeBSD.org> <60071.1299936937@critter.freebsd.dk> <AANLkTinOrNfq5FBOPkXcExjN=mzZCKazxeG8BMJNFVer@mail.gmail.com> <31138978.post@talk.nabble.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Mar 13, 2011 at 3:19 PM, Jakub Lach <jakub_lach@mailplus.pl> wrote:
>
>
> Vin=C3=ADcius Zavam wrote:
> >
> >
> > i'm still curious about things like CPUTYPE=3D and -march=3D configured=
as
> > native, gentlemen.
> > is it the "golden egg" to use with our system or not? why "natives"
> > aren't in the benchs?
> >
> > /me feels confused.
> >
> >
> > --
> > Vin=C3=ADcius Zavam
> > profiles.google.com/egypcio
> >
>
> Apparently -march=3Dnative would equal -march=3Dcore2
> with 65nm generation Core2s, this is not the case with
> Penryns.. But there are none in the test?
>
> However, I agree that testing with -march=3Dnative
> would be simpler and more straightforward.
>
> regards,
> - Jakub Lach
> --
>
The compilers Clang and GCC may also be compared with the following design
because ( on the same computer , multiple parameters are measured ) :
Clang Version x
------------------------------------
Repeated Measures --->
p(1) p(2) p(3) p(4) and other parameter=
s
up to p(m)
----- ----- ----- -----
Computer 1 value value value value .....
Computer 2 value value value value .....
.
.
.
Computer n value value value value .....
GCC Version x
------------------------------------
Repeated Measures --->
p(1) p(2) p(3) p(4) and other
parameters up to p(m)
----- ----- ----- -----
Computer n+1 value value value value .....
Computer n+2 value value value value .....
.
.
.
Computer n+n value value value value .....
For each compiler the same number of computers are used ( This is called
balanced design ) .
Evaluation of unbalanced designs may not be available in used statistical
packages , and
theoretically , they are NOT very good .
Here factors are :
(1) Compilers ( CLang , GCC )
(2) Measured parameters as Compilation Parameters
such as
( No optimization )
( Optimization Level 1 )
( Optimization Level 2 )
( Processor Kind 1 )
( Processor Kind 2 ) and
( Code Generation Kind 1 )
( Code Generation Kind 2 ) and/or
any number of other parameters
Number of computers as n should be greater than Number of parameters as m .
Subjects are the computers which no one of them is equal to the other .
Measured parameters are also called treatments .
In statistical analysis packages and
books on experimental designs
this design is called
two-factor experiment with repeated measures on one factor .
Also the other names may be used :
Repeated measures design , or
within-subjects factorial design , or
multifactor experiments having repeated measures on the same elements
Inclusion of two GCC versions into the above table as another compiler may
NOT be very well ,
because GCC compilers are likely VERY CORRELATED with each other ( because
they are using
the same code generator perhaps some patches applied to distinguish the
versions ).
To obtain an idea about correlation strength of the GCC compilers ,
CANONICAL correlation analysis may be used when there are multiple
parameters
( do NOT use two-by-two correlation coefficients when there are more than
two parameters ) , or ,
simple correlation when there are only two parameters ( one for each
compiler ) .
Design is as follows :
GCC Version x GCC version y
--------------------- ---------------------
p(1) P(2) ... p(k) p(1) (p2) ... p(k)
Computer 1 v v v v v v
computer 2 v v v v v v
.
.
.
Computer n v v v v v v
where
p(1) , p(2) , ... , p(k) are the measured parameters ,
k : Number of parameters for each block individually
( There may be different parameter sets , but
for our purposes equivalent parameter sets are required )
n : Number of observations , where each computer should be different
from
the others .
v : Value measured for a parameter
When there is a significant CANONICAL correlation between two compiler
related
values blocks :
(i) it is NOT possible to include the two compilers in the above
repeated measures design because of high collinearity .
(2) Selection of BEST compiler is possible because two compilers are very
similar
( there are no difference between them other than performance level ) =
.
When the CANONICAL correlation is NOT significant ,
the other GCC compiler may be included
as a third compiler into the repeated measures design .
http://en.wikipedia.org/wiki/Repeated_measures_design
http://en.wikipedia.org/wiki/Category:Experimental_design
http://en.wikipedia.org/wiki/Canonical_correlation
http://en.wikipedia.org/wiki/Category:Multivariate_statistics
Thank you very much .
Mehmet Erol Sanliturk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTiniz2zNjZrtCBxZkPhqpG7gdQYYn3LoWAraWyvU>