Date: Fri, 15 Dec 2006 16:09:52 +0100 From: Stefan Ehmann <shoesoft@gmx.net> To: "Poul-Henning Kamp" <phk@phk.freebsd.dk> Cc: Peter Jeremy <peterjeremy@optushome.com.au>, freebsd-current@freebsd.org, Steve Kargl <sgk@troutmask.apl.washington.edu> Subject: Re: Let's use gcc-4.2, not 4.1 -- OpenMP Message-ID: <200612151609.53750.shoesoft@gmx.net> In-Reply-To: <23331.1166189988@critter.freebsd.dk> References: <23331.1166189988@critter.freebsd.dk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday 15 December 2006 14:39, Poul-Henning Kamp wrote: > In message <200612151250.10033.shoesoft@gmx.net>, Stefan Ehmann writes: > >Settings/Compiler | gcc-3.4 | gcc-4.1 | gcc-4.2 > >----------------------------+---------+---------+--------- > >-O2 | 13.1bn | 13.8bn | 13.5bn > >-O2 -funroll-loops | 9.6bn | 9.3bn | 9.2bn > >-O2 -march=athlon-xp -fun.. | 9.7bn | 10.6bn | 10.7bn > >-O3 | 11.5bn | 9.5bn | 9.6bn > >-O3 -funroll-loops | 8.4bn | 9.2bn | 9.4bn > >-O3 -march=athlon-xp -fun.. | 8.8bn | 10.6bn | 11.1bn > > I love benchmarks. > > It's great when people benchmark things. > > Unfortunately, that is not what you have done, because you have > not indicated what the standard deviation on your numbers are, > so they are totally worthless. I've done 3 runs on an otherwise pretty idle system with a maximum deviation of maybe 1 million instructions. So I figured that accurately calculating the standard deviation would overshoot the mark for this primitive test. IMHO the much weaker point in my benchmark is using a single program and only instruction count. What I wanted to show is whether gcc4 can still be worse than gcc34 in some cases. Sometimes performance counters can vary a lot (I've seen double the instructions on the p4 machine using papiex). So here are the results for the "-O3 -funroll-loops" row (using the output of 100 runs). Going on further seems pretty pointless to me. Using a 99.7 confidence interval, I get these results: -O3 -funroll-loops: gcc-3.4: 8362606323 +/- 440336 gcc-4.1: 9246505378 +/- 531302 gcc-4.2: 9401195544 +/- 784106
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200612151609.53750.shoesoft>