From owner-freebsd-current@FreeBSD.ORG Fri Dec 15 15:09:59 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7925116A417 for ; Fri, 15 Dec 2006 15:09:59 +0000 (UTC) (envelope-from shoesoft@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.FreeBSD.org (Postfix) with SMTP id A5A0143CA1 for ; Fri, 15 Dec 2006 15:08:16 +0000 (GMT) (envelope-from shoesoft@gmx.net) Received: (qmail invoked by alias); 15 Dec 2006 15:09:54 -0000 Received: from h081217095052.dyn.cm.kabsi.at (EHLO [192.168.0.1]) [81.217.95.52] by mail.gmx.net (mp001) with SMTP; 15 Dec 2006 16:09:54 +0100 X-Authenticated: #16703784 From: Stefan Ehmann To: "Poul-Henning Kamp" Date: Fri, 15 Dec 2006 16:09:52 +0100 User-Agent: KMail/1.9.4 References: <23331.1166189988@critter.freebsd.dk> In-Reply-To: <23331.1166189988@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200612151609.53750.shoesoft@gmx.net> X-Y-GMX-Trusted: 0 Cc: Peter Jeremy , freebsd-current@freebsd.org, Steve Kargl Subject: Re: Let's use gcc-4.2, not 4.1 -- OpenMP X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2006 15:09:59 -0000 On Friday 15 December 2006 14:39, Poul-Henning Kamp wrote: > In message <200612151250.10033.shoesoft@gmx.net>, Stefan Ehmann writes: > >Settings/Compiler | gcc-3.4 | gcc-4.1 | gcc-4.2 > >----------------------------+---------+---------+--------- > >-O2 | 13.1bn | 13.8bn | 13.5bn > >-O2 -funroll-loops | 9.6bn | 9.3bn | 9.2bn > >-O2 -march=athlon-xp -fun.. | 9.7bn | 10.6bn | 10.7bn > >-O3 | 11.5bn | 9.5bn | 9.6bn > >-O3 -funroll-loops | 8.4bn | 9.2bn | 9.4bn > >-O3 -march=athlon-xp -fun.. | 8.8bn | 10.6bn | 11.1bn > > I love benchmarks. > > It's great when people benchmark things. > > Unfortunately, that is not what you have done, because you have > not indicated what the standard deviation on your numbers are, > so they are totally worthless. I've done 3 runs on an otherwise pretty idle system with a maximum deviation of maybe 1 million instructions. So I figured that accurately calculating the standard deviation would overshoot the mark for this primitive test. IMHO the much weaker point in my benchmark is using a single program and only instruction count. What I wanted to show is whether gcc4 can still be worse than gcc34 in some cases. Sometimes performance counters can vary a lot (I've seen double the instructions on the p4 machine using papiex). So here are the results for the "-O3 -funroll-loops" row (using the output of 100 runs). Going on further seems pretty pointless to me. Using a 99.7 confidence interval, I get these results: -O3 -funroll-loops: gcc-3.4: 8362606323 +/- 440336 gcc-4.1: 9246505378 +/- 531302 gcc-4.2: 9401195544 +/- 784106