Date: Fri, 11 Mar 2011 16:46:18 +0000 From: "Poul-Henning Kamp" <phk@phk.freebsd.dk> To: Martin Matuska <mm@FreeBSD.org> Cc: freebsd-performance@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: FreeBSD Compiler Benchmark: gcc-base vs. gcc-ports vs. clang Message-ID: <98496.1299861978@critter.freebsd.dk> In-Reply-To: Your message of "Fri, 11 Mar 2011 16:42:04 %2B0100." <4D7A42CC.8020807@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <4D7A42CC.8020807@FreeBSD.org>, Martin Matuska writes: >But what I can say, e.g. for the Intel Atom processor, if there are >performance gains in all but one test (that falls 2% behind), generic >perl code (the routines benchmarked) on this processor is very likely to >run faster with that setup. No, actually you cannot say that, unless you run all the tests at least three times for each compiler(+flag), calculate the average and standard deviation of all the tests, and see which, if any of the results are statistically significant. Until you do that, you numbers are meaningless, because we have no idea what the signal/noise ratio is. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?98496.1299861978>