From owner-freebsd-current@FreeBSD.ORG Mon Dec 19 11:49:19 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A55B1065675; Mon, 19 Dec 2011 11:49:19 +0000 (UTC) (envelope-from sjg@evilcode.net) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id B249D8FC08; Mon, 19 Dec 2011 11:49:18 +0000 (UTC) Received: by qcse13 with SMTP id e13so4457735qcs.13 for ; Mon, 19 Dec 2011 03:49:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.229.106.230 with SMTP id y38mr5306405qco.83.1324295357936; Mon, 19 Dec 2011 03:49:17 -0800 (PST) Received: by 10.229.6.142 with HTTP; Mon, 19 Dec 2011 03:49:17 -0800 (PST) In-Reply-To: <6140271.20111219122721@serebryakov.spb.ru> References: <4EE1EAFE.3070408@m5p.com> <4EE2AE64.9060802@m5p.com> <4EE88343.2050302@m5p.com> <4EE933C6.4020209@zedat.fu-berlin.de> <20111215024249.GA13557@icarus.home.lan> <4EE9A2A0.80607@zedat.fu-berlin.de> <6140271.20111219122721@serebryakov.spb.ru> Date: Mon, 19 Dec 2011 04:49:17 -0700 Message-ID: From: "Samuel J. Greear" To: lev@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Mon, 19 Dec 2011 12:09:29 +0000 Cc: Adrian Chadd , FreeBSD Stable Mailing List , Current FreeBSD , freebsd-performance@freebsd.org, "O. Hartmann" , Jeremy Chadwick Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 11:49:19 -0000 2011/12/19 Lev Serebryakov : > Hello, Samuel. > You wrote 15 =D0=B4=D0=B5=D0=BA=D0=B0=D0=B1=D1=80=D1=8F 2011 =D0=B3., 16:= 32:47: > >> Other benchmarks in the Phoronix suite and their representations are >> similarly flawed, _ALL_ of these results should be ignored and no time >> should be wasted by any FreeBSD committer further evaluating this >> garbage. (Yes, I have been down this rabbit hole). > =C2=A0Here is one problem: we have choice from three items: > > (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD > > (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix > (communication with them, convincing, that they benchamrks are unfare > / meaningless, ets) > > (3) Lose [potential] userbase. > > =C2=A0You know, that these benchmarks are bad. I know. But potential (and > =C2=A0even some current!) user doesn't. And it seems, that these benchmar= ks > =C2=A0become popular over Internet. > > -- > // Black Lion AKA Lev Serebryakov > Here is where you completely derail the train, let me paste again what I said before. ... Take the first test as an example, Blogbench read. This doesn't raise any red flags, right? At least not until you realize that Blogbench isn't a read test, it's a read/write test. So what they have done here is run a read/write test and then thrown away the write results for both platforms and reported only the read results. If you dig down into the actual results, http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will see two Blogbench numbers, one for read and another for write. These were both taken from the same Blogbench run, so FreeBSD optimizes writes over reads, that's probably a good thing for your data but a bad thing when someone totally misrepresents benchmark results. ... FreeBSD actually does _BETTER_ (subjectively) in this test than the Linux system when you look at what is really going on. FreeBSD is favoring writes, which is _GOOD_. FreeBSD does not need to be fixed, the benchmarks need to be fixed to represent reality rather than throwing half of the results in the trash. To be quite frank, "fixing" FreeBSD to look good on this benchmark will make it a worse real-world OS. But you guys go ahead and foot-shoot over these ridiculous benchmarks all you want. Sam