From owner-freebsd-stable@FreeBSD.ORG Wed Jan 4 23:52:10 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F327106566C; Wed, 4 Jan 2012 23:52:10 +0000 (UTC) (envelope-from matthew@phoronix.com) Received: from phx1.phoronix.com (173.192.77.202-static.reverse.softlayer.com [173.192.77.202]) by mx1.freebsd.org (Postfix) with ESMTP id 58A9D8FC13; Wed, 4 Jan 2012 23:52:10 +0000 (UTC) Received: from palm-64-28-152-131.palm.com ([64.28.152.131] helo=LT740055CZ0L1.palm1.palmone.com) by phx1.phoronix.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1Riacr-0006BB-Bb; Wed, 04 Jan 2012 17:52:09 -0600 Message-ID: <4F04E626.5040406@phoronix.com> Date: Wed, 04 Jan 2012 15:52:06 -0800 From: Matthew Tippett User-Agent: Mozilla/5.0 (X11; Linux i686; rv:10.0) Gecko/20111222 Thunderbird/10.0 MIME-Version: 1.0 To: Alexander Kabaev References: <20120104223158.911B11065678@hub.freebsd.org> <20120104184910.0d604240@kan.dyndns.org> In-Reply-To: <20120104184910.0d604240@kan.dyndns.org> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - phx1.phoronix.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - phoronix.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Adrian Chadd , FreeBSD Stable Mailing List , Joe Holden , Michael Larabel , Current FreeBSD , Arnaud Lacombe Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2012 23:52:10 -0000 Hmm... No sure what happened there again. What I sent (pulled from my "Sent" folder... === Thanks for the comment Arnaud. For comparative benchmarking on Phoronix.com , Michael invariable leaves it in the default configuration 'in the way the developers or vendor wanted it for production'. This is by rule. However, invariable the community or vendor for platforms that post poor scores on benchmark cry foul about using the default config. 'it should be tuned, no-one deploys an untuned system' or 'the system is configured for a different workload'. The response from us to this comes in two forms. 1) If it is the wrong workload for the platform, do a public post explaining and analysing the results. Highlighting the rationale for the concious reduction in performance (ie: journaling filesystems with barriers suffer in some write benchmarks for the sake of filesystem integrity. 2) If tuning can have a material impact on the results, post a tuning guide with step by step and rationale. Ie: educate the community and users. Michael and I have had many discussions with vendors and communities on this. In almost all cases, the vendor has either changed the default configuration or accepted the results as valid. As a service to the community or vendor that publishes the tuning guide, Michael is more than willing to redo a tuned vs untuned comparison. To date, the communities have never taken us up on that offer. In part, this affects Phoronix.com 's perception in the public, but that is more of a result of a one sided discussion by a party external to a particular community (with a healthy touch of journalisticly pumped compare & contrast). For the FreeBSD community, who else outside of the FreeBSD community actually runs public comparisons of FreeBSD against anything? Matthew === On 01/04/2012 03:49 PM, Alexander Kabaev wrote: > On Wed, 04 Jan 2012 14:31:55 -0800 > wrote: > >> Thanks for the comment Arnaud. For comparative benchmarking >> on [1]Phoronix.com, Michael inva configuration 'in the way the >> developers or production'. This is by rule. However, i poor >> scores on be 'it should be tuned, is configured for a diffe The >> response from us to this comes in two forms.&nb 1) If it is the >> wrong workload for the platform, do a public pos explaining and >> analysing the results. Highlighting the rationale fo r the >> concious reduction in performance (ie: journaling filesystems with >> ba filesystem integrity 2) If tuning can have a material impact >> on the results, post a t uning guide with step by step and >> rationale. Ie: educate the communit Michael and I have had many >> discussions with vendors an on this. In almost all cases, the >> vendor has either cha default configuration or accepted the results >> as valid. As guide, Micha comparison. To dat offer. In part, >> thi public, but that is more of a result of a one sided d party >> external to a particular community (with a healthy tou >> journalisticly pumped compare& contrast). For the FreeBSD >> community, who else outside of the FreeBSD community actually runs >> public c Matthew > Not really related to the discussion on hand, but the above about the > most unreadable email I am yet to read on the public mailing list. >