Date: Wed, 04 Jan 2012 14:31:55 -0800 From: <matthew@phoronix.com> To: "Arnaud Lacombe" <lacombar@gmail.com> Cc: Adrian Chadd <adrian@freebsd.org>, FreeBSD Stable Mailing List <freebsd-stable@freebsd.org>, Joe Holden <lists@rewt.org.uk>, Michael Larabel <Michael@phoronix.com>, Current FreeBSD <freebsd-current@freebsd.org>, freebsd-performance@freebsd.org, "O. Hartmann" <ohartman@zedat.fu-berlin.de>, Jeremy Chadwick <freebsd@jdc.parodius.com> Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server Message-ID: <20120104223158.911B11065678@hub.freebsd.org> In-Reply-To: <CACqU3MXz-vCt8Agkq=z7zyr7ptMgthRBsETxyaFQahX9X1uzPg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for the comment Arnaud. For comparative benchmarking on [1]Phoronix.com, Michael inva= riable leaves it in the default
configuration 'in the way the developers or= vendor wanted it for
production'. This is by rule.
However, i= nvariable the community or vendor for platforms that post
poor scores on be= nchmark cry foul about using the default config.
'it should be tuned,= no-one deploys an untuned system' or 'the system
is configured for a diffe= rent workload'.
The response from us to this comes in two forms. &nb= sp;
1) If it is the wrong workload for the platform, do a public pos= t
explaining and analysing the results. Highlighting the rationale fo r the concious reduction in performance (ie: journaling filesystems
with ba= rriers suffer in some write benchmarks for the sake of
filesystem integrity= .
2) If tuning can have a material impact on the results, post a t uning guide with step by step and rationale. Ie: educate the
communit= y and users.
Michael and I have had many discussions with vendors an= d communities
on this. In almost all cases, the vendor has either cha= nged the
default configuration or accepted the results as valid.
As = a service to the community or vendor that publishes the tuning
guide, Micha= el is more than willing to redo a tuned vs untuned
comparison. To dat= e, the communities have never taken us up on that
offer. In part, thi= s affects [2]Phoronix.com's perception in the
public, but that is more of a result of a one sided d= iscussion by a
party external to a particular community (with a healthy tou= ch of
journalisticly pumped compare & contrast). For the FreeBSD community, who else outside of the FreeBSD community actually runs
public c= omparisons of FreeBSD against anything?
Matthew
-- Sent from my HP Pre3
_________________________________________________________________
On Jan 4, 2012 1:58 PM, Arnaud Lacombe <lacombar@gmail.com&g= t;
wrote:
Hi,
On Fri, Dec 16, 2011 at 12:16 PM,= <matthew@phoronix.com> wrote:
> Thanks.
>
&= gt; My request for the person documenting the tunings also runs the
benchma= rk to
> ensure expected behaviour.
>
Why should you= have to tune anything ? Did you tune the Oracle
Server
install ? If = not, you should not have to tune the FreeBSD
install,
that wouldn't b= e fair. If you tune FreeBSD, you should tune the
Oracle
Server instal= l too. It is pretty easy to win at least 30% in
performance for certa= in workload by choosing the right kernel
configuration.
= - Arnaud
> The installation, execution and comparison agai= nst the benchmarks
in the
> article is fairly simple.
>=
> Note that some tuning may not be relevant or recommended (ie: s= ome
of the fs
> benchmarks are sensitive to barriers and other syn= chronous
operations). I'd
> recommend bowing out of a benchm= ark with a 'we're going to be
slower since
> the default configura= tion is this way for the following reason' if
this is
> the case.=
>
> Thanks 'someone'.
>
> Matthew
>
>
> Dec 16, 2011 8:46 AM, Adrian Chadd <a= drian@freebsd.org>
wrote:
>
> Can someone please write= up a nice, concise blog post somewhere
> outlining all of this?=
>
> Extra bonus points if it's a blog that is picked up = by
> blogs.freebsdish.org and/or some of the other BSD sites.
>
> Guys/girls/fuzzy things - this is 2011; people look at sh= iny
blog
> sites with graphs rather than mailing lists. Sorry, we = lost
that
> battle. :)
>
>
>
>= Adrian
> _______________________________________________
&g= t; freebsd-performance@freebsd.org mailing list
> http://lists.fre= ebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, se= nd any mail to
> "freebsd-performance-unsubscribe@freebsd.org"
<= br>
References
1. 3D"http://Phoronix.com"/
2. 3D"http://Phoronix.com"/
_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120104223158.911B11065678>