Date: Wed, 04 Jan 2012 14:31:55 -0800 From: <matthew@phoronix.com> To: "Arnaud Lacombe" <lacombar@gmail.com> Cc: Adrian Chadd <adrian@freebsd.org>, FreeBSD Stable Mailing List <freebsd-stable@freebsd.org>, Current FreeBSD <freebsd-current@freebsd.org>, Joe Holden <lists@rewt.org.uk>, Michael Larabel <Michael@phoronix.com>, Stefan Esser <se@freebsd.org>, freebsd-performance@freebsd.org, "O. Hartmann" <ohartman@zedat.fu-berlin.de>, Jeremy Chadwick <freebsd@jdc.parodius.com> Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server Message-ID: <20120104223158.911B11065678@hub.freebsd.org> In-Reply-To: <CACqU3MXz-vCt8Agkq=z7zyr7ptMgthRBsETxyaFQahX9X1uzPg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for the comment Arnaud. For comparative benchmarking on =
[1]Phoronix.com, Michael inva= riable leaves it in the default
configuration 'in the way the developers or= vendor wanted it for
production'. This is by rule.
However, i= nvariable the community or vendor for platforms that post
poor scores on be= nchmark cry foul about using the default config.
'it should be tuned,= no-one deploys an untuned system' or 'the system
is configured for a diffe= rent workload'.
The response from us to this comes in two forms. &nb= sp;
1) If it is the wrong workload for the platform, do a public pos= t
explaining and analysing the results. Highlighting the rationale fo=
r the concious reduction in performance (ie: journaling filesystems
with ba= rriers suffer in some write benchmarks for the sake of
filesystem integrity= =2E
2) If tuning can have a material impact on the results, post a t=
uning guide with step by step and rationale. Ie: educate the
communit= y and users.
Michael and I have had many discussions with vendors an= d communities
on this. In almost all cases, the vendor has either cha= nged the
default configuration or accepted the results as valid.
As = a service to the community or vendor that publishes the tuning
guide, Micha= el is more than willing to redo a tuned vs untuned
comparison. To dat= e, the communities have never taken us up on that
offer. In part, thi= s affects [2]Phoronix.com's perception in the
public, but that is more of a result of a one sided d= iscussion by a
party external to a particular community (with a healthy tou= ch of
journalisticly pumped compare & contrast). For the FreeBSD =
community, who else outside of the FreeBSD community actually runs
public c= omparisons of FreeBSD against anything?
Matthew
-- Sent from my HP Pre3
_________________________________________________________________
On Jan 4, 2012 1:58 PM, Arnaud Lacombe <lacombar@gmail.com&g= t;
wrote:
Hi,=0D
=0D
On Fri, Dec 16, 2011 at 12:16 PM,= <matthew@phoronix.com> wrote:=0D
> Thanks.=0D
>=0D
&= gt; My request for the person documenting the tunings also runs the
benchma= rk to=0D
> ensure expected behaviour.=0D
>=0D
Why should you= have to tune anything ? Did you tune the Oracle
Server=0D
install ? If = not, you should not have to tune the FreeBSD
install,=0D
that wouldn't b= e fair. If you tune FreeBSD, you should tune the
Oracle=0D
Server instal= l too. It is pretty easy to win at least 30% in=0D
performance for certa= in workload by choosing the right kernel=0D
configuration.=0D
=0D
= - Arnaud=0D
=0D
> The installation, execution and comparison agai= nst the benchmarks
in the=0D
> article is fairly simple.=0D
>= =0D
> Note that some tuning may not be relevant or recommended (ie: s= ome
of the fs=0D
> benchmarks are sensitive to barriers and other syn= chronous
operations). =C2=A0I'd=0D
> recommend bowing out of a benchm= ark with a 'we're going to be
slower since=0D
> the default configura= tion is this way for the following reason' if
this is=0D
> the case.= =0D
>=0D
> Thanks 'someone'.=0D
>=0D
> Matthew=0D>=0D
>=0D
> =C2=A0Dec 16, 2011 8:46 AM, Adrian Chadd <a= drian@freebsd.org>
wrote:=0D
>=0D
> Can someone please write= up a nice, concise blog post somewhere=0D
> outlining all of this?= =0D
>=0D
> Extra bonus points if it's a blog that is picked up = by=0D
> blogs.freebsdish.org and/or some of the other BSD sites.=0D>=0D
> Guys/girls/fuzzy things - this is 2011; people look at sh= iny
blog=0D
> sites with graphs rather than mailing lists. Sorry, we = lost
that=0D
> battle. :)=0D
>=0D
>=0D
>=0D
>= Adrian=0D
> _______________________________________________=0D
&g= t; freebsd-performance@freebsd.org mailing list=0D
> http://lists.fre= ebsd.org/mailman/listinfo/freebsd-performance=0D
> To unsubscribe, se= nd any mail to=0D
> "freebsd-performance-unsubscribe@freebsd.org"=0D<= br>
References
1. 3D"http://Phoronix.com"/
2. 3D"http://Phoronix.com"/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120104223158.911B11065678>