From owner-freebsd-stable@FreeBSD.ORG Thu Dec 15 13:36:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34DC31065670; Thu, 15 Dec 2011 13:36:23 +0000 (UTC) (envelope-from michael.larabel@phoronix.com) Received: from phx1.phoronix.com (173.192.77.202-static.reverse.softlayer.com [173.192.77.202]) by mx1.freebsd.org (Postfix) with ESMTP id C962E8FC0C; Thu, 15 Dec 2011 13:36:22 +0000 (UTC) Received: from c-98-193-96-120.hsd1.il.comcast.net ([98.193.96.120] helo=[172.16.93.133]) by phx1.phoronix.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1RbBTx-0005eR-HR; Thu, 15 Dec 2011 07:36:21 -0600 Message-ID: <4EE9F7D2.4050607@phoronix.com> Date: Thu, 15 Dec 2011 07:36:18 -0600 From: Michael Larabel User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Stefan Esser References: <4EE1EAFE.3070408@m5p.com> <4EE2AE64.9060802@m5p.com> <4EE88343.2050302@m5p.com> <4EE933C6.4020209@zedat.fu-berlin.de> <20111215024249.GA13557@icarus.home.lan> <4EE9A2A0.80607@zedat.fu-berlin.de> <4EE9C79B.7080607@phoronix.com> <4EE9F546.6060503@freebsd.org> In-Reply-To: <4EE9F546.6060503@freebsd.org> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - phx1.phoronix.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - phoronix.com Cc: FreeBSD Stable Mailing List , Current FreeBSD , Michael Ross , freebsd-performance@freebsd.org, "O. Hartmann" , Jeremy Chadwick Subject: Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 13:36:23 -0000 On 12/15/2011 07:25 AM, Stefan Esser wrote: > Am 15.12.2011 11:10, schrieb Michael Larabel: >> No, the same hardware was used for each OS. >> >> In terms of the software, the stock software stack for each OS was used. > Just curious: Why did you choose ZFS on FreeBSD, while UFS2 (with > journaling enabled) should be an obvious choice since it is more similar > in concept to ext4 and since that is what most FreeBSD users will use > with FreeBSD? I was running some ZFS vs. UFS tests as well and this happened to have ZFS on when I was running some other tests. > > Did you tune the ZFS ARC (e.g. vfs.zfs.arc_max="6G") for the tests? The OS was left in its stock configuration. > > And BTW: Did your measured run times account for the effect, that Linux > keeps much more dirty data in the buffer cache (FreeBSD has a low limit > on dirty buffers since under realistic load the already cached data is > much more likely to be reused and thus more valuable than freshly > written data; aggressively caching dirty data would significantly reduce > throughput and responsiveness under high load). Given the hardware specs > of the test system, I guess that Linux accepts at least 100 times the > dirty data in the buffer cache, compared to FreeBSD (where this number > is at most in the tens of megabyte range). > > If you did not, then your results do not represent a server load (which > I'd expect relevant, if you are testing against Oracle Linux 6.1 > server), where continuous performance is required. Tests that run on an > idle system starting in a clean state and ignoring background flushing > of the buffer cache after the timed program has stopped are perhaps > useful for a very lowly loaded PC, but not for a system with high load > average as the default. > > I bet that if you compared the systems under higher load (which > admittedly makes it much harder to get sensible numbers for the program > under test) or with reduced buffer cache size (or raise the dirty buffer > limit in FreeBSD accordingly, which ought to be possible with sysctl > and/or boot time tuneables, e.g. "vfs.hidirtybuffers"). > > And a last remark: Single benchmark runs do not provide reliable data. > FreeBSD comes with "ministat" to check the significance of benchmark > results. Each test should be repeated at least 5 times for meaningful > averages with acceptable confidence level. The Phoronix Test Suite runs most tests a minimum of three times and if the standard deviation exceeds 3.5% the run count is dynamically increased, among other safeguards. -- Michael > > Regards, STefan >