From owner-freebsd-arch@FreeBSD.ORG  Tue Feb  3 15:40:22 2009
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: arch@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id C1A7F1065697
	for <arch@FreeBSD.org>; Tue,  3 Feb 2009 15:40:22 +0000 (UTC)
	(envelope-from obrien@NUXI.org)
Received: from dragon.nuxi.org (trang.nuxi.org [74.95.12.85])
	by mx1.freebsd.org (Postfix) with ESMTP id C6E648FC20
	for <arch@FreeBSD.org>; Tue,  3 Feb 2009 15:40:21 +0000 (UTC)
	(envelope-from obrien@NUXI.org)
Received: from dragon.nuxi.org (obrien@localhost [127.0.0.1])
	by dragon.nuxi.org (8.14.2/8.14.2) with ESMTP id n13FeEpR035463;
	Tue, 3 Feb 2009 07:40:14 -0800 (PST)
	(envelope-from obrien@dragon.nuxi.org)
Received: (from obrien@localhost)
	by dragon.nuxi.org (8.14.2/8.14.2/Submit) id n13FeDg1035462;
	Tue, 3 Feb 2009 07:40:13 -0800 (PST) (envelope-from obrien)
Date: Tue, 3 Feb 2009 07:40:13 -0800
From: "David O'Brien" <obrien@FreeBSD.org>
To: Poul-Henning Kamp <phk@phk.freebsd.dk>
Message-ID: <20090203154013.GA33520@dragon.NUXI.org>
Mail-Followup-To: obrien@freebsd.org, Poul-Henning Kamp <phk@phk.freebsd.dk>,
	"M. Warner Losh" <imp@bsdimp.com>, arch@FreeBSD.org
References: <20090202220628.GA76833@dragon.NUXI.org>
	<9061.1233646810@critter.freebsd.dk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <9061.1233646810@critter.freebsd.dk>
X-Operating-System: FreeBSD 8.0-CURRENT
User-Agent: Mutt/1.5.16 (2007-06-09)
Cc: arch@FreeBSD.org
Subject: Re: svn commit: r187132 - head/usr.bin/make
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: obrien@FreeBSD.org
List-Id: Discussion related to FreeBSD architecture <freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Feb 2009 15:40:24 -0000

On Tue, Feb 03, 2009 at 07:40:10AM +0000, Poul-Henning Kamp wrote:
> In message <20090202220628.GA76833@dragon.NUXI.org>, "David O'Brien" writes:
> I am disappointed.
> You of all people here should know better than making such a mess
> out of benchmarks.

Poul-Henning,
I fully know these results are not stringent.

Warner made a baseless 10% performance claim and used it as the
bases for a commit.

It took rounds of emails to get any detail from him, none of which
were the actual times, standard deviation, etc...  The only purpose
of my measurements were to see if I could reproduce anything close
to the claimed 10%.

> Second you totally bungle your data collection, by not eliminating
> cache-effects.

I would have accepted Warner's 10% if I had gotten that just once,
regardless if it was due to cache efforts or not.  It would have
backed up that, without totally cooking the runs, you could see a
10% time difference.

-- David