From owner-freebsd-stable@FreeBSD.ORG  Thu Nov 18 13:02:13 2004
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
Delivered-To: freebsd-stable@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id 0C5FE16A4CE; Thu, 18 Nov 2004 13:02:13 +0000 (GMT)
Received: from fledge.watson.org (fledge.watson.org [204.156.12.50])
	by mx1.FreeBSD.org (Postfix) with ESMTP
	id 6821F43D58; Thu, 18 Nov 2004 13:02:12 +0000 (GMT)
	(envelope-from robert@fledge.watson.org)
Received: from fledge.watson.org (localhost [127.0.0.1])
	by fledge.watson.org (8.13.1/8.13.1) with ESMTP id iAID0dGC069640;
	Thu, 18 Nov 2004 08:00:39 -0500 (EST)
	(envelope-from robert@fledge.watson.org)
Received: from localhost (robert@localhost)iAID0cLp069637;
	Thu, 18 Nov 2004 13:00:38 GMT
	(envelope-from robert@fledge.watson.org)
Date: Thu, 18 Nov 2004 13:00:38 +0000 (GMT)
From: Robert Watson <rwatson@freebsd.org>
X-Sender: robert@fledge.watson.org
To: Wilko Bulte <wb@freebie.xs4all.nl>
In-Reply-To: <20041118124902.GC75559@freebie.xs4all.nl>
Message-ID: <Pine.NEB.3.96L.1041118125822.66045D-100000@fledge.watson.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
cc: Emanuel Strobl <Emanuel.Strobl@gmx.net>
cc: freebsd-current@freebsd.org
cc: freebsd-stable@freebsd.org
Subject: Re: serious networking (em) performance (ggate and NFS) problem
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Production branch of FreeBSD source code
	<freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
	<mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
	<mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Nov 2004 13:02:13 -0000


On Thu, 18 Nov 2004, Wilko Bulte wrote:

> On Thu, Nov 18, 2004 at 12:27:44PM +0000, Robert Watson wrote..
> > 
> > On Wed, 17 Nov 2004, Emanuel Strobl wrote:
> > 
> > > I really love 5.3 in many ways but here're some unbelievable transfer
> > > rates, after I went out and bought a pair of Intel GigaBit Ethernet
> > > Cards to solve my performance problem (*laugh*): 
> > 
> > I think the first thing you want to do is to try and determine whether the
> > problem is a link layer problem, network layer problem, or application
> > (file sharing) layer problem.  Here's where I'd start looking:
> 
> And you definitely want to look at polling(4) 

He did, but he set the HZ at 256, which is sufficiently low as to
guarantee a substantial increase in latency, and likely guarantee
interface and socket queue overruns (although I haven't done the math to
verify that is the case).  Between the very finite sizes of ifnet send
queues, socket buffers, and if_em descriptors, and on-board buffers on the
card, high latency polling can result in lots of packet loss and delay
under load.  Hence the recommendation of a relatively high value of HZ so
that the queues in the driver are drained regularly, and sends
acknowledged so that the sent mbufs can be reclaimed and reused.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org      Principal Research Scientist, McAfee Research