From owner-freebsd-stable@FreeBSD.ORG  Wed Mar 22 00:46:50 2006
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
X-Original-To: stable@freebsd.org
Delivered-To: freebsd-stable@FreeBSD.ORG
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id D2EE016A424;
	Wed, 22 Mar 2006 00:46:50 +0000 (UTC)
	(envelope-from jd@ugcs.caltech.edu)
Received: from ngwee.ugcs.caltech.edu (ngwee.ugcs.caltech.edu
	[131.215.176.116])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 6C44B43D70;
	Wed, 22 Mar 2006 00:46:47 +0000 (GMT)
	(envelope-from jd@ugcs.caltech.edu)
Received: by ngwee.ugcs.caltech.edu (Postfix, from userid 3640)
	id 8AD4CCC065; Tue, 21 Mar 2006 16:46:46 -0800 (PST)
Date: Tue, 21 Mar 2006 16:46:46 -0800
From: Jon Dama <jon@ugcs.caltech.edu>
To: Mikhail Teterin <mi+mx@aldan.algebra.com>
Message-ID: <20060322004646.GA5929@ngwee.ugcs.caltech.edu>
References: <200603211607.30372.mi+mx@aldan.algebra.com>
	<200603211831.54172.mi+mx@aldan.algebra.com>
	<20060321234856.GA24194@hugo10.ka.punkt.de>
	<200603211858.02801.mi+mx@aldan.algebra.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <200603211858.02801.mi+mx@aldan.algebra.com>
Sender: jd@ugcs.caltech.edu
Cc: alc@freebsd.org, stable@freebsd.org
Subject: Re: more weird bugs with mmap-ing via NFS
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>, 
	<mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
	<mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Mar 2006 00:46:50 -0000

>From Mikhail Teterin <mi+mx@aldan.algebra.com>, Tue, Mar 21, 2006 at 06:58:01PM -0500:
> I'll try the TCP mount, workaround. If it helps, we can assume, our UDP NFS is 
> broken for sustained high bandwidth writes :-(

What?  I think you misunderstood.  UDP NFS fairs poorly under 
network congestion; it has nothing to do with FreeBSD.  Any 
iota of frame-loss will induce costly time-outs and hang-ups
and quickly become intolerable.

Let us consider a few cases:
1) a server attached with 1Gb ethernet and clients with 100Mb 
   connections.  Server begins sending data to a client.   The
   client pipe is narrower and the switch begins to buffer the
   frames.  Switches _do not_ have very much buffer space.  A 
   high-end switch may have 2MB.  Likely some of this can be
   shared between ports; some will be reserved per port.  Let us
   assume all of the memory is on.

   d(Memory_Used)/dt = 1000Mb/s - 100Mb/s = 900MB/s.  Thus... in 
   less than 10ms the switch will run out of buffer space... and
   then *drop* frames.

   oops.  bye-bye UDP NFS.

2) a server is attached with 1Gb ethernet and the clients with 1Gb
   ethernet.  Multiple clients write to the NFS server simultanteously...

Okay, okay.  It isn't quite this bad because there will only be so many 
outstanding requests, but it doesn't take many clients actively contending 
for a shared resource (the link to the server) to cause glitches...
   
This is precisely the problem TCP was intended to solve.  NFS UDP is
an antiquated solution to TCP overhead on the underpowered machines of 
yesteryear.  NFS UDP has essentially no use on modern hardware.
   
                 -Jon