From owner-freebsd-current@FreeBSD.ORG Mon Nov 8 03:29:15 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 983C716A4CE for ; Mon, 8 Nov 2004 03:29:15 +0000 (GMT) Received: from mail.gmx.net (pop.gmx.de [213.165.64.20]) by mx1.FreeBSD.org (Postfix) with SMTP id 94B3343D49 for ; Mon, 8 Nov 2004 03:29:14 +0000 (GMT) (envelope-from Emanuel.Strobl@gmx.net) Received: (qmail 24937 invoked by uid 65534); 8 Nov 2004 03:29:13 -0000 Received: from flb.schmalzbauer.de (EHLO cale.flintsbach.schmalzbauer.de) (62.245.232.135) by mail.gmx.net (mp024) with SMTP; 08 Nov 2004 04:29:13 +0100 X-Authenticated: #301138 From: Emanuel Strobl To: freebsd-current@freebsd.org Date: Mon, 8 Nov 2004 04:29:11 +0100 User-Agent: KMail/1.7 References: <20041102105534.K63929@carver.gumbysoft.com> In-Reply-To: <20041102105534.K63929@carver.gumbysoft.com> X-OS: FreeBSD X-Birthday: 10/06/72 X-Address: Munich, 80686 X-Tel: +49 89 18947781 X-CelPhone: +49 173 9967781 X-Country: Germany MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1243660.JDKRhvq51c"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200411080429.12846.Emanuel.Strobl@gmx.net> cc: Robert Watson Subject: Re: asymmetric NFS transfer rates X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Nov 2004 03:29:15 -0000 --nextPart1243660.JDKRhvq51c Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Am Dienstag, 2. November 2004 19:56 schrieb Doug White: > On Tue, 2 Nov 2004, Robert Watson wrote: > > On Tue, 2 Nov 2004, Emanuel Strobl wrote: > > > It's a IDE Raid controller (3ware 7506-4, a real one) and the file is > > > indeed huge, but not abnormally. I have a harddisk video recorder, so= I > > > have lots of 700MB files. Also if I copy my photo collection from the > > > server it takes 5 Minutes but copying _to_ the server it takes almost > > > 15 Minutes and the average file size is 5 MB. Fast Ethernet isn't > > > really suitable for my needs, but at least the 10MB/s should be > > > reached. I can't imagine I get better speeds when I upgrade to GbE, > > > (which the important boxes are already, just not the switch) because > > > NFS in it's current state isn't able to saturate a 100baseTX line, at > > > least in one direction. That's the real anstonishing thing for me. Why > > > does reading staurate 100BaseTX but writes only a third? > > > > Have you tried using tcpdump/ethereal to see if there's any significant > > packet loss (for good reasons or not) going on? Lots of RPC retransmits > > would certainly explain the lower performance, and if that's not it, it > > would be good to rule out. The traces might also provide some insight > > into the specific I/O operations, letting you see what block sizes are = in > > use, etc. I've found that dumping to a file with tcpdump and reading > > with ethereal is a really good way to get a picture of what's going on > > with NFS: ethereal does a very nice job decoding the RPCs, as well as > > figuring out what packets are related to each other, etc. > > It'd also be nice to know the mount options (nfs blocksizes in > particular). I haven't done intensive wire-dumps yet, but I figured out some oddities. My main problem seems to be the 3ware controller in combination with NFS. I= f I=20 create a malloc backed md0 I can push more than 9MB/s to it with UDP and mo= re=20 that 10MB/s with TCP (both without modifying r/w-size). I can also copy a 100M file from twed0s1d to twed0s1e (so from and to the s= ame=20 RAID5 array which is worst rate) with 15MB/s so the array can't be the=20 bottleneck. Only when I push to the RAID5 array via NFS I only get 4MB/s, no matter if = I=20 use UDP, TCP or nonstandard r/w-sizes. Next thing I found is that if I tune -w to anything higher than the standar= d=20 8192 the average transfer rate of one big file degrades with UDP but=20 increases with TCP (like I would expect). UDP transfer seems to hic-up with -w tuned, transfer rates peak at 8MB/s bu= t=20 the next second they stay at 0-2MB/s (watched with systat -vm 1) but with T= CP=20 everything runs smooth, regardless of the -w value. Now back to my real problem: Can you imagine that NFS and twe are blocking= =20 each other or something like that? Why do I get such really bad transfer=20 rates when both parts are in use but every single part on its own seems to= =20 work fine? Thanks for any help, =2DHarry --nextPart1243660.JDKRhvq51c Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQBBjugIBylq0S4AzzwRAnX+AJ0TC2LI6GsiX/L3SHfxdQWwzfdvDwCdEMhq Ndcd6c3XokaY1ksXnJ2jRcU= =pPRL -----END PGP SIGNATURE----- --nextPart1243660.JDKRhvq51c--