Date: Mon, 8 Nov 2004 04:29:11 +0100 From: Emanuel Strobl <Emanuel.Strobl@gmx.net> To: freebsd-current@freebsd.org Cc: Robert Watson <rwatson@freebsd.org> Subject: Re: asymmetric NFS transfer rates Message-ID: <200411080429.12846.Emanuel.Strobl@gmx.net> In-Reply-To: <20041102105534.K63929@carver.gumbysoft.com> References: <Pine.NEB.3.96L.1041102131322.21044C-100000@fledge.watson.org> <20041102105534.K63929@carver.gumbysoft.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart1243660.JDKRhvq51c Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Am Dienstag, 2. November 2004 19:56 schrieb Doug White: > On Tue, 2 Nov 2004, Robert Watson wrote: > > On Tue, 2 Nov 2004, Emanuel Strobl wrote: > > > It's a IDE Raid controller (3ware 7506-4, a real one) and the file is > > > indeed huge, but not abnormally. I have a harddisk video recorder, so= I > > > have lots of 700MB files. Also if I copy my photo collection from the > > > server it takes 5 Minutes but copying _to_ the server it takes almost > > > 15 Minutes and the average file size is 5 MB. Fast Ethernet isn't > > > really suitable for my needs, but at least the 10MB/s should be > > > reached. I can't imagine I get better speeds when I upgrade to GbE, > > > (which the important boxes are already, just not the switch) because > > > NFS in it's current state isn't able to saturate a 100baseTX line, at > > > least in one direction. That's the real anstonishing thing for me. Why > > > does reading staurate 100BaseTX but writes only a third? > > > > Have you tried using tcpdump/ethereal to see if there's any significant > > packet loss (for good reasons or not) going on? Lots of RPC retransmits > > would certainly explain the lower performance, and if that's not it, it > > would be good to rule out. The traces might also provide some insight > > into the specific I/O operations, letting you see what block sizes are = in > > use, etc. I've found that dumping to a file with tcpdump and reading > > with ethereal is a really good way to get a picture of what's going on > > with NFS: ethereal does a very nice job decoding the RPCs, as well as > > figuring out what packets are related to each other, etc. > > It'd also be nice to know the mount options (nfs blocksizes in > particular). I haven't done intensive wire-dumps yet, but I figured out some oddities. My main problem seems to be the 3ware controller in combination with NFS. I= f I=20 create a malloc backed md0 I can push more than 9MB/s to it with UDP and mo= re=20 that 10MB/s with TCP (both without modifying r/w-size). I can also copy a 100M file from twed0s1d to twed0s1e (so from and to the s= ame=20 RAID5 array which is worst rate) with 15MB/s so the array can't be the=20 bottleneck. Only when I push to the RAID5 array via NFS I only get 4MB/s, no matter if = I=20 use UDP, TCP or nonstandard r/w-sizes. Next thing I found is that if I tune -w to anything higher than the standar= d=20 8192 the average transfer rate of one big file degrades with UDP but=20 increases with TCP (like I would expect). UDP transfer seems to hic-up with -w tuned, transfer rates peak at 8MB/s bu= t=20 the next second they stay at 0-2MB/s (watched with systat -vm 1) but with T= CP=20 everything runs smooth, regardless of the -w value. Now back to my real problem: Can you imagine that NFS and twe are blocking= =20 each other or something like that? Why do I get such really bad transfer=20 rates when both parts are in use but every single part on its own seems to= =20 work fine? Thanks for any help, =2DHarry --nextPart1243660.JDKRhvq51c Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQBBjugIBylq0S4AzzwRAnX+AJ0TC2LI6GsiX/L3SHfxdQWwzfdvDwCdEMhq Ndcd6c3XokaY1ksXnJ2jRcU= =pPRL -----END PGP SIGNATURE----- --nextPart1243660.JDKRhvq51c--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200411080429.12846.Emanuel.Strobl>