From owner-freebsd-stable@FreeBSD.ORG Thu Oct 5 15:07:30 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C4CD816A412; Thu, 5 Oct 2006 15:07:30 +0000 (UTC) (envelope-from rebehn@ant.uni-bremen.de) Received: from antsrv1.ant.uni-bremen.de (antsrv1.ant.uni-bremen.de [134.102.176.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBCF943D6D; Thu, 5 Oct 2006 15:07:21 +0000 (GMT) (envelope-from rebehn@ant.uni-bremen.de) Received: from bremerhaven.ant.uni-bremen.de ([134.102.176.10]) by antsrv1.ant.uni-bremen.de with esmtp (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GVUoh-0002Ka-QQ; Thu, 05 Oct 2006 17:07:19 +0200 Message-ID: <45251FA7.6050000@ant.uni-bremen.de> Date: Thu, 05 Oct 2006 17:07:19 +0200 From: Heinrich Rebehn User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060910 SeaMonkey/1.0.5 MIME-Version: 1.0 To: Heinrich Rebehn References: <4523764E.3070309@ant.uni-bremen.de> In-Reply-To: <4523764E.3070309@ant.uni-bremen.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 05 Oct 2006 15:12:27 +0000 Cc: freebsd-amd64@freebsd.org Subject: Re: NFS client slow on amd64 6.2-PRERELEASE #2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2006 15:07:30 -0000 Heinrich Rebehn wrote: > Hi list, > > despite recent improvements with the nfs code, client performance still > seems to be a problem. I am getting < 2 MB/sec where i would expect at > least 10 MB/sec. > > My Setup: > > Machine | ANTSRV1 | ANTSRV2 > ================================================== > Board | Tyan 2882-D | ASUS A7V8X-X > CPU | Opteron 246 | Athlon XP1800+ > Mem | 2G | 1G > NIC | Broadcom BCM5704C | Intel PRO/1000MT > > OS:6.2-PRERELEASE #2 as of 01-OCT-2006 > 64Bit OS on ANTSRV1, 32Bit on ANTSRV2 > > The 2 machines are connected via a 1GB switch > > ============================ > Trying UDP first > ============================ > root@antsrv1 [/tmp] # mount_nfs -U -L antsrv2:/export/disk2 /mnt > root@antsrv1 [/tmp] # rm /mnt/x/* > root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M > count=10 > 10+0 records in > 10+0 records out > 10485760 bytes transferred in 4.967248 secs (2110980 bytes/sec) > > real 0m5.253s > user 0m0.005s > sys 0m0.022s > root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M > count=100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 69.020366 secs (1519227 bytes/sec) > > real 1m9.276s > user 0m0.028s > sys 0m0.211s > > ============================== > Now TCP > ============================== > root@antsrv1 [/tmp] # umount /mnt > root@antsrv1 [/tmp] # mount_nfs -T -L antsrv2:/export/disk2 /mnt > root@antsrv1 [/tmp] # rm /mnt/x/* > root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M > count=10 > 10+0 records in > 10+0 records out > 10485760 bytes transferred in 5.289492 secs (1982376 bytes/sec) > > real 0m5.312s > user 0m0.001s > sys 0m0.023s > root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M > count=100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 58.715595 secs (1785856 bytes/sec) > > real 0m58.740s > user 0m0.000s > sys 0m0.204s > > =============================== > SCP > =============================== > root@antsrv1 [/tmp] # scp 100MB.dat antsrv2:/tmp/x > Password: > 100MB.dat 100% 100MB > 11.1MB/s 00:09 > > =============================== > Iperf > =============================== > root@antsrv1 [~] # iperf -c antsrv2 > ------------------------------------------------------------ > Client connecting to antsrv2, TCP port 5001 > TCP window size: 32.5 KByte (default) > ------------------------------------------------------------ > [ 3] local 134.102.176.16 port 59123 connected with 134.102.176.17 port > 5001 > [ 3] 0.0-10.0 sec 743 MBytes 623 Mbits/sec > > NFS from a Linux machine (Kernel 2.6.17.8) (connected via 10Mb LAN) > yields a constant value of ~10MB/sec (TCP and UDP), so the FreeBSD NFS > server seems to be ok. > > ================================= > Reversing the setup > ================================= > root@antsrv2 [~] # mount_nfs -L -U antsrv1:/export/huge/x /mnt > root@antsrv2 [~] # time dd if=/dev/zero of=/mnt/10MB.dat bs=1M count=10 > 10+0 records in > 10+0 records out > 10485760 bytes transferred in 0.533470 secs (19655758 bytes/sec) > > real 0m0.547s > user 0m0.001s > sys 0m0.085s > root@antsrv2 [~] # time dd if=/dev/zero of=/mnt/100MB.dat bs=1M count=100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 7.647475 secs (13711401 bytes/sec) > > real 0m7.673s > user 0m0.000s > sys 0m0.920s > root@antsrv2 [~] # umount /mnt > root@antsrv2 [~] # mount_nfs -L -T antsrv1:/export/huge/x /mnt > root@antsrv2 [~] # rm /mnt/* > root@antsrv2 [~] # time dd if=/dev/zero of=/mnt/10MB.dat bs=1M count=10 > 10+0 records in > 10+0 records out > 10485760 bytes transferred in 0.524917 secs (19976030 bytes/sec) > > real 0m0.557s > user 0m0.000s > sys 0m0.085s > root@antsrv2 [~] # time dd if=/dev/zero of=/mnt/100MB.dat bs=1M count=100 > 100+0 records in > 100+0 records out > 104857600 bytes transferred in 5.723110 secs (18321787 bytes/sec) > > real 0m5.756s > user 0m0.000s > sys 0m0.704s > > > This yields much better results. Could this be a 64bit-specific problem? > > Both machines have statd and lockd running. > > Any ideas where i could tune? > Hmm, no one? Is 2MB/s on a Gbit link all i can expect on FreeBSD? --Heinrich