Date: Wed, 31 Dec 2008 10:31:42 +0100 From: Bernard Dugas <bernard@dugas-family.org> To: Wojciech Puchar <wojtek@wojtek.tensor.gdynia.pl> Cc: freebsd-questions@freebsd.org, Vincent Hoffman <vince@unsane.co.uk> Subject: Re: Optimising NFS for system files Message-ID: <495B3BFE.1040404@dugas-family.org> In-Reply-To: <20081231002557.D45571@wojtek.tensor.gdynia.pl> References: <20081216094719.EDCEE1065675@hub.freebsd.org> <495930E4.1030501@dugas-family.org> <20081229230115.F68805@wojtek.tensor.gdynia.pl> <4959DFEF.5090407@dugas-family.org> <20081230150507.U87347@wojtek.tensor.gdynia.pl> <495A3957.9050002@dugas-family.org> <495A3C72.5070802@unsane.co.uk> <20081230201351.L12391@wojtek.tensor.gdynia.pl> <495A9595.6080201@dugas-family.org> <20081231002557.D45571@wojtek.tensor.gdynia.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
I am trying a memory disk on server to see the effect of hard drive performances, and also discovering the function :-) The conclusion is that memory disk is faster that this drive ;-) 45MB/s vs 10Mb/s But the NFS access to the memory drive is still 5MB/s :-( As there is no more hard drive involved, we know that there is a bottleneck at 5MB in NFS layer on this system... Where ? Thanks a lot for any help on the method to find/diagnose this ! ----------------------------------------Details are below : nfsserver# uname -a FreeBSD nfsserver 7.1-RC1 FreeBSD 7.1-RC1 #0: Sun Dec 7 00:38:13 UTC 2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 nfsserver# mdconfig -a -t swap -s 600m -o reserve -u 7 nfsserver# ls /dev/md* /dev/md7 /dev/mdctl nfsserver# newfs -i 2048 /dev/md7 Reduced frags per cylinder group from 60864 to 59928 to enlarge last cyl group /dev/md7: 600.0MB (1228800 sectors) block size 16384, fragment size 2048 using 6 cylinder groups of 117.05MB, 7491 blks, 59968 inodes. super-block backups (for fsck -b #) at: 160, 239872, 479584, 719296, 959008, 1198720 nfsserver# mkdir /tstnfs nfsserver# mount /dev/md7 /tstnfs nfsserver# date ; time tar -cf - /nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports > /dev/null ; date ; Wed Dec 31 09:11:08 CET 2008 tar: Removing leading '/' from member names 3.794u 8.766s 0:46.40 27.0% 71+1406k 123375+0io 0pf+0w Wed Dec 31 09:11:54 CET 2008 That makes 498MB / 46s = 10.8MB/s for disk drive. nfsserver# date ; cp -r -p /nfsro/commun/clientusr-amd64-7.2-RC2-20081230/ports /tstnfs/ports ; date Wed Dec 31 09:33:09 CET 2008 Wed Dec 31 09:34:46 CET 2008 df -h /dev/md7 512M 498M -27M 106% /tstnfs nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:36:59 CET 2008 tar: Removing leading '/' from member names 2.947u 6.218s 0:10.61 86.2% 74+1463k 104885+0io 0pf+0w Wed Dec 31 09:37:10 CET 2008 nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:37:12 CET 2008 tar: Removing leading '/' from member names 2.895u 6.487s 0:11.01 85.1% 74+1466k 112838+0io 0pf+0w Wed Dec 31 09:37:23 CET 2008 nfsserver# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:40:22 CET 2008 tar: Removing leading '/' from member names 2.902u 6.610s 0:11.10 85.6% 75+1483k 113393+0io 0pf+0w Wed Dec 31 09:40:33 CET 2008 That makes 498MB / 11s = 45MB/s : better that 10MB/s for disk, but not exceptional. Now on the client : client9# uname -a FreeBSD client9 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 11:42:13 UTC 2008 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 client9# mount -o ro nfsserver:/tstnfs /tstnfs client9# df -h nfsserver:/tstnfs 512M 498M 0B 100% /tstnfs client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:50:39 CET 2008 tar: Removing leading '/' from member names 2.896u 13.020s 1:35.22 16.7% 75+1483k 0+0io 2pf+0w Wed Dec 31 09:52:14 CET 2008 client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:52:22 CET 2008 tar: Removing leading '/' from member names 2.700u 12.755s 1:27.78 17.6% 76+1498k 0+0io 0pf+0w Wed Dec 31 09:53:50 CET 2008 client9# date ; time tar -cf - /tstnfs/ports > /dev/null ; date ; Wed Dec 31 09:55:02 CET 2008 tar: Removing leading '/' from member names 2.681u 12.688s 1:28.15 17.4% 74+1464k 0+0io 0pf+0w Wed Dec 31 09:56:30 CET 2008 That makes between 95s and 87s, then 498MB / 95s = 5,2MB/s and 5.7MB/s, like previous test from hard drive NFS export. Top is showing around 100MB of free memory while taring on client9, so i don't think tar is paging on network : last pid: 3318; load averages: 0.17, 0.11, 0.04 up 0+11:14:27 10:08:10 30 processes: 1 running, 29 sleeping CPU: 0.8% user, 0.0% nice, 9.0% system, 0.0% interrupt, 90.2% idle Mem: 19M Active, 720M Inact, 136M Wired, 240K Cache, 110M Buf, 98M Free Best regards, -- Bernard DUGAS Mobile +33 615 333 770
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?495B3BFE.1040404>