Date: Sat, 28 Aug 2010 22:22:52 -0500 From: "Rick C. Petty" <rick-freebsd2009@kiwi-computer.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: freebsd-stable@freebsd.org Subject: Re: Why is NFSv4 so slow? Message-ID: <20100829032252.GA81736@rix.kiwi-computer.com> In-Reply-To: <Pine.GSO.4.63.1007052332020.7410@muncher.cs.uoguelph.ca> References: <20100627221607.GA31646@kay.kiwi-computer.com> <Pine.GSO.4.63.1007052332020.7410@muncher.cs.uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi. I'm still having problems with NFSv4 being very laggy on one client. When the NFSv4 server is at 50% idle CPU and the disks are < 1% busy, I am getting horrible throughput on an idle client. Using dd(1) with 1 MB block size, when I try to read a > 100 MB file from the client, I'm getting around 300-500 KiB/s. On another client, I see upwards of 20 MiB/s with the same test (on a different file). On the broken client: # uname -mv FreeBSD 8.1-STABLE #5 r211534M: Sat Aug 28 15:53:10 CDT 2010 user@example.com:/usr/obj/usr/src/sys/GENERIC i386 # ifconfig re0 re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC> ether 00:e0:4c:xx:yy:zz inet xx.yy.zz.3 netmask 0xffffff00 broadcast xx.yy.zz.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active # netstat -m 267/768/1035 mbufs in use (current/cache/total) 263/389/652/25600 mbuf clusters in use (current/cache/total/max) 263/377 mbuf+clusters out of packet secondary zone in use (current/cache) 0/20/20/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 592K/1050K/1642K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines # netstat -idn Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop re0 1500 <Link#1> 00:e0:4c:xx:yy:zz 232135 0 0 68984 0 0 0 re0 1500 xx.yy.zz.0/2 xx.yy.zz.3 232127 - - 68979 - - - nfe0* 1500 <Link#2> 00:22:15:xx:yy:zz 0 0 0 0 0 0 0 plip0 1500 <Link#3> 0 0 0 0 0 0 0 lo0 16384 <Link#4> 42 0 0 42 0 0 0 lo0 16384 fe80:4::1/64 fe80:4::1 0 - - 0 - - - lo0 16384 ::1/128 ::1 0 - - 0 - - - lo0 16384 127.0.0.0/8 127.0.0.1 42 - - 42 - - - # sysctl kern.ipc.maxsockbuf kern.ipc.maxsockbuf: 1048576 # sysctl net.inet.tcp.sendbuf_max net.inet.tcp.sendbuf_max: 16777216 # sysctl net.inet.tcp.recvbuf_max net.inet.tcp.recvbuf_max: 16777216 # sysctl net.inet.tcp.sendspace net.inet.tcp.sendspace: 65536 # sysctl net.inet.tcp.recvspace net.inet.tcp.recvspace: 131072 # sysctl hw.pci | grep msi hw.pci.honor_msi_blacklist: 1 hw.pci.enable_msix: 1 hw.pci.enable_msi: 1 # vmstat -i interrupt total rate irq14: ata0 47 0 irq16: re0 219278 191 irq21: ohci0+ 5939 5 irq22: vgapci0+ 77990 67 cpu0: timer 2294451 1998 irq256: hdac0 44069 38 cpu1: timer 2293983 1998 Total 4935757 4299 Any ideas? -- Rick C. Petty
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100829032252.GA81736>