Date: Wed, 6 Feb 2008 11:39:04 +0300 From: Nicholas Kulikov <lostar@rambler.ru> To: freebsd-net@freebsd.org Subject: Re: 1 Gb network NFS issues Message-ID: <57F09B0B-015A-42F2-9C92-4E5A73E3C309@rambler.ru> In-Reply-To: <6FBCC65B-734C-4CF5-9F44-2ECECBFD18EC@rambler.ru> References: <6FBCC65B-734C-4CF5-9F44-2ECECBFD18EC@rambler.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Sorry, i've mistyped: > en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 8000 > media: autoselect (100baseTX <full-duplex>) status: active It is interface configuration from other location (100Mbps lan) so in my home network the same but media is 1Gb link. On 06.02.2008, at 9:11, Nicholas Kulikov wrote: > Hello all, > > I'm trying setup NFS in 1 Gb home network and found some strange > issues: > > Current NFS performance is about 30-35 Mb/s but few days ago i had > 40-45 Mb/s. I've made some changes in server configuration (moved to > FreeBSD 6.3 and changed network driver to nfe) and I can't catch > what is wrong... > > Network structure: > > > NFS server (FreeBSD 6.3) > | > | 1 Gb link > | > NFS client (Mac OS X 10.5.1) > > Server configuration > ----------------------------- > nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 8000 > options=48<VLAN_MTU,POLLING> > > Client configuration > ------------------------ > en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 8000 > media: autoselect (100baseTX <full-duplex>) status: active > > client mount share by using the following parameters: tcp vers=3 > rsize=40960 wsize=40960 readahead=16 rdirplus nolocks intr noatime > > > I've tried copy big file (about 3 Gb) and tcdump-ed client and > server packets: > > 07:11:08.350744 IP (tos 0x0, ttl 64, id 51461, offset 0, flags [DF], > proto TCP (6), length 176) 192.168.254.249.nfs > > 192.168.254.254.2468561782: reply ok 124 access attr: REG 660 ids > 1001/1006 sz 3958224896 nlink 1 rdev 133/134414584 fsid 5e fileid > 2013005 a/m/ctime 1199479211.000000 1199479280.000000 > 1201346449.000000 c 001f > 07:11:08.350793 IP (tos 0x0, ttl 64, id 46719, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.254.49158 > > 192.168.254.249.nfsd: ., cksum 0x7f70 (incorrect (-> 0xd9e5), > 22844:22844(0) ack 27581 win 65535 <nop,nop,timestamp 67511239 > 30749984> > 07:11:08.350988 IP (tos 0x0, ttl 64, id 9526, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561783 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ 0 > 07:11:08.351069 IP (tos 0x0, ttl 64, id 8470, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561784 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 32768 > 07:11:08.351145 IP (tos 0x0, ttl 64, id 65343, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561785 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 65536 > 07:11:08.351242 IP (tos 0x0, ttl 64, id 11022, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561786 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 98304 > 07:11:08.351314 IP (tos 0x0, ttl 64, id 59209, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561787 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 131072 > 07:11:08.351383 IP (tos 0x0, ttl 64, id 39733, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561788 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 163840 > 07:11:08.351450 IP (tos 0x0, ttl 64, id 33325, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561789 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 196608 > 07:11:08.351516 IP (tos 0x0, ttl 64, id 50481, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561790 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 229376 > 07:11:08.351582 IP (tos 0x0, ttl 64, id 50464, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561791 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 262144 > 07:11:08.351658 IP (tos 0x0, ttl 64, id 37215, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561792 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 294912 > 07:11:08.351739 IP (tos 0x0, ttl 64, id 6496, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561793 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 327680 > 07:11:08.351750 IP (tos 0x0, ttl 64, id 49495, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.249.nfsd > > 192.168.254.254.49158: ., cksum 0x15e5 (correct), 27581:27581(0) ack > 23076 win 49943 <nop,nop,timestamp 30749985 67511239> > 07:11:08.351754 IP (tos 0x0, ttl 64, id 36632, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.249.nfsd > > 192.168.254.254.49158: ., cksum 0x1501 (correct), 27581:27581(0) ack > 23308 win 49939 <nop,nop,timestamp 30749985 67511239> > 07:11:08.351758 IP (tos 0x0, ttl 64, id 21313, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.249.nfsd > > 192.168.254.254.49158: ., cksum 0x141c (correct), 27581:27581(0) ack > 23540 win 49936 <nop,nop,timestamp 30749985 67511239> > 07:11:08.351783 IP (tos 0x0, ttl 64, id 50759, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.249.nfsd > > 192.168.254.254.49158: ., cksum 0x1338 (correct), 27581:27581(0) ack > 23772 win 49932 <nop,nop,timestamp 30749985 67511239> > 07:11:08.351801 IP (tos 0x0, ttl 64, id 56942, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.249.nfsd > > 192.168.254.254.49158: ., cksum 0x1254 (correct), 27581:27581(0) ack > 24004 win 49928 <nop,nop,timestamp 30749985 67511239> > 07:11:08.351818 IP (tos 0x0, ttl 64, id 13319, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561794 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 360448 > 07:11:08.351904 IP (tos 0x0, ttl 64, id 17022, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561795 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 393216 > 07:11:08.351981 IP (tos 0x0, ttl 64, id 45600, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561796 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 425984 > 07:11:08.351999 IP (tos 0x0, ttl 64, id 31024, offset 0, flags [DF], > proto TCP (6), length 6196) 192.168.254.249.nfs > > 192.168.254.254.2468561783: reply ok 6144 read REG 660 ids 1001/1006 > sz 3958224896 nlink 1 rdev 133/134414584 fsid 5e fileid 2013005 a/m/ > ctime 1199479211.000000 1199479280.000000 1201346449.000000 32768 > bytes > 07:11:08.352014 IP (tos 0x0, ttl 64, id 13078, offset 0, flags [DF], > proto TCP (6), length 6196) 192.168.254.249.nfs > 192.168.254.254.0: > reply ok 6144 > 07:11:08.352074 IP (tos 0x0, ttl 64, id 9792, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561797 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 458752 > 07:11:08.352085 IP (tos 0x0, ttl 64, id 61207, offset 0, flags [DF], > proto TCP (6), length 6196) 192.168.254.249.nfs > 192.168.254.254.0: > reply ok 6144 > 07:11:08.352143 IP (tos 0x0, ttl 64, id 63816, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.254.49158 > > 192.168.254.249.nfsd: ., cksum 0x7f70 (incorrect (-> 0x8b18), > 24584:24584(0) ack 46013 win 65535 <nop,nop,timestamp 67511239 > 30749985> > 07:11:08.352162 IP (tos 0x0, ttl 64, id 60945, offset 0, flags [DF], > proto TCP (6), length 52) 192.168.254.254.49158 > > 192.168.254.249.nfsd: ., cksum 0x7f70 (incorrect (-> 0x8b18), > 24584:24584(0) ack 46013 win 65535 <nop,nop,timestamp 67511239 > 30749985> > 07:11:08.352177 IP (tos 0x0, ttl 64, id 3701, offset 0, flags [DF], > proto TCP (6), length 168) 192.168.254.254.2468561798 > > 192.168.254.249.nfs: 116 read fh 1143,951878/33632261 32768 bytes @ > 491520 > 07:11:08.352198 IP (tos 0x0, ttl 64, id 3073, offset 0, flags [DF], > proto TCP (6), length 6196) 192.168.254.249.nfs > > 192.168.254.254.3681401976: reply Unknown rpc response > code=3722304989 6144 > 07:11:08.352203 IP (tos 0x0, ttl 64, id 2122, offset 0, flags [DF], > proto TCP (6), length 6196) 192.168.254.249.nfs > > 192.168.254.254.3067833788: reply Unknown rpc response > code=3722304989 6144 > 07:11:08.352207 IP (tos 0x0, ttl 64, id 61294, offset 0, flags [DF], > proto TCP (6), length 2232) 192.168.254.249.nfs > > 192.168.254.254.3067833788: reply Unknown rpc response > code=3722304989 2180 > > There are strange errors: > 192.168.254.254.3681401976: reply Unknown rpc response > > Looks like server successfully sent first piece of data > "07:11:08.351999...length 6196) 192.168.254.249.nfs > > 192.168.254.254.2468561783: reply ok 6144...32768 bytes" and then > report about errors in the following packets: > length 6196) 192.168.254.249.nfs > 192.168.254.254.3681401976: reply > Unknown rpc response code=3722304989 6144 > length 6196) 192.168.254.249.nfs > 192.168.254.254.3067833788: reply > Unknown rpc response code=3722304989 6144 > length 2232) 192.168.254.249.nfs > 192.168.254.254.3067833788: reply > Unknown rpc response code=3722304989 2180 > > There is a lot of such errors in tcpdump logs. > > NFS3 specification do not contains this error code description... > > p.s. And why server is limited with read/write data size 32760? Is > there a possibility to change it? > > > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?57F09B0B-015A-42F2-9C92-4E5A73E3C309>