Date: Tue, 30 Apr 2013 10:08:43 -0700 From: "Marc G. Fournier" <scrappy@hub.org> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: freebsd-fs@freebsd.org Subject: Re: Initial NFS Test: Linux vs FreeBSD (769% slower) Message-ID: <531F8BBE-1476-4591-BABD-EA6B230ADB44@hub.org> In-Reply-To: <311431419.1611.1367324960508.JavaMail.root@erie.cs.uoguelph.ca> References: <311431419.1611.1367324960508.JavaMail.root@erie.cs.uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2013-04-30, at 05:29 , Rick Macklem <rmacklem@uoguelph.ca> wrote:
> Marc G. Fournier wrote:
>> If you want, you can email me startup.pcap as an attachment and I'll
>> take
>> a look, but wireshark is pretty good at spotting TCP retransmits, =
etc.
>>=20
>>=20
>> 'k, at 4.x Gig in size, doubt your mail server will handle me sending
>> this to you :) Even compressed, it was going over 400M =85 I can if =
you
>> want it though =85
>>=20
>>=20
>> Baring that, if you want to give me pointers as to what I should be
>> looking for? I have Wireshark installed / and the startup.pcap data
>> loaded =85 the only thing that is jumping out at me is a bunch of =
lines
>> where 'Length' is 32982 while most are 218 =85 highlighted in 'black
>> background, red font' =85 for example:
>>=20
> The big one is a write RPC and it will be a little more than 32768, if
> you've set wsize=3D32768.
>=20
> This can't be a capture for the "nfsstat" numbers you emailed the last =
time.
> (For one thing, that one didn't have any write RPCs counted.) Try and =
get
> a capture for the case where there are few NFS RPCs. (Did you capture =
for
> the first time doing the startup after doing a mount vs do an =
"nfsstat"
> for a subsequent startup?) Or, is this client doing something else on =
the
> network while the startup is happening?
>=20
> I may take a look at it, to see if I can spot anything weird, but a =
capture
> when it only does 88 RPCs is going to be much easier to look at.
'k, here is what I just ran now =85 igb1 is the private IP network where =
the NFS mount is running from =85 host 192.168.1.5 is the host IP of the =
server that I'm using for testing =85 and the results again have WRITE =
calls in it =85=20
note that when I run the tcpdump command, I'm in /tmp on the local file =
system, so I'm not writing that log to the NFS server =85 and there is =
nothing else running on the NFS mount, since all tha tis on it is =
/usr/local/jboss=85 =85 the rest of /usr/local is on the local drives =
also =85 the WRITEs are to /usr/local/jboss/standalone/logs =85 so not =
sure why RPC Counts for Writes is showing 0 change =85
I'm building a new bz2 file right now, but it looks pretty similar to =
the one I already sent you a URL for ...
root@server04:/tmp # nfsstat -c; tcpdump -i igb1 -s 0 -w startup.pcap =
host 192.168.1.5
Client Info:
Rpc Counts:
Getattr Setattr Lookup Readlink Read Write Create =
Remove
2746536 821481 974263 18 2230948 2098303 160726 =
4954
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus =
Access
1862 0 0 14724 950 16272 0 =
330261
Mknod Fsstat Fsinfo PathConf Commit
12 30926 6 0 0
Rpc Info:
TimedOut Invalid X Replies Retries Requests
0 0 0 0 9432366
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits =
Misses
26331524 2746535 20540473 974231 2380075 2225802 2618800 =
2097243
BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits =
Misses
1262 18 46863 15678 29941 0 22516391 =
330264
tcpdump: listening on igb1, link-type EN10MB (Ethernet), capture size =
65535 bytes
^C5454815 packets captured
5458144 packets received by filter
0 packets dropped by kernel
root@server04:/tmp # nfsstat -c
Client Info:
Rpc Counts:
Getattr Setattr Lookup Readlink Read Write Create =
Remove
2746603 821481 974276 18 2230953 2098303 160726 =
4954
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus =
Access
1862 0 0 14724 950 16272 0 =
330275
Mknod Fsstat Fsinfo PathConf Commit
12 30926 6 0 0
Rpc Info:
TimedOut Invalid X Replies Retries Requests
0 0 0 0 9432465
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits =
Misses
26332545 2746602 20540712 974244 2380801 2225807 2618800 =
2097243
BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits =
Misses
1262 18 46863 15678 29941 0 22516700 =
330278
>=20
> rick
>=20
>>=20
>>=20
>>=20
>>=20
>> i am doing a bzip2 compressed file right now, and will make it
>> available via HTTP, if you are interested =85
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>> The only other thing I can suggest is taking the "soft,intr" options
>> off
>> your mount and see if that has any effect. Maybe some syscall is
>> returning
>> EINTR and confusing jboss?
>>=20
>>=20
>> Tried removing soft,intr =85 no change, still around 240s =85
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?531F8BBE-1476-4591-BABD-EA6B230ADB44>