Date: Fri, 03 Apr 2015 16:33:32 -0500 From: Adam Guimont <aguimont@tezzaron.com> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: freebsd-fs@freebsd.org Subject: Re: NFSD high CPU usage Message-ID: <551F072C.1000505@tezzaron.com> In-Reply-To: <1199661815.10758124.1427941695874.JavaMail.root@uoguelph.ca> References: <1199661815.10758124.1427941695874.JavaMail.root@uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Rick Macklem wrote: > I can think of two explanations for this. > 1 - The server nfsd threads get confused when the TCP recv Q fills > and start looping around. > OR > 2 - The client is sending massive #s of RPCs (or crap that is > incomplete RPCs). > > To get a better idea w.r.t. what is going on, I'd suggest that > you capture packets (for a relatively short period) when the > server is 100% CPU busy. > # tcpdump -s 0 -w out.pcap host <nfs-client> > - run on the server should do it. > Then look at out.pcap in wireshark and see what the packets > look like. (wireshark understands NFS, whereas tcpdump doesn't) > If #1, I'd guess very little traffic (maybe TCP layer stuff), > if #2, I'd guess you'll see a lot of RPC requests or garbage > that isn't a valid request. (This latter case would suggest a > CentOS problem.) > > If you capture the packets but can't look at them in wireshark, > you could email me the packet capture as an attachment and I > can look at it after Apr. 10, when I get home. > > rick > Thanks Rick, I was able to capture this today while it was happening. The capture is for about 100 seconds. I took a look at it in wireshark and to me it appears like the #2 situation you were describing. If you would like to confirm that I've uploaded the pcap file here: https://www.dropbox.com/s/pdhwj5z5tz7iwou/out.pcap.20150403 I will continue running some tests and trying to gather as much data as I can. Regards, Adam Guimont
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?551F072C.1000505>