Date: Tue, 2 Aug 2011 12:35:52 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Steven Hartland" <killing@multiplay.co.uk>, "Andre Oppermann" <andre@freebsd.org> Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? Message-ID: <C706DEE346684B8DB06CFC090F556E72@multiplay.co.uk> References: <E18D678F05BB4F3B93ADEB304CCA8504@multiplay.co.uk><1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message ----- From: "Steven Hartland" > ----- Original Message ----- > From: "Andre Oppermann" > ... >>> I believe this is tcps_rcvmemdrop in tcp_reass.c to which there's the >>> following comment:- >>> >>> * XXXLAS: Using sbspace(so->so_rcv) instead of so->so_rcv.sb_hiwat >>> * should work but causes packets to be dropped when they shouldn't. >>> >>> I notice this code is relatively new, so I'm wondering if this may be >>> something to do with what we're seeing, possibly still dropping packets >>> it shouldn't? >>> >>> @Lawrence apologies' for the direct mail, but I believe you where the >>> original author this particular change so wondered if you may be >>> able to shed any light on this? >> >> You could be onto something here. Please try this patch: >> http://people.freebsd.org/~andre/tcp_reass.c-logdebug-20110802.diff >> >> You can enable the log output with >> sysctl net.inet.tcp.log_debug=1 >> and report the log output (comes at LOG_DEBUG level). > > Thanks for the response Andre, I've applied the patch and I'm seeing > lots of the following during the test which is:- > 1. scp from local host (10.10.1.30) -> tcptest (10.10.1.20) reciever > which gets ~ 64MB/s > 2. scp from remote host (10.10.1.10) -> tcptest (10.10.1.20) reciever > which gets ~ 10MB/s (line has packet loss) > > Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10<ACK>; tcp_reass: global zone limit > reached, segment dropped > Aug 2 11:08:50 tcptest kernel: TCP: [10.10.1.30]:60811 to [10.10.1.20]:22 tcpflags 0x10<ACK>; tcp_reass: global zone limit > reached, segment dropped Hmm, based on this are we seeing something similar to the following? http://www.freebsd.org/cgi/query-pr.cgi?pr=155407 Other potentially useful info:- vmstat -z | head -1 ; vmstat -z | grep -i tcp ITEM SIZE LIMIT USED FREE REQUESTS FAILURES tcp_inpcb: 336, 25608, 115, 556, 707, 0 tcpcb: 880, 25600, 115, 405, 707, 0 tcptw: 72, 5150, 0, 600, 188, 0 tcpreass: 40, 1680, 106, 1574, 185926, 4414 sysctl net.inet.tcp.reass net.inet.tcp.reass.overflows: 0 net.inet.tcp.reass.cursegments: 106 net.inet.tcp.reass.maxsegments: 1680 netstat -s -f inet -p tcp | grep "discarded due" 4414 discarded due to memory problems net.inet.tcp.reass.maxsegments: 1680 sysctl kern.ipc.nmbclusters kern.ipc.nmbclusters: 25600 The default value of nmbclusters on the target machine explains the value of net.inet.tcp.reass.maxsegments which defaults to nmbclusters / 16 Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the tests appears to result in a solid 14MB/s, its still running a full soak test but looking very promising :) So I suppose the question is should maxsegments be larger by default due to the recent changes e.g. - V_tcp_reass_maxseg = nmbclusters / 16; + V_tcp_reass_maxseg = nmbclusters / 8; or is the correct fix something more involved? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C706DEE346684B8DB06CFC090F556E72>