Date: Tue, 2 Aug 2011 13:32:53 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Andre Oppermann" <andre@freebsd.org> Cc: freebsd-net@freebsd.org, lstewart@freebsd.org Subject: Re: tcp failing to recover from a packet loss under 8.2-RELEASE? Message-ID: <BF7A6AD38BBC4BBFA85609E9560F52BC@multiplay.co.uk> References: <E18D678F05BB4F3B93ADEB304CCA8504@multiplay.co.uk><1F95A4C2D54E4F369830143CBDB5FF86@multiplay.co.uk><4E37C0F2.4080004@freebsd.org> <2B063B6D95AA4C27B004C50D96393F91@multiplay.co.uk> <C706DEE346684B8DB06CFC090F556E72@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message -----
From: "Steven Hartland"
> Setting net.inet.tcp.reass.maxsegments=8148 and rerunning the
> tests appears to result in a solid 14MB/s, its still running a
> full soak test but looking very promising :)
Ok so full test has completed with over 134GB of data transferred
without a hitch.
I haven't been able to complete this test without error on this
target machine before the change to maxsegments; so packets being
dropped in the tcp reassembly code due to tcp_reass global zone
exhaustion is almost certainly the cause of the stalls we're
seeing, which is good news :)
I suspect there are a few contributing factors at play causing
this.
1. There is known to be a dodgy fibre on the test routing, which
is scheduled for cleaning, but is currently causing a small amount
of packet loss between the sender and receiver.
2. The target machine has 24 cores and is running 8 queues on
the igb interface which could result in the requirement to reorder
packets even if they arrived in order on the wire. I say this as
disabling msix on igb0, which results in just one queue, did reduce
the occurrence rate of the problem.
3. Combining a low latency (<1ms) high throughput connection
~64MB/s with a lower throughput but still relatively high bandwidth
~14MB/s with a 10ms latency.
I look forward to hearing peoples thoughts on what the actual fix
should be: increased default nmbclusters, decreased nmbclusters =>
maxsegments divisor, or something else?
Regards
Steve
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BF7A6AD38BBC4BBFA85609E9560F52BC>