Date: Sun, 13 Nov 2011 14:48:19 -0700 From: Jason Wolfe <nitroboost@gmail.com> To: Erich Weiler <weiler@soe.ucsc.edu> Cc: freebsd-net@freebsd.org Subject: Re: Arg. TCP slow start killing me. Message-ID: <CAAAm0r3ur3YcKU51bhfzZspJuzOUVyhPf0zAxfa66ZChtwfh%2Bg@mail.gmail.com> In-Reply-To: <4EC033B7.5080609@soe.ucsc.edu> References: <4EC033B7.5080609@soe.ucsc.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Erich, Slow start is actually just the initial ramp up limited by RFC 3390 being enabled by default (usually 3/4 packets), but this is only the case for the first few seconds of the stream. You can effectively speed that up with something like this though: net.inet.tcp.rfc3390=0 net.inet.tcp.slowstart_flightsize=10 net.inet.tcp.sendspace=262144 net.inet.tcp.recvspace=262144 The first 2 allow 10 packets to be sent before an ACK, and the 2nd 2 just bump as the starting window size. With your memory and the massive max you set no reason to force them to slowly step up from such a low initial size. Looks like the numbers you used for initial are actually the default increment/step size of the window growth. Also since you mentioned latency playing a factor here, try this sysctl. If overruns are an issue you'll likely see a bit of an increase in retransmits, but could potentially show a sizable positive impact in the saw tooth. net.inet.tcp.inflight.enable=0 Is it possible to upgrade to 8.2-STABLE? Cubic has shown some really great improvement in my latent paths, a steady 10% overall increase in same cases. Jason Wolfe On Sun, Nov 13, 2011 at 2:16 PM, Erich Weiler <weiler@soe.ucsc.edu> wrote: > So, I have a FreeBSD 8.1 box that I'm using as a firewall (pfSense 2.0 > really, which uses 8.1 as a base), and I'm filtering packets inbound and > I'm seeing a typical sawtooth pattern where I get high bandwidth, then a > packet drops somewhere, and the TCP connections back off a *lot*, then > slowly get faster, then backoff, etc. These are all higher latency WAN > connections. > > I get an average of 1.5 - 2.0 Gb/s incoming, but I see it spike to like > 3Gb/s every once in a while, then drop again. I'm trying to maintain that > 3Gb/s for as long as possible between it dropping. > > Given that 8.1 does not have the more advanced TCP congestion algorithms > like cubic and H-TPC that might help that to some degree, I'm trying to > "fake it". ;) > > My box has 24GB RAM on it. Is there some tunable I can set that would > effectively buffer incoming packets, even though the buffers would > eventually fill up, just to "delay" the TCP dropped packet signal telling > the hosts on the internet to back off? Like, could I effectively buffer > 10GB of packets in the queue before it sent the backoff signal? Would > setting kern.ipc.nmbclusters or something similar help? > > Right now I have: > > loader.conf.local: > > vm.kmem_size_max=12G > vm.kmem_size=10G > > sysctl.conf: > > kern.ipc.maxsockbuf=16777216 > kern.ipc.nmbclusters=262144 > net.inet.tcp.recvbuf_max=16777216 > net.inet.tcp.recvspace=8192 > net.inet.tcp.sendbuf_max=16777216 > net.inet.tcp.sendspace=16384 > > I guess the goal is to keep the bandwidth high without dropoffs for as > long as possible, with out as many TCP resets on the streams. > > Any help much appreciated! I'm probably missing a key point, but that's > why I'm posting to the list. ;) > > cheers, > erich > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAAm0r3ur3YcKU51bhfzZspJuzOUVyhPf0zAxfa66ZChtwfh%2Bg>