Date: Fri, 25 Jan 2013 09:00:38 -0500 From: John Baldwin <jhb@freebsd.org> To: freebsd-net@freebsd.org Cc: Lawrence Stewart <lstewart@freebsd.org> Subject: Re: Some questions about the new TCP congestion control code Message-ID: <201301250900.38871.jhb@freebsd.org> In-Reply-To: <5101C377.7010907@freebsd.org> References: <201301141604.29864.jhb@freebsd.org> <51014150.50101@networx.ch> <5101C377.7010907@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thursday, January 24, 2013 6:27:51 pm Lawrence Stewart wrote: > On 01/25/13 01:12, Andre Oppermann wrote: > > On 24.01.2013 14:28, Lawrence Stewart wrote: > >> On 01/16/13 06:27, John Baldwin wrote: > >>> One other thing I noticed which is may or may not be odd during this, > >>> is that > >>> if you have a connection with TCP_NODELAY enabled and you fill your > >>> cwnd and > >>> then you get an ACK back for an earlier small segment (less than > >>> MSS), TCP > >>> will not send out a "short" segment for the amount of window space > >>> released. > >>> Instead, it will wait until a full MSS of space is available before > >>> sending > >>> a packet. I'm not sure if that is the correct behavior with > >>> TCP_NODELAY or > >>> if we should send "short" segments in that case. > >> > >> We try fairly hard not to send runt segments irrespective of NODELAY, > >> but I would be happy to see that change. I'm not aware of any "correct > >> behaviour" we have to adhere to - I think it would be perfectly > >> reasonable to have a sysctl set the lowest number of bytes we'd be > >> willing to send a runt segment for and then key off TCP_NODELAY as to > >> whether we try hard to send an MSS worth or send as soon as we have the > >> min number of bytes worth of window available. > > > > This is classic silly window syndrome prevention applied to the CWND. > > Yes, but I think we could provide knobs to relax the behaviour where the > latency vs header/payload overhead tradeoff swings in favour of latency. > > I guess, John, I should first ask if you know why you were only getting > such small ACKs back? Were you sending full MSS segments in the first > place or doing some sort of PUSH to try and expedite getting some > smaller chunk of data to the other end which triggered a small segment > and corresponding small ACK? In general we only send very small segments as we have TCP_NODELAY on and are effectively using this to forward small datagrams. Thus, in the usual case we have a lot of small segments (much smaller than MSS). When we fill the congestion window, the stack starts waiting for a full MSS and that requires several ACKs from the earlier small segments. > > Sending a small segment when the window opens just a bit isn't going to help > > much and > > I wouldn't be game to make such a blanket statement - that very much > depends on the situation. I think John's use case is relevant and we > currently aren't very helpful towards it. I think in the case of TCP_NODELAY, the user is explicitly asking for lower latency. Think about the "typical" use case for TCP_NODELAY of a shell session. If you manage to build up a cwnd of backlog you'd rather have the oldest characters get out to the remote machine as soon as possible rather than waiting for enough ACKs to build up a full MSS. However, there might also be cases where it's useful to fall back to throttling to full MSS in this case. If you were to implement my suggestion and the local sender started spamming a bunch of data, you could be stuck permamently sending small frames if you always responded to each sub-MSS ACK with an equivalently-sized frame. It may be that in the case of filling your cwnd, this is the more common case and that the better fix for my use case is to have a more accurate cwnd (and thus the TCP_IGNOREIDLE change). I have no idea what other stacks do in this regard. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201301250900.38871.jhb>