Date: Mon, 23 Jun 2008 09:42:42 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: =?ISO-8859-15?Q?Stefan_E=DFer?= <se@freebsd.org> Cc: Jerahmy Pocott <quakenet1@optusnet.com.au>, freebsd-stable@freebsd.org Subject: Re: Sysctl knob(s) to set TCP 'nagle' time-out? Message-ID: <200806231642.m5NGggV8089156@apollo.backplane.com> References: <0222EAC1-A278-41D2-9566-C9CF19811068@optusnet.com.au> <200806230827.m5N8RBlW085475@apollo.backplane.com> <485F7576.5070104@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
:One possibility I see is a statistic about DelACKs per TCP connection,
:counting those that were rightfully delayed (with hindsight). I.e.,
:if an ACK is delayed, but there was no chance to piggy-back it or to
:combine it with another ACK, it could have been sent without delay.
:Only those delayed ACKs that reduce load are "good", all others cause
:additional state to be maintained and may increase latencies for no
:good reason.
:
:...
:consideration. And to me, automatic setting of TCP_NODELAY seems
:more useful than automatic clearing (after delayed ACKs had been
:found to be of no use for a window of say 8 or 16 ACKs).
:
:The implementation would be quite simple: Whenever a delayed ACK
:is sent, check whether it is sent on its own (bad) or whether it
:could be piggy-backed (good). If, say, 7 of 8 delayed ACKs had to
:be sent as ACK-only packets, anyway, set TCP_NODELAY and do not
:bother to keep on deciding whether delayed ACKs had become useful
:in a different phase of the communication. If you want to be able
:to automatically disable TCP_NODELAY, then just set a time-stamp
:...
:Regards, STefan
That's an interesting approach. I think it would catch some
of the cases, but not enough of them. If the round-trip in
the server-relaying case is less then the delayed-ack, the acks
will still wind up piggy-backed on return traffic but the latency
will also still remain horrible.
It should be noted that Nagle can cause high latencies even when
delayed acks are turned off. Nagle's delay is not timed... in its
simplest description it prevents packets from being transmitted
for new data coming from userland if the data already in the
sockbuf (and presumably already transmitted) has not yet been
acknowledged.
For interactive traffic this means that Nagle is putting the screws
on the packet stream even if the acks aren't delayed, simply from the
ack latency. With delayed acks turned off the latency is lower, but
not 0, so interactive traffic is still being held up by Nagle. The
effect is noticeable even on a LAN. Jerahmy brought up Samba... that
is an excellent example. NFS-over-TCP would be another good example.
Any protocol which multiplexes multiple commands from different
sources over the same connection gets really messed up (slowed down)
by Nagle.
On the flip side, Nagle can't just be turned off by default because
it would cause streaming connections from user programs which do tiny
writes to generate a lot of unnecessarily tiny packets. This can become
apparent when using SSH over a slow link. Numerous programs run from
a shell generate fairly ineffcient packets which could have easily
been batched when operating over SSH. The result can be sludgy
performance for output which ought be batched up by TCP but isn't because
SSH turns off Nagle unconditionally.
-Matt
Matthew Dillon
<dillon@backplane.com>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200806231642.m5NGggV8089156>