Date: Mon, 11 Aug 2014 10:06:06 -0700 From: John-Mark Gurney <jmg@funkthat.com> To: Vlad Zolotarov <vladz@cloudius-systems.com> Cc: freebsd-net@freebsd.org Subject: Re: TCP Rx window auto sizing relies on TCP timestamp option? Message-ID: <20140811170606.GV83475@funkthat.com> In-Reply-To: <53E8B424.2000904@cloudius-systems.com> References: <53E8B424.2000904@cloudius-systems.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Vlad Zolotarov wrote this message on Mon, Aug 11, 2014 at 15:16 +0300: > Hi, I have the most strange question about the TCP Rx window auto sizing > implementation in a FreeBSD networking stack. > When I looked at the FreeBSD code (hash > 9abce0e567c9a5a0520cdd94d5c633c7baf9a184) I noticed that > the mentioned above feature will not be "enabled" if there isn't a TCP > timestamp option present in the current TCP session: > > See sys/netinet/tcp_input.c: line 1813 in tcp_do_segment() function: > > if (V_tcp_do_autorcvbuf && > *to.to_tsecr* && <-------- this is what I'm > talking about > (so->so_rcv.sb_flags & SB_AUTOSIZE)) > > So, if i read the code correctly, if there isn't a TS option (negotiated > and thus present in every received packet) the receive socket buffer > won't grow thus preventing the growth of the Rx window. > If that's the case this is very strange since TS option is not promised > and even more - in many cases it won't be present. > For example in Linux this feature is disabled by default (controlled by > /proc/sys/net/ipv4/tcp_timestamps). > This is how I actually noticed the problem the first place: I ran iperf > test where Linux was an initiator and a transmitter (iperf -c) FreeBSD > box was a receiver (iperf -s) and I noticed that the Rx window wasn't > opening up because Linux box hasn't negotiated the TS option in the SYN. > As a result, the throughput numbers were significantly lower compared to > Linux-to-Linux setup (Linux uses a Dynamic Right-Sizing (DRS) algorithm > http://public.lanl.gov/radiant/pubs.html#DRS, which doesn't rely on TS). > > Could anybody comment on this, pls.? > Did I miss anything? > Is it true that FreeBSD assumes that TS option is always present and if > not how can I cause an Rx Window to open up when TS option hasn't been > negotiated? This means the receive buffer won't grow beyond the default of 64k... But, as the comment says: * On the receive side the socket buffer memory is only rarely * used to any significant extent. This allows us to be much The receive buffer will only get used if the application takes too long to read it's buffer, or it isn't currently waiting... If that's the case, then the application should be fixed to be able to process the data as quickly as it comes in... So, I don't see much of an issue w/ the code you pointed out, yes, the receive buffer won't grow, but there are options that you can set (sysctl net.inet.tcp.recvspace) and SO_RCVBUF in the application that will address it otherwise... Obviously setting the default too large will just waste memory... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140811170606.GV83475>