Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 7 Jun 2018 20:58:21 -0400
From:      Randall Stewart <rrs@netflix.com>
To:        hiren panchasara <hiren@strugglingcoder.info>
Cc:        Randall Ray Stewart <rrs@FreeBSD.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org
Subject:   Re: svn commit: r334804 - in head/sys: kern modules/tcp modules/tcp/rack netinet netinet/tcp_stacks sys
Message-ID:  <40CCC8B4-5667-4D4B-A4B4-7AD5779FE011@netflix.com>
In-Reply-To: <20180607220121.GC62424@strugglingcoder.info>
References:  <201806071818.w57IIENp080093@repo.freebsd.org> <20180607220121.GC62424@strugglingcoder.info>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help


> On Jun 7, 2018, at 6:01 PM, hiren panchasara =
<hiren@strugglingcoder.info> wrote:
>=20
> On 06/07/18 at 06:18P, Randall Stewart wrote:
>> Author: rrs
>> Date: Thu Jun  7 18:18:13 2018
>> New Revision: 334804
>> URL: https://svnweb.freebsd.org/changeset/base/334804
>>=20
>> Log:
>>  This commit brings in a new refactored TCP stack called Rack.
>>  Rack includes the following features:
>>   - A different SACK processing scheme (the old sack structures are =
not used).
>>   - RACK (Recent acknowledgment) where counting dup-acks is no longer =
done
>>          instead time is used to knwo when to retransmit. (see the =
I-D)
>>   - TLP (Tail Loss Probe) where we will probe for tail-losses to =
attempt
>>          to try not to take a retransmit time-out. (see the I-D)
>>   - Burst mitigation using TCPHTPS
>>   - PRR (partial rate reduction) see the RFC.
>>=20
>>  Once built into your kernel, you can select this stack by either
>>  socket option with the name of the stack is "rack" or by setting
>>  the global sysctl so the default is rack.
>>=20
>>  Note that any connection that does not support SACK will be kicked
>>  back to the "default" base  FreeBSD stack (currently known as =
"default").
>>=20
>>  To build this into your kernel you will need to enable in your
>>  kernel:
>>     makeoptions WITH_EXTRA_TCP_STACKS=3D1
>>     options TCPHPTS
>>=20
>>  Sponsored by:	Netflix Inc.
>>  Differential Revision:		=
https://reviews.freebsd.org/D15525
>>=20
>> Added:
>>  head/sys/modules/tcp/rack/
>>  head/sys/modules/tcp/rack/Makefile   (contents, props changed)
>>  head/sys/netinet/tcp_stacks/rack.c   (contents, props changed)
>>  head/sys/netinet/tcp_stacks/rack_bbr_common.h   (contents, props =
changed)
>>  head/sys/netinet/tcp_stacks/sack_filter.c   (contents, props =
changed)
>>  head/sys/netinet/tcp_stacks/sack_filter.h   (contents, props =
changed)
>>  head/sys/netinet/tcp_stacks/tcp_rack.h   (contents, props changed)
>> Modified:
>>  head/sys/kern/uipc_sockbuf.c
>>  head/sys/modules/tcp/Makefile
>>  head/sys/netinet/tcp.h
>>  head/sys/netinet/tcp_log_buf.h
>>  head/sys/netinet/tcp_output.c
>>  head/sys/netinet/tcp_stacks/fastpath.c
>>  head/sys/netinet/tcp_timer.c
>>  head/sys/netinet/tcp_timer.h
>>  head/sys/netinet/tcp_var.h
>>  head/sys/sys/mbuf.h
>>  head/sys/sys/queue.h
>>  head/sys/sys/sockbuf.h
>>  head/sys/sys/time.h
>=20
> I thought we'd have more time to review/test this. Looks like BSDCan
> commit-spree in effect. :-)

The Phabricator review has been up since May 22nd. Thats over 2.5 weeks,
this was also discussed on the Thursday conference calls.
>=20
> A few questions:
> 1) Does RACK work reliably without HPTS? If yes, has that config been
> tested?
>=20
No it requires the pacer.

> 2) It looks like PRR is tied to RACK. Why did we go that route?
> Shouldn't it be easily used with the 'default' stack also?
>=20

It is what I developed.. and I had no desire to work with the default =
stack. That
is a fifth rail that no one wants touched.

> 3) Can new SACK be used with the traditional stack?

Well if you want to rework the base stack you might be able to do that =
:)

It would be quite some effort.. I think Robert wants eventually the old
stack to be de-composed and then slowly work at getting more common
code between them until eventually you can have a diff and somehow
figure out how to integrate the two.

>=20
> 4) Where should manpage like info for RACK go? a new man-page or
> extending tcp(4)? Info like how to enable system-wide or per socket
> should go here.
>=20

The enable/disable or per-socket I think is in with the pluggable stack
stuff. We might want a Rack man page.. have to think about it.



> 5) Any perf numbers to go along with this commit? Synthetic or
> production numbers showing improvements in transfer speed or any other
> impact on CPU usage (specially with HPTS) that you can share?
>=20

CPU will be more but we see close to a drop in rebuffers by about 12% I =
am told.

> 6) In your testing, have you found cases where RACK does poorly =
compared
> to the 'default' stack? Any recommendations on when should RACK be
> enabled? (Something like this could go in the manpage.)

Nope.=20

R

>=20
> Glad to finally see this in -head!
>=20
> Cheers,
> Hiren

--------
Randall Stewart
rrs@netflix.com
803-317-4952








Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?40CCC8B4-5667-4D4B-A4B4-7AD5779FE011>