Date: Thu, 7 Jun 2018 18:57:40 -0700 From: hiren panchasara <hiren@strugglingcoder.info> To: Randall Stewart <rrs@netflix.com> Cc: Randall Ray Stewart <rrs@FreeBSD.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r334804 - in head/sys: kern modules/tcp modules/tcp/rack netinet netinet/tcp_stacks sys Message-ID: <20180608015740.GD62424@strugglingcoder.info> In-Reply-To: <40CCC8B4-5667-4D4B-A4B4-7AD5779FE011@netflix.com> References: <201806071818.w57IIENp080093@repo.freebsd.org> <20180607220121.GC62424@strugglingcoder.info> <40CCC8B4-5667-4D4B-A4B4-7AD5779FE011@netflix.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--jL2BoiuKMElzg3CS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 06/07/18 at 08:58P, Randall Stewart wrote: >=20 >=20 > > On Jun 7, 2018, at 6:01 PM, hiren panchasara <hiren@strugglingcoder.inf= o> wrote: > >=20 > > On 06/07/18 at 06:18P, Randall Stewart wrote: > >> Author: rrs > >> Date: Thu Jun 7 18:18:13 2018 > >> New Revision: 334804 > >> URL: https://svnweb.freebsd.org/changeset/base/334804 > >>=20 > >> Log: > >> This commit brings in a new refactored TCP stack called Rack. > >> Rack includes the following features: > >> - A different SACK processing scheme (the old sack structures are no= t used). > >> - RACK (Recent acknowledgment) where counting dup-acks is no longer = done > >> instead time is used to knwo when to retransmit. (see the I-D) > >> - TLP (Tail Loss Probe) where we will probe for tail-losses to attem= pt > >> to try not to take a retransmit time-out. (see the I-D) > >> - Burst mitigation using TCPHTPS > >> - PRR (partial rate reduction) see the RFC. > >>=20 > >> Once built into your kernel, you can select this stack by either > >> socket option with the name of the stack is "rack" or by setting > >> the global sysctl so the default is rack. > >>=20 > >> Note that any connection that does not support SACK will be kicked > >> back to the "default" base FreeBSD stack (currently known as "defaul= t"). > >>=20 > >> To build this into your kernel you will need to enable in your > >> kernel: > >> makeoptions WITH_EXTRA_TCP_STACKS=3D1 > >> options TCPHPTS > >>=20 > >> Sponsored by: Netflix Inc. > >> Differential Revision: https://reviews.freebsd.org/D15525 > >>=20 > >> Added: > >> head/sys/modules/tcp/rack/ > >> head/sys/modules/tcp/rack/Makefile (contents, props changed) > >> head/sys/netinet/tcp_stacks/rack.c (contents, props changed) > >> head/sys/netinet/tcp_stacks/rack_bbr_common.h (contents, props chan= ged) > >> head/sys/netinet/tcp_stacks/sack_filter.c (contents, props changed) > >> head/sys/netinet/tcp_stacks/sack_filter.h (contents, props changed) > >> head/sys/netinet/tcp_stacks/tcp_rack.h (contents, props changed) > >> Modified: > >> head/sys/kern/uipc_sockbuf.c > >> head/sys/modules/tcp/Makefile > >> head/sys/netinet/tcp.h > >> head/sys/netinet/tcp_log_buf.h > >> head/sys/netinet/tcp_output.c > >> head/sys/netinet/tcp_stacks/fastpath.c > >> head/sys/netinet/tcp_timer.c > >> head/sys/netinet/tcp_timer.h > >> head/sys/netinet/tcp_var.h > >> head/sys/sys/mbuf.h > >> head/sys/sys/queue.h > >> head/sys/sys/sockbuf.h > >> head/sys/sys/time.h > >=20 > > I thought we'd have more time to review/test this. Looks like BSDCan > > commit-spree in effect. :-) >=20 > The Phabricator review has been up since May 22nd. Thats over 2.5 weeks, > this was also discussed on the Thursday conference calls. Fair enough. (I am out of touch a little so shouldn't really complain ;-)) > >=20 > > A few questions: > > 1) Does RACK work reliably without HPTS? If yes, has that config been > > tested? > >=20 > No it requires the pacer. >=20 > > 2) It looks like PRR is tied to RACK. Why did we go that route? > > Shouldn't it be easily used with the 'default' stack also? > >=20 >=20 > It is what I developed.. and I had no desire to work with the default sta= ck. That > is a fifth rail that no one wants touched. >=20 > > 3) Can new SACK be used with the traditional stack? >=20 > Well if you want to rework the base stack you might be able to do that :) >=20 > It would be quite some effort.. I think Robert wants eventually the old > stack to be de-composed and then slowly work at getting more common > code between them until eventually you can have a diff and somehow > figure out how to integrate the two. Okay. I believe there might be situations where we may want to still keep the 'default' stack alive. I know Windows doesn't yet use RACK when rtt is lesser than 10ms (or something like that), as an example. Such optimizations (PRR, better SACK) should be made available to that also if we see non-RACK having a viable future. >=20 > >=20 > > 4) Where should manpage like info for RACK go? a new man-page or > > extending tcp(4)? Info like how to enable system-wide or per socket > > should go here. > >=20 >=20 > The enable/disable or per-socket I think is in with the pluggable stack > stuff. We might want a Rack man page.. have to think about it. >=20 > > 5) Any perf numbers to go along with this commit? Synthetic or > > production numbers showing improvements in transfer speed or any other > > impact on CPU usage (specially with HPTS) that you can share? > >=20 >=20 > CPU will be more but we see close to a drop in rebuffers by about 12% I a= m told. Great. Thanks. >=20 > > 6) In your testing, have you found cases where RACK does poorly compared > > to the 'default' stack? Any recommendations on when should RACK be > > enabled? (Something like this could go in the manpage.) >=20 > Nope.=20 Okay. I assuming the 2 places this got tested at (NFLX and LLNW) would have provided enough variety in terms of traffic types. Thanks again, Hiren --jL2BoiuKMElzg3CS Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJbGeKRXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/ljPsH/ilifGCroYcPYnC/NNauw9Dy qtb16sdkFURaVX2qktda8QOFQoA6NuiEx8g1DOA88JwaFkWAS6xArNX8FahD5LmJ z/0OeX/PZZY7SBI+9sGXxa1/SD+RaOR+FfTFbmz5fZlnuBn7gqJvItiX/koVSVBn +Zg8CarG4YhBA8eKumAfOxFaM/LOkI5lmb1F25vU7vsJ9iGiNrbukdoK4t8bY1Va WXwj5Wj6cZv0o5b0AiY5LVfZxg0+Ka0pjHVAsVonYce3WFAF0TS34NPpztOSnUKp nLztOBPzhQpLpPUzdK/+dqyBjytBBBpwTUzwGjDtLFQS3iQo295cdN2b/R857OM= =9wYv -----END PGP SIGNATURE----- --jL2BoiuKMElzg3CS--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180608015740.GD62424>