Date: Tue, 27 Jun 2017 12:05:06 +0000 From: "Youssef GHORBAL" <youssef.ghorbal@pasteur.fr> To: "sthaug@nethelp.no" <sthaug@nethelp.no> Cc: "matt.joras@gmail.com" <matt.joras@gmail.com>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, "nparhar@gmail.com" <nparhar@gmail.com> Subject: Re: Sporadic TCP/RST sent to client Message-ID: <CB4FA7F4-6DAC-4231-8CE1-AB83525F0F23@pasteur.fr> In-Reply-To: <20170627.125426.74697078.sthaug@nethelp.no> References: <CAPFoGT8sthFOm=vOiFb9%2B2BM=%2BBqtREz1SrOm_UiVge81CrYrw@mail.gmail.com> <CADdTf%2BgeCy4naC5jJ6UhTm7-n9c6XKpgRs96EsXGq-UVjSn8MQ@mail.gmail.com> <5ABA962E-A90A-4C25-A5A7-EE5CF66FFDD4@pasteur.fr> <20170627.125426.74697078.sthaug@nethelp.no>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 27 Jun 2017, at 12:54, sthaug@nethelp.no wrote: >=20 >> Imagine this set up : >>=20 >> freebsd host port0 <-> switch 1 <-> linux host port0 >> freebsd host port1 <-> switch 2 <-> linux host port1 >>=20 >> On the linux box, port 0&1 are enslaved in a bond with a RR algorithm (R= ound Robin) >> On the freebsd box, port 0&1 are enslaved in a lagg. >>=20 >> switchs 1&2 are configured for doing MLAG. >>=20 >> The Linux box disapatchs packets on both NICs (since the RR algo dictate= s that) packets are dispatched in order. >> Packets outgoing on port0 gets handled by switch1 and hits the freebsd b= ox on port 0 >> Packets outgoing on port1 gets handled by switch2 and hits the freebsd b= ox on port 1 >>=20 >> As I stated earlier, from the tcpdump traces I've done on the freebsd bo= x (both on the lagg interface and the actual ports) packets do arrive order= ed but on different NICs and it works great until the elapes times start to= be around microsecond. >>=20 >> I don't really have control over the Linux box to make them use other ha= sh algo (but I'm stil trying) >=20 > If the Linux box is using round robin you shouldn't expect to be able > to "fix" the problem at the FreeBSD end. There is nothing in the 802.3ad that mandates stickiness of flows per NIC, = the only thing explicit is that hash algorithm needs to maintain packet ord= er. In this case, strictly speaking, it's : Packets do leave in "order" and= do arrive in "order". > On routers and switches (which is what I normally work with) the hash > algorithm used for LAG connections ensures that one "flow" always uses > the same path, thus no reordering. A typical hash algorithm uses a > 5-tuple with (src ip, src port, dst ip, dst port, protocol) as input. >=20 > So the advice in this case is simple - don't use round robin! Yes, I > understand you don't control the Linux box. Sure, I was just wondering if the FreeBSD network stack was built with the = fact that each flow needs to arrive on the same NIC and the system was desi= gned with this assumption in mind or not. I reported it here, thinking that maybe it's a subtle buggy corner case and= maybe the community was interesting to know about and maybe fix : - If the stack is working as expected and was built with the assumption tha= t each incoming flow needs to stick to a NIC during it's lifetime, maybe do= cumentation needs to be more explicit regarding this situation. In that cas= e I'll file documentation enhancement bug report. - If the stack is misbehaving, maybe help the community identify the root c= ause and help fixing it Youssef
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CB4FA7F4-6DAC-4231-8CE1-AB83525F0F23>