Date: Mon, 02 Sep 2013 01:02:31 +0400 From: "Alexander V. Chernikov" <melifaro@ipfw.ru> To: Adrian Chadd <adrian@freebsd.org> Cc: Jack F Vogel <jfv@freebsd.org>, "Justin T. Gibbs" <gibbs@freebsd.org>, Alan Somers <asomers@freebsd.org>, "net@freebsd.org" <net@freebsd.org> Subject: Re: Flow ID, LACP, and igb Message-ID: <5223AB67.60207@ipfw.ru> In-Reply-To: <CAJ-Vmo=%2B__t_8xMsVjtz_ZeKEDyQHRb_kDikEVJO%2Bj2iFZYsgw@mail.gmail.com> References: <D01A0CB2-B1E3-4F4B-97FA-4C821C0E3FD2@FreeBSD.org> <5222F661.1090601@ipfw.ru> <CAJ-Vmo=%2B__t_8xMsVjtz_ZeKEDyQHRb_kDikEVJO%2Bj2iFZYsgw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02.09.2013 00:45, Adrian Chadd wrote: > > > Not sure about igb, but ixgbe (according to advanced RX descriptor > format, 7.1.6.2 @ 82599 datasheet) can provide 'real' RSS value > which can be used in m_flowid instead of NIC queue id. > > (And, by the way, another RSS-related problem: there are cases when > setting flowid does more harm, for example - PPPoE frames always > being received at Q0. Partially this can be solved by analyzing RSS > type from the same RX descriptor format (e.g. don't set flowid for > RSS type 0x0), but there are other cases like GRE tunneling (where > you probably want to perform deeper inspection in SW). > > So, can we have some kind of per-NIC sysctl disabling setting > flowid on given port? > > (Yes, this should be in some kind of `ethtool` binary but we still > don't have it..) ) > > > What specifically are you asking for? Disabling the flowid tagging > of I'm talking about some small ixgbe (and maybe igb?) changes related to the generation of mbuf flowid. More specifically: 1) As far as I understand, ixgbe generates u16 hash which is then used to compute receive queue number. It seems that this value can be set by NIC in per-packet advanced RX descriptor, so it can be used as better flowid value (which should be optional). 2) There are cases when we shouldn't simply mark all packets as received by q0 (since NIC hash doesn't known how to hash them), so disabling setting m_flowid for given port can help a lot. > mbufs? Or changing the LACP hashing? It is not related to lagg directly. Actually, it is, but for the very special case like 'routing on the stick' when we're forwarding packets back to the same lagg interface. In this case, we can set (pre-computed) static RX queue flowids to force forwarder packet fall to the same NIC and the same TX queue id. This approach minimizes egress mutex contention (I forgot to mention patch implementing this stuff in my 'Network stack changes' letter). > > You can configure LACP to not use the flowid from the NIC and do > the hashing yourself. Yup, I'm aware of that :) > > > -adrian > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIjq2cACgkQwcJ4iSZ1q2ncAACfVmiVTtyno7hcxG59HZs8cSyq umwAnjQ6r4V3UCO8T0uE3gMZmeMveUMB =thS0 -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5223AB67.60207>