Date: Thu, 8 Sep 2011 20:13:50 -0400 From: George Neville-Neil <gnn@neville-neil.com> To: Navdeep Parhar <np@FreeBSD.ORG> Cc: Ben Hutchings <bhutchings@solarflare.com>, freebsd-net@freebsd.org, jfv@freebsd.org, John Baldwin <jhb@freebsd.org>, Takuya ASADA <syuu@dokukino.com> Subject: Re: Adding Flow Director sysctls to ixgbe(4) Message-ID: <37419C45-4436-4738-851B-2B765BC2C60F@neville-neil.com> In-Reply-To: <20110908184928.GA87872@hub.freebsd.org> References: <CALG4x-W99OZxd=1ZDvW4=MBqeE3RPOazc7jc_3O30X-Pou3k8Q@mail.gmail.com> <1315221674.3092.282.camel@deadeye> <201109080834.11607.jhb@freebsd.org> <20110908184928.GA87872@hub.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sep 8, 2011, at 14:49 , Navdeep Parhar wrote: > On Thu, Sep 08, 2011 at 08:34:11AM -0400, John Baldwin wrote: >> On Monday, September 05, 2011 7:21:12 am Ben Hutchings wrote: >>> On Mon, 2011-09-05 at 15:51 +0900, Takuya ASADA wrote: >>>> Hi, >>>>=20 >>>> I implemented Ethernet Flow Director sysctls to ixgbe(4), here's a = detail: >>>>=20 >>>> - Adding removing signature filter >>>> On linux version of ixgbe driver, it has ability to set/remove = perfect >>>> filter from userland using ethtool command. >>>> I implemented similar feature, but on sysctl, and not perfect = filter >>>> but signature filter(which means hash collision may occurs). >>> [...] >>>=20 >>> Linux also has a generic interface to RX filtering and hashing >>> (ethtool_rxnfc) which ixgbe supports; wouldn't it be better for = FreeBSD >>> to support something like that? >>=20 >> Some sort of shared interface might be nice. The cxgb(4) and = cxgbe(4) drivers >> both provide their own tools to manipulate filters, though they do = not >> provide explicit steering IIRC. >=20 > Both of them can filter as well as steer (and the tools let you do = that). > cxgbe(4) can do a lot more (rewrite + switch, replicate, etc.) but = those > features are perhaps too specialized to be configurable via a general > purpose tool. >=20 >>=20 >> We would need to come up with some sort of standard interface = (ioctls?) for=20 >> adding filters however. >=20 > +1 for a standard interface. >=20 > imho the kernel needs to be aware of the rx and tx queues of a NIC, = and > not just for steering. But that's a separate discussion. >=20 Well I do think this is actually all of a part. Most of us realize by = now that high speed (e.g. 10G and higher) NICs only make sense if you can steer = traffic and pin queues to cores etc. Without those pieces in place it's impossible = to get the full utilization out of the NIC because things like cache misses = just kill your performance. Having looked at a few 10G NICs in the last couple of = years the one thing I notice is that everyone wants to do things like offload and = steering but that the specifics are quite different between different cards. = Some of that has to do with the libraries that expose these things to the kernel and = user programs, but some has to do with how the device is modeled. What this means is = that we have a failure of abstraction. Abstraction has a cost, and some of the = people who want access to low level queues are not interested in paying an extra = abstraction cost. I think that some of the abstractions we need are tied up in the work = that Takuya did for SoC and some of it is in the work done by Luigi on netmap. I'd go = so far as to say that what we should do is try to combine those two pieces of code into a = set of low level APIs for programs to interact with high speed NICs. The one = thing most people do not talk about is extending our socket API to do two things = that I think would be a win for 80% of our users. If a socket, and also a kqueue, could be = pinned to a CPU as well as a NIC queue that should improve overall bandwidth = for a large number of our users. The API there is definitely an ioctl() and the = hard part is doing the tying together. To do this we need to also work out our low = level story. Best, George
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?37419C45-4436-4738-851B-2B765BC2C60F>