Date: Fri, 27 Sep 2013 21:47:17 +0200 From: Luigi Rizzo <rizzo@iet.unipi.it> To: chintu hetam <rometoroam@gmail.com> Cc: Michio Honda <micchie@outlook.com>, hiren panchasara <hiren.panchasara@gmail.com>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: netmap: traffic distribution Message-ID: <CA%2BhQ2%2Bi3%2B4Jps-jy_8EaM9PWBLAMUTg30JRH6mg=dZs2QgS-DA@mail.gmail.com> In-Reply-To: <CAOH8Hto7675LNvE56rrzdEog5%2BwGx2B5i4rYwSmF-uK-u6uwjA@mail.gmail.com> References: <CALCpEUFhfcZO9keQ_fpmCQhg5DVA5DNrG5Neogm5TV_QiAYPRw@mail.gmail.com> <CA%2BhQ2%2Bid6tEcmtRWu4%2BLFZDmthwto3rAdqrSyJrY-r%2BJx=vhhg@mail.gmail.com> <CALCpEUENg%2BVq0Z0Evpw=7VEBTLq_T2eZQ9UgSpsxUn5OOjd3eQ@mail.gmail.com> <CA%2BhQ2%2Bg4%2B2P_L8kLLL0Svky3xrvxJW7zjc07xSTgNknEqNvr0A@mail.gmail.com> <CALCpEUHLLfvakMzDJPL=tPym1d6m13Wgbid7NOWEoMxZ1%2B6kGg@mail.gmail.com> <BAY176-W227BF61C5551E2295DC646BC280@phx.gbl> <CALCpEUGjCvL5UEiXaOnBKaHSf%2B%2BHRGR4m-57SABGjEpiHmCZWA@mail.gmail.com> <CAOH8Htqz%2BU6AguXTJbdf9FWDGMvA=xUJFJVfJ0Tu-4X6mfbESw@mail.gmail.com> <CALCpEUGFA7vU1TV1dMFA2UAHkKn_giPDyOdH%2BdSR6zX=97sCuQ@mail.gmail.com> <CAOH8Hto7675LNvE56rrzdEog5%2BwGx2B5i4rYwSmF-uK-u6uwjA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Sep 27, 2013 at 8:16 PM, chintu hetam <rometoroam@gmail.com> wrote: > As far as i know, flow director is Intel terminology it addresses both RSS > and RFS. I think FreeBSD implementation is RFS. > > Luigi, you touched upon SW de-multiplexer, i would like to know why it's > necessary. the sw demux is only necessary if you expect to have traffic with heavy processing requirements for which you cannot easily guarantee that the RSS more or less balances the queues. Since the number of RSS queues is small, unbalance is not unlikely; in software you could in principle demux each flow to its own queue and then have one worker process per thread process queues. Note though that scheduling may become complicated if you care for CPU locality, but if the sw demux only reads buffers, then the situation is not too bad. > let say i have 82599 ixgbe driver (RSS enabled)configured with 5 tuple hash. > My application reads from netmap queue 0-7(1-8), for testing purpose i will > define tuples in such a way that i know for sure that each packet will be > filtered to specific hw queue(0-7). Now is it safe to assume that in netmap > mode in user space i will receive the packet as in ixgbe hw queue? yes. > > Michio, reason i asked for performance values > http://arxiv.org/ftp/arxiv/papers/1106/1106.0443.pdf The problem that the paper describes is due to the incorrect implementation (at least according to the description give in the paper) of the Flow Director (FD) feature in linux. The hardware demux (RSS or RFS) per se does not cause reordering unless it is misued (as described in the paper). Basically when FD is active, the RFS entry in the NIC is generated by looking at the core used for the transmit side of a flow, and (according to what the paper says; not sure if it has been fixed since), the association is updated dynamically and not generated just once at the beginning of the flow. So if the userspace thread floats between CPUs, the RFS entry keepfloating, and incoming packets for the same flow can be delivered to different queues. cheers luigi -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+-------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BhQ2%2Bi3%2B4Jps-jy_8EaM9PWBLAMUTg30JRH6mg=dZs2QgS-DA>