Date: Mon, 10 Nov 2014 21:12:05 -0800 From: Adrian Chadd <adrian@freebsd.org> To: J David <j.david.lists@gmail.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, "freebsd-questions@freebsd.org" <freebsd-questions@freebsd.org> Subject: Re: How thread-friendly is kevent? Message-ID: <CAJ-Vmo=nGu0ZpAuDrxH=A8PH0DBSYgAGOEm2aPLK5JJA_c%2By0Q@mail.gmail.com> In-Reply-To: <CABXB=RStLz6J9L3--KsM308-0h0N5ZeZZvw1GbDi%2BZvKO4U64g@mail.gmail.com> References: <CABXB=RQWxu-d30raZ%2BFcrnrGsr5gG2Za_=cx8-jCnLSgJDSF=Q@mail.gmail.com> <20141110071353.GO24601@funkthat.com> <CABXB=RStLz6J9L3--KsM308-0h0N5ZeZZvw1GbDi%2BZvKO4U64g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10 November 2014 19:00, J David <j.david.lists@gmail.com> wrote: > On Mon, Nov 10, 2014 at 2:13 AM, John-Mark Gurney <jmg@funkthat.com> wrote: >> you >> really need to use one of _DISPATCH or _ONESHOT to ensure that the >> event only gets delivered to a single thread.... > > That's what one would expect, which is why the observed behavior was > so surprising. After increasing the testing load considerably, it did > behave as expected (waking more than one thread for one event). But > even so, the occurrences were very rare. It would wake up at most one > "extra" thread in slightly less than 1 out of 100,000 events. > >> Though if you mean how many threads will be woken up in the kernel >> and find that there are no events remaining as one of the other kernel >> threads has delivered the event, then yes, I have looked at the code, >> and there will be a thundering herd problem... > > Thanks for that, that's exactly the kind of information I was hoping to find. > > Is that something that can happen without any usermode-visible > effects? I.e. all the threads wake up, but they almost all go back to > sleep without leaving the kevent() syscall since they can see there's > nothing to do anymore. If so, that would match the observed behavior, > but could add up to a lot of hidden overhead. > >> And if you do, it would make more sense to >> use the recent RSS work that Adrian has been working on, and have one >> kq per CPU w/ the proper cpu binding for that set of sockets... > > The most recent information I was able to find: > > http://adrianchadd.blogspot.com/2014/10/more-rss-udp-tests-this-time-on-dell.html > > suggests that this work, while admirable and important, is quite some > ways away from being production-stable for usermode code: > > "hopefully I can get my network / rss library up and running enough to > prototype an RSS-aware memcached and see if it'll handle this > particular workload." > > It's definitely something to keep an eye on, but probably not a viable > approach for us right now. Why? It's likely going to be heading in the direction that you'd want it to for scaling networking workloads and I could do with more users / feedback on it. (Ie, you should totally use it, because this is the direction things are likely going to head in the future, and you right now get to help shape its direction. :) -adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=nGu0ZpAuDrxH=A8PH0DBSYgAGOEm2aPLK5JJA_c%2By0Q>