From owner-freebsd-questions@FreeBSD.ORG Tue Nov 11 03:00:51 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4ADABCDC; Tue, 11 Nov 2014 03:00:51 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10E6D23F; Tue, 11 Nov 2014 03:00:51 +0000 (UTC) Received: by mail-ig0-f177.google.com with SMTP id hl2so246641igb.16 for ; Mon, 10 Nov 2014 19:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=G5HBwQRVPIWPPMKfqNrE2zJ/k1S0hpnaIOeu7+GZYEc=; b=Jx/weeNVhJQpwD+K7uLEqLH+LFULYTiUHHHH3cUU82UcP4TIJwL4dgFI3fZw88kuAI +3N76QXH2wK/Q69z/S9alNquuITjkU6I76TQkxKWy4xA0faI+KcFNMOuzN8sEOq88O+D L8m/zRYmjhGJPdUgD5knRDr0w1nLur6r41MTfTI7N07PpCMW0w1rq75tUM5kV66RN/L6 wWA2ZpiDnmBReojD6yZZAtVWvrN2hH2+fU07VOJ9Oa3/Yr72+acHOmeXOM8VcYj1u0u8 RRkr/H3ne0GKVaApayvPMIphwO+eQ31a19xbiJizmoAFhf2pUbDRryXRSzs4bUimD/WF aHFw== MIME-Version: 1.0 X-Received: by 10.43.99.3 with SMTP id cq3mr4225318icc.49.1415674850388; Mon, 10 Nov 2014 19:00:50 -0800 (PST) Sender: jdavidlists@gmail.com Received: by 10.43.96.202 with HTTP; Mon, 10 Nov 2014 19:00:50 -0800 (PST) In-Reply-To: <20141110071353.GO24601@funkthat.com> References: <20141110071353.GO24601@funkthat.com> Date: Mon, 10 Nov 2014 22:00:50 -0500 X-Google-Sender-Auth: HoG40Do3LP_Nr6J8r9y8a_rEkQQ Message-ID: Subject: Re: How thread-friendly is kevent? From: J David To: J David , "freebsd-hackers@freebsd.org" , "freebsd-questions@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 03:00:51 -0000 On Mon, Nov 10, 2014 at 2:13 AM, John-Mark Gurney wrote: > you > really need to use one of _DISPATCH or _ONESHOT to ensure that the > event only gets delivered to a single thread.... That's what one would expect, which is why the observed behavior was so surprising. After increasing the testing load considerably, it did behave as expected (waking more than one thread for one event). But even so, the occurrences were very rare. It would wake up at most one "extra" thread in slightly less than 1 out of 100,000 events. > Though if you mean how many threads will be woken up in the kernel > and find that there are no events remaining as one of the other kernel > threads has delivered the event, then yes, I have looked at the code, > and there will be a thundering herd problem... Thanks for that, that's exactly the kind of information I was hoping to find. Is that something that can happen without any usermode-visible effects? I.e. all the threads wake up, but they almost all go back to sleep without leaving the kevent() syscall since they can see there's nothing to do anymore. If so, that would match the observed behavior, but could add up to a lot of hidden overhead. > And if you do, it would make more sense to > use the recent RSS work that Adrian has been working on, and have one > kq per CPU w/ the proper cpu binding for that set of sockets... The most recent information I was able to find: http://adrianchadd.blogspot.com/2014/10/more-rss-udp-tests-this-time-on-dell.html suggests that this work, while admirable and important, is quite some ways away from being production-stable for usermode code: "hopefully I can get my network / rss library up and running enough to prototype an RSS-aware memcached and see if it'll handle this particular workload." It's definitely something to keep an eye on, but probably not a viable approach for us right now. Thanks!