From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 8 18:12:47 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 490F2EF4; Tue, 8 Apr 2014 18:12:47 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D99410D6; Tue, 8 Apr 2014 18:12:47 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D2CEEB953; Tue, 8 Apr 2014 14:12:44 -0400 (EDT) From: John Baldwin To: Edward Tomasz =?iso-8859-2?q?Napiera=B3a?= Subject: Re: Multiple locks and missing wakeup. Date: Tue, 8 Apr 2014 14:12:10 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <0D69A6A8-43D1-41FB-8C2D-00F5CAD9C86E@FreeBSD.org> <201404081001.31219.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Message-Id: <201404081412.10066.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 08 Apr 2014 14:12:44 -0400 (EDT) Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 18:12:47 -0000 On Tuesday, April 08, 2014 12:45:49 pm Edward Tomasz Napiera=B3a wrote: > Wiadomo=B6=E6 napisana przez John Baldwin w dniu 8 kwi 2014, o godz. 16:0= 1: >=20 > > On Tuesday, April 08, 2014 2:34:30 am Edward Tomasz Napiera=B3a wrote: > >> Let's say I have a kernel thread processing elements from a queue, > >> sleeping until there is work to do; something like this: > >>=20 > >> mtx_lock(&mtx1); > >> for (;;) { > >> while (!LIST_EMPTY(&list1)) { > >> elt =3D LIST_FIRST(&list1); > >> do_stuff(elt); > >> LIST_REMOVE(&list1, elt); > >> } > >> sleep(&list1, &mtx1); > >> } > >> mtx_unlock(&mtx1); > >>=20 > >> Now, is there some way to make it work with two lists, protected > >> by different mutexes? The mutex part is crucial here; the whole > >> point of this is to reduce lock contention on one of the lists. The > >> following code would result in a missing wakeup: > >=20 > > All our sleep primitives in the kernel only accept a single wait channe= l. > > It sounds like you want something more like select() or poll() where you > > can specify multiple wait channels. There isn't a good way to do that > > currently. You could write one, but it would be a bit hard to do > > correctly. >=20 > Perhaps I should have been more clear: I'm ok with a single wait > channel. The problem is that there is no way to pass more than one > mutex to the sleep() function, so we can miss wakeup for the list > protected by the second lock, if something gets enqueued between > releasing mtx2 and calling sleep(). >=20 > > In practice you'd end up implementing something that boiled > > down to having a single wait channel with a common lock that protected > > it so you could do something like: >=20 > The whole purpose of this is to avoid locking mtx1 in the the enqueue > routine for the second list, for contention reasons. Ah, but note that I didn't lock mtx1 in the enqueue routine, I marked the 'combo_mtx' which is only used for the sleep/wakeup. > [..] >=20 > > Another way to do this would be to be a bit more poll-like (e.g. if > > you wanted a generic mechanism for this) where you have some sort of > > 'poller' structure and you set a flag before starting a scan of all > > your backends. Any wakeup that occurs while scanning clears the > > flag, and you only sleep if the flag is still set at the end of the > > scan, etc. >=20 > But the flag would have to be protected by the mutex we pass > to sleep(), and would require grabbing that mutex in both enqueue > routines, right? Yep. =2D-=20 John Baldwin