From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 18:08:26 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB7D716A418; Tue, 21 Aug 2007 18:08:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from speedfactory.net (mail6.speedfactory.net [66.23.216.219]) by mx1.freebsd.org (Postfix) with ESMTP id 4EDEF13C4E9; Tue, 21 Aug 2007 18:08:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (unverified [66.23.211.162]) by speedfactory.net (SurgeMail 3.8k) with ESMTP id 204501075-1834499 for multiple; Tue, 21 Aug 2007 14:08:35 -0400 Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id l7LI8A1P000145; Tue, 21 Aug 2007 14:08:11 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 21 Aug 2007 14:03:28 -0400 User-Agent: KMail/1.9.6 References: <20070818120056.GA6498@garage.freebsd.pl> <20070818155041.GY90381@elvis.mu.org> <20070818161449.GE6498@garage.freebsd.pl> In-Reply-To: <20070818161449.GE6498@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200708211403.29293.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 21 Aug 2007 14:08:11 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/4021/Tue Aug 21 11:42:43 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Alfred Perlstein , Pawel Jakub Dawidek Subject: Re: Lockless uidinfo. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2007 18:08:26 -0000 On Saturday 18 August 2007 12:14:49 pm Pawel Jakub Dawidek wrote: > On Sat, Aug 18, 2007 at 08:50:41AM -0700, Alfred Perlstein wrote: > > * Pawel Jakub Dawidek [070818 07:59] wrote: > > > Yes, to lookup uidinfo you need to hold uihashtbl_mtx mutex, so once you > > > hold it and ui_ref is 0, noone will be able to reference it, because it > > > has to wait to look it up. > > > > And the field doesn't need to be volatile to prevent cached/opportunitic > > reads? > > The only chance of something like this will be the scenario below: > > thread1 (uifind) thread2 (uifree) > ---------------- ---------------- > refcount_release(&uip->ui_ref)) > /* ui_ref == 0 */ > mtx_lock(&uihashtbl_mtx); > refcount_acquire(&uip->ui_ref); > /* ui_ref == 1 */ > mtx_unlock(&uihashtbl_mtx); > mtx_lock(&uihashtbl_mtx); > if (uip->ui_ref > 0) { > mtx_unlock(&uihashtbl_mtx); > return; > } > > Now, you suggest that ui_ref in 'if (uip->ui_ref > 0)' may still have > cached 0? I don't think it is possible, first refcount_acquire() uses > read memory bariers (but we may still need ui_ref to volatile for this > to make any difference) and second, think of ui_ref as a field protected > by uihashtbl_mtx mutex in this very case. > > Is my thinking correct? Memory barriers on another CPU don't mean anything about the CPU thread 2 is on. Memory barriers do not flush caches on other CPUs, etc. Normally when objects are refcounted in a table, the table holds a reference on the object, but that doesn't seem to be the case here. Have you tried doing something very simple in uifree(): { mtx_lock(&uihashtbl_mtx); if (refcount_release(...)) { LIST_REMOVE(); mtx_unlock(&uihashtbl_mtx); ... free(); } else mtx_unlock(&uihashtbl_mtx); } I wouldn't use a more complex algo in uifree() unless the simple one is shown to perform badly. Needless complexity is a hindrance to future maintenance. Also, even if you do go with the more complex route, I'd rather you reduce diffs with the current code by keeping the test as 'uip->ui_ref == 0' and keeping the removal code in the if-block. In chgproccnt() you should use atomic_fetchadd_long() to avoid a race when reading ui_proccnt. old = atomic_fetchadd_long(&uip->ui_proccnt, diff); if (old + diff < 0) printf("...."); OTOH, atomic_fetchadd_long() doesn't yet exist, so you will need to fix that, or just always use an atomic_cmpset() loop. -- John Baldwin