From owner-freebsd-amd64@FreeBSD.ORG Mon Jul 8 21:26:09 2013 Return-Path: Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DB49F70E; Mon, 8 Jul 2013 21:26:09 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) by mx1.freebsd.org (Postfix) with ESMTP id 871441A11; Mon, 8 Jul 2013 21:26:09 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E461AB94B; Mon, 8 Jul 2013 17:26:08 -0400 (EDT) From: John Baldwin To: freebsd-amd64@freebsd.org Subject: Re: Accessing struct pthread from kernel Date: Mon, 8 Jul 2013 14:56:54 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <20130707213458.GH91021@kib.kiev.ua> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201307081456.54944.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 08 Jul 2013 17:26:09 -0400 (EDT) Cc: Davide Italiano , freebsd-threads@freebsd.org X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 21:26:09 -0000 On Sunday, July 07, 2013 6:22:18 pm Davide Italiano wrote: > On Sun, Jul 7, 2013 at 2:34 PM, Konstantin Belousov wrote: > > On Sat, Jul 06, 2013 at 01:22:05AM +0200, Davide Italiano wrote: > >> Hi, > >> as a preliminary step in the implementation of adaptive spinning for > >> umtx, I'm switching the pthread/umtx code so that a thread that > >> acquires a pthread_mutex writes the address of struct pthread in the > >> owner field of the lock instead of the thread id (tid). This is > >> because having struct pthread pointer allows easily to access > >> informations of the thread, and makes easy to get the state of the > >> thread exported from the kernel (once this will be implemented). > >> > >> For what concerns the libthr side, the internal function > >> _get_curthread() goes into the TLS to obtain the struct field of > >> curthread, so I'm done. > >> OTOH, I'm quite unsure instead about how to get address of struct > >> pthread for curthread from the kernel side (for example, in > >> do_lock_umutex() sys/kern/kern_umtx.c). > > You should not, see below. > > > >> > >> I guess I need to write some MD code because the TLS is different on > >> the various architecture supported in FreeBSD, and as a first step I > >> focused on amd64. > >> It looks like from the SDM that %%fs register points to the base of > >> the TLS, so I think that accessing using curthread->td_pcb->pcb_fsbase > >> (and then adding the proper offset to acces the right field) is a > >> viable solution to do this. Am I correct? > >> In particular what worries me is if the read of 'struct pthread' for > >> curthread from the TLS register is atomic with respect to preemptions. > >> > >> Alternatively, is there an (easier way) to accomplish this task? > > > > Coupling the libthr thread structure and kernel makes the ABI cast in > > stone and avoids most possibilities of changing the libthr internals. > > The same is true for kernel accessing the userspace TLS area of the thread. > > > > If you want kernel<->usermode communication of the thread run state, > > and possibly also a way to advisory prevent a preemption of the > > spinning usermode thread, you should create a dedicated parameter block > > communicated from usermode to kernel on thread creation. For the main > > thread, the block could be allocated by kernel by image activator, > > placed on the stack and its address passed to the usermode by auxv. > > > > Note that you cannot access the usermode from the context switch > > code. Wiring the corresponding page is very wasteful (think about a > > process with 10,000 threads) and still does not provide any guarantees > > since usermode can unmap or remap the range. You might see my 'fast > > sigprocmask' patches where somewhat similar idea was implemented. > > > > I think the tecnique you used for sigprocmask is neat and could be > reused for sharing thread state between kernel and userland, with some > modifications, thanks. > > That said, the problem I faced out before was slightly different. > In order to implement adaptive spinning 'efficiently'[1], threads > waiting for a lock held by some other thread should be able to access > easily the state owner. If I understand the kernel locking code > properly, to accomplish this, once a thread acquire a lock it writes > the address of its struct thread in the owner field of the lock, so > that other threads can easily access to his state. > > Now, it looks like applying such a tecnique for userspace is just > impossible for the reasons you mentioned in your previous mail. > The two alternatives that came up to the top of my mind are: > 1) maintain an hashtable that keep track of the mapping > tid->curthread, so that other thread perform a lookup on the hash > table to get curthread access. If you have a good hash, this should be close to O(1) in the common case, so the lookup shouldn't be bad. However, you have to handle the race of a thread being added or removed to the table while you are spinning, and that seems more problematic / expensive. You only have to do a best effort though, so you could perhaps never remove items from the table (just mark them as dead somehow, though even that isn't completely safe). > 2) having an array indexed by tid where the state is stuck so that > other threads can check what is the state of the lock owner. This might not be a bad idea, though I'd possibly make it a hash table. It handles the remaining race from above since an exiting thread can just tag its tid with a "dead" state and you don't have to worry about threads trying to deref a free'd pointer. -- John Baldwin