From owner-freebsd-threads@FreeBSD.ORG Mon Jun 19 11:03:16 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CE6CB16A6E5 for ; Mon, 19 Jun 2006 11:03:16 +0000 (UTC) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 85E1E43D48 for ; Mon, 19 Jun 2006 11:03:16 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k5JB3G13064367 for ; Mon, 19 Jun 2006 11:03:16 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k5JB3FWX064363 for freebsd-threads@freebsd.org; Mon, 19 Jun 2006 11:03:15 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 19 Jun 2006 11:03:15 GMT Message-Id: <200606191103.k5JB3FWX064363@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 11:03:17 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- s [2005/01/26] threads/76690threads fork hang in child for (-lc_r & -lthr) 1 problem total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can s [2001/01/20] threads/24472threads libc_r does not honor SO_SNDTIMEO/SO_RCVT s [2001/01/25] threads/24632threads libc_r delicate deviation from libc in ha s [2001/11/26] bin/32295 threads pthread dont dequeue signals s [2002/02/01] threads/34536threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe s [2002/06/27] threads/39922threads [threads] [patch] Threaded applications e s [2003/03/02] threads/48856threads Setting SIGCHLD to SIG_IGN still leaves z s [2003/03/10] threads/49087threads Signals lost in programs linked with libc s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/08/26] threads/70975threads unexpected and unreliable behaviour when o [2004/10/05] threads/72353threads Assertion fails in /usr/src/lib/libpthrea o [2004/10/07] threads/72429threads threads blocked in stdio (fgets, etc) are o [2004/10/21] threads/72953threads fork() unblocks blocked signals w/o PTHRE o [2004/12/19] threads/75273threads FBSD 5.3 libpthread (KSE) bug o [2004/12/21] threads/75374threads pthread_kill() ignores SA_SIGINFO flag s [2005/01/26] threads/76694threads fork cause hang in dup()/close() function o [2005/04/08] threads/79683threads svctcp_create() fails if multiple threads o [2005/04/28] threads/80435threads panic on high loads o [2005/05/19] threads/81258threads Thread specific data is sometimes assigne o [2005/07/22] threads/83914threads [libc] popen() doesn't work in static thr s [2005/08/02] threads/84483threads problems with devel/nspr and -lc_r on 4.x o [2005/08/20] threads/85160threads [libthr] [patch] libobjc + libpthread/lib p [2005/11/19] threads/89262threads [kernel] [patch] multi-threaded process h o [2005/12/12] threads/90278threads libthr, ULE and -current produces >100% W o [2006/01/03] kern/91266 threads [threads] Trying sleep, but thread marked s [2006/03/15] threads/94467threads send(), sendto() and sendmsg() are not co o [2006/06/01] threads/98256threads gnome-system-monitor core dumps from pthr 28 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- s [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything s [2000/10/21] kern/22190 threads A threaded read(2) from a socketpair(2) f s [2001/09/09] threads/30464threads pthread mutex attributes -- pshared s [2002/05/02] threads/37676threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] threads/40671threads pthread_cancel doesn't remove thread from s [2004/07/13] threads/69020threads pthreads library leaks _gc_mutex o [2004/09/21] threads/71966threads Mlnet Core Dumped : Fatal error '_pq_inse o [2004/11/21] threads/74180threads KSE problem. Applications those riched ma o [2005/04/13] threads/79887threads [patch] freopen() isn't thread-safe o [2005/05/13] threads/80992threads abort() sometimes not caught by gdb depen o [2005/05/26] threads/81534threads [libc_r] [patch] libc_r close() will fail 11 problems total. From owner-freebsd-threads@FreeBSD.ORG Tue Jun 20 12:10:01 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8422016A47E; Tue, 20 Jun 2006 12:10:01 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.10.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 139F043D6D; Tue, 20 Jun 2006 12:09:54 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.7/8.13.7) with ESMTP id k5KC9mdQ008494 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Tue, 20 Jun 2006 14:09:49 +0200 (CEST) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.7/8.13.3/Submit) id k5KC9mEK008493; Tue, 20 Jun 2006 14:09:48 +0200 (CEST) Date: Tue, 20 Jun 2006 14:09:48 +0200 From: Divacky Roman To: hackers@freebsd.org Message-ID: <20060620120948.GA8288@stud.fit.vutbr.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2i X-Scanned-By: MIMEDefang 2.54 on 147.229.10.14 Cc: threads@freebsd.org Subject: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 12:10:01 -0000 Hi I am student working on SoC project - extending linuxolator, now I am working on implementing linux TLS in FreeBSD. Here is what I think/know and I like you to comment on this, thnx. Roman ------------------------------------------------- Linux and FreeBSD TLS implementation - comparison Both systems use per-thread setting of where is the tls area stored. This setting is loaded into active threads GDT and can be accessed via %gs register. This GDT setup is done on every context switch. Linux uses strict 1:1 threading so every thread is in fact process, so thread creation is done using plain clone()/fork(). FreeBSD uses M:N (including 1:1) threading. Threads are created via pthread_create() call to threading library. In kernel there's thr_new() syscall or thread_create() syscall. I didnt find the connection between threading library and kernel but I assume its using one of the syscalls For setting up the GDT for the thread Linux uses syscall set_thread_area() (TODO - how exactly? its unclear what it does). I dont know how FreeBSD does it but I think it might be done via params to the syscalls (TODO - how is it done?) Remaining questions: clone() - 2.6.x glibc fork() implementation uses clone() syscall. is it supposed to create a thread or just a process? I think its process but why is the binary (ls, date and probably some other) linked to pthread library? is it just Linux "strangeness"? I dont see a reason for ls to be threaded... does anyone see? set/get tid - does it relate to TLS at all? I dont think so but you never know. The tid thing is unclear to me. The clone() syscall is passed CLONE_CHILD_SETTID & CLONE_CHILD_CLEARTID which should be mutually exclusive. I dont believe much its a mistake.. but the code is clear: p->set_child_tid = (clone_flags & CLONE_CHILD_SETTID) ? child_tidptr : NULL; p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? child_tidptr: NULL; kostik belousov pointed out that this is used for futexes, so not interesting for this Possible mapping from Linux to FreeBSD: To me it seems that the the set_thread_area() syscall is used in the process of thread creation to set where the tls is stored. In FreeBSD we use cpu_set_user_tls() for this. So it might be enough to just wrap call to cpu_set_user_tls() into the syscall. From owner-freebsd-threads@FreeBSD.ORG Tue Jun 20 22:58:37 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 1994016A47A; Tue, 20 Jun 2006 22:58:37 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-threads@freebsd.org Date: Wed, 21 Jun 2006 06:58:24 +0800 User-Agent: KMail/1.8.2 References: <20060620120948.GA8288@stud.fit.vutbr.cz> In-Reply-To: <20060620120948.GA8288@stud.fit.vutbr.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200606210658.24741.davidxu@freebsd.org> Cc: threads@freebsd.org, Divacky Roman , hackers@freebsd.org Subject: Re: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 22:58:37 -0000 On Tuesday 20 June 2006 20:09, Divacky Roman wrote: > Hi > > I am student working on SoC project - extending linuxolator, now I am > working on implementing linux TLS in FreeBSD. Here is what I think/know and > I like you to comment on this, thnx. Roman > > ------------------------------------------------- > > Linux and FreeBSD TLS implementation - comparison > > Both systems use per-thread setting of where is the tls area stored. This > setting is loaded into active threads GDT and can be accessed via %gs > register. This GDT setup is done on every context switch. > Yes. > Linux uses strict 1:1 threading so every thread is in fact process, so > thread creation is done using plain clone()/fork(). FreeBSD uses M:N > (including 1:1) threading. Threads are created via pthread_create() call to > threading library. In kernel there's thr_new() syscall or thread_create() > syscall. I didnt find the connection between threading library and kernel > but I assume its using one of the syscalls > The M:N and 1:1 threading in FreeBSD use different mechanisms to implement TLS, M:N implements it in userland, while 1:1 implements it in kernel. the thr_new or thr_create are used for 1:1 threading, right now libthr uses thr_new to atomically setup a thread, this includes, storing TID, setting TLS, and maybe signal mask( not implemented ) , cpu affinity mask etcs(not implemented), scheduling scope, in one word, it is intended to map most part of pthread_attr into kernel world. > For setting up the GDT for the thread Linux uses syscall set_thread_area() > (TODO - how exactly? its unclear what it does). I dont know how FreeBSD > does it but I think it might be done via params to the syscalls (TODO - how > is it done?) > If you use thr_new, it is not necessary to use set_thread_area, I am not sure you need to change TLS pointer again after the thread is created, I think only main thread may need this feature, in FreeBSD, setting thread's TLS pointer is via libc function: _set_tp(void *tp). > Remaining questions: > > clone() - 2.6.x glibc fork() implementation uses clone() syscall. is it > supposed to create a thread or just a process? I think its process but why > is the binary (ls, date and probably some other) linked to pthread library? > is it just Linux "strangeness"? I dont see a reason for ls to be > threaded... does anyone see? > Dunno. > set/get tid - does it relate to TLS at all? I dont think so but you never > know. The tid thing is unclear to me. The clone() syscall is passed > CLONE_CHILD_SETTID & CLONE_CHILD_CLEARTID which should be mutually > exclusive. I dont believe much its a mistake.. but the code is clear: > p->set_child_tid = (clone_flags & CLONE_CHILD_SETTID) ? child_tidptr : > NULL; p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? > child_tidptr: NULL; > > kostik belousov pointed out that this is used for futexes, so not > interesting for this > > I think it is used for futex, and the childtid is use to implement pthread_join and garbage collection in thread library, the parent tid pointer (if I recall correctly) is used by parent thread to retrieve child tid. > Possible mapping from Linux to FreeBSD: > > To me it seems that the the set_thread_area() syscall is used in the > process of thread creation to set where the tls is stored. In FreeBSD we > use cpu_set_user_tls() for this. So it might be enough to just wrap call to > cpu_set_user_tls() into the syscall. cpu_set_user_tls is used by thr_new syscall internally to setup TLS pointer before executing user code, the thr_new syscall's only user is libthr. From owner-freebsd-threads@FreeBSD.ORG Tue Jun 20 22:58:37 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 1994016A47A; Tue, 20 Jun 2006 22:58:37 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-threads@freebsd.org Date: Wed, 21 Jun 2006 06:58:24 +0800 User-Agent: KMail/1.8.2 References: <20060620120948.GA8288@stud.fit.vutbr.cz> In-Reply-To: <20060620120948.GA8288@stud.fit.vutbr.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200606210658.24741.davidxu@freebsd.org> Cc: threads@freebsd.org, Divacky Roman , hackers@freebsd.org Subject: Re: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 22:58:37 -0000 On Tuesday 20 June 2006 20:09, Divacky Roman wrote: > Hi > > I am student working on SoC project - extending linuxolator, now I am > working on implementing linux TLS in FreeBSD. Here is what I think/know and > I like you to comment on this, thnx. Roman > > ------------------------------------------------- > > Linux and FreeBSD TLS implementation - comparison > > Both systems use per-thread setting of where is the tls area stored. This > setting is loaded into active threads GDT and can be accessed via %gs > register. This GDT setup is done on every context switch. > Yes. > Linux uses strict 1:1 threading so every thread is in fact process, so > thread creation is done using plain clone()/fork(). FreeBSD uses M:N > (including 1:1) threading. Threads are created via pthread_create() call to > threading library. In kernel there's thr_new() syscall or thread_create() > syscall. I didnt find the connection between threading library and kernel > but I assume its using one of the syscalls > The M:N and 1:1 threading in FreeBSD use different mechanisms to implement TLS, M:N implements it in userland, while 1:1 implements it in kernel. the thr_new or thr_create are used for 1:1 threading, right now libthr uses thr_new to atomically setup a thread, this includes, storing TID, setting TLS, and maybe signal mask( not implemented ) , cpu affinity mask etcs(not implemented), scheduling scope, in one word, it is intended to map most part of pthread_attr into kernel world. > For setting up the GDT for the thread Linux uses syscall set_thread_area() > (TODO - how exactly? its unclear what it does). I dont know how FreeBSD > does it but I think it might be done via params to the syscalls (TODO - how > is it done?) > If you use thr_new, it is not necessary to use set_thread_area, I am not sure you need to change TLS pointer again after the thread is created, I think only main thread may need this feature, in FreeBSD, setting thread's TLS pointer is via libc function: _set_tp(void *tp). > Remaining questions: > > clone() - 2.6.x glibc fork() implementation uses clone() syscall. is it > supposed to create a thread or just a process? I think its process but why > is the binary (ls, date and probably some other) linked to pthread library? > is it just Linux "strangeness"? I dont see a reason for ls to be > threaded... does anyone see? > Dunno. > set/get tid - does it relate to TLS at all? I dont think so but you never > know. The tid thing is unclear to me. The clone() syscall is passed > CLONE_CHILD_SETTID & CLONE_CHILD_CLEARTID which should be mutually > exclusive. I dont believe much its a mistake.. but the code is clear: > p->set_child_tid = (clone_flags & CLONE_CHILD_SETTID) ? child_tidptr : > NULL; p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? > child_tidptr: NULL; > > kostik belousov pointed out that this is used for futexes, so not > interesting for this > > I think it is used for futex, and the childtid is use to implement pthread_join and garbage collection in thread library, the parent tid pointer (if I recall correctly) is used by parent thread to retrieve child tid. > Possible mapping from Linux to FreeBSD: > > To me it seems that the the set_thread_area() syscall is used in the > process of thread creation to set where the tls is stored. In FreeBSD we > use cpu_set_user_tls() for this. So it might be enough to just wrap call to > cpu_set_user_tls() into the syscall. cpu_set_user_tls is used by thr_new syscall internally to setup TLS pointer before executing user code, the thr_new syscall's only user is libthr. From owner-freebsd-threads@FreeBSD.ORG Wed Jun 21 08:09:23 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 133CE16A47D; Wed, 21 Jun 2006 08:09:23 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.10.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 67B0643D49; Wed, 21 Jun 2006 08:09:21 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.7/8.13.7) with ESMTP id k5L89GO4082110 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Wed, 21 Jun 2006 10:09:16 +0200 (CEST) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.7/8.13.3/Submit) id k5L89GYM082109; Wed, 21 Jun 2006 10:09:16 +0200 (CEST) Date: Wed, 21 Jun 2006 10:09:16 +0200 From: Divacky Roman To: David Xu Message-ID: <20060621080916.GA81422@stud.fit.vutbr.cz> References: <20060620120948.GA8288@stud.fit.vutbr.cz> <200606210658.24741.davidxu@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200606210658.24741.davidxu@freebsd.org> User-Agent: Mutt/1.4.2i X-Scanned-By: MIMEDefang 2.54 on 147.229.10.14 Cc: threads@freebsd.org, hackers@freebsd.org, freebsd-threads@freebsd.org Subject: Re: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 08:09:23 -0000 > > Linux uses strict 1:1 threading so every thread is in fact process, so > > thread creation is done using plain clone()/fork(). FreeBSD uses M:N > > (including 1:1) threading. Threads are created via pthread_create() call to > > threading library. In kernel there's thr_new() syscall or thread_create() > > syscall. I didnt find the connection between threading library and kernel > > but I assume its using one of the syscalls > > > The M:N and 1:1 threading in FreeBSD use different mechanisms to > implement TLS, M:N implements it in userland, while 1:1 implements it in > kernel. the thr_new or thr_create are used for 1:1 threading, right > now libthr uses thr_new to atomically setup a thread, this includes, > storing TID, setting TLS, and maybe signal mask( not implemented ) , > cpu affinity mask etcs(not implemented), scheduling scope, in one word, > it is intended to map most part of pthread_attr into kernel world. but on the kernel level the implementation must be the same.. I mean the mangling of %gs. right? > > For setting up the GDT for the thread Linux uses syscall set_thread_area() > > (TODO - how exactly? its unclear what it does). I dont know how FreeBSD > > does it but I think it might be done via params to the syscalls (TODO - how > > is it done?) > > > If you use thr_new, it is not necessary to use set_thread_area, I am not sure > you need to change TLS pointer again after the thread is created, I think > only main thread may need this feature, in FreeBSD, setting thread's TLS > pointer is via libc function: _set_tp(void *tp). well.. in linux the thread creation and setting up the tls is done using separate syscalls. I plan to extend clone() syscall to use thr_create() or thr_new() (if the flags tell me its thread) but I am afraid I'l have to modify those syscalls to not to setup TLS (some flag) because linux wants to set it separately. > > set/get tid - does it relate to TLS at all? I dont think so but you never > > know. The tid thing is unclear to me. The clone() syscall is passed > > CLONE_CHILD_SETTID & CLONE_CHILD_CLEARTID which should be mutually > > exclusive. I dont believe much its a mistake.. but the code is clear: > > p->set_child_tid = (clone_flags & CLONE_CHILD_SETTID) ? child_tidptr : > > NULL; p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? > > child_tidptr: NULL; > > > > kostik belousov pointed out that this is used for futexes, so not > > interesting for this > > > > > I think it is used for futex, and the childtid is use to implement > pthread_join and garbage collection in thread library, the parent tid > pointer (if I recall correctly) is used by parent thread to retrieve > child tid. this is the next step... I think all the magic is done in their libc (or somewhere) and I basically just need to malloc some space for this info and clear/set it on proces creation/exit > > Possible mapping from Linux to FreeBSD: > > > > To me it seems that the the set_thread_area() syscall is used in the > > process of thread creation to set where the tls is stored. In FreeBSD we > > use cpu_set_user_tls() for this. So it might be enough to just wrap call to > > cpu_set_user_tls() into the syscall. > cpu_set_user_tls is used by thr_new syscall internally to setup TLS pointer > before executing user code, the thr_new syscall's only user is libthr. the cpu_set_user_tls() is then what I need I think... maybe some modifications needed but it shuold be basiscally the thing. the linux syscall set_thread_area() just loads GDT with that info.. thats the same like ours cpu_set_use_tls(), right? thnx for your information! roman From owner-freebsd-threads@FreeBSD.ORG Wed Jun 21 08:09:23 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 133CE16A47D; Wed, 21 Jun 2006 08:09:23 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.10.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 67B0643D49; Wed, 21 Jun 2006 08:09:21 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.7/8.13.7) with ESMTP id k5L89GO4082110 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Wed, 21 Jun 2006 10:09:16 +0200 (CEST) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.7/8.13.3/Submit) id k5L89GYM082109; Wed, 21 Jun 2006 10:09:16 +0200 (CEST) Date: Wed, 21 Jun 2006 10:09:16 +0200 From: Divacky Roman To: David Xu Message-ID: <20060621080916.GA81422@stud.fit.vutbr.cz> References: <20060620120948.GA8288@stud.fit.vutbr.cz> <200606210658.24741.davidxu@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200606210658.24741.davidxu@freebsd.org> User-Agent: Mutt/1.4.2i X-Scanned-By: MIMEDefang 2.54 on 147.229.10.14 Cc: threads@freebsd.org, hackers@freebsd.org, freebsd-threads@freebsd.org Subject: Re: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 08:09:23 -0000 > > Linux uses strict 1:1 threading so every thread is in fact process, so > > thread creation is done using plain clone()/fork(). FreeBSD uses M:N > > (including 1:1) threading. Threads are created via pthread_create() call to > > threading library. In kernel there's thr_new() syscall or thread_create() > > syscall. I didnt find the connection between threading library and kernel > > but I assume its using one of the syscalls > > > The M:N and 1:1 threading in FreeBSD use different mechanisms to > implement TLS, M:N implements it in userland, while 1:1 implements it in > kernel. the thr_new or thr_create are used for 1:1 threading, right > now libthr uses thr_new to atomically setup a thread, this includes, > storing TID, setting TLS, and maybe signal mask( not implemented ) , > cpu affinity mask etcs(not implemented), scheduling scope, in one word, > it is intended to map most part of pthread_attr into kernel world. but on the kernel level the implementation must be the same.. I mean the mangling of %gs. right? > > For setting up the GDT for the thread Linux uses syscall set_thread_area() > > (TODO - how exactly? its unclear what it does). I dont know how FreeBSD > > does it but I think it might be done via params to the syscalls (TODO - how > > is it done?) > > > If you use thr_new, it is not necessary to use set_thread_area, I am not sure > you need to change TLS pointer again after the thread is created, I think > only main thread may need this feature, in FreeBSD, setting thread's TLS > pointer is via libc function: _set_tp(void *tp). well.. in linux the thread creation and setting up the tls is done using separate syscalls. I plan to extend clone() syscall to use thr_create() or thr_new() (if the flags tell me its thread) but I am afraid I'l have to modify those syscalls to not to setup TLS (some flag) because linux wants to set it separately. > > set/get tid - does it relate to TLS at all? I dont think so but you never > > know. The tid thing is unclear to me. The clone() syscall is passed > > CLONE_CHILD_SETTID & CLONE_CHILD_CLEARTID which should be mutually > > exclusive. I dont believe much its a mistake.. but the code is clear: > > p->set_child_tid = (clone_flags & CLONE_CHILD_SETTID) ? child_tidptr : > > NULL; p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? > > child_tidptr: NULL; > > > > kostik belousov pointed out that this is used for futexes, so not > > interesting for this > > > > > I think it is used for futex, and the childtid is use to implement > pthread_join and garbage collection in thread library, the parent tid > pointer (if I recall correctly) is used by parent thread to retrieve > child tid. this is the next step... I think all the magic is done in their libc (or somewhere) and I basically just need to malloc some space for this info and clear/set it on proces creation/exit > > Possible mapping from Linux to FreeBSD: > > > > To me it seems that the the set_thread_area() syscall is used in the > > process of thread creation to set where the tls is stored. In FreeBSD we > > use cpu_set_user_tls() for this. So it might be enough to just wrap call to > > cpu_set_user_tls() into the syscall. > cpu_set_user_tls is used by thr_new syscall internally to setup TLS pointer > before executing user code, the thr_new syscall's only user is libthr. the cpu_set_user_tls() is then what I need I think... maybe some modifications needed but it shuold be basiscally the thing. the linux syscall set_thread_area() just loads GDT with that info.. thats the same like ours cpu_set_use_tls(), right? thnx for your information! roman From owner-freebsd-threads@FreeBSD.ORG Wed Jun 21 10:12:24 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 5208516A479; Wed, 21 Jun 2006 10:12:22 +0000 (UTC) (envelope-from davidxu@freebsd.org) Message-ID: <44991B87.1020104@freebsd.org> Date: Wed, 21 Jun 2006 18:12:23 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060519 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Divacky Roman References: <20060620120948.GA8288@stud.fit.vutbr.cz> <200606210658.24741.davidxu@freebsd.org> <20060621080916.GA81422@stud.fit.vutbr.cz> In-Reply-To: <20060621080916.GA81422@stud.fit.vutbr.cz> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org, hackers@freebsd.org, freebsd-threads@freebsd.org Subject: Re: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 10:12:24 -0000 Divacky Roman wrote: >>The M:N and 1:1 threading in FreeBSD use different mechanisms to >>implement TLS, M:N implements it in userland, while 1:1 implements it in >>kernel. the thr_new or thr_create are used for 1:1 threading, right >>now libthr uses thr_new to atomically setup a thread, this includes, >>storing TID, setting TLS, and maybe signal mask( not implemented ) , >>cpu affinity mask etcs(not implemented), scheduling scope, in one word, >>it is intended to map most part of pthread_attr into kernel world. > > > but on the kernel level the implementation must be the same.. I mean the > mangling of %gs. right? > > There is no such standard that a kernel must implement it in that way, we happens to implement it in kernel with GDT, before this, thread libraries were using LDT. The offical TLS standard only defined ABI in userspace: http://people.redhat.com/drepper/tls.pdf M:N thread library only set GDT entry once, for Variant II TLS (x86), the userland scheduler just replaces some pointers in TCB, it does not have to set TLS via syscall later. but 1:1 thread library will just let kernel context switch code to update it for next thread. > > well.. in linux the thread creation and setting up the tls is done using > separate syscalls. I plan to extend clone() syscall to use thr_create() or > thr_new() (if the flags tell me its thread) but I am afraid I'l have to modify > those syscalls to not to setup TLS (some flag) because linux wants to set it > separately. > You can try, but the thr_xx syscalls were not designed to implement linux clone() syscall, they are only used by libthr to implement 1:1 threading. >>I think it is used for futex, and the childtid is use to implement >>pthread_join and garbage collection in thread library, the parent tid >>pointer (if I recall correctly) is used by parent thread to retrieve >>child tid. > > > this is the next step... I think all the magic is done in their libc (or > somewhere) and I basically just need to malloc some space for this info > and clear/set it on proces creation/exit > > we don't save childtid pointer and clear it at thread exiting time like Linux did, we use thr_exit() which passes a pointer to let kernel write a value into the address, this lets libthr's garbage collection code work. the thr syscalls may be extented to save childtid pointer somewhere in kernel by adding another flag, and clear it to zero when thread is exiting like Linux did. > the cpu_set_user_tls() is then what I need I think... maybe some modifications > needed but it shuold be basiscally the thing. > > the linux syscall set_thread_area() just loads GDT with that info.. thats the > same like ours cpu_set_use_tls(), right? > Right. > thnx for your information! > > roman From owner-freebsd-threads@FreeBSD.ORG Wed Jun 21 10:12:24 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from [127.0.0.1] (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 5208516A479; Wed, 21 Jun 2006 10:12:22 +0000 (UTC) (envelope-from davidxu@freebsd.org) Message-ID: <44991B87.1020104@freebsd.org> Date: Wed, 21 Jun 2006 18:12:23 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060519 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Divacky Roman References: <20060620120948.GA8288@stud.fit.vutbr.cz> <200606210658.24741.davidxu@freebsd.org> <20060621080916.GA81422@stud.fit.vutbr.cz> In-Reply-To: <20060621080916.GA81422@stud.fit.vutbr.cz> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org, hackers@freebsd.org, freebsd-threads@freebsd.org Subject: Re: TLS - implementing linux one in fbsd X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 10:12:24 -0000 Divacky Roman wrote: >>The M:N and 1:1 threading in FreeBSD use different mechanisms to >>implement TLS, M:N implements it in userland, while 1:1 implements it in >>kernel. the thr_new or thr_create are used for 1:1 threading, right >>now libthr uses thr_new to atomically setup a thread, this includes, >>storing TID, setting TLS, and maybe signal mask( not implemented ) , >>cpu affinity mask etcs(not implemented), scheduling scope, in one word, >>it is intended to map most part of pthread_attr into kernel world. > > > but on the kernel level the implementation must be the same.. I mean the > mangling of %gs. right? > > There is no such standard that a kernel must implement it in that way, we happens to implement it in kernel with GDT, before this, thread libraries were using LDT. The offical TLS standard only defined ABI in userspace: http://people.redhat.com/drepper/tls.pdf M:N thread library only set GDT entry once, for Variant II TLS (x86), the userland scheduler just replaces some pointers in TCB, it does not have to set TLS via syscall later. but 1:1 thread library will just let kernel context switch code to update it for next thread. > > well.. in linux the thread creation and setting up the tls is done using > separate syscalls. I plan to extend clone() syscall to use thr_create() or > thr_new() (if the flags tell me its thread) but I am afraid I'l have to modify > those syscalls to not to setup TLS (some flag) because linux wants to set it > separately. > You can try, but the thr_xx syscalls were not designed to implement linux clone() syscall, they are only used by libthr to implement 1:1 threading. >>I think it is used for futex, and the childtid is use to implement >>pthread_join and garbage collection in thread library, the parent tid >>pointer (if I recall correctly) is used by parent thread to retrieve >>child tid. > > > this is the next step... I think all the magic is done in their libc (or > somewhere) and I basically just need to malloc some space for this info > and clear/set it on proces creation/exit > > we don't save childtid pointer and clear it at thread exiting time like Linux did, we use thr_exit() which passes a pointer to let kernel write a value into the address, this lets libthr's garbage collection code work. the thr syscalls may be extented to save childtid pointer somewhere in kernel by adding another flag, and clear it to zero when thread is exiting like Linux did. > the cpu_set_user_tls() is then what I need I think... maybe some modifications > needed but it shuold be basiscally the thing. > > the linux syscall set_thread_area() just loads GDT with that info.. thats the > same like ours cpu_set_use_tls(), right? > Right. > thnx for your information! > > roman