From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 11:07:02 2009 Return-Path: Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F080C1065678 for ; Mon, 30 Mar 2009 11:07:01 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D52728FC3A for ; Mon, 30 Mar 2009 11:07:01 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2UB71ZA054930 for ; Mon, 30 Mar 2009 11:07:01 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2UB71ZV054926 for freebsd-threads@FreeBSD.org; Mon, 30 Mar 2009 11:07:01 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 30 Mar 2009 11:07:01 GMT Message-Id: <200903301107.n2UB71ZV054926@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 11:07:02 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/83914 threads [libc] popen() doesn't work in static threaded program o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/80435 threads panic on high loads o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/70975 threads [sysvipc] unexpected and unreliable behaviour when usi s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 37 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 21:10:27 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C78810656C6 for ; Mon, 30 Mar 2009 21:10:27 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 0FCB58FC19 for ; Mon, 30 Mar 2009 21:10:26 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2ULAXhI049863 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Mon, 30 Mar 2009 17:10:33 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> From: Randall Stewart To: threads@freebsd.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 30 Mar 2009 17:10:25 -0400 X-Mailer: Apple Mail (2.930.3) Cc: Subject: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 21:10:28 -0000 Hi all: I have recently written a small set of routines that allow two process to have a "mutex" between them.. actually it allows all of the threads in any set of processes to have mutexes between themselves ;-) Anyway it seems to be working fairly well.. I still have to write a man page for it (documentation always last).. and eventually I would like to port in some of the WITNESS type features since the mutex's have names.. I probably should also think about scaling it up a bit.. right now its really more for a small scale (100 or less mutexes)... Who should I talk to about getting this in... having it reviewed etc. I think it belongs in libthr since it really needs the tid of the pthreads from the pthread_t type... and for now I have a horrible hack in to get it ;-) Thanks R ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 21:16:48 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA223106566B for ; Mon, 30 Mar 2009 21:16:48 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5EBBB8FC08 for ; Mon, 30 Mar 2009 21:16:48 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [211.166.10.233]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 420D728448 for ; Tue, 31 Mar 2009 05:16:47 +0800 (CST) Received: from localhost (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id E54A9EB7261; Tue, 31 Mar 2009 05:16:46 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by localhost (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with ESMTP id zBarfh+AhXb1; Tue, 31 Mar 2009 05:16:42 +0800 (CST) Received: from charlie.delphij.net (adsl-76-237-33-62.dsl.pltn13.sbcglobal.net [76.237.33.62]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id CE230EB6F10; Tue, 31 Mar 2009 05:16:39 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=suqxa8LxWN4lDwuG4gQ2sWDwdISPD18e2zSnZTwv9bM2FbV6CMjvjI+STt2uVqX0g ZtoEexXr72cL8VFUKoNqg== Message-ID: <49D136B1.6060809@delphij.net> Date: Mon, 30 Mar 2009 14:16:33 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.21 (X11/20090324) MIME-Version: 1.0 To: Randall Stewart References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> In-Reply-To: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> X-Enigmail-Version: 0.95.7 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 21:16:49 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Randall Stewart wrote: > Hi all: > > I have recently written a small set of routines that allow > two process to have a "mutex" between them.. actually it allows > all of the threads in any set of processes to have mutexes between > themselves ;-) > > Anyway it seems to be working fairly well.. I still have to write a man > page > for it (documentation always last).. and eventually I would like to port in > some of the WITNESS type features since the mutex's have names.. > > I probably should also think about scaling it up a bit.. right now its > really > more for a small scale (100 or less mutexes)... > > Who should I talk to about getting this in... having it reviewed etc. I > think > it belongs in libthr since it really needs the tid of the pthreads from the > pthread_t type... and for now I have a horrible hack in to get it ;-) I think davidxu@ deischen@ and julian@? BTW. How do you handle with one process exit (abnormally) without releasing the mutex? Just curious :) Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo 6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH =JL/U -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 21:22:54 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 902FF106567B for ; Mon, 30 Mar 2009 21:22:54 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 50F608FC26 for ; Mon, 30 Mar 2009 21:22:54 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id n2ULMr7L028452; Mon, 30 Mar 2009 17:22:53 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Mon, 30 Mar 2009 17:22:53 -0400 (EDT) Date: Mon, 30 Mar 2009 17:22:53 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Randall Stewart In-Reply-To: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> Message-ID: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 21:22:55 -0000 On Mon, 30 Mar 2009, Randall Stewart wrote: > Hi all: > > I have recently written a small set of routines that allow > two process to have a "mutex" between them.. actually it allows > all of the threads in any set of processes to have mutexes between themselves > ;-) > > Anyway it seems to be working fairly well.. I still have to write a man page > for it (documentation always last).. and eventually I would like to port in > some of the WITNESS type features since the mutex's have names.. > > I probably should also think about scaling it up a bit.. right now its really > more for a small scale (100 or less mutexes)... > > Who should I talk to about getting this in... having it reviewed etc. I think > it belongs in libthr since it really needs the tid of the pthreads from the > pthread_t type... and for now I have a horrible hack in to get it ;-) The real way to do this is to support PTHREAD_PROCESS_SHARED mutexes within our normal mutex, and to change our current mutex (and cv) types to be structs instead of pointers. The current API, other than the type change, shouldn't change at all. -- DE From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 23:29:21 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3B9A106566B for ; Mon, 30 Mar 2009 23:29:21 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 651698FC0A for ; Mon, 30 Mar 2009 23:29:21 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2UNTIKr055101 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Mon, 30 Mar 2009 19:29:18 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> From: Randall Stewart To: d@delphij.net In-Reply-To: <49D136B1.6060809@delphij.net> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 30 Mar 2009 19:29:10 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 23:29:22 -0000 On Mar 30, 2009, at 5:16 PM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Randall Stewart wrote: >> Hi all: >> >> I have recently written a small set of routines that allow >> two process to have a "mutex" between them.. actually it allows >> all of the threads in any set of processes to have mutexes between >> themselves ;-) >> >> Anyway it seems to be working fairly well.. I still have to write a >> man >> page >> for it (documentation always last).. and eventually I would like to >> port in >> some of the WITNESS type features since the mutex's have names.. >> >> I probably should also think about scaling it up a bit.. right now >> its >> really >> more for a small scale (100 or less mutexes)... >> >> Who should I talk to about getting this in... having it reviewed >> etc. I >> think >> it belongs in libthr since it really needs the tid of the pthreads >> from the >> pthread_t type... and for now I have a horrible hack in to get it ;-) > > I think davidxu@ deischen@ and julian@? > > BTW. How do you handle with one process exit (abnormally) without > releasing the mutex? Just curious :) I have a couple of ways of dealing with this.. 1) Of course the initialization routine calls atexit() to get a "cleanup handler" in place. 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. or some such. When you attach the memory, an audit is done. The audit will validate that the pid is still alive and has the particular tid in it. Of course this is not 100% but as long as the tid's have not rolled over it should work. The function is also public (need to add that and many things to the manual pages ;-D) so that one can call it whenever one wants :-) I will ping Julian and the others... R > > > Cheers, > - -- > Xin LI http://www.delphij.net/ > FreeBSD - The Power to Serve! > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.11 (FreeBSD) > > iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo > 6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH > =JL/U > -----END PGP SIGNATURE----- > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 23:30:48 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE54C1065677; Mon, 30 Mar 2009 23:30:48 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 7E4228FC12; Mon, 30 Mar 2009 23:30:48 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2UNUtu5055176 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Mon, 30 Mar 2009 19:30:55 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: From: Randall Stewart To: Daniel Eischen In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 30 Mar 2009 19:30:47 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 23:30:49 -0000 On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: > On Mon, 30 Mar 2009, Randall Stewart wrote: > >> Hi all: >> >> I have recently written a small set of routines that allow >> two process to have a "mutex" between them.. actually it allows >> all of the threads in any set of processes to have mutexes between >> themselves ;-) >> >> Anyway it seems to be working fairly well.. I still have to write a >> man page >> for it (documentation always last).. and eventually I would like to >> port in >> some of the WITNESS type features since the mutex's have names.. >> >> I probably should also think about scaling it up a bit.. right now >> its really >> more for a small scale (100 or less mutexes)... >> >> Who should I talk to about getting this in... having it reviewed >> etc. I think >> it belongs in libthr since it really needs the tid of the pthreads >> from the >> pthread_t type... and for now I have a horrible hack in to get it ;-) > > The real way to do this is to support PTHREAD_PROCESS_SHARED > mutexes within our normal mutex, and to change our current > mutex (and cv) types to be structs instead of pointers. > The current API, other than the type change, shouldn't > change at all. So how do you propose to name the mutex's so that two disparate process can locate the same mutex? I don't see how a pthread_mutex can suffice... we need more than just the current mutex... What am I missing? R > > > -- > DE > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Mon Mar 30 23:56:12 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B82C2106566B for ; Mon, 30 Mar 2009 23:56:12 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 723C58FC0C for ; Mon, 30 Mar 2009 23:56:12 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id n2UNuBF5008464; Mon, 30 Mar 2009 19:56:11 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Mon, 30 Mar 2009 19:56:11 -0400 (EDT) Date: Mon, 30 Mar 2009 19:56:11 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Randall Stewart In-Reply-To: Message-ID: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2009 23:56:13 -0000 On Mon, 30 Mar 2009, Randall Stewart wrote: > > On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: > >> On Mon, 30 Mar 2009, Randall Stewart wrote: >> >>> Hi all: >>> >>> I have recently written a small set of routines that allow >>> two process to have a "mutex" between them.. actually it allows >>> all of the threads in any set of processes to have mutexes between >>> themselves ;-) >>> >>> Anyway it seems to be working fairly well.. I still have to write a man >>> page >>> for it (documentation always last).. and eventually I would like to port >>> in >>> some of the WITNESS type features since the mutex's have names.. >>> >>> I probably should also think about scaling it up a bit.. right now its >>> really >>> more for a small scale (100 or less mutexes)... >>> >>> Who should I talk to about getting this in... having it reviewed etc. I >>> think >>> it belongs in libthr since it really needs the tid of the pthreads from >>> the >>> pthread_t type... and for now I have a horrible hack in to get it ;-) >> >> The real way to do this is to support PTHREAD_PROCESS_SHARED >> mutexes within our normal mutex, and to change our current >> mutex (and cv) types to be structs instead of pointers. >> The current API, other than the type change, shouldn't >> change at all. > > > So how do you propose to name the mutex's so that two disparate > process can locate the same mutex? They are placed in shared memory, according to POSIX. > I don't see how a pthread_mutex can suffice... we need more than > just the current mutex... > > What am I missing? As far as I know, David Xu implemented the kernel hooks for umtx (the underlying mutex in pthread mutex) to be shared. As soon as you can place the entire userland pthread_mutex_t struct in shared memory, it should all just work (with probably some trivial changes in libthr). The harder part is versioning all the symbols that currently think pthread_mutex_t, pthread_cond_t, etc, are pointers, and defining the structs with enough foresight so that it is unlikely we have to modify them in the future (causing a future ABI breakage), and also aligning them nicely for the various archs. You should really look at how POSIX defines process shared mutex, cvs, etc. See: pthread_barrierattr_[gs]etpshared() pthread_condattr_[gs]etpshared() pthread_mutexattr_[gs]etpshared() pthread_wrlockattr_[gs]etsphared() You can use this as a starting point: http://www.opengroup.org/onlinepubs/009695399/ http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 01:05:33 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12A2E106566C for ; Tue, 31 Mar 2009 01:05:33 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outI.internet-mail-service.net (outi.internet-mail-service.net [216.240.47.232]) by mx1.freebsd.org (Postfix) with ESMTP id ECE968FC22 for ; Tue, 31 Mar 2009 01:05:32 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id ABE0BCD38; Mon, 30 Mar 2009 17:55:23 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id EF0C32D606A; Mon, 30 Mar 2009 17:55:16 -0700 (PDT) Message-ID: <49D16A0F.4000404@elischer.org> Date: Mon, 30 Mar 2009 17:55:43 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Randall Stewart References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> In-Reply-To: <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org, d@delphij.net Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 01:05:33 -0000 Randall Stewart wrote: > >> >> I think davidxu@ deischen@ and julian@? >> >> BTW. How do you handle with one process exit (abnormally) without >> releasing the mutex? Just curious :) > > I have a couple of ways of dealing with this.. > > 1) Of course the initialization routine calls atexit() to get a > "cleanup handler" in place. this is not really sufficient for a system supplied service. > 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. > or some such. When you attach the memory, an audit is done. The > audit will validate that the pid is still alive and has the > particular tid in it. Of course this is not 100% but as long as the > tid's have not rolled over it should work. The function is also > public (need to add that and many things to the manual pages ;-D) > so that onecan call it whenever one wants :-) TIDs do roll over the last I looked.. (this may have changed) did you say man page? goodie.. lets' see it.. There have been a lot of IPC and mutex implementations but the trick always comes with the requirement that they handle process/thread death. David has done some recent work in this space.. > > I will ping Julian and the others... > From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 02:18:32 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9597B1065670; Tue, 31 Mar 2009 02:18:32 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 802DF8FC08; Tue, 31 Mar 2009 02:18:32 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from apple.my.domain (root@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2V2ISMd088215; Tue, 31 Mar 2009 02:18:29 GMT (envelope-from davidxu@freebsd.org) Message-ID: <49D17D76.5060309@freebsd.org> Date: Tue, 31 Mar 2009 10:18:30 +0800 From: David Xu User-Agent: Thunderbird 2.0.0.9 (X11/20080612) MIME-Version: 1.0 To: Daniel Eischen References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 02:18:32 -0000 Daniel Eischen wrote: > On Mon, 30 Mar 2009, Randall Stewart wrote: > >> >> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: >> >>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>> >>>> Hi all: >>>> >>>> I have recently written a small set of routines that allow >>>> two process to have a "mutex" between them.. actually it allows >>>> all of the threads in any set of processes to have mutexes between >>>> themselves ;-) >>>> >>>> Anyway it seems to be working fairly well.. I still have to write a >>>> man page >>>> for it (documentation always last).. and eventually I would like to >>>> port in >>>> some of the WITNESS type features since the mutex's have names.. >>>> >>>> I probably should also think about scaling it up a bit.. right now >>>> its really >>>> more for a small scale (100 or less mutexes)... >>>> >>>> Who should I talk to about getting this in... having it reviewed >>>> etc. I think >>>> it belongs in libthr since it really needs the tid of the pthreads >>>> from the >>>> pthread_t type... and for now I have a horrible hack in to get it ;-) >>> >>> The real way to do this is to support PTHREAD_PROCESS_SHARED >>> mutexes within our normal mutex, and to change our current >>> mutex (and cv) types to be structs instead of pointers. >>> The current API, other than the type change, shouldn't >>> change at all. >> >> >> So how do you propose to name the mutex's so that two disparate >> process can locate the same mutex? > > They are placed in shared memory, according to POSIX. > >> I don't see how a pthread_mutex can suffice... we need more than >> just the current mutex... >> >> What am I missing? > > As far as I know, David Xu implemented the kernel hooks > for umtx (the underlying mutex in pthread mutex) to be > shared. As soon as you can place the entire userland > pthread_mutex_t struct in shared memory, it should all > just work (with probably some trivial changes in libthr). > The harder part is versioning all the symbols that > currently think pthread_mutex_t, pthread_cond_t, etc, > are pointers, and defining the structs with enough > foresight so that it is unlikely we have to modify > them in the future (causing a future ABI breakage), > and also aligning them nicely for the various archs. > > You should really look at how POSIX defines process > shared mutex, cvs, etc. See: > > pthread_barrierattr_[gs]etpshared() > pthread_condattr_[gs]etpshared() > pthread_mutexattr_[gs]etpshared() > pthread_wrlockattr_[gs]etsphared() > > You can use this as a starting point: > > http://www.opengroup.org/onlinepubs/009695399/ > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html > > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html > > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html > > > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html > > You are right. umtx is ready for process-shared mutex, condition variable and rwlock. We are blocked by our pthread_mutex_t and pthread_cond_t definitions which are pointers, mmap()ing it into shared memory and calling pthread API will not work correctly, they should be defined as a block of memory. Recent POSIX standard introduces robust mutex type which can detects mutex owner's death, but in theory, shared memory model will never be robust. Regards, David Xu From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 04:44:33 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F2461065673; Tue, 31 Mar 2009 04:44:33 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 2E0568FC0A; Tue, 31 Mar 2009 04:44:33 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2V4idE6066419 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 00:44:39 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: From: Randall Stewart To: David Xu In-Reply-To: <49D17D76.5060309@freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 00:44:32 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D17D76.5060309@freebsd.org> X-Mailer: Apple Mail (2.930.3) Cc: Daniel Eischen , threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 04:44:34 -0000 On Mar 30, 2009, at 10:18 PM, David Xu wrote: > Daniel Eischen wrote: >> On Mon, 30 Mar 2009, Randall Stewart wrote: >>> >>> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: >>> >>>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>>> >>>>> Hi all: >>>>> >>>>> I have recently written a small set of routines that allow >>>>> two process to have a "mutex" between them.. actually it allows >>>>> all of the threads in any set of processes to have mutexes >>>>> between themselves ;-) >>>>> >>>>> Anyway it seems to be working fairly well.. I still have to >>>>> write a man page >>>>> for it (documentation always last).. and eventually I would like >>>>> to port in >>>>> some of the WITNESS type features since the mutex's have names.. >>>>> >>>>> I probably should also think about scaling it up a bit.. right >>>>> now its really >>>>> more for a small scale (100 or less mutexes)... >>>>> >>>>> Who should I talk to about getting this in... having it reviewed >>>>> etc. I think >>>>> it belongs in libthr since it really needs the tid of the >>>>> pthreads from the >>>>> pthread_t type... and for now I have a horrible hack in to get >>>>> it ;-) >>>> >>>> The real way to do this is to support PTHREAD_PROCESS_SHARED >>>> mutexes within our normal mutex, and to change our current >>>> mutex (and cv) types to be structs instead of pointers. >>>> The current API, other than the type change, shouldn't >>>> change at all. >>> >>> >>> So how do you propose to name the mutex's so that two disparate >>> process can locate the same mutex? >> They are placed in shared memory, according to POSIX. >>> I don't see how a pthread_mutex can suffice... we need more than >>> just the current mutex... >>> >>> What am I missing? >> As far as I know, David Xu implemented the kernel hooks >> for umtx (the underlying mutex in pthread mutex) to be >> shared. As soon as you can place the entire userland >> pthread_mutex_t struct in shared memory, it should all >> just work (with probably some trivial changes in libthr). >> The harder part is versioning all the symbols that >> currently think pthread_mutex_t, pthread_cond_t, etc, >> are pointers, and defining the structs with enough >> foresight so that it is unlikely we have to modify >> them in the future (causing a future ABI breakage), >> and also aligning them nicely for the various archs. >> You should really look at how POSIX defines process >> shared mutex, cvs, etc. See: >> pthread_barrierattr_[gs]etpshared() >> pthread_condattr_[gs]etpshared() >> pthread_mutexattr_[gs]etpshared() >> pthread_wrlockattr_[gs]etsphared() >> You can use this as a starting point: >> http://www.opengroup.org/onlinepubs/009695399/ >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html > > You are right. umtx is ready for process-shared mutex, condition > variable and rwlock. We are blocked by our pthread_mutex_t > and pthread_cond_t definitions which are pointers, mmap()ing it into > shared memory and calling pthread API will not work correctly, they > should be defined as a block of memory. Yes, the stuff I have been playing with uses umtx... it works seamlessly... > > Recent POSIX standard introduces robust mutex type which can detects > mutex owner's death, but in theory, shared memory model will never > be robust. I agree, but its something someone I interviewed with was asking about... and they were busy going about making kernel hacks to add shared mutex's which led me down the path of looking what's there and not there.. I will go poke around and look at the posix stuff... I wonder if some sort of extensions in the kernel might be a good thing to do with a shared memory model to deal with the "robustness" issue. Something to think about for sure. R > > > Regards, > David Xu > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 04:47:11 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA8F0106564A; Tue, 31 Mar 2009 04:47:11 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 456538FC12; Tue, 31 Mar 2009 04:47:11 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2V4lDr2066515 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 00:47:13 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: <59795ADA-892B-4FAF-8506-82007D317C12@lakerest.net> From: Randall Stewart To: Alfred Perlstein In-Reply-To: <20090331043245.GZ92757@elvis.mu.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 00:47:05 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> <20090331043245.GZ92757@elvis.mu.org> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org, d@delphij.net Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 04:47:11 -0000 The actual take I took is that both the pid and first tid are recorded. When you start up you validate each pid listed and make sure the main tid matches and the process is alive. Now of course there is a chance that both the tid and pid will wrap.. but both would have to wrap and you would have to have the new pid be assigned the same tid during the wrap. Maybe this would be common.. don't know .. but its a start. Without adding some sort of hook at process death to detect and cleanup the shared memory.. R On Mar 31, 2009, at 12:32 AM, Alfred Perlstein wrote: > One trick to handling pid wrap is to also record process > start time. I sort of wish our signalling code allowed > this as a optional thing to make _really sure_ you weren't > signalling the wrong process. > > -Alfred > > * Randall Stewart [090330 16:29] wrote: >> >> On Mar 30, 2009, at 5:16 PM, Xin LI wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Randall Stewart wrote: >>>> Hi all: >>>> >>>> I have recently written a small set of routines that allow >>>> two process to have a "mutex" between them.. actually it allows >>>> all of the threads in any set of processes to have mutexes between >>>> themselves ;-) >>>> >>>> Anyway it seems to be working fairly well.. I still have to write a >>>> man >>>> page >>>> for it (documentation always last).. and eventually I would like to >>>> port in >>>> some of the WITNESS type features since the mutex's have names.. >>>> >>>> I probably should also think about scaling it up a bit.. right now >>>> its >>>> really >>>> more for a small scale (100 or less mutexes)... >>>> >>>> Who should I talk to about getting this in... having it reviewed >>>> etc. I >>>> think >>>> it belongs in libthr since it really needs the tid of the pthreads >>>> from the >>>> pthread_t type... and for now I have a horrible hack in to get >>>> it ;-) >>> >>> I think davidxu@ deischen@ and julian@? >>> >>> BTW. How do you handle with one process exit (abnormally) without >>> releasing the mutex? Just curious :) >> >> I have a couple of ways of dealing with this.. >> >> 1) Of course the initialization routine calls atexit() to get a >> "cleanup handler" in place. >> 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. or >> some such. When you >> attach the memory, an audit is done. The audit will validate that >> the pid is still alive >> and has the particular tid in it. Of course this is not 100% but >> as long as the tid's have >> not rolled over it should work. The function is also public (need >> to add that and many things >> to the manual pages ;-D) so that one can call it whenever one >> wants :-) >> >> I will ping Julian and the others... >> >> R >> >>> >>> >>> Cheers, >>> - -- >>> Xin LI http://www.delphij.net/ >>> FreeBSD - The Power to Serve! >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v2.0.11 (FreeBSD) >>> >>> iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo >>> 6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH >>> =JL/U >>> -----END PGP SIGNATURE----- >>> >> >> ------------------------------ >> Randall Stewart >> 803-317-4952 (cell) >> 803-345-0391(direct) >> >> _______________________________________________ >> freebsd-threads@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-threads >> To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org >> " > > -- > - Alfred Perlstein > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 04:52:34 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5082106566B for ; Tue, 31 Mar 2009 04:52:34 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id C27138FC1C for ; Tue, 31 Mar 2009 04:52:34 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id CB7C51A3C3E; Mon, 30 Mar 2009 21:32:45 -0700 (PDT) Date: Mon, 30 Mar 2009 21:32:45 -0700 From: Alfred Perlstein To: Randall Stewart Message-ID: <20090331043245.GZ92757@elvis.mu.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D136B1.6060809@delphij.net> <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <78DBBDDA-5A39-4CEB-8289-F36EFB96461D@lakerest.net> User-Agent: Mutt/1.4.2.3i Cc: threads@freebsd.org, d@delphij.net Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 04:52:35 -0000 One trick to handling pid wrap is to also record process start time. I sort of wish our signalling code allowed this as a optional thing to make _really sure_ you weren't signalling the wrong process. -Alfred * Randall Stewart [090330 16:29] wrote: > > On Mar 30, 2009, at 5:16 PM, Xin LI wrote: > > >-----BEGIN PGP SIGNED MESSAGE----- > >Hash: SHA1 > > > >Randall Stewart wrote: > >>Hi all: > >> > >>I have recently written a small set of routines that allow > >>two process to have a "mutex" between them.. actually it allows > >>all of the threads in any set of processes to have mutexes between > >>themselves ;-) > >> > >>Anyway it seems to be working fairly well.. I still have to write a > >>man > >>page > >>for it (documentation always last).. and eventually I would like to > >>port in > >>some of the WITNESS type features since the mutex's have names.. > >> > >>I probably should also think about scaling it up a bit.. right now > >>its > >>really > >>more for a small scale (100 or less mutexes)... > >> > >>Who should I talk to about getting this in... having it reviewed > >>etc. I > >>think > >>it belongs in libthr since it really needs the tid of the pthreads > >>from the > >>pthread_t type... and for now I have a horrible hack in to get it ;-) > > > >I think davidxu@ deischen@ and julian@? > > > >BTW. How do you handle with one process exit (abnormally) without > >releasing the mutex? Just curious :) > > I have a couple of ways of dealing with this.. > > 1) Of course the initialization routine calls atexit() to get a > "cleanup handler" in place. > 2) Often times, of course, this can fail e.g. you get a SIGSEGV.. or > some such. When you > attach the memory, an audit is done. The audit will validate that > the pid is still alive > and has the particular tid in it. Of course this is not 100% but > as long as the tid's have > not rolled over it should work. The function is also public (need > to add that and many things > to the manual pages ;-D) so that one can call it whenever one > wants :-) > > I will ping Julian and the others... > > R > > > > > > >Cheers, > >- -- > >Xin LI http://www.delphij.net/ > >FreeBSD - The Power to Serve! > >-----BEGIN PGP SIGNATURE----- > >Version: GnuPG v2.0.11 (FreeBSD) > > > >iEYEARECAAYFAknRNrEACgkQi+vbBBjt66DIswCbBWRMJN55c60UTBBIZMRCY4zo > >6hcAnixfVXdtdnn0fT/Z31v0EdyVCVlH > >=JL/U > >-----END PGP SIGNATURE----- > > > > ------------------------------ > Randall Stewart > 803-317-4952 (cell) > 803-345-0391(direct) > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" -- - Alfred Perlstein From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 05:04:43 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2B5F106566B; Tue, 31 Mar 2009 05:04:43 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 524D78FC14; Tue, 31 Mar 2009 05:04:43 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2V54niQ067221 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 01:04:50 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> From: Randall Stewart To: Daniel Eischen In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 01:04:42 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 05:04:43 -0000 Daniel: In-line :-) On Mar 30, 2009, at 7:56 PM, Daniel Eischen wrote: > On Mon, 30 Mar 2009, Randall Stewart wrote: > >> >> On Mar 30, 2009, at 5:22 PM, Daniel Eischen wrote: >> >>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>>> Hi all: >>>> I have recently written a small set of routines that allow >>>> two process to have a "mutex" between them.. actually it allows >>>> all of the threads in any set of processes to have mutexes >>>> between themselves ;-) >>>> Anyway it seems to be working fairly well.. I still have to write >>>> a man page >>>> for it (documentation always last).. and eventually I would like >>>> to port in >>>> some of the WITNESS type features since the mutex's have names.. >>>> I probably should also think about scaling it up a bit.. right >>>> now its really >>>> more for a small scale (100 or less mutexes)... >>>> Who should I talk to about getting this in... having it reviewed >>>> etc. I think >>>> it belongs in libthr since it really needs the tid of the >>>> pthreads from the >>>> pthread_t type... and for now I have a horrible hack in to get >>>> it ;-) >>> The real way to do this is to support PTHREAD_PROCESS_SHARED >>> mutexes within our normal mutex, and to change our current >>> mutex (and cv) types to be structs instead of pointers. >>> The current API, other than the type change, shouldn't >>> change at all. >> >> >> So how do you propose to name the mutex's so that two disparate >> process can locate the same mutex? > > They are placed in shared memory, according to POSIX. > >> I don't see how a pthread_mutex can suffice... we need more than >> just the current mutex... >> >> What am I missing? > > As far as I know, David Xu implemented the kernel hooks > for umtx (the underlying mutex in pthread mutex) to be > shared. As soon as you can place the entire userland > pthread_mutex_t struct in shared memory, it should all > just work (with probably some trivial changes in libthr). > The harder part is versioning all the symbols that > currently think pthread_mutex_t, pthread_cond_t, etc, > are pointers, and defining the structs with enough > foresight so that it is unlikely we have to modify > them in the future (causing a future ABI breakage), > and also aligning them nicely for the various archs. > > You should really look at how POSIX defines process > shared mutex, cvs, etc. See: > > pthread_barrierattr_[gs]etpshared() > pthread_condattr_[gs]etpshared() > pthread_mutexattr_[gs]etpshared() > pthread_wrlockattr_[gs]etsphared() > > You can use this as a starting point: > > http://www.opengroup.org/onlinepubs/009695399/ > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html Ok, I have poked around at these... all the mutex attributes defined here do is set the attributes to shared. There does not seem to be any standard naming mechanism. In fact following the set attributes stuff it gives examples of a condition variable and defines "new local methods" to get a shared semaphore. Creating the actual naming semantics in the new local methods. All that they do on the mutex side is set the attributes to "shared" and basically do the very same thing that I was playing with... i.e. mmap() the file after initializing it... Now granted I did not use the pthread_mutex_*() calls themselves but instead used the umtx() calls directly on the shared memory. Not sure if there is much difference there.. but in any event there is no declaration here in posix on calls for setting "names" so one could then expand the stuff and add witness etc. It looks to me like its more or less a "left open" for future work.. I just love standards bodies ;-D R > > > -- > DE > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 07:01:38 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76D41106564A for ; Tue, 31 Mar 2009 07:01:38 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 432898FC28 for ; Tue, 31 Mar 2009 07:01:38 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id n2V6oRV7019827; Tue, 31 Mar 2009 02:50:27 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Tue, 31 Mar 2009 02:50:27 -0400 (EDT) Date: Tue, 31 Mar 2009 02:50:27 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Randall Stewart In-Reply-To: <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> Message-ID: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 07:01:38 -0000 On Tue, 31 Mar 2009, Randall Stewart wrote: > Daniel: > > In-line :-) > > > On Mar 30, 2009, at 7:56 PM, Daniel Eischen wrote: > >> On Mon, 30 Mar 2009, Randall Stewart wrote: >> >>> >>> What am I missing? >> >> As far as I know, David Xu implemented the kernel hooks >> for umtx (the underlying mutex in pthread mutex) to be >> shared. As soon as you can place the entire userland >> pthread_mutex_t struct in shared memory, it should all >> just work (with probably some trivial changes in libthr). >> The harder part is versioning all the symbols that >> currently think pthread_mutex_t, pthread_cond_t, etc, >> are pointers, and defining the structs with enough >> foresight so that it is unlikely we have to modify >> them in the future (causing a future ABI breakage), >> and also aligning them nicely for the various archs. >> >> You should really look at how POSIX defines process >> shared mutex, cvs, etc. See: >> >> pthread_barrierattr_[gs]etpshared() >> pthread_condattr_[gs]etpshared() >> pthread_mutexattr_[gs]etpshared() >> pthread_wrlockattr_[gs]etsphared() >> >> You can use this as a starting point: >> >> http://www.opengroup.org/onlinepubs/009695399/ >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html >> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html > > Ok, I have poked around at these... all the mutex attributes defined here > do is set the attributes to shared. There does not seem to be any standard > naming mechanism. Naming mechanism for what? Names shouldn't be needed for anything, nor do I think it is desired. > In fact following the set attributes stuff it gives examples of a condition > variable and defines "new local methods" to get a shared semaphore. Creating > the actual naming semantics in the new local methods. All that they > do on the mutex side is set the attributes to "shared" and basically do > the very same thing that I was playing with... i.e. mmap() the file > after initializing it... They define the API. We should not be making new APIs for something that already exists, that applications already know how to use, etc. > Now granted I did not use the pthread_mutex_*() calls themselves but instead > used the umtx() calls directly on the shared memory. Not sure if there is > much difference there.. but in any event there is no declaration here > in posix on calls for setting "names" so one could then expand the stuff > and add witness etc. It looks to me like its more or less a "left open" > for future work.. See above. The proper way to do this is to define the pthread_foo types, mark them as pshared, and have libthr make appropriate umtx calls when they are marked as shared. It is up to the application to define the shared memory segment and place the pthread types in the shared memory. There is no need for "names" on umtx, mutex, whatever. The kernel umtx, as David already pointed out, already handles process shared umtx. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 07:29:24 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C418106566C for ; Tue, 31 Mar 2009 07:29:22 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outR.internet-mail-service.net (outr.internet-mail-service.net [216.240.47.241]) by mx1.freebsd.org (Postfix) with ESMTP id 3D0308FC24 for ; Tue, 31 Mar 2009 07:29:22 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 7AA222DA6E; Tue, 31 Mar 2009 00:29:23 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 836952D6090; Tue, 31 Mar 2009 00:29:18 -0700 (PDT) Message-ID: <49D1C669.5030809@elischer.org> Date: Tue, 31 Mar 2009 00:29:45 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Randall Stewart References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <49D17D76.5060309@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Daniel Eischen , threads@freebsd.org, David Xu Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 07:29:24 -0000 Randall Stewart wrote: > > > > I agree, but its something someone I interviewed with was asking > about... and well I told Huawei to talk to david about it when they asked me. From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 09:30:29 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD32E106564A for ; Tue, 31 Mar 2009 09:30:29 +0000 (UTC) (envelope-from srinivasganji@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.31]) by mx1.freebsd.org (Postfix) with ESMTP id 697198FC08 for ; Tue, 31 Mar 2009 09:30:29 +0000 (UTC) (envelope-from srinivasganji@gmail.com) Received: by yw-out-2324.google.com with SMTP id 5so1620062ywh.13 for ; Tue, 31 Mar 2009 02:30:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=mzElXj3z8GKvWT3RPEjqIocPnDFd5b2SWYy2d8sKeTE=; b=sJKR5AgY5Ihnkqv57cH6cv5tsWbkZY9WMM/4vvFgK62VU8gKl1mFAQviyf212xCou3 BEKKeCDkNJDdm+w2xyo0zEGFRDW8v6oKpYeKW3TnYPK6WlNCsaY9+2P90vAl7NhqFdpq ByHCdXGzsGJySKQ4b7TPHpe200yvRM4g5FGw8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=SdqRzSySe9PwwUA39H/gVhM+8Xpv2NPvO7H9oysu/bN0eBKPvT+/y4Yiqizq1cETbG s2BPHngmG1YYJ3GVu39EB+vrITFKLFIvYFZzKQ//N8df8wIZx4G2hnRccCCtXHelCo3e So4dxcS0xUZNg02pp8q9nHOp6oN7IhZ/oKbpI= MIME-Version: 1.0 Received: by 10.150.143.12 with SMTP id q12mr11884173ybd.153.1238489856083; Tue, 31 Mar 2009 01:57:36 -0700 (PDT) Date: Tue, 31 Mar 2009 14:27:36 +0530 Message-ID: From: Srinivas Ganji To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Is it possible to use the libthr.a file on a Redhat Linux? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 09:30:29 -0000 Dear All, I have tried to use the libthr.a library for compiling an application which is working fine on Redhat system with libpthread library. However, I end up with the following errors. ../lib/linux/libthr.a(thr_sem.o): In function `_sem_init': thr_sem.c:(.text+0x100): undefined reference to `ksem_init' thr_sem.c:(.text+0x115): undefined reference to `ksem_destroy' ../lib/linux/libthr.a(thr_sem.o): In function `_sem_destroy': thr_sem.c:(.text+0x216): undefined reference to `ksem_destroy' ../lib/linux/libthr.a(thr_sem.o): In function `_sem_timedwait': thr_sem.c:(.text+0x2ad): undefined reference to `ksem_timedwait' ../lib/linux/libthr.a(thr_sem.o): In function `_sem_wait': .... .... .... collect2: ld returned 1 exit status make: *** [target] Error 1 So, I have also mentioned the libc.so.7(This is also a FreeBSD libc library) library in our application to remove the above undefined references. So, at that time I got the following errors. /usr/bin/ld: errno@@FBSD_1.0: TLS definition in /lib/libc.so.6 section .tbss mismatches non-TLS definition in ../lib/linux/libc.so section .bss /lib/libc.so.6: could not read symbols: Bad value Here, the lib/libc.so.6 is a Redhat libc library where as ../lib/linux/libc.so is a FreeBSD library (libc.so.7). My question is: Is it possible to use the FreeBSD libthr.a library on a Redhat Linux distribution? Thanks in advance. With Regards, Srinivas G From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 11:05:20 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E939A1065674 for ; Tue, 31 Mar 2009 11:05:20 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 9A0118FC39 for ; Tue, 31 Mar 2009 11:05:20 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2VB5QNj081387 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 07:05:26 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: <081E4C0E-4DD0-45DA-BDFE-89FC2388E1AE@lakerest.net> From: Randall Stewart To: Daniel Eischen In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 07:05:18 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 11:05:21 -0000 On Mar 31, 2009, at 2:50 AM, Daniel Eischen wrote: > On Tue, 31 Mar 2009, Randall Stewart wrote: > >> Daniel: >> >> In-line :-) >> >> >> On Mar 30, 2009, at 7:56 PM, Daniel Eischen wrote: >> >>> On Mon, 30 Mar 2009, Randall Stewart wrote: >>>> What am I missing? >>> As far as I know, David Xu implemented the kernel hooks >>> for umtx (the underlying mutex in pthread mutex) to be >>> shared. As soon as you can place the entire userland >>> pthread_mutex_t struct in shared memory, it should all >>> just work (with probably some trivial changes in libthr). >>> The harder part is versioning all the symbols that >>> currently think pthread_mutex_t, pthread_cond_t, etc, >>> are pointers, and defining the structs with enough >>> foresight so that it is unlikely we have to modify >>> them in the future (causing a future ABI breakage), >>> and also aligning them nicely for the various archs. >>> You should really look at how POSIX defines process >>> shared mutex, cvs, etc. See: >>> pthread_barrierattr_[gs]etpshared() >>> pthread_condattr_[gs]etpshared() >>> pthread_mutexattr_[gs]etpshared() >>> pthread_wrlockattr_[gs]etsphared() >>> You can use this as a starting point: >>> http://www.opengroup.org/onlinepubs/009695399/ >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_barrierattr_setpshared.html >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_condattr_setpshared.html >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_mutexattr_setpshared.html >>> http://www.opengroup.org/onlinepubs/009695399/functions/pthread_rwlockattr_setpshared.html >> >> Ok, I have poked around at these... all the mutex attributes >> defined here >> do is set the attributes to shared. There does not seem to be any >> standard >> naming mechanism. > > Naming mechanism for what? Names shouldn't be needed for anything, > nor do I think it is desired. > >> In fact following the set attributes stuff it gives examples of a >> condition >> variable and defines "new local methods" to get a shared semaphore. >> Creating >> the actual naming semantics in the new local methods. All that they >> do on the mutex side is set the attributes to "shared" and >> basically do >> the very same thing that I was playing with... i.e. mmap() the file >> after initializing it... > > They define the API. We should not be making new APIs for something > that already exists, that applications already know how to use, etc. So what you are saying is ... just let the application do it. Provide nothing but the ability to "mark" a mutex as shared. And let the app figure it out. Hmm.. If one company is asking for this ability i.e. easily do shared mutexs I am sure other folks have wanted it as well. Now rolling your own is a valid thing to do.. but it seems to me providing something for general use is not a bad idea either. The pages you pointed out even show such a mechanism for semaphores... i.e. there definition of semaphore_create(char *shared_name) semaphore_open(char *shared_name) semaphore_post(..) and kin. Curious place for it though.. under pthread_mutex_destroy() ;-) And of course as pointed out this does not solve the quick death syndrome (for that matter neither did I yet but I am thinking about this one ;D)... which is the real hard problem.. and really does require assistance beyond what an application can generally do... IMO having a general library function available is a good thing. If you are not interested in seeing it in libthr where I think it would belong.. thats fine I can build a port or other such... I will send Julian the manual page after I get it built through :-D R > > >> Now granted I did not use the pthread_mutex_*() calls themselves >> but instead >> used the umtx() calls directly on the shared memory. Not sure if >> there is >> much difference there.. but in any event there is no declaration here >> in posix on calls for setting "names" so one could then expand the >> stuff >> and add witness etc. It looks to me like its more or less a "left >> open" >> for future work.. > > See above. The proper way to do this is to define the pthread_foo > types, mark them as pshared, and have libthr make appropriate umtx > calls when they are marked as shared. It is up to the application > to define the shared memory segment and place the pthread types in > the shared memory. There is no need for "names" on umtx, mutex, > whatever. > > The kernel umtx, as David already pointed out, already handles > process shared umtx. > > -- > DE > > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 13:05:11 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E05C1065690 for ; Tue, 31 Mar 2009 13:05:11 +0000 (UTC) (envelope-from gofdt-freebsd-threads@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 1BBD38FC20 for ; Tue, 31 Mar 2009 13:05:11 +0000 (UTC) (envelope-from gofdt-freebsd-threads@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1LodUd-0008ES-DF for freebsd-threads@freebsd.org; Tue, 31 Mar 2009 12:55:03 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 31 Mar 2009 12:55:03 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 31 Mar 2009 12:55:03 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-threads@freebsd.org From: Ivan Voras Date: Tue, 31 Mar 2009 14:53:00 +0200 Lines: 42 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig3547E0B762CDB0B179F65D9A" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090318) In-Reply-To: X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Is it possible to use the libthr.a file on a Redhat Linux? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 13:05:12 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig3547E0B762CDB0B179F65D9A Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Srinivas Ganji wrote: > Dear All, >=20 >=20 >=20 > I have tried to use the libthr.a library for compiling an application w= hich > is working fine on Redhat system with libpthread library. However, I en= d up > with the following errors. > My question is: Is it possible to use the FreeBSD libthr.a library on a= > Redhat Linux distribution? Just to clarify things: you are asking if you can use a system library tightly integrated with its operating system on a completely different, unrelated operating system? --------------enig3547E0B762CDB0B179F65D9A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ0hIsldnAQVacBcgRArsqAJ4sUz/zfV6weGBMH6z/ie9ZZEn7ZwCg3dZS jU3qWVZfKb9EmBecR92WvgI= =xeKI -----END PGP SIGNATURE----- --------------enig3547E0B762CDB0B179F65D9A-- From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 13:12:03 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E374D10656C4 for ; Tue, 31 Mar 2009 13:12:03 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 820F48FC16 for ; Tue, 31 Mar 2009 13:12:03 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id n2VDC2CT028653; Tue, 31 Mar 2009 09:12:02 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Tue, 31 Mar 2009 09:12:02 -0400 (EDT) Date: Tue, 31 Mar 2009 09:12:01 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Randall Stewart In-Reply-To: <081E4C0E-4DD0-45DA-BDFE-89FC2388E1AE@lakerest.net> Message-ID: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <081E4C0E-4DD0-45DA-BDFE-89FC2388E1AE@lakerest.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org Subject: Re: A mutex for inter-process ;-) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 13:12:04 -0000 On Tue, 31 Mar 2009, Randall Stewart wrote: > > On Mar 31, 2009, at 2:50 AM, Daniel Eischen wrote: > >> >> They define the API. We should not be making new APIs for something >> that already exists, that applications already know how to use, etc. > > So what you are saying is ... just let the application do it. Provide > nothing but the ability to "mark" a mutex as shared. And let the > app figure it out. Correct. We do not need nor want any more SYS V IPC stuff and have more utilities like ipcrm, ipcs, etc to deal with them. POSIX already defines the API for us and tells us how to use process shared mutexes, et al. > Hmm.. If one company is asking for this ability i.e. easily > do shared mutexs I am sure other folks have wanted it as well. > Now rolling your own is a valid thing to do.. but it seems to > me providing something for general use is not a bad idea either. There already is something for general usage, see POSIX ;-) > The pages you pointed out even show such a mechanism for > semaphores... i.e. there definition of > > semaphore_create(char *shared_name) > semaphore_open(char *shared_name) > semaphore_post(..) > > and kin. > > > Curious place for it though.. under pthread_mutex_destroy() ;-) They are also in the POSIX spec under their own entries. > And of course as pointed out this does not solve the quick death > syndrome (for that matter neither did I yet but I am thinking > about this one ;D)... which is the real hard problem.. and really > does require assistance beyond what an application can generally > do... No, the kernel can do it under the existing umutex API. You should really be asking David Xu this stuff, but the kernel can remove any of its own resources (if it has any allocated) when the shared memory is removed, or it may be possible to have POSIX mutex robustness by having the kernel unlock or deallocate umutex resources upon process termination. The point is, it is possible for the kernel to do this, if it already doesn't, using the existing umutex APIs. > IMO having a general library function available is a good thing. If > you are not interested in seeing it in libthr where I think it would > belong.. thats fine I can build a port or other such... > > I will send Julian the manual page after I get it built through :-D There already is an API for doing what you want, and sorry, but no, I don't think adding a BSD-only API that is different from something POSIX already defines is a good thing ;-) -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 15:03:35 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B572106566B; Tue, 31 Mar 2009 15:03:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 394658FC26; Tue, 31 Mar 2009 15:03:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id D414D46B51; Tue, 31 Mar 2009 11:03:34 -0400 (EDT) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n2VF3Na1032302; Tue, 31 Mar 2009 11:03:29 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-threads@freebsd.org Date: Tue, 31 Mar 2009 10:38:43 -0400 User-Agent: KMail/1.9.7 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200903311038.43401.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 31 Mar 2009 11:03:29 -0400 (EDT) X-Virus-Scanned: ClamAV 0.94.2/9186/Tue Mar 31 05:51:33 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: threads@freebsd.org Subject: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 15:03:36 -0000 On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > Ok, I have poked around at these... all the mutex attributes defined here > > do is set the attributes to shared. There does not seem to be any standard > > naming mechanism. > > Naming mechanism for what? Names shouldn't be needed for anything, > nor do I think it is desired. Off topic: names would be very helpful to port witness to pthreads. The thoughts I have had for doing this though would be to add a new _np attribute to set the name. I actually would like to write a 'libwitness' that basically overrides the various symbols and provides the name_np attribute and implement witness in the shared library on top of whatever pthreads library is in use. This would also allow it to be portable to other OS's. (Well, it could break pshared mutexes, but using the pointer-style types, you could have the libwitness allocate its own "mutex" structure which has a "real" mutex inside of it along with the name and other per-lock data it tracks. It would then forward mutex operations to the real pthreads library after performing LOR checks, etc.). -- John Baldwin From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 15:03:35 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B572106566B; Tue, 31 Mar 2009 15:03:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 394658FC26; Tue, 31 Mar 2009 15:03:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id D414D46B51; Tue, 31 Mar 2009 11:03:34 -0400 (EDT) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n2VF3Na1032302; Tue, 31 Mar 2009 11:03:29 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-threads@freebsd.org Date: Tue, 31 Mar 2009 10:38:43 -0400 User-Agent: KMail/1.9.7 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200903311038.43401.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 31 Mar 2009 11:03:29 -0400 (EDT) X-Virus-Scanned: ClamAV 0.94.2/9186/Tue Mar 31 05:51:33 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: threads@freebsd.org Subject: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 15:03:36 -0000 On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > Ok, I have poked around at these... all the mutex attributes defined here > > do is set the attributes to shared. There does not seem to be any standard > > naming mechanism. > > Naming mechanism for what? Names shouldn't be needed for anything, > nor do I think it is desired. Off topic: names would be very helpful to port witness to pthreads. The thoughts I have had for doing this though would be to add a new _np attribute to set the name. I actually would like to write a 'libwitness' that basically overrides the various symbols and provides the name_np attribute and implement witness in the shared library on top of whatever pthreads library is in use. This would also allow it to be portable to other OS's. (Well, it could break pshared mutexes, but using the pointer-style types, you could have the libwitness allocate its own "mutex" structure which has a "real" mutex inside of it along with the name and other per-lock data it tracks. It would then forward mutex operations to the real pthreads library after performing LOR checks, etc.). -- John Baldwin From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 15:11:04 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98372106568B; Tue, 31 Mar 2009 15:11:04 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 1E5EF8FC15; Tue, 31 Mar 2009 15:11:04 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2VFBAne091217 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 11:11:10 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: From: Randall Stewart To: John Baldwin In-Reply-To: <200903311038.43401.jhb@freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 11:11:02 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 15:11:05 -0000 This was one of the places I was heading (as I wrote privately to Daniel ;-D) I suppose I can share it all i.e. the pthread mutex stuff will of course work with shared mutexe's but it won't: a) Build an easy to use semantic for the app to agree on sharing memory.. i.e. you have left undefined how the process figure out what they are sharing. There is some value in setting up a easy semantic for app dev's to use. b) What happens when a process exits or hits a core dump while holding one of these mutex's? Is this what you are thinking the PROCESS_SHARED would do?? c) If you build something to do so you have some nice way of naming mutex's you can do something similar to our WITNESS option in the kernel... this is something the few times I have played in user space recently that I have missed... having LOR warnings and such can be a useful tool. You can't have this without IMO. I was am interested in a/b but one of my long term intents is to do ;-) R On Mar 31, 2009, at 10:38 AM, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes >>> defined here >>> do is set the attributes to shared. There does not seem to be any >>> standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. > The > thoughts I have had for doing this though would be to add a new _np > attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np > attribute > and implement witness in the shared library on top of whatever > pthreads > library is in use. This would also allow it to be portable to other > OS's. > (Well, it could break pshared mutexes, but using the pointer-style > types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock > data it > tracks. It would then forward mutex operations to the real pthreads > library > after performing LOR checks, etc.). > > -- > John Baldwin > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 15:11:04 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98372106568B; Tue, 31 Mar 2009 15:11:04 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id 1E5EF8FC15; Tue, 31 Mar 2009 15:11:04 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2VFBAne091217 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 11:11:10 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: From: Randall Stewart To: John Baldwin In-Reply-To: <200903311038.43401.jhb@freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 11:11:02 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> X-Mailer: Apple Mail (2.930.3) Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 15:11:05 -0000 This was one of the places I was heading (as I wrote privately to Daniel ;-D) I suppose I can share it all i.e. the pthread mutex stuff will of course work with shared mutexe's but it won't: a) Build an easy to use semantic for the app to agree on sharing memory.. i.e. you have left undefined how the process figure out what they are sharing. There is some value in setting up a easy semantic for app dev's to use. b) What happens when a process exits or hits a core dump while holding one of these mutex's? Is this what you are thinking the PROCESS_SHARED would do?? c) If you build something to do so you have some nice way of naming mutex's you can do something similar to our WITNESS option in the kernel... this is something the few times I have played in user space recently that I have missed... having LOR warnings and such can be a useful tool. You can't have this without IMO. I was am interested in a/b but one of my long term intents is to do ;-) R On Mar 31, 2009, at 10:38 AM, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes >>> defined here >>> do is set the attributes to shared. There does not seem to be any >>> standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. > The > thoughts I have had for doing this though would be to add a new _np > attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np > attribute > and implement witness in the shared library on top of whatever > pthreads > library is in use. This would also allow it to be portable to other > OS's. > (Well, it could break pshared mutexes, but using the pointer-style > types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock > data it > tracks. It would then forward mutex operations to the real pthreads > library > after performing LOR checks, etc.). > > -- > John Baldwin > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 15:34:37 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 792571065721 for ; Tue, 31 Mar 2009 15:34:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 459918FC19 for ; Tue, 31 Mar 2009 15:34:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (pool-98-109-39-197.nwrknj.fios.verizon.net [98.109.39.197]) by cyrus.watson.org (Postfix) with ESMTPSA id C6A4546B1A; Tue, 31 Mar 2009 11:34:36 -0400 (EDT) Received: from localhost (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id n2VFYUYW032521; Tue, 31 Mar 2009 11:34:30 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Randall Stewart Date: Tue, 31 Mar 2009 11:27:06 -0400 User-Agent: KMail/1.9.7 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <200903311038.43401.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200903311127.06447.jhb@freebsd.org> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Tue, 31 Mar 2009 11:34:31 -0400 (EDT) X-Virus-Scanned: ClamAV 0.94.2/9186/Tue Mar 31 05:51:33 2009 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 15:34:39 -0000 On Tuesday 31 March 2009 11:11:02 am Randall Stewart wrote: > This was one of the places I was heading (as I wrote privately to > Daniel ;-D) > > I suppose I can share it all i.e. the pthread mutex stuff > will of course work with shared mutexe's but it won't: > > a) Build an easy to use semantic for the app to agree on sharing > memory.. i.e. you > have left undefined how the process figure out what they are > sharing. There is > some value in setting up a easy semantic for app dev's to use. You can use shm_open() to share memory regions by name and then create mutexes and condvars in that. > interface> > > b) What happens when a process exits or hits a core dump while holding > one > of these mutex's? Is this what you are thinking the PROCESS_SHARED > would > do?? There is a "robust" mutex extension David Xu mentioned. Presumably though what would happen is that when one thread went to block on a mutex, the kernel (in the umtx code) would see if the current owning thread had exited, and if so, do something "appropriate" (break the lock, etc.) at that time. I think a (pid, tid, process starttime) tuple would work ok for detecting this. > the > PROCESS_SHARED could be made to help here> > > c) If you build something to do so you have some nice way of naming > mutex's you can do something similar to our WITNESS option in the > kernel... this is something the few times I have played in user > space recently that I have missed... having LOR warnings and such > can be a useful tool. You can't have this without IMO. > > > I was am interested in a/b but one of my long term intents is to do > ;-) All my WITNESS thoughts are completely separate from PROCESS_SHARED mutexes and I think actually break PROCESS_SHARED mutexes. (Though perhaps they can still be made to work but using something far more invasive where WITNESS defines its own pthread_mutex structure that the app has to be compiled against.) -- John Baldwin From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 17:32:59 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD51510656DC; Tue, 31 Mar 2009 17:32:59 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 6D8948FC1F; Tue, 31 Mar 2009 17:32:59 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id n2VHWvo9008470; Tue, 31 Mar 2009 13:32:58 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Tue, 31 Mar 2009 13:32:58 -0400 (EDT) Date: Tue, 31 Mar 2009 13:32:57 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: John Baldwin In-Reply-To: <200903311038.43401.jhb@freebsd.org> Message-ID: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 17:33:02 -0000 On Tue, 31 Mar 2009, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes defined here >>> do is set the attributes to shared. There does not seem to be any standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I think this is all overkill when we don't even have proper pthread synchronization primitives in libthr that can be used in shared memory. And if we also implement robust mutexes, then you have additional error-checking. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 17:32:59 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD51510656DC; Tue, 31 Mar 2009 17:32:59 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 6D8948FC1F; Tue, 31 Mar 2009 17:32:59 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.3/8.14.3/NETPLEX) with ESMTP id n2VHWvo9008470; Tue, 31 Mar 2009 13:32:58 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Tue, 31 Mar 2009 13:32:58 -0400 (EDT) Date: Tue, 31 Mar 2009 13:32:57 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: John Baldwin In-Reply-To: <200903311038.43401.jhb@freebsd.org> Message-ID: References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 17:33:02 -0000 On Tue, 31 Mar 2009, John Baldwin wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: >>> Ok, I have poked around at these... all the mutex attributes defined here >>> do is set the attributes to shared. There does not seem to be any standard >>> naming mechanism. >> >> Naming mechanism for what? Names shouldn't be needed for anything, >> nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I think this is all overkill when we don't even have proper pthread synchronization primitives in libthr that can be used in shared memory. And if we also implement robust mutexes, then you have additional error-checking. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 17:52:35 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 016DB1065740 for ; Tue, 31 Mar 2009 17:52:35 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id DB9788FC12 for ; Tue, 31 Mar 2009 17:52:34 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id CBE0C1A3C39; Tue, 31 Mar 2009 10:44:01 -0700 (PDT) Date: Tue, 31 Mar 2009 10:44:01 -0700 From: Alfred Perlstein To: John Baldwin Message-ID: <20090331174401.GD92757@elvis.mu.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200903311038.43401.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 17:52:35 -0000 * John Baldwin [090331 08:03] wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > > Ok, I have poked around at these... all the mutex attributes defined here > > > do is set the attributes to shared. There does not seem to be any standard > > > naming mechanism. > > > > Naming mechanism for what? Names shouldn't be needed for anything, > > nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I've heard of this work being done by multiple other places in house. so you have a great idea, if you have time to run with it, it would likely eb greatly appreciated and give FreeBSD a big bump as a development platform. -alfred From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 18:22:35 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23C4110657BB; Tue, 31 Mar 2009 18:22:35 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 100388FC0A; Tue, 31 Mar 2009 18:22:34 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id CBE0C1A3C39; Tue, 31 Mar 2009 10:44:01 -0700 (PDT) Date: Tue, 31 Mar 2009 10:44:01 -0700 From: Alfred Perlstein To: John Baldwin Message-ID: <20090331174401.GD92757@elvis.mu.org> References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200903311038.43401.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 18:22:35 -0000 * John Baldwin [090331 08:03] wrote: > On Tuesday 31 March 2009 2:50:27 am Daniel Eischen wrote: > > > Ok, I have poked around at these... all the mutex attributes defined here > > > do is set the attributes to shared. There does not seem to be any standard > > > naming mechanism. > > > > Naming mechanism for what? Names shouldn't be needed for anything, > > nor do I think it is desired. > > Off topic: names would be very helpful to port witness to pthreads. The > thoughts I have had for doing this though would be to add a new _np attribute > to set the name. I actually would like to write a 'libwitness' that > basically overrides the various symbols and provides the name_np attribute > and implement witness in the shared library on top of whatever pthreads > library is in use. This would also allow it to be portable to other OS's. > (Well, it could break pshared mutexes, but using the pointer-style types, you > could have the libwitness allocate its own "mutex" structure which has > a "real" mutex inside of it along with the name and other per-lock data it > tracks. It would then forward mutex operations to the real pthreads library > after performing LOR checks, etc.). I've heard of this work being done by multiple other places in house. so you have a great idea, if you have time to run with it, it would likely eb greatly appreciated and give FreeBSD a big bump as a development platform. -alfred From owner-freebsd-threads@FreeBSD.ORG Tue Mar 31 19:56:10 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 643F7106564A; Tue, 31 Mar 2009 19:56:10 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from lakerest.net (unknown [IPv6:2001:240:585:2:203:6dff:fe1a:4ddc]) by mx1.freebsd.org (Postfix) with ESMTP id DD9F38FC16; Tue, 31 Mar 2009 19:56:09 +0000 (UTC) (envelope-from rrs@lakerest.net) Received: from [10.1.1.53] ([10.1.1.53]) (authenticated bits=0) by lakerest.net (8.14.3/8.14.3) with ESMTP id n2VJuFvl002628 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 31 Mar 2009 15:56:16 -0400 (EDT) (envelope-from rrs@lakerest.net) Message-Id: <989B6D28-243F-4A13-8C9D-F9C1CD5C2D77@lakerest.net> From: Randall Stewart To: John Baldwin In-Reply-To: <200903311127.06447.jhb@freebsd.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 31 Mar 2009 15:56:07 -0400 References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <200903311038.43401.jhb@freebsd.org> <200903311127.06447.jhb@freebsd.org> X-Mailer: Apple Mail (2.930.3) Cc: freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2009 19:56:12 -0000 On Mar 31, 2009, at 11:27 AM, John Baldwin wrote: > On Tuesday 31 March 2009 11:11:02 am Randall Stewart wrote: >> This was one of the places I was heading (as I wrote privately to >> Daniel ;-D) >> >> I suppose I can share it all i.e. the pthread mutex stuff >> will of course work with shared mutexe's but it won't: >> >> a) Build an easy to use semantic for the app to agree on sharing >> memory.. i.e. you >> have left undefined how the process figure out what they are >> sharing. There is >> some value in setting up a easy semantic for app dev's to use. > > You can use shm_open() to share memory regions by name and then > create mutexes > and condvars in that. Thats what my little ipc_mutex...() functions do ;-) > > >> > interface> >> >> b) What happens when a process exits or hits a core dump while >> holding >> one >> of these mutex's? Is this what you are thinking the PROCESS_SHARED >> would >> do?? > > There is a "robust" mutex extension David Xu mentioned. Presumably > though > what would happen is that when one thread went to block on a mutex, > the > kernel (in the umtx code) would see if the current owning thread had > exited, > and if so, do something "appropriate" (break the lock, etc.) at that > time. I > think a (pid, tid, process starttime) tuple would work ok for > detecting this. If that is implemented.. I need to go look into this.. what I found when I was doing some digging is that umtx was used.. and I saw no way to make them "robust".. it may be something that needs adding.. > > >> > the >> PROCESS_SHARED could be made to help here> >> >> c) If you build something to do so you have some nice way of >> naming >> mutex's you can do something similar to our WITNESS option in the >> kernel... this is something the few times I have played in user >> space recently that I have missed... having LOR warnings and such >> can be a useful tool. You can't have this without IMO. >> >> >> I was am interested in a/b but one of my long term intents is to do >> ;-) > > All my WITNESS thoughts are completely separate from PROCESS_SHARED > mutexes > and I think actually break PROCESS_SHARED mutexes. (Though perhaps > they can > still be made to work but using something far more invasive where > WITNESS > defines its own pthread_mutex structure that the app has to be > compiled > against.) Which could also be put in shared memory so that you could learn the lock ordering across multiple processes ;-) R > > > -- > John Baldwin > ------------------------------ Randall Stewart 803-317-4952 (cell) 803-345-0391(direct) From owner-freebsd-threads@FreeBSD.ORG Wed Apr 1 01:34:43 2009 Return-Path: Delivered-To: threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24CDA1065673; Wed, 1 Apr 2009 01:34:43 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E84AE8FC0A; Wed, 1 Apr 2009 01:34:42 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from apple.my.domain (root@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n311YcqO016994; Wed, 1 Apr 2009 01:34:40 GMT (envelope-from davidxu@freebsd.org) Message-ID: <49D2C4B0.2020805@freebsd.org> Date: Wed, 01 Apr 2009 09:34:40 +0800 From: David Xu User-Agent: Thunderbird 2.0.0.9 (X11/20080612) MIME-Version: 1.0 To: Randall Stewart References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2009 01:34:43 -0000 Randall Stewart wrote: > This was one of the places I was heading (as I wrote privately to Daniel > ;-D) > > I suppose I can share it all i.e. the pthread mutex stuff > will of course work with shared mutexe's but it won't: > > a) Build an easy to use semantic for the app to agree on sharing > memory.. i.e. you > have left undefined how the process figure out what they are sharing. > There is > some value in setting up a easy semantic for app dev's to use. > > interface> > > b) What happens when a process exits or hits a core dump while holding one > of these mutex's? Is this what you are thinking the PROCESS_SHARED would > do?? > > PROCESS_SHARED could be made to help here> > Yes, kernel has to involve in this area, maybe all locking and unlocking for PROCESS_SHARED mutex should be done in kernel, so kernel can remember a list of mutex the current thread owned, when the thread exits for whatever reason, kernel should reset the mutexes to a state and wake up threads waiting on these mutexes. It seems that Solaris does it in this way, another way is setting a mutex list pointer in kernel, but the list itself is in user address space, it is a bit tricky for kernel to figure out the list's intermediate states when the thread is killed and fix the mutex states, the benefit is locking and unlocking are fast because they are done by userland when possible, it seems Linux does it in this way. > c) If you build something to do so you have some nice way of naming > mutex's you can do something similar to our WITNESS option in the > kernel... this is something the few times I have played in user > space recently that I have missed... having LOR warnings and such > can be a useful tool. You can't have this without IMO. > > > I was am interested in a/b but one of my long term intents is to do ;-) > > > R From owner-freebsd-threads@FreeBSD.ORG Wed Apr 1 01:34:43 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24CDA1065673; Wed, 1 Apr 2009 01:34:43 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E84AE8FC0A; Wed, 1 Apr 2009 01:34:42 +0000 (UTC) (envelope-from davidxu@freebsd.org) Received: from apple.my.domain (root@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n311YcqO016994; Wed, 1 Apr 2009 01:34:40 GMT (envelope-from davidxu@freebsd.org) Message-ID: <49D2C4B0.2020805@freebsd.org> Date: Wed, 01 Apr 2009 09:34:40 +0800 From: David Xu User-Agent: Thunderbird 2.0.0.9 (X11/20080612) MIME-Version: 1.0 To: Randall Stewart References: <7D4F6788-0F12-4863-9635-7FADA9115D16@lakerest.net> <9157F968-5CCF-451C-9BA0-E12A957D6B38@lakerest.net> <200903311038.43401.jhb@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: threads@freebsd.org, freebsd-threads@freebsd.org Subject: Re: WITNESS for pthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2009 01:34:43 -0000 Randall Stewart wrote: > This was one of the places I was heading (as I wrote privately to Daniel > ;-D) > > I suppose I can share it all i.e. the pthread mutex stuff > will of course work with shared mutexe's but it won't: > > a) Build an easy to use semantic for the app to agree on sharing > memory.. i.e. you > have left undefined how the process figure out what they are sharing. > There is > some value in setting up a easy semantic for app dev's to use. > > interface> > > b) What happens when a process exits or hits a core dump while holding one > of these mutex's? Is this what you are thinking the PROCESS_SHARED would > do?? > > PROCESS_SHARED could be made to help here> > Yes, kernel has to involve in this area, maybe all locking and unlocking for PROCESS_SHARED mutex should be done in kernel, so kernel can remember a list of mutex the current thread owned, when the thread exits for whatever reason, kernel should reset the mutexes to a state and wake up threads waiting on these mutexes. It seems that Solaris does it in this way, another way is setting a mutex list pointer in kernel, but the list itself is in user address space, it is a bit tricky for kernel to figure out the list's intermediate states when the thread is killed and fix the mutex states, the benefit is locking and unlocking are fast because they are done by userland when possible, it seems Linux does it in this way. > c) If you build something to do so you have some nice way of naming > mutex's you can do something similar to our WITNESS option in the > kernel... this is something the few times I have played in user > space recently that I have missed... having LOR warnings and such > can be a useful tool. You can't have this without IMO. > > > I was am interested in a/b but one of my long term intents is to do ;-) > > > R From owner-freebsd-threads@FreeBSD.ORG Thu Apr 2 11:16:11 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28DE5106564A for ; Thu, 2 Apr 2009 11:16:11 +0000 (UTC) (envelope-from fdeliege@gmail.com) Received: from mail-ew0-f171.google.com (mail-ew0-f171.google.com [209.85.219.171]) by mx1.freebsd.org (Postfix) with ESMTP id 874CC8FC14 for ; Thu, 2 Apr 2009 11:16:10 +0000 (UTC) (envelope-from fdeliege@gmail.com) Received: by ewy19 with SMTP id 19so452149ewy.43 for ; Thu, 02 Apr 2009 04:16:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=TRHNXB4mo7b1LBOl/cPzj7dFMFr3b/V0WdkJ4tkpeqI=; b=TSRx8SIRTlpdl5pqaLiEcp51jOZTcHdWip4vI6EgZsdTJ6iX99hJbZSMyVNoEGH2ly lIRYN8EjI9BF4YvNltqhexJYVX2ElTO22R3rDqRRo9jeK9RwJlJcCAg2RYJVO/e+vdPV nckuri00XaV8GcYRwmTgRWE6DKMcwEgp7tPUA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=CZFZ8GFFYuuGOINwqEooeTW9aaO5Nc1f2s82PSq93f1w0pFtSO2SkrCQ+NsznYkkjh d7kfJi7peaRVrRkp1/KDASL4JRy/kh7UF0Z2fAUjvxFjUGXwwTpYC++LVPz1hKJ5RXM0 hSyjNituHHYp7DFoxa2T8StxW/u+lVj0C8kGs= MIME-Version: 1.0 Received: by 10.210.16.10 with SMTP id 10mr6792948ebp.35.1238669156536; Thu, 02 Apr 2009 03:45:56 -0700 (PDT) Date: Thu, 2 Apr 2009 12:45:56 +0200 Message-ID: <92c2d900904020345x1541daefy85705fa049b54d8e@mail.gmail.com> From: =?ISO-8859-1?Q?Fran=E7ois_Deli=E8ge?= To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: SPI openmp parallel critical X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2009 11:16:11 -0000 Hi list, I am writing a C user defined function that is loading values obtained from an SPI table and that has to perform some operation on each of these values. The values are binary and have a variable size. Since CPU is a bottleneck and since I have multiple cores, I would like the operations to be performed in parallel. Everything goes fine if I define as a critical segment the part where I get the data from tuptable. However, if I remove the critical section I get this error: ERROR: lock 53 is not held I don't always get the error, but it always crashes if I start to have a few values in the SPI table. I would like to eliminate that critical section. Any advice ? I guess someone else already ran in this kind of problem. :-) SPI_connect(); ret = SPI_execute(command, 1, 0); proc = SPI_processed; #pragma omp parallel for shared(proc, SPI_tuptable ) private( mydata, isnull) for (j = 0; j < proc; j++) { // returns datum from the first element #pragma omp critical // would like to eliminate this { mydata = (mydatatype *)PG_DETOAST_DATUM(SPI_getbinval(SPI_tuptable->vals[j], SPI_tuptable->tupdesc, 1, &isnull)); } ... // do something with mydata } SPI_finish(); Cheers, Francois