From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 11:03:58 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 71FF216A4CE for ; Mon, 5 Jan 2004 11:03:58 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0495D43D93 for ; Mon, 5 Jan 2004 11:01:43 -0800 (PST) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) i05J1hFR016162 for ; Mon, 5 Jan 2004 11:01:43 -0800 (PST) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.10/8.12.10/Submit) id i05J1gG3016156 for freebsd-threads@freebsd.org; Mon, 5 Jan 2004 11:01:42 -0800 (PST) (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 5 Jan 2004 11:01:42 -0800 (PST) Message-Id: <200401051901.i05J1gG3016156@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 19:03:58 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2002/01/16] kern/33951 threads pthread_cancel is ignored 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2000/08/26] misc/20861 threads libc_r does not honor socket timeouts o [2001/01/19] bin/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] bin/24632 threads libc_r delicate deviation from libc in ha o [2001/01/25] misc/24641 threads pthread_rwlock_rdlock can deadlock o [2001/04/02] bin/26307 threads libc_r aborts when using the KDE media pl o [2001/10/31] bin/31661 threads pthread_kill signal handler doesn't get s o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/02/01] i386/34536 threads accept() blocks other threads o [2002/03/07] bin/35622 threads sigaltstack is missing in libc_r o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/06/27] bin/39922 threads [PATCH?] Threaded applications executed w o [2002/08/04] misc/41331 threads Pthread library open sets O_NONBLOCK flag o [2002/10/10] kern/43887 threads abnormal CPU useage when use pthread_mute o [2003/03/02] bin/48856 threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] bin/49087 threads Signals lost in programs linked with libc a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin o [2003/05/07] bin/51949 threads thread in accept cannot be cancelled o [2003/05/30] kern/52817 threads top(1) shows garbage for threaded process 19 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/05/25] misc/18824 threads gethostbyname is not thread safe o [2000/10/21] misc/22190 threads A threaded read(2) from a socketpair(2) f o [2001/09/09] bin/30464 threads pthread mutex attributes -- pshared o [2002/05/02] bin/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwri o [2002/07/16] misc/40671 threads pthread_cancel doesn't remove thread from 5 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 11:20:35 2004 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A6D416A4CE for ; Mon, 5 Jan 2004 11:20:35 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 195DA43D2D for ; Mon, 5 Jan 2004 11:20:23 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) i05JKMFR022598 for ; Mon, 5 Jan 2004 11:20:22 -0800 (PST) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.10/8.12.10/Submit) id i05JKM6A022597; Mon, 5 Jan 2004 11:20:22 -0800 (PST) (envelope-from gnats) Date: Mon, 5 Jan 2004 11:20:22 -0800 (PST) Message-Id: <200401051920.i05JKM6A022597@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Earl Chew Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Earl Chew List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 19:20:35 -0000 The following reply was made to PR misc/24641; it has been noted by GNATS. From: Earl Chew To: The Thodes Cc: freebsd-gnats-submit@FreeBSD.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock Date: Mon, 05 Jan 2004 11:11:40 -0800 The Thodes wrote: > First, Earl, GCC objected to your test program. Apologies. I compiled this as a C++ program (g++) hence the missing parameter names. > The patch given is needed to make it compile (in my case, at least). Yes, your patch will fill in the "junk" names, and allow the C compiler to parse the source successfully. > Also, it (your test program) > doesn't even get to trying to acquire the second read lock. It hangs > trying to acquire the write lock. The output is provided in this (long) > follow-up. Oops... there was a cut-and-paste error. Please look at the amended program below. Earl -- #include #include #include #include static pthread_rwlock_t rwlock1 = PTHREAD_RWLOCK_INITIALIZER; static volatile int wrStarted; void * wrfunc(void *unused) { printf("Attempt to acquire write lock\n"); assert(pthread_rwlock_wrlock(&rwlock1) == 0); printf("Acquired write lock\n"); assert(pthread_rwlock_unlock(&rwlock1) == 0); return 0; } static volatile int rdStarted; void * rdfunc(void *unused) { printf("Attempt to acquire read lock first\n"); assert(pthread_rwlock_rdlock(&rwlock1) == 0); printf("Acquired read lock first\n"); rdStarted = 1; while (wrStarted == 0) sleep(1); printf("Attempt to acquire read lock second\n"); assert(pthread_rwlock_rdlock(&rwlock1) == 0); printf("Acquired read lock second\n"); assert(pthread_rwlock_unlock(&rwlock1) == 0); assert(pthread_rwlock_unlock(&rwlock1) == 0); return 0; } int main(int argc, char **argv) { pthread_t wrt; pthread_t rdt; pthread_attr_t a; assert(pthread_rwlock_init(&rwlock1, 0) == 0); assert(pthread_attr_init(&a) == 0); assert(pthread_create(&rdt, &a, rdfunc, NULL) == 0); while (rdStarted == 0) sleep(1); assert(pthread_create(&wrt, &a, wrfunc, NULL) == 0); assert(pthread_join(wrt, 0) == 0); assert(pthread_join(rdt, 0) == 0); assert(pthread_rwlock_destroy(&rwlock1) == 0); assert(pthread_detach(wrt) == 0); assert(pthread_detach(rdt) == 0); return 0; } From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 14:30:25 2004 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0171C16A4D1 for ; Mon, 5 Jan 2004 14:30:25 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8402443D4C for ; Mon, 5 Jan 2004 14:30:14 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) i05MUEFR039024 for ; Mon, 5 Jan 2004 14:30:14 -0800 (PST) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.10/8.12.10/Submit) id i05MUEov039020; Mon, 5 Jan 2004 14:30:14 -0800 (PST) (envelope-from gnats) Date: Mon, 5 Jan 2004 14:30:14 -0800 (PST) Message-Id: <200401052230.i05MUEov039020@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Earl Chew Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Earl Chew List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 22:30:25 -0000 The following reply was made to PR misc/24641; it has been noted by GNATS. From: Earl Chew To: aspiesrule@mcleodusa.net Cc: freebsd-gnats-submit@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock Date: Mon, 05 Jan 2004 14:21:00 -0800 aspiesrule@mcleodusa.net wrote: > Same results, except for the fact that stepping thru your code in GDB yields > a segfault in KERNEL32!IsBadWritePtr. It must be the holidays! Ok, I've added the missing line at the beginning of wrfunc(). Earl -- #include #include #include #include static pthread_rwlock_t rwlock1 = PTHREAD_RWLOCK_INITIALIZER; static volatile int wrStarted; void * wrfunc(void *unused) { wrStarted = 1; printf("Attempt to acquire write lock\n"); assert(pthread_rwlock_wrlock(&rwlock1) == 0); printf("Acquired write lock\n"); assert(pthread_rwlock_unlock(&rwlock1) == 0); return 0; } static volatile int rdStarted; void * rdfunc(void *unused) { printf("Attempt to acquire read lock first\n"); assert(pthread_rwlock_rdlock(&rwlock1) == 0); printf("Acquired read lock first\n"); rdStarted = 1; while (wrStarted == 0) sleep(1); printf("Attempt to acquire read lock second\n"); assert(pthread_rwlock_rdlock(&rwlock1) == 0); printf("Acquired read lock second\n"); assert(pthread_rwlock_unlock(&rwlock1) == 0); assert(pthread_rwlock_unlock(&rwlock1) == 0); return 0; } int main(int argc, char **argv) { pthread_t wrt; pthread_t rdt; pthread_attr_t a; assert(pthread_rwlock_init(&rwlock1, 0) == 0); assert(pthread_attr_init(&a) == 0); assert(pthread_create(&rdt, &a, rdfunc, NULL) == 0); while (rdStarted == 0) sleep(1); assert(pthread_create(&wrt, &a, wrfunc, NULL) == 0); assert(pthread_join(wrt, 0) == 0); assert(pthread_join(rdt, 0) == 0); assert(pthread_rwlock_destroy(&rwlock1) == 0); assert(pthread_detach(wrt) == 0); assert(pthread_detach(rdt) == 0); return 0; } From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 15:30:19 2004 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F003016A4CE for ; Mon, 5 Jan 2004 15:30:19 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 99B0B43D67 for ; Mon, 5 Jan 2004 15:30:14 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) i05NUEFR044746 for ; Mon, 5 Jan 2004 15:30:14 -0800 (PST) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.10/8.12.10/Submit) id i05NUEMd044745; Mon, 5 Jan 2004 15:30:14 -0800 (PST) (envelope-from gnats) Date: Mon, 5 Jan 2004 15:30:14 -0800 (PST) Message-Id: <200401052330.i05NUEMd044745@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Earl Chew Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Earl Chew List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 23:30:20 -0000 The following reply was made to PR misc/24641; it has been noted by GNATS. From: Earl Chew To: aspiesrule@mcleodusa.net Cc: freebsd-gnats-submit@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock Date: Mon, 05 Jan 2004 15:26:25 -0800 aspiesrule@mcleodusa.net wrote: > Oops, running your new program yields an assertion failure at line 37 and a > core dump. The output is as follows: > > --snip-- > Attempt to acquire read lock first > Acquired read lock first > Attempt to acquire write lock > Attempt to acquire read lock second > assertion "pthread_rwlock_rdlock(&rwlock1) == 0" failed: file "test.c", line > 37 > Aborted (core dumped) Right. The test is (correctly) showing an inadequacy in the rwlock implementation. I refer you to the original bug report: http://www.freebsd.org/cgi/query-pr.cgi?pr=24641 and the referenced articles: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=slrn87nusa.rsv.kaz%40ashi.FootPrints.net http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=38828D22.7A98%40LambdaCS.com Earl From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 16:28:52 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 38DE516A4CE for ; Mon, 5 Jan 2004 16:28:52 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B9DE43D48 for ; Mon, 5 Jan 2004 16:28:50 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i060Smiw022429; Mon, 5 Jan 2004 19:28:48 -0500 (EST) Date: Mon, 5 Jan 2004 19:28:48 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Earl Chew In-Reply-To: <200401052230.i05MUEov039020@freefall.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 00:28:52 -0000 On Mon, 5 Jan 2004, Earl Chew wrote: > The following reply was made to PR misc/24641; it has been noted by GNATS. > > From: Earl Chew > To: aspiesrule@mcleodusa.net > Cc: freebsd-gnats-submit@freebsd.org > Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock > Date: Mon, 05 Jan 2004 14:21:00 -0800 > > aspiesrule@mcleodusa.net wrote: > > Same results, except for the fact that stepping thru your code in GDB yields > > a segfault in KERNEL32!IsBadWritePtr. > > It must be the holidays! Ok, I've added the missing line at the > beginning of wrfunc(). POSIX says that: If the Thread Execution Scheduling option is supported, and the threads involved in the lock are executing with the scheduling policies SCHED_FIFO or SCHED_RR, the calling thread shall not acquire the lock if a writer holds the lock or if writers of higher or equal priority are blocked on the lock; otherwise, the calling thread shall acquire the lock. Forget about the writer's priority for a moment; we don't currently honor that. And FYI, SCHED_OTHER in FreeBSD is SCHED_RR. POSIX also says that: A thread may hold multiple concurrent read locks on rwlock (that is, successfully call the pthread_rwlock_rdlock() function n times). If so, the application shall ensure that the thread performs matching unlocks (that is, it calls the pthread_rwlock_unlock() function n times). It isn't clear to me that this means that a subsequent rdlock request while there are writers blocked on the lock succeeds. It may seem trivial to implement this, but when you consider that there may be a lot of readers then it is not so trivial. In order to implement it, you need to know whether a thread owns the rdlock and have a recurse count for it. And threads may own multiple read locks. You would have to track all of them and it would add a bit of overhead having to search the owned read locks (and they don't have to be taken and released in order so you'd have to search the entire queue). From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 17:03:33 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3688916A4CE for ; Mon, 5 Jan 2004 17:03:33 -0800 (PST) Received: from msgbas2x.cos.agilent.com (msgbas2x.cos.agilent.com [192.25.240.37]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A92F43D49 for ; Mon, 5 Jan 2004 17:03:28 -0800 (PST) (envelope-from earl_chew@agilent.com) Received: from relcos1.cos.agilent.com (relcos1.cos.agilent.com [130.29.152.239]) by msgbas2x.cos.agilent.com (Postfix) with ESMTP id C16F369FD; Mon, 5 Jan 2004 18:03:27 -0700 (MST) Received: from websvr.canada.agilent.com (websvr.canada.agilent.com [141.184.122.102]) by relcos1.cos.agilent.com (Postfix) with ESMTP id BDB791DC; Mon, 5 Jan 2004 18:03:26 -0700 (MST) Received: from agilent.com (dhcp6burnaby.canada.agilent.com [141.184.123.147]) SMKit7.1.1_Agilent) with ESMTP id i0612vo28414; Mon, 5 Jan 2004 17:02:58 -0800 (PST) Message-ID: <3FFA0958.3020307@agilent.com> Date: Mon, 05 Jan 2004 17:03:20 -0800 From: Earl Chew Organization: Agilent Technologies User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 01:03:33 -0000 Daniel Eischen wrote: > POSIX says that: > > If the Thread Execution Scheduling option is supported, and > the threads involved in the lock are executing with the scheduling > policies SCHED_FIFO or SCHED_RR, the calling thread shall not > acquire the lock if a writer holds the lock or if writers of > higher or equal priority are blocked on the lock; otherwise, > the calling thread shall acquire the lock. Call this P1. > Forget about the writer's priority for a moment; we don't > currently honor that. And FYI, SCHED_OTHER in FreeBSD is > SCHED_RR. > > POSIX also says that: > > A thread may hold multiple concurrent read locks on rwlock (that > is, successfully call the pthread_rwlock_rdlock() function n > times). If so, the application shall ensure that the thread > performs matching unlocks (that is, it calls the > pthread_rwlock_unlock() function n times). Call this P2. I interpret P2 as requiring rdlock() to be recursive. That is, once a thread acquires a rdlock(), it can do so repeatedly, as long as it performs the correct number of matching unlocks(). I interpret P1 as giving wrlock()s preference over rdlocks() for a particular lock. > It isn't clear to me that this means that a subsequent rdlock > request while there are writers blocked on the lock succeeds. Hmm... reading P2 again, I can see how you could be right. However, it appears counter-intuitive to me that if a thread has already acquired a rdlock(), and the rdlock() is recursive, that a subsequent call should fail. Failure should indicate that the lock could not be acquired. But the thread _already_ has the lock! So it's hard to explain why the subsequent call should fail (given the requirement that the rdlock be recursive). > It may seem trivial to implement this, but when you consider > that there may be a lot of readers then it is not so trivial. > In order to implement it, you need to know whether a thread > owns the rdlock and have a recurse count for it. Hmm... doesn't P2 already require you to support recursion? > And threads > may own multiple read locks. You would have to track all > of them and it would add a bit of overhead having to search > the owned read locks (and they don't have to be taken and > released in order so you'd have to search the entire queue). I don't see how either P1 or P2 requires this behaviour. What am I missing? Earl From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 17:45:37 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F38EF16A4CE for ; Mon, 5 Jan 2004 17:45:36 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3206543D2D for ; Mon, 5 Jan 2004 17:45:35 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i061jXiw009218; Mon, 5 Jan 2004 20:45:33 -0500 (EST) Date: Mon, 5 Jan 2004 20:45:33 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Earl Chew In-Reply-To: <3FFA0958.3020307@agilent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 01:45:37 -0000 On Mon, 5 Jan 2004, Earl Chew wrote: > Daniel Eischen wrote: > > POSIX says that: > > > > If the Thread Execution Scheduling option is supported, and > > the threads involved in the lock are executing with the scheduling > > policies SCHED_FIFO or SCHED_RR, the calling thread shall not > > acquire the lock if a writer holds the lock or if writers of > > higher or equal priority are blocked on the lock; otherwise, > > the calling thread shall acquire the lock. > > Call this P1. > > > Forget about the writer's priority for a moment; we don't > > currently honor that. And FYI, SCHED_OTHER in FreeBSD is > > SCHED_RR. > > > > POSIX also says that: > > > > A thread may hold multiple concurrent read locks on rwlock (that > > is, successfully call the pthread_rwlock_rdlock() function n > > times). If so, the application shall ensure that the thread > > performs matching unlocks (that is, it calls the > > pthread_rwlock_unlock() function n times). > > Call this P2. > > I interpret P2 as requiring rdlock() to be recursive. That is, once > a thread acquires a rdlock(), it can do so repeatedly, as long as it > performs the correct number of matching unlocks(). > > I interpret P1 as giving wrlock()s preference over rdlocks() for > a particular lock. > > > It isn't clear to me that this means that a subsequent rdlock > > request while there are writers blocked on the lock succeeds. > > Hmm... reading P2 again, I can see how you could be right. > > However, it appears counter-intuitive to me that if a thread has already > acquired a rdlock(), and the rdlock() is recursive, that a subsequent > call should fail. > > Failure should indicate that the lock could not be acquired. But the > thread _already_ has the lock! So it's hard to explain why the > subsequent call should fail (given the requirement that the > rdlock be recursive). It shouldn't fail; it should deadlock. > > It may seem trivial to implement this, but when you consider > > that there may be a lot of readers then it is not so trivial. > > In order to implement it, you need to know whether a thread > > owns the rdlock and have a recurse count for it. > > Hmm... doesn't P2 already require you to support recursion? The implementation just keeps one count of total readers (i.e., recursion by a group of readers); this is stored in the rwlock. Recursive rdlocks by the same thread count as multiple readers. There is no per-thread counter. For recursive _mutexes_ this is simple because there can be only one owner so the one recurse count stored in the mutex is sufficient. But for rwlocks there can be lots of readers, and each reader may hold multiple rwlocks. You either need a queue to hold the rwlock status (pointer to rwlock and recurse count) for each owned rwlock in each thread, or you need a queue of thread status (pointer to thread and recurse count) for each owning thread in each rwlock. And regardless of where you store the queue, you need to seach it to see if the thread already owns the rdlock (and then again on the rdunlock). Think about it. What if you have 100 threads rdlock recursing on 1 rwlock? How do you tell if threads already own a rwlock? And how many times have they recursed on the rwlock? And what if threads want to own multiple rwlocks? Our current implementation is rather simple, but it does avoid us from having to build in limits as to how many rwlocks can be owned by a thread or how many threads can own a rwlock. I'll add this to my TODO list for libkse but it will come at a bit of a performance & storage cost and there will have to be some built-in limits. > > And threads > > may own multiple read locks. You would have to track all > > of them and it would add a bit of overhead having to search > > the owned read locks (and they don't have to be taken and > > released in order so you'd have to search the entire queue). > > I don't see how either P1 or P2 requires this behaviour. What am > I missing? See above :) From owner-freebsd-threads@FreeBSD.ORG Mon Jan 5 22:53:55 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4AA7D16A4CE for ; Mon, 5 Jan 2004 22:53:55 -0800 (PST) Received: from bento.FreeBSD.org (bento.freebsd.org [216.136.204.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 278E443D48; Mon, 5 Jan 2004 22:53:54 -0800 (PST) (envelope-from davidxu@freebsd.org) Received: from freebsd.org (localhost [127.0.0.1]) by bento.FreeBSD.org (8.12.10/8.12.10) with ESMTP id i066rnfx042944; Mon, 5 Jan 2004 22:53:50 -0800 (PST) (envelope-from davidxu@freebsd.org) Message-ID: <3FFA5B61.8070109@freebsd.org> Date: Tue, 06 Jan 2004 14:53:21 +0800 From: David Xu User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5b) Gecko/20030723 Thunderbird/0.1 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: Earl Chew cc: freebsd-threads@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 06:53:55 -0000 Daniel Eischen wrote: >It shouldn't fail; it should deadlock. > > > >>>It may seem trivial to implement this, but when you consider >>>that there may be a lot of readers then it is not so trivial. >>>In order to implement it, you need to know whether a thread >>>owns the rdlock and have a recurse count for it. >>> >>> >>Hmm... doesn't P2 already require you to support recursion? >> >> > >The implementation just keeps one count of total readers (i.e., >recursion by a group of readers); this is stored in the rwlock. >Recursive rdlocks by the same thread count as multiple readers. >There is no per-thread counter. For recursive _mutexes_ this >is simple because there can be only one owner so the one recurse >count stored in the mutex is sufficient. But for rwlocks there >can be lots of readers, and each reader may hold multiple rwlocks. >You either need a queue to hold the rwlock status (pointer to >rwlock and recurse count) for each owned rwlock in each thread, >or you need a queue of thread status (pointer to thread and >recurse count) for each owning thread in each rwlock. And >regardless of where you store the queue, you need to seach it >to see if the thread already owns the rdlock (and then again >on the rdunlock). > >Think about it. What if you have 100 threads rdlock recursing >on 1 rwlock? How do you tell if threads already own a rwlock? >And how many times have they recursed on the rwlock? And what >if threads want to own multiple rwlocks? > >Our current implementation is rather simple, but it does >avoid us from having to build in limits as to how many >rwlocks can be owned by a thread or how many threads can >own a rwlock. I'll add this to my TODO list for libkse >but it will come at a bit of a performance & storage >cost and there will have to be some built-in limits. > > These information should be written into manual, so nobody will repeatly ask this question. :-) From owner-freebsd-threads@FreeBSD.ORG Tue Jan 6 10:18:14 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0258716A4D0 for ; Tue, 6 Jan 2004 10:18:14 -0800 (PST) Received: from msgbas2x.cos.agilent.com (msgbas2x.cos.agilent.com [192.25.240.37]) by mx1.FreeBSD.org (Postfix) with ESMTP id 877EB43D48 for ; Tue, 6 Jan 2004 10:18:10 -0800 (PST) (envelope-from earl_chew@agilent.com) Received: from relcos1.cos.agilent.com (relcos1.cos.agilent.com [130.29.152.239]) by msgbas2x.cos.agilent.com (Postfix) with ESMTP id DF5FB6CC2; Tue, 6 Jan 2004 11:18:09 -0700 (MST) Received: from websvr.canada.agilent.com (websvr.canada.agilent.com [141.184.122.102]) by relcos1.cos.agilent.com (Postfix) with ESMTP id DDA7A102; Tue, 6 Jan 2004 11:18:08 -0700 (MST) Received: from agilent.com (dhcp6burnaby.canada.agilent.com [141.184.123.147]) SMKit7.1.1_Agilent) with ESMTP id i06IHdo27989; Tue, 6 Jan 2004 10:17:39 -0800 (PST) Message-ID: <3FFAFBD7.2030601@agilent.com> Date: Tue, 06 Jan 2004 10:17:59 -0800 From: Earl Chew Organization: Agilent Technologies User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 18:18:14 -0000 Daniel Eischen wrote: > Our current implementation is rather simple, but it does > avoid us from having to build in limits as to how many > rwlocks can be owned by a thread or how many threads can > own a rwlock. I'll add this to my TODO list for libkse > but it will come at a bit of a performance & storage > cost and there will have to be some built-in limits. Thanks for the detailed explanation. Earl From owner-freebsd-threads@FreeBSD.ORG Tue Jan 6 23:04:43 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7EE0816A4D0 for ; Tue, 6 Jan 2004 23:04:43 -0800 (PST) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id B460743D45 for ; Tue, 6 Jan 2004 23:04:41 -0800 (PST) (envelope-from rodrigc@crodrigues.org) Received: from h00609772adf0.ne.client2.attbi.com ([66.31.45.197]) by comcast.net (rwcrmhc12) with ESMTP id <2004010707044101400phdsbe>; Wed, 7 Jan 2004 07:04:41 +0000 Received: from h00609772adf0.ne.client2.attbi.com (localhost.crodrigues.org [127.0.0.1])i0774g7d034561 for ; Wed, 7 Jan 2004 02:04:42 -0500 (EST) (envelope-from rodrigc@h00609772adf0.ne.client2.attbi.com) Received: (from rodrigc@localhost)i0774g9L034560 for freebsd-threads@freebsd.org; Wed, 7 Jan 2004 02:04:42 -0500 (EST) (envelope-from rodrigc) Date: Wed, 7 Jan 2004 02:04:42 -0500 From: Craig Rodrigues To: freebsd-threads@freebsd.org Message-ID: <20040107070442.GB34511@crodrigues.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 07:04:43 -0000 Hi, I asked this question to re@, but didn't get a clear answer. What is are the release criteria for switching libkse to libpthread? I assume that this is going to be targeted for 5.3, and not 5.2. Dan Eischen has been periodically testing with ACE, which works quite well with libkse. Are there other representative multithreaded applications which need to be tested? If so, which ones? re@ has mentioned that there are problems with ports and PTHREAD_CFLAGS and PTHREAD_LIBS which currently prevent switching libkse to libpthread. How many ports need to be fixed? Does every single port need to be fixed before the switch is made? If not, how many ports can be fixed to have an acceptable state of affairs for the switch? I am very eager to have a more robust pthreads implementation for FreeBSD, so that I can use FreeBSD as a drop-in replacement for Linux for several projects that I work on. I think we are pretty close... Thanks. -- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 03:27:32 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24A2916A4CE for ; Wed, 7 Jan 2004 03:27:32 -0800 (PST) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F24E43D2F for ; Wed, 7 Jan 2004 03:27:30 -0800 (PST) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (localhost [127.0.0.1]) by silver.he.iki.fi (8.12.9p2/8.11.4) with ESMTP id i07BRNgr061356; Wed, 7 Jan 2004 13:27:24 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <3FFBECEC.8090709@he.iki.fi> Date: Wed, 07 Jan 2004 13:26:36 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Craig Rodrigues References: <20040107070442.GB34511@crodrigues.org> In-Reply-To: <20040107070442.GB34511@crodrigues.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 11:27:32 -0000 Craig Rodrigues wrote: >Hi, > >Dan Eischen has been periodically testing with ACE, which works quite well >with libkse. Are there other representative multithreaded applications >which need to be tested? If so, which ones? > > > Iīve been experimenting with running mysql-4.0 and it still seems to have issues with signal handling. If the database is idle, shutdown works properly but if there is load on the database, shutting down never happens unless forced with SIGKILL. Unfortunately I havenīt had the time to debug this further, just periodically updated both mysql and world to see if this would go away but it hasnīt. Iīd be happy to provide more detail if somebody would point me to the direction where to look at. Pete From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 04:28:31 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D279C16A4CE for ; Wed, 7 Jan 2004 04:28:31 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C25843D53 for ; Wed, 7 Jan 2004 04:28:23 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i07CSKiw027472; Wed, 7 Jan 2004 07:28:20 -0500 (EST) Date: Wed, 7 Jan 2004 07:28:20 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Petri Helenius In-Reply-To: <3FFBECEC.8090709@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 12:28:32 -0000 On Wed, 7 Jan 2004, Petri Helenius wrote: > Craig Rodrigues wrote: >=20 > >Hi, > > > >Dan Eischen has been periodically testing with ACE, which works quite we= ll > >with libkse. Are there other representative multithreaded applications= =20 > >which need to be tested? If so, which ones? > > > > =20 > > > I=B4ve been experimenting with running mysql-4.0 and it still seems to=20 > have issues > with signal handling. If the database is idle, shutdown works properly bu= t > if there is load on the database, shutting down never happens unless > forced with SIGKILL. >=20 > Unfortunately I haven=B4t had the time to debug this further, just=20 > periodically > updated both mysql and world to see if this would go away but it hasn=B4t= =2E >=20 > I=B4d be happy to provide more detail if somebody would point me to the= =20 > direction > where to look at. Send it a SIGINFO and look at the dumped thread states (/tmp/pthread.dump.x= xx). From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 07:20:46 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 898EF16A4CE for ; Wed, 7 Jan 2004 07:20:46 -0800 (PST) Received: from miramanee.icarz.com (miramanee.icarz.com [207.99.22.55]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3583243D31 for ; Wed, 7 Jan 2004 07:20:34 -0800 (PST) (envelope-from kenm@icarz.com) Received: from deanna.icarz.com (deanna.icarz.com [207.99.22.19]) by miramanee.icarz.com (8.12.10/8.12.10) with ESMTP id i07FKXTt004753; Wed, 7 Jan 2004 10:20:33 -0500 (EST) (envelope-from kenm@icarz.com) Received: from kenxp (netb-178.icarz.com [209.123.219.178]) by deanna.icarz.com (8.12.9p2/8.12.9) with SMTP id i07FKWiM079835; Wed, 7 Jan 2004 10:20:32 -0500 (EST) (envelope-from kenm@icarz.com) Message-ID: <08b401c3d530$92923df0$b2db7bd1@icarz.com> From: "Ken Menzel" To: "Petri Helenius" References: <3FEAB27E.90306@he.iki.fi> <073201c3ce2c$28304200$b2db7bd1@icarz.com> <3FF124B0.7050309@he.iki.fi> Date: Wed, 7 Jan 2004 10:11:48 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 X-Scanned-By: MIMEDefang 2.38 cc: freebsd-threads@freebsd.org Subject: Re: mysql support with kse X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 15:20:46 -0000 Hi Pete, Sorry for the long delayed answer I have been away. The extra libs you have are for SSL support in MySQL. I did not compile in SSL support. Even after several days I am not having shut down problems. However I have upgraded to Dec 29th FreeBSD 5.2 because performance seems to be better. Maybe the problem with shutting down is related to SSL and not threading? I hope you have nailed it by now anyway! Ken ----- Original Message ----- From: "Petri Helenius" To: "Ken Menzel" Cc: Sent: Tuesday, December 30, 2003 2:09 AM Subject: Re: mysql support with kse > Ken Menzel wrote: > > >No problems here, but I am not in production yet! > > > >riker# ldd /usr/local/libexec/mysqld > >/usr/local/libexec/mysqld: > > libz.so.2 => /lib/libz.so.2 (0x68b35000) > > libcrypt.so.2 => /lib/libcrypt.so.2 (0x68b43000) > > libm.so.2 => /lib/libm.so.2 (0x68b5c000) > > libc_r.so.5 => /usr/lib/libkse.so.1 (0x68b75000) > > libc.so.5 => /lib/libc.so.5 (0x68b99000) > >riker# mysqld_safe& > > > > > You have awfully few libraries compared to my mysql 4.0.17 installed > from ports; > /usr/local/libexec/mysqld: > libwrap.so.3 => /usr/lib/libwrap.so.3 (0x28371000) > libssl.so.3 => /usr/lib/libssl.so.3 (0x28379000) > libcrypto.so.3 => /lib/libcrypto.so.3 (0x283b4000) > libz.so.2 => /lib/libz.so.2 (0x284c2000) > libcrypt.so.2 => /lib/libcrypt.so.2 (0x284d0000) > libc_r.so.5 => /usr/lib/libkse.so.1 (0x284e9000) > libstdc++.so.4 => /usr/lib/libstdc++.so.4 (0x2850d000) > libm.so.2 => /lib/libm.so.2 (0x285c9000) > libc.so.5 => /lib/libc.so.5 (0x285e2000) > > It seems shutdown works if the mysqld process has been running only for > a moment > but if it has been running for a longer time (just tried overnight), > shutdown has > no effect. > > Pete > > > From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 09:05:39 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC15F16A4CE for ; Wed, 7 Jan 2004 09:05:39 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 32FE443D55 for ; Wed, 7 Jan 2004 09:05:38 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i07H3xUd007771; Wed, 7 Jan 2004 12:03:59 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i07H3xE5007768; Wed, 7 Jan 2004 12:03:59 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 7 Jan 2004 12:03:59 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Craig Rodrigues In-Reply-To: <20040107070442.GB34511@crodrigues.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 17:05:39 -0000 On Wed, 7 Jan 2004, Craig Rodrigues wrote: > I asked this question to re@, but didn't get a clear answer. > > What is are the release criteria for switching libkse to libpthread? I > assume that this is going to be targeted for 5.3, and not 5.2. > > Dan Eischen has been periodically testing with ACE, which works quite > well with libkse. Are there other representative multithreaded > applications which need to be tested? If so, which ones? > > re@ has mentioned that there are problems with ports and PTHREAD_CFLAGS > and PTHREAD_LIBS which currently prevent switching libkse to libpthread. > How many ports need to be fixed? Does every single port need to be > fixed before the switch is made? If not, how many ports can be fixed to > have an acceptable state of affairs for the switch? > > I am very eager to have a more robust pthreads implementation for > FreeBSD, so that I can use FreeBSD as a drop-in replacement for Linux > for several projects that I work on. > > I think we are pretty close... I'm not sure there's a specific documented set of criteria at this point, other than that the issue probably got addressed too late in the 5.2 release process. There are a number of running concerns, including: (1) Avoid building a binary library name dependency into all the pre-built packages we distribute. Also, resolve any lasting concerns about how build processes should say "And I want threads, dammit". (2) Have services like process debugging, profiling available and known fully functional (or close). I agree that we're very close. I was under the vague impression that we planned to throw the switch once 5.2 was officially out the door. Many of us are running KSE as our libc_r via libmap.conf on all our machines, and have been for many months, and it appears to hold up quite well :-). Resolving how best to declare threading support in binaries will also facilitate shipping the JDK linked against KSE. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 14:47:38 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D24816A4D0 for ; Wed, 7 Jan 2004 14:47:38 -0800 (PST) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id A852B43D3F for ; Wed, 7 Jan 2004 14:47:34 -0800 (PST) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (localhost [127.0.0.1]) by silver.he.iki.fi (8.12.9p2/8.11.4) with ESMTP id i07MlTgr065021; Thu, 8 Jan 2004 00:47:30 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <3FFC8C51.7030203@he.iki.fi> Date: Thu, 08 Jan 2004 00:46:41 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 22:47:38 -0000 Daniel Eischen wrote: >Send it a SIGINFO and look at the dumped thread states (/tmp/pthread.dump.xxx). > > > > No files get written to /tmp (or anywhere else I looked) when I send SIGINFO to the mysqld process. Is there some specific compilation options / preparations that should be made in advance? Pete From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 14:52:48 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9357216A4F8 for ; Wed, 7 Jan 2004 14:52:48 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 92ADD43D49 for ; Wed, 7 Jan 2004 14:52:46 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i07Mqiiw022321; Wed, 7 Jan 2004 17:52:44 -0500 (EST) Date: Wed, 7 Jan 2004 17:52:44 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Petri Helenius In-Reply-To: <3FFC8C51.7030203@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 22:52:48 -0000 On Thu, 8 Jan 2004, Petri Helenius wrote: > Daniel Eischen wrote: > > >Send it a SIGINFO and look at the dumped thread states (/tmp/pthread.dump.xxx). > > > > > > > > > No files get written to /tmp (or anywhere else I looked) when I send > SIGINFO to the mysqld > process. > > Is there some specific compilation options / preparations that should be > made in advance? No, that should do it. It sounds like something is hung up. Are you sure that whatever libraries mysql is using are thread-safe? You can define DEBUG_SIGNAL when building libkse to show signal debugging info. Either edit libpthread/thread/thr_sig.c and uncomment line ~97 or add it to your CFLAGS. -- Dan From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 15:12:08 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A21116A4CE for ; Wed, 7 Jan 2004 15:12:08 -0800 (PST) Received: from dust.freshx.de (freshx.de [80.190.100.215]) by mx1.FreeBSD.org (Postfix) with ESMTP id C21E343D55 for ; Wed, 7 Jan 2004 15:12:05 -0800 (PST) (envelope-from kai@freshx.de) Received: from localhost (LOCALHOST [127.0.0.1]) by dust.freshx.de (Postfix) with ESMTP id 4CF9815E2FE; Thu, 8 Jan 2004 00:11:54 +0100 (CET) Received: from localhost (LOCALHOST [127.0.0.1]) by dust.freshx.de (Postfix) with ESMTP id 856C415E2E3; Thu, 8 Jan 2004 00:11:53 +0100 (CET) Received: from 127.0.0.1 ( [127.0.0.1]) as user dust0005@localhost by localhost with HTTP; Thu, 8 Jan 2004 00:11:53 +0100 Message-ID: <1073517113.3ffc923968db8@localhost> Date: Thu, 8 Jan 2004 00:11:53 +0100 From: Kai Mosebach To: Daniel Eischen References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.0 X-Virus-Scanned: by AMaViS 0.3.12 cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 23:12:08 -0000 > On Thu, 8 Jan 2004, Petri Helenius wrote: > > > Daniel Eischen wrote: > > > > >Send it a SIGINFO and look at the dumped thread states > (/tmp/pthread.dump.xxx). > > > > > > > > > > > > > > No files get written to /tmp (or anywhere else I looked) when I send > > SIGINFO to the mysqld > > process. > > > > Is there some specific compilation options / preparations that should be > > made in advance? > > No, that should do it. It sounds like something is hung up. > Are you sure that whatever libraries mysql is using are > thread-safe? Sorrowly i also cannot see any pthread* files in my /tmp directory, when i send -SIGINFO to my SAPDB kernel ... regards Kai From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 17:09:00 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1B30616A4CE for ; Wed, 7 Jan 2004 17:09:00 -0800 (PST) Received: from smtp3.adl2.internode.on.net (smtp3.adl2.internode.on.net [203.16.214.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id ABE2343D3F for ; Wed, 7 Jan 2004 17:08:56 -0800 (PST) (envelope-from doconnor@gsoft.com.au) Received: from midget.dons.net.au (ppp108-195.lns1.adl1.internode.on.net [150.101.108.195])i0818sqR093865; Thu, 8 Jan 2004 11:38:54 +1030 (CST) Received: from chowder.gsoft.com.au (root@localhost.dons.net.au [127.0.0.1]) by midget.dons.net.au (8.12.9/8.12.9) with ESMTP id i0818paW099585; Thu, 8 Jan 2004 11:38:52 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: threads@freebsd.org Date: Thu, 8 Jan 2004 11:38:50 +1030 User-Agent: KMail/1.5.4 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401081138.50146.doconnor@gsoft.com.au> X-Spam-Score: -2.9 () SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01,USER_AGENT,USER_AGENT_KMAIL X-Scanned-By: MIMEDefang 2.26 (www . roaringpenguin . com / mimedefang) Subject: KDE and KSE X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 01:09:00 -0000 Hi, I am trying to use KSE with KDE but I'm not having much luck :( The upgrade process was a bit dodgy unfortunately, I upgraded KDE using portupgrade -PP, but rebuilt Qt and fontconfig from source. I am using libmap.conf to get KSE, if I have no libmap.conf file (ie use libc_r) KDE works fine. libthr doesn't work either but that's a different story (different failure mode too) After I do startx -> XFree86 Version 4.3.0 Release Date: 27 February 2003 X Protocol Version 11, Revision 0, Release 6.6 Build Operating System: FreeBSD 4.7 i386 [ELF] Build Date: 17 March 2003 Before reporting problems, check http://www.XFree86.Org/ to make sure that you have the latest version. Module Loader present Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. (==) Log file: "/var/log/XFree86.0.log", Time: Thu Jan 8 11:14:41 2004 (==) Using config file: "/etc/XF86Config" Samsung LTN141P2 Color, Single, TFT Warning: font renderer for ".pcf" already registered at priority 0 Warning: font renderer for ".pcf.Z" already registered at priority 0 Warning: font renderer for ".pcf.gz" already registered at priority 0 Warning: font renderer for ".snf" already registered at priority 0 Warning: font renderer for ".snf.Z" already registered at priority 0 Warning: font renderer for ".snf.gz" already registered at priority 0 Warning: font renderer for ".bdf" already registered at priority 0 Warning: font renderer for ".bdf.Z" already registered at priority 0 Warning: font renderer for ".bdf.gz" already registered at priority 0 Warning: font renderer for ".pmf" already registered at priority 0 :5: invalid preprocessing directive #XPilot :770: invalid preprocessing directive #xearth localhost being added to access control list startkde: Starting up... Warning: connect() failed: : Connection refused Segmentation fault (core dumped) startkde: Shutting down... Warning: connect() failed: : Connection refused Error: Can't contact kdeinit! startkde: Running shutdown scripts... startkde: Done. waiting for X server to shut down I end up with 3 core files -> -rw------- 1 darius 1007 1384448 Jan 8 11:14 kdeinit.core -rw------- 1 darius 1007 1388544 Jan 8 11:14 ksmserver.core -rw------- 1 darius 1007 1380352 Jan 8 11:14 ksplash.core kdeinit back trace -> Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)... done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x290d014d in xdr_opaque () from /lib/libc.so.5 (gdb) bt #0 0x290d014d in xdr_opaque () from /lib/libc.so.5 #1 0x290ada04 in xdr_ypbind_binding () from /lib/libc.so.5 #2 0x290a8d06 in .cerror () from /lib/libc.so.5 #3 0x287c7c5d in waitpid () from /usr/lib/libkse.so.1 #4 0x0804d8d8 in execpath_avoid_loops(QCString const&, int, char const*, bool) () #5 0x0804f045 in main () #6 0x0804b362 in _start () ksmserver back trace -> Loaded symbols for /libexec/ld-elf.so.1 ---Type to continue, or q to quit--- #0 0x28dd514d in xdr_opaque () from /lib/libc.so.5 (gdb) (gdb) bt #0 0x28dd514d in xdr_opaque () from /lib/libc.so.5 #1 0x28db2a04 in xdr_ypbind_binding () from /lib/libc.so.5 #2 0x28dadd06 in .cerror () from /lib/libc.so.5 #3 0x28f7a668 in FcDirCacheReadDir () from /usr/X11R6/lib/libfontconfig.so.1 #4 0x28f8060a in FcDirScanConfig () from /usr/X11R6/lib/libfontconfig.so.1 #5 0x28f7b1f5 in FcConfigBuildFonts () from /usr/X11R6/lib/libfontconfig.so.1 #6 0x28f8247f in FcInitLoadConfigAndFonts () from /usr/X11R6/lib/libfontconfig.so.1 #7 0x28f824d4 in FcInit () from /usr/X11R6/lib/libfontconfig.so.1 #8 0x28efa560 in XftInit () from /usr/X11R6/lib/libXft.so.2 #9 0x286974b3 in QApplication::x11_apply_settings() () from /usr/X11R6/lib/libqt-mt.so.3 #10 0x28697c4d in qt_set_x11_resources(char const*, char const*, char const*, char const*) () from /usr/X11R6/lib/libqt-mt.so.3 #11 0x2869b9da in qt_init_internal(int*, char**, _XDisplay*, unsigned long, unsigned long) () from /usr/X11R6/lib/libqt-mt.so.3 #12 0x2869c148 in qt_init(int*, char**, QApplication::Type) () from /usr/X11R6/lib/libqt-mt.so.3 #13 0x286f7d98 in QApplication::construct(int&, char**, QApplication::Type) () from /usr/X11R6/lib/libqt-mt.so.3 #14 0x286f7b67 in QApplication::QApplication(int&, char**, bool) () from /usr/X11R6/lib/libqt-mt.so.3 #15 0x283522bb in KApplication::KApplication(bool, bool) () from /usr/local/lib/libkdecore.so.5 #16 0x2808a94f in main () from /usr/local/lib/ksmserver.so #17 0x080485b2 in _start () ksplash back trace -> Loaded symbols for /libexec/ld-elf.so.1 #0 0x28dbf14d in xdr_opaque () from /lib/libc.so.5 (gdb) bt #0 0x28dbf14d in xdr_opaque () from /lib/libc.so.5 #1 0x28d9ca04 in xdr_ypbind_binding () from /lib/libc.so.5 #2 0x28d97d06 in .cerror () from /lib/libc.so.5 #3 0x28f64668 in FcDirCacheReadDir () from /usr/X11R6/lib/libfontconfig.so.1 #4 0x28f6a60a in FcDirScanConfig () from /usr/X11R6/lib/libfontconfig.so.1 #5 0x28f651f5 in FcConfigBuildFonts () from /usr/X11R6/lib/libfontconfig.so.1 #6 0x28f6c47f in FcInitLoadConfigAndFonts () from /usr/X11R6/lib/libfontconfig.so.1 #7 0x28f6c4d4 in FcInit () from /usr/X11R6/lib/libfontconfig.so.1 #8 0x28ee4560 in XftInit () from /usr/X11R6/lib/libXft.so.2 #9 0x286814b3 in QApplication::x11_apply_settings() () from /usr/X11R6/lib/libqt-mt.so.3 #10 0x28681c4d in qt_set_x11_resources(char const*, char const*, char const*, char const*) () from /usr/X11R6/lib/libqt-mt.so.3 #11 0x286859da in qt_init_internal(int*, char**, _XDisplay*, unsigned long, unsigned long) () from /usr/X11R6/lib/libqt-mt.so.3 #12 0x28686148 in qt_init(int*, char**, QApplication::Type) () from /usr/X11R6/lib/libqt-mt.so.3 #13 0x286e1d98 in QApplication::construct(int&, char**, QApplication::Type) () from /usr/X11R6/lib/libqt-mt.so.3 #14 0x286e1b67 in QApplication::QApplication(int&, char**, bool) () from /usr/X11R6/lib/libqt-mt.so.3 #15 0x2833c2bb in KApplication::KApplication(bool, bool) () from /usr/local/lib/libkdecore.so.5 #16 0x0804ff0a in main () #17 0x0804db52 in _start () -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 9A8C 569F 685A D928 5140 AE4B 319B 41F4 5D17 FDD5 From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 17:20:12 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D0B1F16A4CE; Wed, 7 Jan 2004 17:20:12 -0800 (PST) Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19B9943D45; Wed, 7 Jan 2004 17:20:11 -0800 (PST) (envelope-from rodrigc@crodrigues.org) Received: from h00609772adf0.ne.client2.attbi.com ([66.31.45.197]) by comcast.net (sccrmhc13) with ESMTP id <2004010801201001600nnde3e>; Thu, 8 Jan 2004 01:20:10 +0000 Received: from h00609772adf0.ne.client2.attbi.com (localhost.crodrigues.org [127.0.0.1])i081KD7d038160; Wed, 7 Jan 2004 20:20:13 -0500 (EST) (envelope-from rodrigc@h00609772adf0.ne.client2.attbi.com) Received: (from rodrigc@localhost)i081KCWb038159; Wed, 7 Jan 2004 20:20:12 -0500 (EST) (envelope-from rodrigc) Date: Wed, 7 Jan 2004 20:20:07 -0500 From: Craig Rodrigues To: Robert Watson Message-ID: <20040108012007.GA38122@crodrigues.org> References: <20040107070442.GB34511@crodrigues.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 01:20:12 -0000 On Wed, Jan 07, 2004 at 12:03:59PM -0500, Robert Watson wrote: > I'm not sure there's a specific documented set of criteria at this point, OK, well let's start putting all our cards on the table, and see what we get. :) > other than that the issue probably got addressed too late in the 5.2 > release process. Well 5.2 has branched, and let's assume that it will be released shortly after the appropriate QA and bugfixing. Are there any issues that would prevent moving to KSE as the default on -current at this very moment, since 5.2 is on its own branch? > There are a number of running concerns, including: > > (1) Avoid building a binary library name dependency into all the pre-built > packages we distribute. Can you explain this? I don't fully understand. > Also, resolve any lasting concerns about how > build processes should say "And I want threads, dammit". This seems to be a point of debate, but it would be nice to resolve this sooner rather than later. My opinion is to do the following: - move libkse to libpthread - inform users that they must use -lpthread link in thread support - remove -pthread in gcc (maybe deprecate it instead) - fix the ports appropriately with PTHREAD_CFLAGS and PTHREAD_LDFLAGS > (2) Have services like process debugging, profiling available and known > fully functional (or close). This stuff is good to have, but should the lack of it prevent moving libkse -> libpthread? It sounds like a lot of work. > us are running KSE as our libc_r via libmap.conf on all our machines, and > have been for many months, and it appears to hold up quite well :-). > Resolving how best to declare threading support in binaries will also > facilitate shipping the JDK linked against KSE. I also use libkse via libmap.conf, and am happy with it. I also agree that having a KSE-friendly version of the JDK will be very important. Are there any other ports that we should focus our efforts on in getting KSE to work with? httpd, mysql, ACE,....? Thanks. -- Craig Rodrigues http://crodrigues.org rodrigc@crodrigues.org From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 17:41:02 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AF99116A4CE for ; Wed, 7 Jan 2004 17:41:02 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B40FB43D1F for ; Wed, 7 Jan 2004 17:41:00 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i081dUUd014337; Wed, 7 Jan 2004 20:39:31 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i081dUkX014334; Wed, 7 Jan 2004 20:39:30 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 7 Jan 2004 20:39:30 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Craig Rodrigues In-Reply-To: <20040108012007.GA38122@crodrigues.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 01:41:02 -0000 On Wed, 7 Jan 2004, Craig Rodrigues wrote: > > other than that the issue probably got addressed too late in the 5.2 > > release process. > > Well 5.2 has branched, and let's assume that it will be released shortly > after the appropriate QA and bugfixing. Are there any issues that would > prevent moving to KSE as the default on -current at this very moment, > since 5.2 is on its own branch? Agreed. I think the waiting was to prevent 5-CURRENT from diverging too much from 5.2 before 5.2 is actually out, but I think now would be a fine time to do it if the other dependencies are met. > > There are a number of running concerns, including: > > > > (1) Avoid building a binary library name dependency into all the pre-built > > packages we distribute. > > Can you explain this? I don't fully understand. When we build packages, etc, that rely on threading, a library name is built into each binary and library depending on it identifying the dependency. Currently, the packages built in the package cluster use "libc_r" as the name to depend on for threads. I think the right long-term answer is for thread-dependent packages to simply link against "libpthread". People can rebuild locally to use something else, rename libraries, or use libmap.conf to select alternatives. What I don't want to see happen is lots of binaries and libraries with conflicting dependencies, and I want to avoid a lot of applications linking against a name that will go away, be insufficiently supported, etc. > > Also, resolve any lasting concerns about how > > build processes should say "And I want threads, dammit". > > This seems to be a point of debate, but it would be nice to resolve > this sooner rather than later. My opinion is to do the following: > - move libkse to libpthread > - inform users that they must use -lpthread link in thread support > - remove -pthread in gcc (maybe deprecate it instead) > - fix the ports appropriately with PTHREAD_CFLAGS and PTHREAD_LDFLAGS There seems to be continuing disagreement on the gcc -pthread thing; I don't really have much of an opinion on it, except that I think we should choose the path of least headache. And that I suspect that path is to keep it around... The one nice thing about dropping -pthread is it will force everyone to go out and update the ports, but that sounds like a technical solution to a social problem. We could as easily take other paths visible only to ports developers in the cluster and shoot fewer users. > > (2) Have services like process debugging, profiling available and known > > fully functional (or close). > > This stuff is good to have, but should the lack of it prevent moving > libkse -> libpthread? It sounds like a lot of work. Well, it means that we have to document, loudly and carefully somewhere, that in the default configuration, gdb on threaded applications is no longer supported...? > > us are running KSE as our libc_r via libmap.conf on all our machines, and > > have been for many months, and it appears to hold up quite well :-). > > Resolving how best to declare threading support in binaries will also > > facilitate shipping the JDK linked against KSE. > > I also use libkse via libmap.conf, and am happy with it. I also agree > that having a KSE-friendly version of the JDK will be very important. > Are there any other ports that we should focus our efforts on in getting > KSE to work with? httpd, mysql, ACE,....? That looks like a good starting list :-). Apache2 + eviljava + mysql sounds like a killer combination with kse. My leaning is we should throw the switch to libkse sooner rather than later to improve exposure, even though it's not quite feature complete. It was basically an accident of timing and intent that we didn't get to the rename before 5.2. For one thing, it will get applications to start linking against libpthread sooner. However, if we're going to throw the switch, it would be good to know the features aren't far behind :-). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 18:09:10 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8A9B116A4CE for ; Wed, 7 Jan 2004 18:09:10 -0800 (PST) Received: from ftp.bjpu.edu.cn (ftp.bjpu.edu.cn [202.112.78.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2244E43D39 for ; Wed, 7 Jan 2004 18:09:08 -0800 (PST) (envelope-from delphij@frontfree.net) Received: by ftp.bjpu.edu.cn (Postfix, from userid 426) id 635AF52D4; Thu, 8 Jan 2004 10:09:06 +0800 (CST) Received: from srv (unknown [192.168.122.253]) by ftp.bjpu.edu.cn (Postfix) with ESMTP id E5C675299; Thu, 8 Jan 2004 10:09:05 +0800 (CST) From: "Xin LI" To: "'Craig Rodrigues'" Date: Thu, 8 Jan 2004 10:09:02 +0800 Organization: Frontfree Technology Network MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0 In-Reply-To: <20040108012007.GA38122@crodrigues.org> Thread-Index: AcPVhZolZAp9L0GhT92Ly3/EYYM35QAAWAdg Message-Id: <20040108020905.E5C675299@ftp.bjpu.edu.cn> cc: freebsd-threads@freebsd.org Subject: RE: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 02:09:10 -0000 > -----Original Message----- > From: owner-freebsd-threads@freebsd.org > [mailto:owner-freebsd-threads@freebsd.org] On Behalf Of Craig > Rodrigues > Sent: Thursday, January 08, 2004 9:20 AM > To: Robert Watson > Cc: freebsd-threads@freebsd.org > Subject: Re: Release criteria for libkse -> libpthread switch? > > > other than that the issue probably got addressed too late > in the 5.2 > > release process. > > Well 5.2 has branched, and let's assume that it will be > released shortly after the appropriate QA and bugfixing. > Are there any issues that would prevent moving to KSE as the > default on -current at this very moment, since 5.2 is on its > own branch? I think this (code freeze) is a common practice in release engineering process, and, personally, I believe it brings good, with a software engineer's view. During the release engineering cycle, it is important to have the code frozen and we have branched RELENG_5_2, which, theorically, is a "frozen" branch as of the branching day after a considerable period of code slush. After the branching point, the main effort of the whole project would focus on fixing known bugs instead of adding some new features or vastly change the behavior. By avoiding the feature and behavioral changes, the release engineer and the rest of the team would have chance to stablize the codebase on a tight timeline, because what we are trying to avoid during the release cycle usually brings unexpected concussion and may result in a drastically prolonged release cycle. You know, it's hard to predicate how much time we will spend on debugging, especially, when the problem appears when some new code was added, because it is not very easy to determine which part has caused the problem, and, as a result, the bug could not be easily tracked down. So now you may ask why while we have the RELENG_5_2 branched down from HEAD, no major functional changes were permitted to hit the tree? IMO this is also reasonable because by allowing massive changes to HEAD before 5.2-RELEASE was finally released, we may have problem when backporting fixes from HEAD to RELENG_5_2. Of course, this may delay the development process for a little period, but I think that's so important to make a successful release, which, personally, is believed to be the current focus of the whole project. I think the release engineering process of FreeBSD is one of what the project should be proud of. The religious of this process sometimes cause delay of a upcoming release, but it will make the release valuable for a user to trust. When we are using a FreeBSD RELEASE, we rarely worry about stablity, performance, nor compability problems it will cause, you see, many problems were addressed before the release goes out. For a conservative user, especially users running critical servers on FreeBSD operating system, this is more important. So I would say, let's just wait for the end of the semi-freeze state, after the 5.2-RELEASE. From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 18:12:07 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BFF0A16A4CE; Wed, 7 Jan 2004 18:12:07 -0800 (PST) Received: from mhub-m5.tc.umn.edu (mhub-m5.tc.umn.edu [160.94.23.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8638843D2F; Wed, 7 Jan 2004 18:12:06 -0800 (PST) (envelope-from ryans@gamersimpact.com) Received: from [24.107.70.120] by mhub-m5.tc.umn.edu with ESMTP; Wed, 7 Jan 2004 20:12:05 -0600 From: Ryan Sommers To: Robert Watson In-Reply-To: References: Content-Type: text/plain Message-Id: <1073527914.650.41.camel@localhost> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.5 Date: Wed, 07 Jan 2004 20:11:55 -0600 Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 02:12:07 -0000 On Wed, 2004-01-07 at 19:39, Robert Watson wrote: > My leaning is we should throw the switch to libkse sooner rather than > later to improve exposure, even though it's not quite feature complete. > It was basically an accident of timing and intent that we didn't get to > the rename before 5.2. For one thing, it will get applications to start > linking against libpthread sooner. However, if we're going to throw the > switch, it would be good to know the features aren't far behind :-). > > Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > robert@fledge.watson.org Senior Research Scientist, McAfee Research My only concern with this is that it might cause some sort of delay in getting to 5-STABLE. I think, as I'm sure most would agree, that achieving a 5-STABLE is the next major milestone for the project. One that, from the lists, some think is somewhat overdue. -- Ryan Sommers ryans@gamersimpact.com From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 18:29:19 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1C33B16A4CE for ; Wed, 7 Jan 2004 18:29:19 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21BED43D5F for ; Wed, 7 Jan 2004 18:29:08 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i082RgUd014929; Wed, 7 Jan 2004 21:27:42 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i082RgbH014926; Wed, 7 Jan 2004 21:27:42 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 7 Jan 2004 21:27:42 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Ryan Sommers In-Reply-To: <1073527914.650.41.camel@localhost> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 02:29:19 -0000 On Wed, 7 Jan 2004, Ryan Sommers wrote: > On Wed, 2004-01-07 at 19:39, Robert Watson wrote: > > My leaning is we should throw the switch to libkse sooner rather than > > later to improve exposure, even though it's not quite feature complete. > > It was basically an accident of timing and intent that we didn't get to > > the rename before 5.2. For one thing, it will get applications to start > > linking against libpthread sooner. However, if we're going to throw the > > switch, it would be good to know the features aren't far behind :-). > > My only concern with this is that it might cause some sort of delay in > getting to 5-STABLE. I think, as I'm sure most would agree, that > achieving a 5-STABLE is the next major milestone for the project. One > that, from the lists, some think is somewhat overdue. On the other hand, M:N threading is one of the big "5-STABLE" features, so if we continue to consider it one of those features, moving to it as the default is a better thing to do sooner than later. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-threads@FreeBSD.ORG Wed Jan 7 21:46:46 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CAC4616A4CE; Wed, 7 Jan 2004 21:46:46 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id A0D6343D41; Wed, 7 Jan 2004 21:46:44 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i085kgiw001046; Thu, 8 Jan 2004 00:46:42 -0500 (EST) Date: Thu, 8 Jan 2004 00:46:42 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Robert Watson In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 05:46:46 -0000 On Wed, 7 Jan 2004, Robert Watson wrote: > > On Wed, 7 Jan 2004, Craig Rodrigues wrote: > > > > other than that the issue probably got addressed too late in the 5.2 > > > release process. > > > > Well 5.2 has branched, and let's assume that it will be released shortly > > after the appropriate QA and bugfixing. Are there any issues that would > > prevent moving to KSE as the default on -current at this very moment, > > since 5.2 is on its own branch? > > Agreed. I think the waiting was to prevent 5-CURRENT from diverging too > much from 5.2 before 5.2 is actually out, but I think now would be a fine > time to do it if the other dependencies are met. > > > > There are a number of running concerns, including: > > > > > > (1) Avoid building a binary library name dependency into all the pre-built > > > packages we distribute. > > > > Can you explain this? I don't fully understand. > > When we build packages, etc, that rely on threading, a library name is > built into each binary and library depending on it identifying the > dependency. Currently, the packages built in the package cluster use > "libc_r" as the name to depend on for threads. I think the right > long-term answer is for thread-dependent packages to simply link against > "libpthread". People can rebuild locally to use something else, rename > libraries, or use libmap.conf to select alternatives. What I don't want > to see happen is lots of binaries and libraries with conflicting > dependencies, and I want to avoid a lot of applications linking against a > name that will go away, be insufficiently supported, etc. Right, I think this is mostly solved by changing gcc -pthread to use -lpthread (at the same time as the libkse switchover) and changing the default PTHREAD_LIBS to -lpthread. That gets you most of the way there, but there may still be ports that know about the existence of libc_r and pick that up as well. I think we need a portbuild run to find possible offenders. Perhaps you could remove libc_r from the system during the run to force those ports to fail. > > > Also, resolve any lasting concerns about how > > > build processes should say "And I want threads, dammit". > > > > This seems to be a point of debate, but it would be nice to resolve > > this sooner rather than later. My opinion is to do the following: > > - move libkse to libpthread > > - inform users that they must use -lpthread link in thread support > > - remove -pthread in gcc (maybe deprecate it instead) > > - fix the ports appropriately with PTHREAD_CFLAGS and PTHREAD_LDFLAGS > > There seems to be continuing disagreement on the gcc -pthread thing; I > don't really have much of an opinion on it, except that I think we should > choose the path of least headache. And that I suspect that path is to > keep it around... The one nice thing about dropping -pthread is it will > force everyone to go out and update the ports, but that sounds like a > technical solution to a social problem. We could as easily take other > paths visible only to ports developers in the cluster and shoot fewer > users. I think we resolved to keep -pthread and have it cause a link to -lc_r (and -lpthread after the switchover). I'd like to noop -pthread when linking shared libraries (so you can have application A use libc_r, application B use libpthread, and both applications use libGL.so) but that could be up for discussion. > > > (2) Have services like process debugging, profiling available and known > > > fully functional (or close). > > > > This stuff is good to have, but should the lack of it prevent moving > > libkse -> libpthread? It sounds like a lot of work. > > Well, it means that we have to document, loudly and carefully somewhere, > that in the default configuration, gdb on threaded applications is no > longer supported...? The goal is to have gdb support for 5.3. > > > us are running KSE as our libc_r via libmap.conf on all our machines, and > > > have been for many months, and it appears to hold up quite well :-). > > > Resolving how best to declare threading support in binaries will also > > > facilitate shipping the JDK linked against KSE. > > > > I also use libkse via libmap.conf, and am happy with it. I also agree > > that having a KSE-friendly version of the JDK will be very important. > > Are there any other ports that we should focus our efforts on in getting > > KSE to work with? httpd, mysql, ACE,....? > > That looks like a good starting list :-). Apache2 + eviljava + mysql > sounds like a killer combination with kse. > > My leaning is we should throw the switch to libkse sooner rather than > later to improve exposure, even though it's not quite feature complete. > It was basically an accident of timing and intent that we didn't get to > the rename before 5.2. For one thing, it will get applications to start > linking against libpthread sooner. However, if we're going to throw the > switch, it would be good to know the features aren't far behind :-). Other than debugging, sparc64 & alpha support, I think it already has more features than libc_r. Judging by the last reaction I got when -pthread was removed, I'd suggest we first try to see how the switchover is going to affect ports. Is Kris due back soon? From owner-freebsd-threads@FreeBSD.ORG Thu Jan 8 01:43:33 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C97516A4CE; Thu, 8 Jan 2004 01:43:33 -0800 (PST) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42BD343D3F; Thu, 8 Jan 2004 01:43:31 -0800 (PST) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (localhost [127.0.0.1]) by silver.he.iki.fi (8.12.9p2/8.11.4) with ESMTP id i089hSgr068863; Thu, 8 Jan 2004 11:43:29 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <3FFD2611.8020206@he.iki.fi> Date: Thu, 08 Jan 2004 11:42:41 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ryan Sommers References: <1073527914.650.41.camel@localhost> In-Reply-To: <1073527914.650.41.camel@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: Robert Watson cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 09:43:33 -0000 Ryan Sommers wrote: >My only concern with this is that it might cause some sort of delay in >getting to 5-STABLE. I think, as I'm sure most would agree, that >achieving a 5-STABLE is the next major milestone for the project. One >that, from the lists, some think is somewhat overdue. > > > The same can be said for M:N threads, they are overdue too and since kse works well with a few signal handling related exceptions, I think the general experience would be significantly improved by retiring current userland thread library. Pete From owner-freebsd-threads@FreeBSD.ORG Thu Jan 8 07:13:23 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1682916A4CE for ; Thu, 8 Jan 2004 07:13:22 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE8DE43D5A for ; Thu, 8 Jan 2004 07:13:21 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i08FDKiw003888; Thu, 8 Jan 2004 10:13:20 -0500 (EST) Date: Thu, 8 Jan 2004 10:13:20 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: "Daniel O'Connor" In-Reply-To: <200401081138.50146.doconnor@gsoft.com.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: KDE and KSE X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 15:13:23 -0000 On Thu, 8 Jan 2004, Daniel O'Connor wrote: > Hi, > I am trying to use KSE with KDE but I'm not having much luck :( > The upgrade process was a bit dodgy unfortunately, I upgraded KDE using > portupgrade -PP, but rebuilt Qt and fontconfig from source. > > I am using libmap.conf to get KSE, if I have no libmap.conf file (ie use > libc_r) KDE works fine. libthr doesn't work either but that's a different story > (different failure mode too) Sounds like something is screwy. KDE > After I do startx -> > XFree86 Version 4.3.0 > Release Date: 27 February 2003 > X Protocol Version 11, Revision 0, Release 6.6 > Build Operating System: FreeBSD 4.7 i386 [ELF] ^^^^^^^^^^^ ? You don't have any NVidia drivers or libraries installed do you? Failing that, I'd try rebuilding all out of date (-aP). Many of us have been using libkse with KDE for a while without any real problems. From owner-freebsd-threads@FreeBSD.ORG Thu Jan 8 07:43:55 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C01FF16A4CE for ; Thu, 8 Jan 2004 07:43:55 -0800 (PST) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id BDB6843D2D for ; Thu, 8 Jan 2004 07:43:54 -0800 (PST) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i08Fhriw010918; Thu, 8 Jan 2004 10:43:53 -0500 (EST) Date: Thu, 8 Jan 2004 10:43:53 -0500 (EST) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Earl Chew In-Reply-To: <3FFA0958.3020307@agilent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: misc/24641: pthread_rwlock_rdlock can deadlock X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 15:43:55 -0000 On Mon, 5 Jan 2004, Earl Chew wrote: > However, it appears counter-intuitive to me that if a thread has already > acquired a rdlock(), and the rdlock() is recursive, that a subsequent > call should fail. > > Failure should indicate that the lock could not be acquired. But the > thread _already_ has the lock! So it's hard to explain why the > subsequent call should fail (given the requirement that the > rdlock be recursive). I committed a fix for this in both libc_r and libkse. From owner-freebsd-threads@FreeBSD.ORG Thu Jan 8 14:44:27 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A9BAD16A4CE for ; Thu, 8 Jan 2004 14:44:27 -0800 (PST) Received: from smtp1.adl2.internode.on.net (smtp1.adl2.internode.on.net [203.16.214.181]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8350143D49 for ; Thu, 8 Jan 2004 14:44:25 -0800 (PST) (envelope-from doconnor@gsoft.com.au) Received: from midget.dons.net.au (ppp108-195.lns1.adl1.internode.on.net [150.101.108.195])i08MiMZC026834; Fri, 9 Jan 2004 09:14:22 +1030 (CST) Received: from chowder.gsoft.com.au (root@localhost.dons.net.au [127.0.0.1]) by midget.dons.net.au (8.12.9/8.12.9) with ESMTP id i08MiJaW010483; Fri, 9 Jan 2004 09:14:21 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: Daniel Eischen Date: Fri, 9 Jan 2004 09:14:18 +1030 User-Agent: KMail/1.5.4 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401090914.18767.doconnor@gsoft.com.au> X-Spam-Score: -5 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01,USER_AGENT,USER_AGENT_KMAIL X-Scanned-By: MIMEDefang 2.26 (www . roaringpenguin . com / mimedefang) cc: threads@freebsd.org Subject: Re: KDE and KSE X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 22:44:27 -0000 On Friday 09 January 2004 01:43, Daniel Eischen wrote: > > After I do startx -> > > XFree86 Version 4.3.0 > > Release Date: 27 February 2003 > > X Protocol Version 11, Revision 0, Release 6.6 > > Build Operating System: FreeBSD 4.7 i386 [ELF] > > ^^^^^^^^^^^ ? > > You don't have any NVidia drivers or libraries installed do you? > Failing that, I'd try rebuilding all out of date (-aP). Many of > us have been using libkse with KDE for a while without any real > problems. Hmm I haven't rebuilt X.. Oops :) I'll try that and see what happens. (Well portupgrade -PP) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 9A8C 569F 685A D928 5140 AE4B 319B 41F4 5D17 FDD5 From owner-freebsd-threads@FreeBSD.ORG Thu Jan 8 15:51:14 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7CEA016A4CE for ; Thu, 8 Jan 2004 15:51:14 -0800 (PST) Received: from smtp3.adl2.internode.on.net (smtp3.adl2.internode.on.net [203.16.214.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7FC7143D3F for ; Thu, 8 Jan 2004 15:50:27 -0800 (PST) (envelope-from doconnor@gsoft.com.au) Received: from midget.dons.net.au (ppp108-195.lns1.adl1.internode.on.net [150.101.108.195])i08NoOqR064449; Fri, 9 Jan 2004 10:20:24 +1030 (CST) Received: from chowder.gsoft.com.au (root@localhost.dons.net.au [127.0.0.1]) by midget.dons.net.au (8.12.9/8.12.9) with ESMTP id i08NoMaW011017; Fri, 9 Jan 2004 10:20:22 +1030 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: Daniel Eischen Date: Fri, 9 Jan 2004 10:20:20 +1030 User-Agent: KMail/1.5.4 References: <200401090914.18767.doconnor@gsoft.com.au> In-Reply-To: <200401090914.18767.doconnor@gsoft.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200401091020.20996.doconnor@gsoft.com.au> X-Spam-Score: -5 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01,USER_AGENT,USER_AGENT_KMAIL X-Scanned-By: MIMEDefang 2.26 (www . roaringpenguin . com / mimedefang) cc: threads@freebsd.org Subject: Re: KDE and KSE X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2004 23:51:14 -0000 On Friday 09 January 2004 09:14, Daniel O'Connor wrote: > On Friday 09 January 2004 01:43, Daniel Eischen wrote: > > You don't have any NVidia drivers or libraries installed do you? > > Failing that, I'd try rebuilding all out of date (-aP). Many of > > us have been using libkse with KDE for a while without any real > > problems. > > Hmm I haven't rebuilt X.. > > Oops :) > > I'll try that and see what happens. > (Well portupgrade -PP) Ahah! We have a winner, thanks :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 9A8C 569F 685A D928 5140 AE4B 319B 41F4 5D17 FDD5 From owner-freebsd-threads@FreeBSD.ORG Thu Jan 8 16:21:30 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EAAF416A4CE for ; Thu, 8 Jan 2004 16:21:30 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 28AEC43D64 for ; Thu, 8 Jan 2004 16:20:54 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i090JSUd025701 for ; Thu, 8 Jan 2004 19:19:28 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i090JSp9025698 for ; Thu, 8 Jan 2004 19:19:28 -0500 (EST) (envelope-from robert@fledge.watson.org) X-Received: from cyrus.watson.org ([unix socket]) by cyrus.watson.org (Cyrus v2.1.12) with LMTP; Thu, 08 Jan 2004 19:14:09 -0500 X-Sieve: CMU Sieve 2.2 X-Received: from mx2.freebsd.org (mx2.freebsd.org [216.136.204.119]) by cyrus.watson.org (Postfix) with ESMTP id AFA89ACB4 for ; Thu, 8 Jan 2004 19:14:08 -0500 (EST) X-Received: from hub.freebsd.org (hub.freebsd.org [216.136.204.18]) by mx2.freebsd.org (Postfix) with ESMTP id E15C456411; Thu, 8 Jan 2004 16:14:06 -0800 (PST) (envelope-from owner-freebsd-current@freebsd.org) X-Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id D829E16A4D7; Thu, 8 Jan 2004 16:14:01 -0800 (PST) Delivered-To: freebsd-current@freebsd.org X-Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 351D816A4CE; Thu, 8 Jan 2004 16:13:48 -0800 (PST) X-Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E81543D31; Thu, 8 Jan 2004 16:13:46 -0800 (PST) (envelope-from robert@fledge.watson.org) X-Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i090CKUd024413; Thu, 8 Jan 2004 19:12:20 -0500 (EST) (envelope-from robert@fledge.watson.org) X-Received: from localhost (robert@localhost)i090CJa0024410; Thu, 8 Jan 2004 19:12:19 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Thu, 8 Jan 2004 19:12:19 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Don Lewis In-Reply-To: <200401081210.i08CAt7E019668@gw.catspoiler.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Sender: owner-freebsd-current@FreeBSD.org Errors-To: owner-freebsd-current@FreeBSD.org ReSent-Date: Thu, 8 Jan 2004 19:19:24 -0500 (EST) Resent-From: robert Resent-To: threads@FreeBSD.org ReSent-Message-ID: cc: current@FreeBSD.org Subject: Re: strace, holding sigacts lock over postsig(), et al. X-BeenThere: freebsd-threads@freebsd.org Reply-To: Robert Watson List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 00:21:31 -0000 On Thu, 8 Jan 2004, Don Lewis wrote: > In both issignal() and postsig() I think it would be safe to drop the > p_sig mutex before _STOPEVENT() and grab the mutex again afterwards. > About the only thing that can happen during the interim would be the > receipt of another signal and I don't think that would be a problem. > Dropping the mutex is how issignal() handles ptracestop() a bit further > down in the code. Alright, a headache or so later, here are my conclusions: (1) I committed the change to drop the sigact mutex around the two calls to stopevent(). I agree it should be safe. (2) Ctrl-T gives some mighty useless output if the thread selected for siginfo() is suspended or in another less common state, so I modified siginfo() some to be more informative. As a couple of people have pointed out, "You should never get the intrwait case" -- yeah, I know. But if I do get it, I want to know about it. (3) There appears to be a problem associated with procfs waiting for a process being debugged to suspend. When strace calls PIOCWAIT using procfs, the debugging proces performs an msleep() on p_stype. However, the child process appears already to be suspended in thread_suspend_check() due to TDS_INHIBITED resulting from P_STOPPED being set. I think this may be a property of conflicting suspension models: stopevent() and the scheduler suspended state. This may be a result of strace using both ptrace() and procfs side-by-side. In any case, the parent strace process will wait forever for the child to hit a stopevent, which the child will never hit. I'm still attempting to wade through the more complex thread/process state machine with KSE and figure out what should be happening. I also need to grab a ktrace of strace doing its thing to figure out what precise sequence of events is kicking off the problem: for example, it could be that the debugging attachment is being done using ptrace(), but strce is then trying to use procfs to wait for events. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Fri Jan 9 09:36:36 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F87C16A4CE for ; Fri, 9 Jan 2004 09:36:36 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7AAB143D3F for ; Fri, 9 Jan 2004 09:36:34 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.10/8.12.10) with ESMTP id i09HZ6Ud064874; Fri, 9 Jan 2004 12:35:06 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i09HZ6bL064871; Fri, 9 Jan 2004 12:35:06 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 9 Jan 2004 12:35:05 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Daniel Eischen In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Release criteria for libkse -> libpthread switch? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jan 2004 17:36:36 -0000 On Thu, 8 Jan 2004, Daniel Eischen wrote: > > When we build packages, etc, that rely on threading, a library name is > > built into each binary and library depending on it identifying the > > dependency. Currently, the packages built in the package cluster use > > "libc_r" as the name to depend on for threads. I think the right > > long-term answer is for thread-dependent packages to simply link against > > "libpthread". People can rebuild locally to use something else, rename > > libraries, or use libmap.conf to select alternatives. What I don't want > > to see happen is lots of binaries and libraries with conflicting > > dependencies, and I want to avoid a lot of applications linking against a > > name that will go away, be insufficiently supported, etc. > > Right, I think this is mostly solved by changing gcc -pthread to use > -lpthread (at the same time as the libkse switchover) and changing the > default PTHREAD_LIBS to -lpthread. That gets you most of the way there, > but there may still be ports that know about the existence of libc_r and > pick that up as well. I think we need a portbuild run to find possible > offenders. Perhaps you could remove libc_r from the system during the > run to force those ports to fail. Sounds good. > I think we resolved to keep -pthread and have it cause a link to -lc_r > (and -lpthread after the switchover). I'd like to noop -pthread when > linking shared libraries (so you can have application A use libc_r, > application B use libpthread, and both applications use libGL.so) but > that could be up for discussion. Also sounds good. > > Well, it means that we have to document, loudly and carefully somewhere, > > that in the default configuration, gdb on threaded applications is no > > longer supported...? > > The goal is to have gdb support for 5.3. Sounds great :-). > Other than debugging, sparc64 & alpha support, I think it already has > more features than libc_r. Judging by the last reaction I got when > -pthread was removed, I'd suggest we first try to see how the switchover > is going to affect ports. Is Kris due back soon? I'm not sure what Kris's schedule is, but I agree that sparc64 and alpha support sound like the last remaining piece. I think we need to move ahead with the default change for the rest of the platforms without waiting for them, however. I think it would be reasonable to throw the switch once the press release is out the door for 5.2 (hopefully very early next week). Has anyone experimented with profiling applications running KSE with multiple CPUs in use? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research