From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 10:47:43 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9126B16A4CE; Mon, 15 Nov 2004 10:47:43 +0000 (GMT) Received: from thekla.de.clara.net (thekla.de.clara.net [212.82.225.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id B8FCD43D48; Mon, 15 Nov 2004 10:47:42 +0000 (GMT) (envelope-from michael.riexinger@de.clara.net) Received: from localhost.de.clara.net ([127.0.0.1] helo=localhost) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTeOb-0002S4-EC; Mon, 15 Nov 2004 11:47:41 +0100 Received: from box.int.de.clara.net ([192.168.0.226]) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTeOb-0002S0-9E; Mon, 15 Nov 2004 11:47:41 +0100 From: Michael Riexinger To: Robert Watson Date: Mon, 15 Nov 2004 11:47:47 +0100 User-Agent: KMail/1.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200411151147.47755.michael.riexinger@de.clara.net> cc: threads@freebsd.org Subject: Re: heavy named problems (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 10:47:43 -0000 On Saturday 13 November 2004 10:17, Robert Watson wrote: > Figured I'd forward this over to threads@ since it sounds like a > potential threading resource leak (or the like). If that's really a threading problem, what's the easiest way to compile the base named without threading? I really want to get rid of that problem as fast as possible. Thanks, Michael Riexinger systems engineer -- claranet gmbh internet service provider tel +49 (0) 69 - 40 80 18 - 300 email: michael.riexinger@de.clara.net http://www.claranet.de/ From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 10:53:00 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C7D916A4CE for ; Mon, 15 Nov 2004 10:53:00 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2F1C43D41 for ; Mon, 15 Nov 2004 10:52:59 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id iAFApOpr079012; Mon, 15 Nov 2004 05:51:24 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)iAFApOWS079009; Mon, 15 Nov 2004 10:51:24 GMT (envelope-from robert@fledge.watson.org) Date: Mon, 15 Nov 2004 10:51:24 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Michael Riexinger In-Reply-To: <200411151147.47755.michael.riexinger@de.clara.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: heavy named problems (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 10:53:00 -0000 On Mon, 15 Nov 2004, Michael Riexinger wrote: > On Saturday 13 November 2004 10:17, Robert Watson wrote: > > Figured I'd forward this over to threads@ since it sounds like a > > potential threading resource leak (or the like). > If that's really a threading problem, what's the easiest way to compile > the base named without threading? I really want to get rid of that > problem as fast as possible. Could you try using libmap.conf to map libpthread to libc_r for named, and see if that side-steps it? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 10:57:30 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C505316A4CE for ; Mon, 15 Nov 2004 10:57:30 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65A7343D3F for ; Mon, 15 Nov 2004 10:57:30 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id iAFAu5Q7079060; Mon, 15 Nov 2004 05:56:05 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)iAFAu5dZ079057; Mon, 15 Nov 2004 10:56:05 GMT (envelope-from robert@fledge.watson.org) Date: Mon, 15 Nov 2004 10:56:05 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Julian Elischer In-Reply-To: <4192763C.2010403@elischer.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: [Fwd: Re: Mysql - Linuxthreads : Still needed?] X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 10:57:30 -0000 On Wed, 10 Nov 2004, Julian Elischer wrote: > send to the right list. So, the good news is that we've successfully dramatically improved the performance of MySQL between 5.2 and 5.3. The mixed news is that clearly we have more work to do. :-) Out of curiousity, has anyone done much in the way of kernel profiling during heavy duty MySQL runs to see if there are specific kernel bottlenecks we can be working on? I've noticed that we're contending a fair amount on UNIX domain socket locking, although with Giant off the stack this is a big improvement over what we saw previously. Although it was a few months ago, my recollection from profiling the mutex use and kernel use is that we're spending a lot of time checking to see if we have to deliver signals to threads or not in kernel. I may have the opportunity to do a bit of profiling today or tomorrow and see what I bump into. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 11:00:26 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2471416A4CE; Mon, 15 Nov 2004 11:00:26 +0000 (GMT) Received: from thekla.de.clara.net (thekla.de.clara.net [212.82.225.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB2C243D45; Mon, 15 Nov 2004 11:00:25 +0000 (GMT) (envelope-from michael.riexinger@de.clara.net) Received: from localhost.de.clara.net ([127.0.0.1] helo=localhost) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTeav-0003kq-33; Mon, 15 Nov 2004 12:00:25 +0100 Received: from box.int.de.clara.net ([192.168.0.226]) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTeau-0003kk-FI; Mon, 15 Nov 2004 12:00:24 +0100 From: Michael Riexinger To: Robert Watson Date: Mon, 15 Nov 2004 12:00:30 +0100 User-Agent: KMail/1.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200411151200.31168.michael.riexinger@de.clara.net> cc: threads@freebsd.org Subject: Re: heavy named problems (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 11:00:26 -0000 On Monday 15 November 2004 11:51, Robert Watson wrote: > On Mon, 15 Nov 2004, Michael Riexinger wrote: > > On Saturday 13 November 2004 10:17, Robert Watson wrote: > > > Figured I'd forward this over to threads@ since it sounds like a > > > potential threading resource leak (or the like). > > > > If that's really a threading problem, what's the easiest way to > > compile the base named without threading? I really want to get rid > > of that problem as fast as possible. > > Could you try using libmap.conf to map libpthread to libc_r for > named, and see if that side-steps it? Ok, done that. The problem appears 1 or 2 times a day, so I'll keep track and hope that it's gone now. Kind regards, Michael Riexinger systems engineer -- claranet gmbh internet service provider tel +49 (0) 69 - 40 80 18 - 300 email: michael.riexinger@de.clara.net http://www.claranet.de/ From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 11:02:19 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 11B1816A4CE for ; Mon, 15 Nov 2004 11:02:19 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id D9D0643D1F for ; Mon, 15 Nov 2004 11:02:18 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id iAFB2IPN074744 for ; Mon, 15 Nov 2004 11:02:18 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id iAFB2IxF074738 for freebsd-threads@freebsd.org; Mon, 15 Nov 2004 11:02:18 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 15 Nov 2004 11:02:18 GMT Message-Id: <200411151102.iAFB2IxF074738@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 11:02:19 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2004/04/22] threads/65883threads libkse's sigwait does not work after fork 1 problem total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2000/08/26] kern/20861 threads libc_r does not honor socket timeouts o [2001/01/20] threads/24472threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] threads/24632threads libc_r delicate deviation from libc in ha o [2001/01/25] kern/24641 threads pthread_rwlock_rdlock can deadlock o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/02/01] threads/34536threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/06/27] threads/39922threads [PATCH?] Threaded applications executed w o [2002/08/04] kern/41331 threads Pthread library open sets O_NONBLOCK flag o [2003/03/02] threads/48856threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] threads/49087threads Signals lost in programs linked with libc o [2003/05/08] threads/51949threads thread in accept cannot be cancelled s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/08/26] threads/70975threads unexpected and unreliable behaviour when o [2004/09/14] threads/71725threads Mysql Crashes frequently giving Sock Erro o [2004/10/05] threads/72353threads Assertion fails in /usr/src/lib/libpthrea o [2004/10/07] threads/72429threads threads blocked in stdio (fgets, etc) are o [2004/10/21] threads/72953threads fork() unblocks blocked signals w/o PTHRE 19 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/05/26] kern/18824 threads gethostbyname is not thread safe o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2000/10/21] kern/22190 threads A threaded read(2) from a socketpair(2) f o [2001/09/09] threads/30464threads pthread mutex attributes -- pshared o [2002/05/02] threads/37676threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] threads/40671threads pthread_cancel doesn't remove thread from o [2004/07/13] threads/69020threads pthreads library leaks _gc_mutex o [2004/09/21] threads/71966threads Mlnet Core Dumped : Fatal error '_pq_inse 8 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 12:23:08 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A835416A4CE; Mon, 15 Nov 2004 12:23:08 +0000 (GMT) Received: from publicd.ub.mng.net (publicd.ub.mng.net [202.179.0.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2B90043D49; Mon, 15 Nov 2004 12:23:04 +0000 (GMT) (envelope-from ganbold@micom.mng.net) Received: from [202.179.0.164] (helo=ganbold.micom.mng.net) by publicd.ub.mng.net with esmtpa (Exim 4.43 (FreeBSD)) id 1CTfs4-0008hy-Q1; Mon, 15 Nov 2004 20:22:13 +0800 Message-Id: <6.2.0.14.2.20041115200900.0301a9b0@202.179.0.80> X-Mailer: QUALCOMM Windows Eudora Version 6.2.0.14 Date: Mon, 15 Nov 2004 20:22:08 +0800 To: Robert Watson From: Ganbold In-Reply-To: References: <4192763C.2010403@elischer.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed cc: threads@freebsd.org cc: julian@elischer.org cc: jhopper@bsdhosting.net Subject: Re: [Fwd: Re: Mysql - Linuxthreads : Still needed?] X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 12:23:08 -0000 Some mysql benchmark I just did... ----------------------------------------------------------------------- System Specs: IBM @server 325 type 8835 Dual AMD Opteron 2.1ghz 4GB (2x2GB) PC-2700 ECC memory 6x36GB hot swap RAID5 Ultra 320 SCSI HDD ServeRAID 6M controller ------------------------------------------------------------------------- OS / Software Configuration: FreeBSD amd.micom.mng.net 5.3-STABLE FreeBSD 5.3-STABLE #6: Thu Nov 11 21:08:50 ULAT 2004 tsgan@amd.micom.mng.net:/usr/obj/usr/src/sys/AMD amd64 MySQL Compiled from mysql40-server port with: BUILD_OPTIMIZED=yes BUILD_STATIC=yes ------------------------------------------------------------------------- kernel without SMP and without PREEMPTION options ------------------------------------------------------------------------- amd# cat test_nosmp_nopreemption Query Barrel Report for client smacker connect: max=7ms min=2ms avg= 4ms from 30 clients Query_type num_queries max_time min_time q_per_s select_index 300000 4 0 3565.71 update_index 300000 4 0 3565.71 ------------------------------------------------------------------------- ------------------------------------------------------------------------- kernel without SMP and with PREEMPTION options ------------------------------------------------------------------------- amd# cat test_nosmp_preemption Query Barrel Report for client smacker connect: max=18ms min=1ms avg= 8ms from 30 clients Query_type num_queries max_time min_time q_per_s select_index 300000 6 0 3568.64 update_index 300000 4 0 3568.64 ------------------------------------------------------------------------- ------------------------------------------------------------------------- kernel with SMP and without PREEMPTION options ------------------------------------------------------------------------- amd# cat test_smp_nopreemption Query Barrel Report for client smacker connect: max=16ms min=5ms avg= 8ms from 30 clients Query_type num_queries max_time min_time q_per_s select_index 300000 4 0 3755.09 update_index 300000 3 0 3755.09 ------------------------------------------------------------------------- ------------------------------------------------------------------------- kernel with SMP and with PREEMPTION options ------------------------------------------------------------------------- amd# cat test_smp_preemption Query Barrel Report for client smacker connect: max=14ms min=9ms avg= 10ms from 30 clients Query_type num_queries max_time min_time q_per_s select_index 300000 4 0 4053.86 update_index 300000 3 0 4053.86 ------------------------------------------------------------------------- It seems like fastest result I get in kernel with SMP and with PREEMPTION options. regards, Ganbold At 06:56 PM 11/15/2004, you wrote: >On Wed, 10 Nov 2004, Julian Elischer wrote: > > > send to the right list. > >So, the good news is that we've successfully dramatically improved the >performance of MySQL between 5.2 and 5.3. The mixed news is that clearly >we have more work to do. :-) Out of curiousity, has anyone done much in >the way of kernel profiling during heavy duty MySQL runs to see if there >are specific kernel bottlenecks we can be working on? > >I've noticed that we're contending a fair amount on UNIX domain socket >locking, although with Giant off the stack this is a big improvement over >what we saw previously. Although it was a few months ago, my recollection >from profiling the mutex use and kernel use is that we're spending a lot >of time checking to see if we have to deliver signals to threads or not in >kernel. I may have the opportunity to do a bit of profiling today or >tomorrow and see what I bump into. > >Robert N M Watson FreeBSD Core Team, TrustedBSD Projects >robert@fledge.watson.org Principal Research Scientist, McAfee Research > > >_______________________________________________ >freebsd-threads@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-threads >To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 12:55:52 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CD83316A4CE; Mon, 15 Nov 2004 12:55:52 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 59FCC43D41; Mon, 15 Nov 2004 12:55:52 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAFCtnsu002500; Mon, 15 Nov 2004 07:55:49 -0500 (EST) Date: Mon, 15 Nov 2004 07:55:49 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Robert Watson In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: threads@freebsd.org cc: Julian Elischer Subject: Re: [Fwd: Re: Mysql - Linuxthreads : Still needed?] X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 12:55:52 -0000 On Mon, 15 Nov 2004, Robert Watson wrote: > > On Wed, 10 Nov 2004, Julian Elischer wrote: > > > send to the right list. > > So, the good news is that we've successfully dramatically improved the > performance of MySQL between 5.2 and 5.3. The mixed news is that clearly > we have more work to do. :-) Out of curiousity, has anyone done much in > the way of kernel profiling during heavy duty MySQL runs to see if there > are specific kernel bottlenecks we can be working on? > > I've noticed that we're contending a fair amount on UNIX domain socket > locking, although with Giant off the stack this is a big improvement over > what we saw previously. Although it was a few months ago, my recollection > from profiling the mutex use and kernel use is that we're spending a lot > of time checking to see if we have to deliver signals to threads or not in > kernel. I may have the opportunity to do a bit of profiling today or > tomorrow and see what I bump into. David did a bit of research and found that in M:N cases there is a lot of contention for upcall allocation (actually I think it is thread allocation) in the kernel. It looks like we completely teardown and free completed threads, then reallocate and set them up again when they unblock and we need a new thread to convert into an upcall. It would probably be faster to cache them (hang them off the KSE group) instead of completely tearing them down and deallocating them. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 16:41:04 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF70716A4CE; Mon, 15 Nov 2004 16:41:04 +0000 (GMT) Received: from thekla.de.clara.net (thekla.de.clara.net [212.82.225.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id 28CFE43D2D; Mon, 15 Nov 2004 16:41:04 +0000 (GMT) (envelope-from michael.riexinger@de.clara.net) Received: from localhost.de.clara.net ([127.0.0.1] helo=localhost) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTjuY-000DPg-OI; Mon, 15 Nov 2004 17:41:02 +0100 Received: from box.int.de.clara.net ([192.168.0.226]) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTjuY-000DPc-BN; Mon, 15 Nov 2004 17:41:02 +0100 From: Michael Riexinger To: freebsd-threads@freebsd.org Date: Mon, 15 Nov 2004 17:41:09 +0100 User-Agent: KMail/1.7 References: <200411151200.31168.michael.riexinger@de.clara.net> In-Reply-To: <200411151200.31168.michael.riexinger@de.clara.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200411151741.09519.michael.riexinger@de.clara.net> cc: threads@freebsd.org cc: Robert Watson Subject: Re: heavy named problems (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 16:41:05 -0000 On Monday 15 November 2004 12:00, Michael Riexinger wrote: > On Monday 15 November 2004 11:51, Robert Watson wrote: > > On Mon, 15 Nov 2004, Michael Riexinger wrote: > > > On Saturday 13 November 2004 10:17, Robert Watson wrote: > > > > Figured I'd forward this over to threads@ since it sounds like > > > > a potential threading resource leak (or the like). > > > > > > If that's really a threading problem, what's the easiest way to > > > compile the base named without threading? I really want to get > > > rid of that problem as fast as possible. > > > > Could you try using libmap.conf to map libpthread to libc_r for > > named, and see if that side-steps it? > > Ok, done that. The problem appears 1 or 2 times a day, so I'll keep > track and hope that it's gone now. Didn't help, few minutes ago, I had to restart the named again.. Kind regards, Michael Riexinger systems engineer -- claranet gmbh internet service provider tel +49 (0) 69 - 40 80 18 - 300 email: michael.riexinger@de.clara.net http://www.claranet.de/ From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 16:41:04 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF70716A4CE; Mon, 15 Nov 2004 16:41:04 +0000 (GMT) Received: from thekla.de.clara.net (thekla.de.clara.net [212.82.225.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id 28CFE43D2D; Mon, 15 Nov 2004 16:41:04 +0000 (GMT) (envelope-from michael.riexinger@de.clara.net) Received: from localhost.de.clara.net ([127.0.0.1] helo=localhost) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTjuY-000DPg-OI; Mon, 15 Nov 2004 17:41:02 +0100 Received: from box.int.de.clara.net ([192.168.0.226]) by thekla.de.clara.net with esmtp (Exim 4.30; FreeBSD) id 1CTjuY-000DPc-BN; Mon, 15 Nov 2004 17:41:02 +0100 From: Michael Riexinger To: freebsd-threads@freebsd.org Date: Mon, 15 Nov 2004 17:41:09 +0100 User-Agent: KMail/1.7 References: <200411151200.31168.michael.riexinger@de.clara.net> In-Reply-To: <200411151200.31168.michael.riexinger@de.clara.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200411151741.09519.michael.riexinger@de.clara.net> cc: threads@freebsd.org cc: Robert Watson Subject: Re: heavy named problems (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 16:41:05 -0000 On Monday 15 November 2004 12:00, Michael Riexinger wrote: > On Monday 15 November 2004 11:51, Robert Watson wrote: > > On Mon, 15 Nov 2004, Michael Riexinger wrote: > > > On Saturday 13 November 2004 10:17, Robert Watson wrote: > > > > Figured I'd forward this over to threads@ since it sounds like > > > > a potential threading resource leak (or the like). > > > > > > If that's really a threading problem, what's the easiest way to > > > compile the base named without threading? I really want to get > > > rid of that problem as fast as possible. > > > > Could you try using libmap.conf to map libpthread to libc_r for > > named, and see if that side-steps it? > > Ok, done that. The problem appears 1 or 2 times a day, so I'll keep > track and hope that it's gone now. Didn't help, few minutes ago, I had to restart the named again.. Kind regards, Michael Riexinger systems engineer -- claranet gmbh internet service provider tel +49 (0) 69 - 40 80 18 - 300 email: michael.riexinger@de.clara.net http://www.claranet.de/ From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 20:43:33 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3CF4B16A4CE for ; Mon, 15 Nov 2004 20:43:33 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id CF36C43D2D for ; Mon, 15 Nov 2004 20:43:30 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAFKhR1i040005 for ; Mon, 15 Nov 2004 22:43:27 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419914F0.1010308@he.iki.fi> Date: Mon, 15 Nov 2004 22:43:28 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 20:43:33 -0000 Now that 5.3 is the first STABLE release, is the future focus on -threads towards performance, so that benchmarks and profiling, etc. would be appreciated somewhat more than they might have been previously when the development was more in the "just get it working right" mode? Momentarily I could provide benchmarks from fairly parallel dual-HTT CPU machines running libpthread code (both proprietary and real-world mysql/apache/mod_perl loads) Pete From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 21:07:21 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB1ED16A4CE for ; Mon, 15 Nov 2004 21:07:21 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 174E443D3F for ; Mon, 15 Nov 2004 21:07:20 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id AE6CA7A403; Mon, 15 Nov 2004 13:07:19 -0800 (PST) Message-ID: <41991A87.6010803@elischer.org> Date: Mon, 15 Nov 2004 13:07:19 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Petri Helenius References: <419914F0.1010308@he.iki.fi> In-Reply-To: <419914F0.1010308@he.iki.fi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 21:07:21 -0000 Petri Helenius wrote: > > Now that 5.3 is the first STABLE release, is the future focus on > -threads towards performance, so that benchmarks and profiling, etc. > would be appreciated somewhat more than they might have been > previously when the development was more in the "just get it working > right" mode? > Basically, yes. there is still work to do in the "get it right" field but it's working "right enough" now that performance is starting to be an issue. There is work proceeding on 3 fronts. * libthr continues to be improved by Mike. * libpthread is being scrutinised for ineffiencies and problems. In particular M:N performance has a lot to be desired. * (from left field) David has been working on a hybrid of libpthread and libthr. As a private project, David is playing w > Momentarily I could provide benchmarks from fairly parallel dual-HTT > CPU machines running libpthread code (both proprietary and real-world > mysql/apache/mod_perl loads) Is that the American or British meaning of Momentarily? (i.e. "IN a moment" or "FOR a moment"? ) :-) What we will really want eventually is KTR output as well as profilings. i.e. output of ktrdump. > > Pete > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads To > unsubscribe, send any mail to > "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 21:11:03 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0911416A4CE for ; Mon, 15 Nov 2004 21:11:03 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDD1943D49 for ; Mon, 15 Nov 2004 21:11:02 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id B2EBE7A403; Mon, 15 Nov 2004 13:11:02 -0800 (PST) Message-ID: <41991B66.4090801@elischer.org> Date: Mon, 15 Nov 2004 13:11:02 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Julian Elischer References: <419914F0.1010308@he.iki.fi> <41991A87.6010803@elischer.org> In-Reply-To: <41991A87.6010803@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 21:11:03 -0000 Julian Elischer wrote: > > > Petri Helenius wrote: > >> >> Now that 5.3 is the first STABLE release, is the future focus on >> -threads towards performance, so that benchmarks and profiling, etc. >> would be appreciated somewhat more than they might have been >> previously when the development was more in the "just get it working >> right" mode? >> > Basically, yes. > there is still work to do in the "get it right" field but > it's working "right enough" now that performance is > starting to be an issue. > > There is work proceeding on 3 fronts. > > * libthr continues to be improved by Mike. > > * libpthread is being scrutinised for ineffiencies and problems. > In particular M:N performance has a lot to be desired. > > * (from left field) David has been working on a hybrid of libpthread and > libthr. As a private project, David is playing w oops accidentally deleted some of my own mail.. -- with a version of libpthread optimised for 1:1 using libthr entrypoints. > > >> Momentarily I could provide benchmarks from fairly parallel dual-HTT >> CPU machines running libpthread code (both proprietary and real-world >> mysql/apache/mod_perl loads) > > > > Is that the American or British meaning of Momentarily? > (i.e. "IN a moment" or "FOR a moment"? ) :-) > > What we will really want eventually is KTR output as well as profilings. > > i.e. output of ktrdump. > > >> >> Pete >> >> _______________________________________________ >> freebsd-threads@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-threads To >> unsubscribe, send any mail to >> "freebsd-threads-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to > "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 21:42:27 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52E2F16A4CE for ; Mon, 15 Nov 2004 21:42:27 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id A237F43D46 for ; Mon, 15 Nov 2004 21:42:26 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAFLgG89041939; Mon, 15 Nov 2004 23:42:16 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419922B9.7050603@he.iki.fi> Date: Mon, 15 Nov 2004 23:42:17 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Julian Elischer References: <419914F0.1010308@he.iki.fi> <41991A87.6010803@elischer.org> <41991B66.4090801@elischer.org> In-Reply-To: <41991B66.4090801@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 21:42:27 -0000 Julian Elischer wrote: > >> >> >> There is work proceeding on 3 fronts. >> >> * libthr continues to be improved by Mike. > Is there a guideline when one should consider using libthr? I haven't tried libthr for quite a while. >> >> * libpthread is being scrutinised for ineffiencies and problems. >> In particular M:N performance has a lot to be desired. > I suppose this requires both userland and kernel tracing/profiling? > >> >> Is that the American or British meaning of Momentarily? >> (i.e. "IN a moment" or "FOR a moment"? ) :-) > American. Do I sound like I talk funny? :-) >> >> What we will really want eventually is KTR output as well as profilings. >> >> i.e. output of ktrdump. > Do you have set of ALQ, MASK etc. you're interested in? It would help if I could get a ready set of options to drop into the kernel... Pete From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 21:47:54 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7627F16A4CE for ; Mon, 15 Nov 2004 21:47:54 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C94A43D39 for ; Mon, 15 Nov 2004 21:47:53 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAFLlk8h041966; Mon, 15 Nov 2004 23:47:46 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <41992403.8040308@he.iki.fi> Date: Mon, 15 Nov 2004 23:47:47 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Julian Elischer References: <419914F0.1010308@he.iki.fi> <41991A87.6010803@elischer.org> <41991B66.4090801@elischer.org> <419922B9.7050603@he.iki.fi> In-Reply-To: <419922B9.7050603@he.iki.fi> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 21:47:54 -0000 Petri Helenius wrote: > Julian Elischer wrote: > >> >>> >>> >>> There is work proceeding on 3 fronts. >>> >>> * libthr continues to be improved by Mike. >> >> > Is there a guideline when one should consider using libthr? I haven't > tried libthr for quite a while. To comment myself, just tried libthr and it seems to get stuck on umtx fairly easily and the application just sits idle. I just libmap.conf:ed pthread->thr. Average lifetime was around five seconds before it hung. On libpthread the same piece of software runs for months uninterrupted. Pete From owner-freebsd-threads@FreeBSD.ORG Mon Nov 15 21:53:15 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 11C4A16A4CE for ; Mon, 15 Nov 2004 21:53:15 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED41543D45 for ; Mon, 15 Nov 2004 21:53:14 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id D64D57A403; Mon, 15 Nov 2004 13:53:14 -0800 (PST) Message-ID: <4199254A.8030607@elischer.org> Date: Mon, 15 Nov 2004 13:53:14 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Petri Helenius References: <419914F0.1010308@he.iki.fi> <41991A87.6010803@elischer.org> <41991B66.4090801@elischer.org> <419922B9.7050603@he.iki.fi> <41992403.8040308@he.iki.fi> In-Reply-To: <41992403.8040308@he.iki.fi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Nov 2004 21:53:15 -0000 Petri Helenius wrote: > Petri Helenius wrote: > > > Julian Elischer wrote: > > > >> > >>> > >>> > >>> There is work proceeding on 3 fronts. > >>> > >>> * libthr continues to be improved by Mike. > >> > >> > >> > > Is there a guideline when one should consider using libthr? I > > haven't tried libthr for quite a while. > > > > To comment myself, just tried libthr and it seems to get stuck on > umtx fairly easily and the application just sits idle. I just > libmap.conf:ed pthread->thr. Average lifetime was around five seconds > before it hung. On libpthread the same piece of software runs for > months uninterrupted. > The good thing about libthr is that, being so different, it tends to provide a good second string for helping diagnose problems.. For those applications that run with it, they tend to run well. It has its own problems as you noticed, but it's always worth checking it now and then. > Pete From owner-freebsd-threads@FreeBSD.ORG Tue Nov 16 00:00:04 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADF6416A4D1 for ; Tue, 16 Nov 2004 00:00:04 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8FD8C43D46; Tue, 16 Nov 2004 00:00:04 +0000 (GMT) (envelope-from davidxu@freebsd.org) Received: from [127.0.0.1] (davidxu@localhost [127.0.0.1]) iAG0032T077129; Tue, 16 Nov 2004 00:00:03 GMT (envelope-from davidxu@freebsd.org) Message-ID: <41994302.6020806@freebsd.org> Date: Tue, 16 Nov 2004 08:00:02 +0800 From: David Xu User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.2) Gecko/20040921 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Julian Elischer References: <419914F0.1010308@he.iki.fi> <41991A87.6010803@elischer.org> <41991B66.4090801@elischer.org> In-Reply-To: <41991B66.4090801@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: performance tuning X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Nov 2004 00:00:04 -0000 Julian Elischer wrote: > > > Julian Elischer wrote: > >> >> >> Petri Helenius wrote: >> >>> >>> Now that 5.3 is the first STABLE release, is the future focus on >>> -threads towards performance, so that benchmarks and profiling, etc. >>> would be appreciated somewhat more than they might have been >>> previously when the development was more in the "just get it working >>> right" mode? >>> >> Basically, yes. >> there is still work to do in the "get it right" field but >> it's working "right enough" now that performance is >> starting to be an issue. >> >> There is work proceeding on 3 fronts. >> >> * libthr continues to be improved by Mike. >> >> * libpthread is being scrutinised for ineffiencies and problems. >> In particular M:N performance has a lot to be desired. >> >> * (from left field) David has been working on a hybrid of libpthread and >> libthr. As a private project, David is playing w > > > oops accidentally deleted some of my own mail.. > > > > -- with a version of libpthread optimised for 1:1 using libthr > entrypoints. > You can try: http://people.freebsd.org/~davidxu/kern_thr.c http://people.freebsd.org/~davidxu/kern_umtx.c http://people.freebsd.org/~davidxu/libfptl.tgz I can run all applications current libpthread can run. Performance still needs to be tuned in kern_umtx.c, but it already beats current libpthread performance using supersmacks with 100 clients. :-) David Xu From owner-freebsd-threads@FreeBSD.ORG Tue Nov 16 18:11:12 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AA8ED16A4CE for ; Tue, 16 Nov 2004 18:11:12 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 886CD43D69 for ; Tue, 16 Nov 2004 18:11:11 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAGIB8rb047530 for ; Tue, 16 Nov 2004 20:11:09 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419A42BE.5090507@he.iki.fi> Date: Tue, 16 Nov 2004 20:11:10 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: thread scheduling X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Nov 2004 18:11:12 -0000 I have an single CPU (2.0GHz P4 Celeron) machine which is practically empty running an application which dispatches detached threads to do various kinds of work. It seems that recently process scope threads seem to have fairly random scheduling latency up to various seconds while system scope threads work as expected. This behaviour seems to have introduced between 5.2.1 and 5.3 but cannot point exactly when. I have printf's next to pthread_create and at the top of the new thread routine and the delay is usually in order of multiple seconds. The application also uses itimers if that makes a difference. Haven't built a simplified application demonstrating the issue yet, but will do that if the problem is not known. Pete From owner-freebsd-threads@FreeBSD.ORG Tue Nov 16 20:38:43 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBA2E16A4CE for ; Tue, 16 Nov 2004 20:38:43 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 72FE243D4C for ; Tue, 16 Nov 2004 20:38:43 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAGKccG5021224; Tue, 16 Nov 2004 15:38:39 -0500 (EST) Date: Tue, 16 Nov 2004 15:38:14 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Petri Helenius In-Reply-To: <419A42BE.5090507@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: thread scheduling X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Nov 2004 20:38:44 -0000 On Tue, 16 Nov 2004, Petri Helenius wrote: > > I have an single CPU (2.0GHz P4 Celeron) machine which is practically > empty running an application which dispatches detached threads to do > various kinds of work. It seems that recently process scope threads seem > to have fairly random scheduling latency up to various seconds while > system scope threads work as expected. This behaviour seems to have > introduced between 5.2.1 and 5.3 but cannot point exactly when. I have > printf's next to pthread_create and at the top of the new thread routine > and the delay is usually in order of multiple seconds. The application > also uses itimers if that makes a difference. > > Haven't built a simplified application demonstrating the issue yet, but > will do that if the problem is not known. I don't know about the problem and haven't really been keeping track of what's been merged to -stable. I would suggest checking out the -6 branch library and trying it on -stable (there should be nothing that needs to change for it to run on -stable). I also have some patches at: http://people.freebsd.org/~deischen/kse/libpthread.diffs -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Nov 17 19:06:43 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C10C16A4CE for ; Wed, 17 Nov 2004 19:06:43 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6193143D2D for ; Wed, 17 Nov 2004 19:06:42 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAHJ6et5054122 for ; Wed, 17 Nov 2004 21:06:40 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419BA142.9000801@he.iki.fi> Date: Wed, 17 Nov 2004 21:06:42 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2004 19:06:43 -0000 Do you feel that mutex performance could be improved from the current 2-3 million lock/unlock operations per second on uncontested mutexes on ~2.4Ghz prescott? Which seems to be about 1000 cycles per lock/unlock. I have a fairly basic producer/consumer application to optimize and I'm trying to decide on the performance-optimal synchronization method. Pete From owner-freebsd-threads@FreeBSD.ORG Wed Nov 17 19:18:10 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AA0A916A4CE for ; Wed, 17 Nov 2004 19:18:10 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57C4043D1D for ; Wed, 17 Nov 2004 19:18:10 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAHJI2IO000872; Wed, 17 Nov 2004 14:18:05 -0500 (EST) Date: Wed, 17 Nov 2004 14:18:02 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Petri Helenius In-Reply-To: <419BA142.9000801@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2004 19:18:10 -0000 On Wed, 17 Nov 2004, Petri Helenius wrote: > > Do you feel that mutex performance could be improved from the current > 2-3 million lock/unlock operations per second on uncontested mutexes on > ~2.4Ghz prescott? Which seems to be about 1000 cycles per lock/unlock. I'm not sure what you're trying to say above. > I have a fairly basic producer/consumer application to optimize and I'm > trying to decide on the performance-optimal synchronization method. Mutex performance is not optimal since I designed the low-level locking so that it would work on 80386 which doesn't have cmpxchg. The mutexes are also pointers instead of actual structures, so there's an extra indirection and checks for NULL. libthr mutexes should be faster in the uncontested case since they do use the atomic compare/set primitives. I want to change libpthread locks to drop the 80386 support and just use the atomic primitives for default mutex types. In 6.0, we'll also change all the mutexes, CVs, and semaphores so they aren't pointers -- that will save an indirection and also allow them to be process shared. -- DE From owner-freebsd-threads@FreeBSD.ORG Wed Nov 17 19:30:02 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C8A716A4CE; Wed, 17 Nov 2004 19:30:02 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C71543D3F; Wed, 17 Nov 2004 19:30:01 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAHJU0LY054202; Wed, 17 Nov 2004 21:30:00 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419BA6BA.4060304@he.iki.fi> Date: Wed, 17 Nov 2004 21:30:02 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2004 19:30:02 -0000 Daniel Eischen wrote: >On Wed, 17 Nov 2004, Petri Helenius wrote: > > > >>Do you feel that mutex performance could be improved from the current >>2-3 million lock/unlock operations per second on uncontested mutexes on >>~2.4Ghz prescott? Which seems to be about 1000 cycles per lock/unlock. >> >> > >I'm not sure what you're trying to say above. > > I have this: #include main() { register int i; pthread_mutexattr_t attr; pthread_mutex_t onepacket; pthread_mutexattr_init (&attr); pthread_mutexattr_settype (&attr,PTHREAD_MUTEX_ERRORCHECK); pthread_mutex_init (&onepacket,&attr); for (i = 0; i < 20000000;i++) { pthread_mutex_lock (&onepacket); pthread_mutex_unlock (&onepacket); } } And run it: > gcc -O3 -std=c99 -g -march=pentiumpro mutextest.c -o mutextest -lthr > time ./mutextest 12.291u 0.000s 0:12.29 100.0% 5+189k 0+0io 0pf+0w > gcc -O3 -std=c99 -g -march=pentiumpro mutextest.c -o mutextest -lpthread > time ./mutextest 7.008u 0.000s 0:07.01 99.8% 5+230k 0+0io 0pf+0w > gcc -O3 -std=c99 -g -march=pentiumpro mutextest.c -o mutextest -lc_r > time ./mutextest 5.887u 0.000s 0:05.88 100.0% 5+389k 0+0io 0pf+0w And I'm wondering how much I need to engineer around mutexes or holding them for longer and releasing just before yielding. I'm not saying it should/must be better, just trying to ask for the feel where to go. >I want to change libpthread locks to drop the 80386 support >and just use the atomic primitives for default mutex types. >In 6.0, we'll also change all the mutexes, CVs, and semaphores >so they aren't pointers -- that will save an indirection and >also allow them to be process shared. > > Do you have patches or is this on planning stage? Pete From owner-freebsd-threads@FreeBSD.ORG Wed Nov 17 19:40:21 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C8E7C16A4CE for ; Wed, 17 Nov 2004 19:40:21 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6506243D45 for ; Wed, 17 Nov 2004 19:40:21 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAHJeHZ1027182; Wed, 17 Nov 2004 14:40:18 -0500 (EST) Date: Wed, 17 Nov 2004 14:40:17 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Petri Helenius In-Reply-To: <419BA6BA.4060304@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2004 19:40:21 -0000 On Wed, 17 Nov 2004, Petri Helenius wrote: > Daniel Eischen wrote: > > >On Wed, 17 Nov 2004, Petri Helenius wrote: > > > >>Do you feel that mutex performance could be improved from the current > >>2-3 million lock/unlock operations per second on uncontested mutexes on > >>~2.4Ghz prescott? Which seems to be about 1000 cycles per lock/unlock. [ ... ] > And I'm wondering how much I need to engineer around mutexes or holding > them for longer and releasing just before yielding. I'm not saying it > should/must be better, just trying to ask for the feel where to go. It should get better, so... > >I want to change libpthread locks to drop the 80386 support > >and just use the atomic primitives for default mutex types. > >In 6.0, we'll also change all the mutexes, CVs, and semaphores > >so they aren't pointers -- that will save an indirection and > >also allow them to be process shared. > > > Do you have patches or is this on planning stage? Just planning. The change from pointer type to structure is a big change; it breaks the ABI and affects libc, libthr, libpthread, libc_r (if we keep it around in 6.0). Changing our current low-level locks to use the atomic functions is less intrusive and will be done first. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Nov 18 20:04:45 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6857416A4CE for ; Thu, 18 Nov 2004 20:04:45 +0000 (GMT) Received: from mail3.bluewin.ch (mail3.bluewin.ch [195.186.1.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 12C6F43D60 for ; Thu, 18 Nov 2004 20:04:45 +0000 (GMT) (envelope-from Federico.Besnard@bluewin.ch) Received: from [192.168.1.33] (83.76.79.163) by mail3.bluewin.ch (Bluewin AG 7.0.030.2) id 41862FF2001CCC57 for freebsd-threads@freebsd.org; Thu, 18 Nov 2004 20:04:40 +0000 Message-ID: <419D003A.4000601@bluewin.ch> Date: Thu, 18 Nov 2004 21:04:10 +0100 From: Federico Galvez-Durand Besnard User-Agent: Mozilla Thunderbird 0.9 (X11/20041108) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: firefox-1.0 crash on 5.3-stable, running flash plugin: possibleproblem in pthreads (Nov 8th) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 20:04:45 -0000 Hello, I found firefox-1.0 crashes with signal 11, Segmentation fault, when using flashplugin. It happens accessing this site: http://www.globo.com. The plugin seems to work well in other sites, but I did not test many. The same site can be accessed without the plugin (no flash effects, of course) and firefox doesn't crash. GDB backtrace points out libpthread functions before crashing. The problem is triggered by the flashplugin, but I am not sure whether it is a pthread bug or just a misuse of the library routines. Federico p.s. : I do not sign this list, please reply with CC to me whenever possible. Additional info: # file /usr/lib/libpthread.so.1 /usr/lib/libpthread.so.1: ELF 32-bit LSB shared object, Intel 80386, version 1 (FreeBSD), stripped # ls -l /usr/lib/libpthread.so.1 -r--r--r-- 1 root wheel 133328 Nov 8 00:34 /usr/lib/libpthread.so.1 # file /usr/X11R6/lib/firefox/plugins/npflash.so /usr/X11R6/lib/firefox/plugins/npflash.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (FreeBSD), stripped # ls -l /usr/X11R6/lib/firefox/plugins/npflash.so -r-xr-xr-x 1 root wheel 14248 Nov 18 12:38 /usr/X11R6/lib/firefox/plugins/npflash.so # uname -v FreeBSD 5.3-STABLE #0: Tue Nov 16 22:09:20 GMT-1 2004 [...] i386 # dmesg | grep -i cpu CPU: Intel Pentium III (646.83-MHz 686-class CPU) # pkg_info | grep flash flashplugin-firefox-0.4.12 A GPL standalone Flash (TM) plugin for FireFox web browser libflash-0.4.12 GPL Flash (TM) Library # pkg_info | grep firefox | grep -v plugin firefox-1.0,1 Web browser based on the browser portion of Mozilla backtrace from gdb: (no debugging symbols found)...Core was generated by `firefox-bin'. Program terminated with signal 11, Segmentation fault. ... (no debugging symbols found)... #0 0x2898df9f in pthread_testcancel () from /usr/lib/libpthread.so.1 #1 0x2897f0b9 in sigaction () from /usr/lib/libpthread.so.1 #2 0x289791e1 in pthread_kill () from /usr/lib/libpthread.so.1 #3 0x28978bb0 in raise () from /usr/lib/libpthread.so.1 #4 0x08055493 in nsProfileLock::FatalSignalHandler () #5 0x2897dc1a in sigaction () from /usr/lib/libpthread.so.1 #6 0x2897da9b in sigaction () from /usr/lib/libpthread.so.1 #7 0x2897e61d in sigaction () from /usr/lib/libpthread.so.1 #8 0x28986b57 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #9 0x28986a46 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #10 0x28a34c5f in _ctx_start () from /lib/libc.so.5 #11 0x00000000 in ?? () #12 0xbfbfd260 in ?? () #13 0xbfbfcfa0 in ?? () #14 0x00000000 in ?? () #15 0x289869e0 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #16 0x2990b882 in Text::execute (this=0x8941e80, gd=0x8a73400, matrix=0xbfbfd4b0, cxform=0x8a81fa8) at text.cc:104 #17 0x298faeb9 in DisplayList::render (this=0x8a66460, gd=0x8a73400, render_matrix=0xbfbfd5d0, cxform=0x0) at displaylist.cc:629 #18 0x2990b519 in Sprite::execute (this=0x17e, gd=0x8a73400, matrix=0xbfbfd5d0, cxform=0x0) at sprite.cc:67 #19 0x298faeb9 in DisplayList::render (this=0x891a1e0, gd=0x8a73400, render_matrix=0x0, cxform=0x0) at displaylist.cc:629 #20 0x29902304 in FlashMovie::renderMovie (this=0x88e9c00) at movie.cc:149 #21 0x29902062 in FlashMovie::processMovie (this=0x88e9c00, gd=0x8a73400, sm=0x88e0dc0) at movie.cc:75 #22 0x298fb353 in FlashExec (flashHandle=0x88e9c00, flag=1, fe=0xbfbfd730, wakeDate=0x88e9bac) at flash.cc:184 #23 0x298ed4ac in NPP_Print () from /usr/X11R6/lib/firefox/plugins/npflash.so #24 0x298ecd08 in NPP_Destroy () from /usr/X11R6/lib/firefox/plugins/npflash.so #25 0x28b355f5 in XtAppProcessEvent () from /usr/X11R6/lib/libXt.so.6 #26 0x28b0fc63 in ?? () from /usr/X11R6/lib/firefox/lib/firefox-1.0/libgtkxtbin.so #27 0x0899f700 in ?? () #28 0x0000000f in ?? () #29 0x419c8dfe in ?? () #30 0x288484ce in g_thread_self () from /usr/local/lib/libglib-2.0.so.400 #31 0x288315f0 in g_timeout_dispatch () from /usr/local/lib/libglib-2.0.so.400 #32 0x2882ef37 in g_main_dispatch () from /usr/local/lib/libglib-2.0.so.400 #33 0x2882fe03 in g_main_context_dispatch () from /usr/local/lib/libglib-2.0.so.400 #34 0x288301e2 in g_main_context_iterate () from /usr/local/lib/libglib-2.0.so.400 #35 0x2883084e in g_main_loop_run () from /usr/local/lib/libglib-2.0.so.400 #36 0x283c7896 in gtk_main () from /usr/X11R6/lib/libgtk-x11-2.0.so.400 #37 0x28b929f8 in nsAppShell::ReleaseGlobals () from /usr/X11R6/lib/firefox/lib/firefox-1.0/components/libwidget_gtk2.so #38 0x28ade872 in nsAppShellService::HandleExitEvent () from /usr/X11R6/lib/firefox/lib/firefox-1.0/components/libnsappshell.so #39 0x080503a8 in xre_main () #40 0x0804c275 in main () From owner-freebsd-threads@FreeBSD.ORG Thu Nov 18 23:27:53 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F9A416A4CE for ; Thu, 18 Nov 2004 23:27:53 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3728943D39 for ; Thu, 18 Nov 2004 23:27:52 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAINRncn076219 for ; Fri, 19 Nov 2004 01:27:50 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419D2FF8.7070602@he.iki.fi> Date: Fri, 19 Nov 2004 01:27:52 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: atomic_64 operations X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 23:27:53 -0000 I found a patch dating back to 2002 implementing atomic 64 bit arithmetic on i386. Is there a reason why that got never committed? (this is related to my quest reducing mutexes, having 64bit add would reduce the requirement nicely) Pete From owner-freebsd-threads@FreeBSD.ORG Thu Nov 18 23:39:51 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D57D816A4CE for ; Thu, 18 Nov 2004 23:39:51 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7174443D46 for ; Thu, 18 Nov 2004 23:39:51 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAINdmQM024966; Thu, 18 Nov 2004 18:39:48 -0500 (EST) Date: Thu, 18 Nov 2004 18:39:48 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Petri Helenius In-Reply-To: <419D2FF8.7070602@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: atomic_64 operations X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 23:39:51 -0000 On Fri, 19 Nov 2004, Petri Helenius wrote: > > I found a patch dating back to 2002 implementing atomic 64 bit > arithmetic on i386. Is there a reason why that got never committed? For ? I'd ask over on -current. > (this is related to my quest reducing mutexes, having 64bit add would > reduce the requirement nicely) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Nov 18 23:41:50 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE8C516A4CE for ; Thu, 18 Nov 2004 23:41:50 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id CACFB43D46 for ; Thu, 18 Nov 2004 23:41:50 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id AC8F67A403; Thu, 18 Nov 2004 15:41:50 -0800 (PST) Message-ID: <419D333E.7000906@elischer.org> Date: Thu, 18 Nov 2004 15:41:50 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Petri Helenius References: <419D2FF8.7070602@he.iki.fi> In-Reply-To: <419D2FF8.7070602@he.iki.fi> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: atomic_64 operations X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 23:41:51 -0000 I don't remember, but if it's here we probably need to have it on all architectures.. The 64 bit archs would be ok I guess.. don't know about ARM etc. Petri Helenius wrote: > > I found a patch dating back to 2002 implementing atomic 64 bit > arithmetic on i386. Is there a reason why that got never committed? > > (this is related to my quest reducing mutexes, having 64bit add would > reduce the requirement nicely) > > Pete > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to > "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 06:16:36 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ACB1B16A4CE; Fri, 19 Nov 2004 06:16:36 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 916C943D3F; Fri, 19 Nov 2004 06:16:35 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAJ6GXub079288; Fri, 19 Nov 2004 08:16:34 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419D8FC5.5040607@he.iki.fi> Date: Fri, 19 Nov 2004 08:16:37 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: atomic_64 operations X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 06:16:36 -0000 Daniel Eischen wrote: >On Fri, 19 Nov 2004, Petri Helenius wrote: > > > >>I found a patch dating back to 2002 implementing atomic 64 bit >>arithmetic on i386. Is there a reason why that got never committed? >> >> > >For ? I'd ask over on -current. > > Google found it at: http://www.csie.nctu.edu.tw/news/article/freebsd.csie.nctu.edu.tw/mailing.freebsd.audit/1929h Pete > > >>(this is related to my quest reducing mutexes, having 64bit add would >>reduce the requirement nicely) >> >> > > > From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 06:17:50 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B0D8116A4CE for ; Fri, 19 Nov 2004 06:17:50 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE53043D46 for ; Fri, 19 Nov 2004 06:17:49 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAJ6HhmS079310; Fri, 19 Nov 2004 08:17:44 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419D900B.7080703@he.iki.fi> Date: Fri, 19 Nov 2004 08:17:47 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Julian Elischer References: <419D2FF8.7070602@he.iki.fi> <419D333E.7000906@elischer.org> In-Reply-To: <419D333E.7000906@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: atomic_64 operations X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 06:17:50 -0000 Julian Elischer wrote: > I don't remember, but if it's here we probably need to have it on all > architectures.. > The 64 bit archs would be ok I guess.. don't know about ARM etc. > The atomic manpage states: The type ``64'' is currently not implemented for any of the atomic opera- tions on the i386 architecture. Which makes me assume all other architechtures should be ok except i386? Pete > > Petri Helenius wrote: > >> >> I found a patch dating back to 2002 implementing atomic 64 bit >> arithmetic on i386. Is there a reason why that got never committed? >> >> (this is related to my quest reducing mutexes, having 64bit add would >> reduce the requirement nicely) >> >> Pete >> >> _______________________________________________ >> freebsd-threads@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-threads >> To unsubscribe, send any mail to >> "freebsd-threads-unsubscribe@freebsd.org" > > > From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 07:55:00 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B75AE16A4CE; Fri, 19 Nov 2004 07:55:00 +0000 (GMT) Received: from smtp2.jp.viruscheck.net (smtp2.jp.viruscheck.net [154.33.69.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5F0ED43D2F; Fri, 19 Nov 2004 07:55:00 +0000 (GMT) (envelope-from bland@freebsd.org) Received: from scan3.jp.viruscheck.net ([154.33.69.38] helo=mail5.jp.viruscheck.net) by smtp2.jp.viruscheck.net with esmtp (Exim 3.36 #1) id 1CV3be-0004SI-00; Fri, 19 Nov 2004 16:54:58 +0900 Received: from [220.108.149.115] (helo=noc.orchid) by mail5.jp.viruscheck.net with esmtp (Exim 3.36 #2) id 1CV3be-0006Mj-00; Fri, 19 Nov 2004 16:54:58 +0900 Received: from [89.60.10.11] (horse.orchid [89.60.10.11]) by noc.orchid (8.12.11/8.12.11) with ESMTP id iAJ7svMp087862; Fri, 19 Nov 2004 16:54:57 +0900 (JST) (envelope-from bland@FreeBSD.org) Message-ID: <419DA6CC.7040002@FreeBSD.org> Date: Fri, 19 Nov 2004 16:54:52 +0900 From: Alexander Nedotsukov User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8a4) Gecko/20040927 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@FreeBSD.org Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit cc: Joe Marcus Clarke Subject: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 07:55:00 -0000 Hey guys, After squashing yet another "too small thread stack size" bug in software developed on Linux. I decided to ask gurus for the comment. Why we still insist that 64K is good enough for 32bit archs? I do understand fact that specs isn't clear about that number and therefore portable application must reserve it's own stack. But reality is sucks. Nobody cares about it for the one simple reason the majority of popular OSes provides at least megabyte of memory for the purpose by default. More other please read what for example Sun tells to developers about stack size allocation: http://docs.sun.com/app/docs/doc/816-5137/6mba5vpjc?a=view#attrib-33670 How much people will care after reading this? If there is no any technical issue which prevent us from bumping default thread stack size I propose to do this. If smaller default stack sizes gains us some significant benefits let's make it system wide or per process tunable (ie use getrlimit(RLIMIT_STACK)). All the best, Alexander. From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 08:00:01 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 51AAA16A4CE for ; Fri, 19 Nov 2004 08:00:01 +0000 (GMT) Received: from psych.st-andrews.ac.uk (psych.st-and.ac.uk [138.251.11.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1DB6D43D1D for ; Fri, 19 Nov 2004 08:00:00 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by psych.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id HAA17350 for ; Fri, 19 Nov 2004 07:59:58 GMT From: "Niall Douglas" To: freebsd-threads@freebsd.org Date: Fri, 19 Nov 2004 07:59:40 -0000 MIME-Version: 1.0 Message-ID: <419DA7EC.20473.1CFB6AE9@localhost> Priority: normal In-reply-to: <419BA142.9000801@he.iki.fi> X-PM-Encryptor: IDWPGP-PM32, 4 X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT Subject: Re: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 08:00:01 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 17 Nov 2004 at 21:06, Petri Helenius wrote: > Do you feel that mutex performance could be improved from the current > 2-3 million lock/unlock operations per second on uncontested mutexes > on ~2.4Ghz prescott? Which seems to be about 1000 cycles per > lock/unlock. > > I have a fairly basic producer/consumer application to optimize and > I'm trying to decide on the performance-optimal synchronization > method. In my library TnFOX (http://www.nedprod.com/TnFOX/) where I've completely rewritten mutexs for speed: FXAtomicInt FXMutex SMP Build, 1 thread : 51203277 18389113 SMP Build, 2 threads: 4793978 5337603 Non-SMP Build, 1 thread : 103305785 27352297 Non-SMP Build, 2 threads: 54929964 10978153 This is on a dual Athon 1700 (1.43Ghz), so that's 77.76 cycles per lock/unlock with SMP build and 52.28 cycles on non-SMP build. The difference between SMP and non-SMP is that the former uses the lock prefix on the x86 instructions. So yes, I think there is some scope for improvement. BTW 64 bit on ARM could be implemented using a spin lock to ensure exclusion during the two 32 bit operations. You just now need three 32 bit words per 64 bit quantity, which is why FXAtomicInt has just such an entity in TnFOX. Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBQZ2n7MEcvDLFGKbPEQKqbgCgltsn6ev7/pt4KIz8/Wm5S5jewEQAniWH 0JHu+0Z68UhS8dqKQpUeQ2aO =WfiD -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 08:07:20 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 633DE16A4CE for ; Fri, 19 Nov 2004 08:07:20 +0000 (GMT) Received: from silver.he.iki.fi (helenius.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5455D43D45 for ; Fri, 19 Nov 2004 08:07:19 +0000 (GMT) (envelope-from pete@he.iki.fi) Received: from [193.64.42.134] (h86.vuokselantie10.fi [193.64.42.134]) by silver.he.iki.fi (8.13.1/8.11.4) with ESMTP id iAJ87ISL079654; Fri, 19 Nov 2004 10:07:18 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <419DA9B9.50105@he.iki.fi> Date: Fri, 19 Nov 2004 10:07:21 +0200 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040803 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Niall Douglas References: <419DA7EC.20473.1CFB6AE9@localhost> In-Reply-To: <419DA7EC.20473.1CFB6AE9@localhost> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 08:07:20 -0000 Niall Douglas wrote: >In my library TnFOX (http://www.nedprod.com/TnFOX/) where I've >completely rewritten mutexs for speed: > > FXAtomicInt FXMutex > SMP Build, 1 thread : 51203277 18389113 > SMP Build, 2 threads: 4793978 5337603 >Non-SMP Build, 1 thread : 103305785 27352297 >Non-SMP Build, 2 threads: 54929964 10978153 > >This is on a dual Athon 1700 (1.43Ghz), so that's 77.76 cycles per >lock/unlock with SMP build and 52.28 cycles on non-SMP build. The >difference between SMP and non-SMP is that the former uses the lock >prefix on the x86 instructions. > >So yes, I think there is some scope for improvement. > > Are these mutexes spinlocks or "real" locks which make the thread actually yield if they have to wait longer? Or should I RTFS? :-) Pete From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 09:19:56 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 96F2216A4CE for ; Fri, 19 Nov 2004 09:19:56 +0000 (GMT) Received: from psych.st-andrews.ac.uk (psych.st-and.ac.uk [138.251.11.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id C497843D58 for ; Fri, 19 Nov 2004 09:19:55 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by psych.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id JAA19965 for ; Fri, 19 Nov 2004 09:19:54 GMT From: "Niall Douglas" To: Petri Helenius Date: Fri, 19 Nov 2004 09:19:36 -0000 MIME-Version: 1.0 Message-ID: <419DBAA8.29867.1D449929@localhost> Priority: normal In-reply-to: <419DA9B9.50105@he.iki.fi> References: <419DA7EC.20473.1CFB6AE9@localhost> X-PM-Encryptor: IDWPGP-PM32, 4 X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT cc: freebsd-threads@freebsd.org Subject: Re: mutex performance X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 09:19:56 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 19 Nov 2004 at 10:07, Petri Helenius wrote: > >This is on a dual Athon 1700 (1.43Ghz), so that's 77.76 cycles per > >lock/unlock with SMP build and 52.28 cycles on non-SMP build. The > >difference between SMP and non-SMP is that the former uses the lock > >prefix on the x86 instructions. > > > >So yes, I think there is some scope for improvement. > > > Are these mutexes spinlocks or "real" locks which make the thread > actually yield if they have to wait longer? Or should I RTFS? :-) You would have had to read the source actually. I wrote them as I was very unhappy with the performance of the locks supplied by Win2k which are far slower than the same code on WinNT 4.0. They're also far faster than most pthread lock implementations. Here's the schema: 1. Is lock free? If so take possession and exit 2. Is lock already held by this thread? If so, increment refcount and exit 3. If on multiprocessor, examine lock's freeness X times (default 4000). If still not free, ask kernel to suspend me. If on uniprocessor or non-SMP build, ask kernel to switch immediately to next thread. The key to the high performance is the use of "lock inc" - a lot of the time we don't care about what the value is, just that it is or isn't zero. By using inc and dec instead of xadd I gained quite a few percent if I remember plus it's a lot faster without the lock prefix. Of course, I have no idea what performance would be like on Intel chips. It's likely that the many branches would penalise code though I have used the pause x86 instruction where possible. Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBQZ26qMEcvDLFGKbPEQIpGQCg124mjqWElk4cP/LpNtRUgEMO7a8An0ye M/Ku1yACotLlZDyvpPdWI1cC =/p1K -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 16:01:56 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93EB216A4CE; Fri, 19 Nov 2004 16:01:56 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F86A43D5E; Fri, 19 Nov 2004 16:01:56 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAJG1ojp014247; Fri, 19 Nov 2004 11:01:50 -0500 (EST) Date: Fri, 19 Nov 2004 11:01:50 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Alexander Nedotsukov In-Reply-To: <419DA6CC.7040002@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: Joe Marcus Clarke cc: freebsd-threads@freebsd.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 16:01:56 -0000 On Fri, 19 Nov 2004, Alexander Nedotsukov wrote: > Hey guys, > > After squashing yet another "too small thread stack size" bug in > software developed on Linux. I decided to ask gurus for the comment. Why > we still insist that 64K is good enough for 32bit archs? I do understand I suggested we double the stack size for 64-bit archs (making it 128K). I could see going to 256K for 32-bit and 512K for 64-bit. We haven't worried too much about stack size since everything has been running with libc_r for years without too many problems and it's been using the same stack size. -- DE From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 17:52:39 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 63DBA16A4CE; Fri, 19 Nov 2004 17:52:39 +0000 (GMT) Received: from rooster.cisco.com (hen.cisco.com [64.102.19.198]) by mx1.FreeBSD.org (Postfix) with ESMTP id B217C43D31; Fri, 19 Nov 2004 17:52:38 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from [171.71.248.255] (dhcp-171-71-248-255.cisco.com [171.71.248.255]) by rooster.cisco.com (8.11.7+Sun/8.8.8) with ESMTP id iAJHqbe28168; Fri, 19 Nov 2004 12:52:37 -0500 (EST) Message-ID: <419E32EC.8070400@FreeBSD.org> Date: Fri, 19 Nov 2004 12:52:44 -0500 From: Joe Marcus Clarke Organization: FreeBSD, Inc. User-Agent: Mozilla Thunderbird 0.9 (Macintosh/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: X-Enigmail-Version: 0.86.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: Alexander Nedotsukov cc: freebsd-threads@FreeBSD.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 17:52:39 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel Eischen wrote: | On Fri, 19 Nov 2004, Alexander Nedotsukov wrote: | | |>Hey guys, |> |>After squashing yet another "too small thread stack size" bug in |>software developed on Linux. I decided to ask gurus for the comment. Why |>we still insist that 64K is good enough for 32bit archs? I do understand | | | I suggested we double the stack size for 64-bit archs (making it | 128K). I could see going to 256K for 32-bit and 512K for 64-bit. ia64 already uses a 256 KB default stack size. However, I argue that is is "too small." Linux has a much higher default (inline with the document bland referenced), and thus, most popular multithreaded applications are developed with that in mind. It has become some problematic for GNOME, for example, that I have hacked glib20 to allocate a default 1 MB stack on all architectures (this is, of course, configurable). | | We haven't worried too much about stack size since everything | has been running with libc_r for years without too many problems | and it's been using the same stack size. I think we should revisit this decision, and do something similar to what is outlined in the Sun document. Going to at least 1 MB on all architectures would ensure more popular software runs out-of-the-box on FreeBSD. Joe | - -- Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBnjLsb2iPiv4Uz4cRAjeiAKCpAIEvg4KPdLMy1DGYQw78fM58lgCcDBt6 8sDV+TOf6dyNs6pVVSnY96Q= =sKab -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 18:09:35 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C8C316A4CE; Fri, 19 Nov 2004 18:09:35 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id A4B1843D41; Fri, 19 Nov 2004 18:09:34 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAJI9V6f006854; Fri, 19 Nov 2004 13:09:31 -0500 (EST) Date: Fri, 19 Nov 2004 13:09:31 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Joe Marcus Clarke In-Reply-To: <419E32EC.8070400@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: Alexander Nedotsukov cc: freebsd-threads@freebsd.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 18:09:35 -0000 On Fri, 19 Nov 2004, Joe Marcus Clarke wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Daniel Eischen wrote: > | On Fri, 19 Nov 2004, Alexander Nedotsukov wrote: > | > | > |>Hey guys, > |> > |>After squashing yet another "too small thread stack size" bug in > |>software developed on Linux. I decided to ask gurus for the comment. Why > |>we still insist that 64K is good enough for 32bit archs? I do understand > | > | > | I suggested we double the stack size for 64-bit archs (making it > | 128K). I could see going to 256K for 32-bit and 512K for 64-bit. > > ia64 already uses a 256 KB default stack size. However, I argue that is > is "too small." Linux has a much higher default (inline with the > document bland referenced), and thus, most popular multithreaded > applications are developed with that in mind. > > It has become some problematic for GNOME, for example, that I have > hacked glib20 to allocate a default 1 MB stack on all architectures > (this is, of course, configurable). The thing I worry about is these piggy applications being the driving force behind our stack size. If they really are designed to need a huge stack size, they should be the ones that change to support it, not the other way around. Do they know their own stack space requirements or do they just ignore it because it isn't a problem so far (on Linux)? What if they need more than 1MB in a few months (Bill Gates -- who's ever going to need more than 64K ;-)? Are we going to change again? I can see raising the default stack size, but 1MB (32-bit) and 2MB (64-bit) seem kinda large. -- DE From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 18:17:02 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AEDAA16A4CE; Fri, 19 Nov 2004 18:17:02 +0000 (GMT) Received: from creme-brulee.marcuscom.com (creme-brulee.marcuscom.com [24.172.16.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id 166A343D3F; Fri, 19 Nov 2004 18:17:02 +0000 (GMT) (envelope-from marcus@marcuscom.com) Received: from [10.2.1.56] (vpn-client-56.marcuscom.com [10.2.1.56]) iAJIH04E007318; Fri, 19 Nov 2004 13:17:00 -0500 (EST) (envelope-from marcus@marcuscom.com) Message-ID: <419E38A0.2030308@marcuscom.com> Date: Fri, 19 Nov 2004 13:17:04 -0500 From: Joe Marcus Clarke Organization: MarcusCom, Inc. User-Agent: Mozilla Thunderbird 0.9 (Macintosh/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: X-Enigmail-Version: 0.86.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-5.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.1 X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on creme-brulee.marcuscom.com cc: Joe Marcus Clarke cc: Alexander Nedotsukov cc: freebsd-threads@freebsd.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 18:17:02 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel Eischen wrote: | On Fri, 19 Nov 2004, Joe Marcus Clarke wrote: | | |>-----BEGIN PGP SIGNED MESSAGE----- |>Hash: SHA1 |> |>Daniel Eischen wrote: |>| On Fri, 19 Nov 2004, Alexander Nedotsukov wrote: |>| |>| |>|>Hey guys, |>|> |>|>After squashing yet another "too small thread stack size" bug in |>|>software developed on Linux. I decided to ask gurus for the comment. Why |>|>we still insist that 64K is good enough for 32bit archs? I do understand |>| |>| |>| I suggested we double the stack size for 64-bit archs (making it |>| 128K). I could see going to 256K for 32-bit and 512K for 64-bit. |> |>ia64 already uses a 256 KB default stack size. However, I argue that is |>is "too small." Linux has a much higher default (inline with the |>document bland referenced), and thus, most popular multithreaded |>applications are developed with that in mind. |> |>It has become some problematic for GNOME, for example, that I have |>hacked glib20 to allocate a default 1 MB stack on all architectures |>(this is, of course, configurable). | | | The thing I worry about is these piggy applications being the | driving force behind our stack size. If they really are designed | to need a huge stack size, they should be the ones that change | to support it, not the other way around. Do they know their own | stack space requirements or do they just ignore it because it | isn't a problem so far (on Linux)? The bottom line is that they don't know their stack requirements, but the OS is accommodating, so they never have to really find out until we submit a bug to them. However, some applications (e.g. gstreamer, libgnomecups, etc.) cannot be fixed without massive architectural changes. Just to be clear, these applications _are_ overrunning the default stack. ~ What if they need more than | 1MB in a few months (Bill Gates -- who's ever going to need | more than 64K ;-)? Are we going to change again? It was 640 K, but I get the point. Honestly, I think we _do_ have to change with the industry. We can't be the Bill Gates, and say a 64 K default stack size should be good enough. | | I can see raising the default stack size, but 1MB (32-bit) and | 2MB (64-bit) seem kinda large. Why? Remember, this is just a _default_. Developers that know their stack requirements can lower this down to 16 K (IIRC). However, the developers that are programming these pigs (popular pigs, mind you), will find that FreeBSD behaves just like Solaris and Linux. That might entice them to continue their efforts on FreeBSD. Speaking for myself, debugging stack overruns aren't the easiest thing, and I would imagine random crashes in one's application might discourage people from developing on FreeBSD. Joe | - -- PGP Key : http://www.marcuscom.com/pgp.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBnjigb2iPiv4Uz4cRAsubAKCX4vu3GGJoKOhV6gTn5yKDoruEQQCeJ2uG 6aMobmkKosPUg6O+vgrG8Ow= =mp6z -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 19:53:11 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5B6BE16A4CE; Fri, 19 Nov 2004 19:53:11 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id EFD5A43D49; Fri, 19 Nov 2004 19:53:10 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) iAJJr9qu004615; Fri, 19 Nov 2004 14:53:09 -0500 (EST) Date: Fri, 19 Nov 2004 14:53:07 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Joe Marcus Clarke In-Reply-To: <419E38A0.2030308@marcuscom.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: Joe Marcus Clarke cc: Alexander Nedotsukov cc: freebsd-threads@freebsd.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 19:53:11 -0000 On Fri, 19 Nov 2004, Joe Marcus Clarke wrote: > Daniel Eischen wrote: > | The thing I worry about is these piggy applications being the > | driving force behind our stack size. If they really are designed > | to need a huge stack size, they should be the ones that change > | to support it, not the other way around. Do they know their own > | stack space requirements or do they just ignore it because it > | isn't a problem so far (on Linux)? > > The bottom line is that they don't know their stack requirements, but > the OS is accommodating, so they never have to really find out until we > submit a bug to them. However, some applications (e.g. gstreamer, > libgnomecups, etc.) cannot be fixed without massive architectural > changes. Just to be clear, these applications _are_ overrunning the > default stack. FYI, I don't suggest they change their stack usage, just that they create threads with thread attributes specifying a larger stack size. If they recognize they have large stack space requirements, it should be easily solved without an architectural overhaul. I assume your patches do exactly this; are the GNOME developers reluctant to incorporate your patches? I'm not going to argue very strongly against changing the default stack size. I just think the onus should be on the larger applications to recognize that they need larger stacks and to explicitly set it. There may also be other applications that create a lot of threads which may need to lower the stack size if the default were changed. -- DE From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 21:17:37 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 483C516A4CE; Fri, 19 Nov 2004 21:17:37 +0000 (GMT) Received: from lakermmtao08.cox.net (lakermmtao08.cox.net [68.230.240.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6CEFF43D48; Fri, 19 Nov 2004 21:17:36 +0000 (GMT) (envelope-from mezz7@cox.net) Received: from smtp.east.cox.net ([172.18.52.54]) by lakermmtao08.cox.net (InterMail vM.6.01.04.00 201-2131-117-20041022) with SMTP <20041119211734.LESF29395.lakermmtao08.cox.net@smtp.east.cox.net>; Fri, 19 Nov 2004 16:17:34 -0500 X-Mailer: Openwave WebEngine, version 2.8.15 (webedge20-101-1103-20040528) From: To: "Daniel Eischen" Date: Fri, 19 Nov 2004 16:17:34 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Message-Id: <20041119211734.LESF29395.lakermmtao08.cox.net@smtp.east.cox.net> cc: Joe Marcus Clarke cc: Alexander Nedotsukov cc: freebsd-threads@freebsd.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 21:17:37 -0000 Had to reply in webmail without followup, because smtp.central.cox.net is down for two days that will not allow me send anything. Anyway...(copy-n-paste) On Fri, 19 Nov 2004 11:01:50 -0500 (EST), Daniel Eischen wrote: > On Fri, 19 Nov 2004, Alexander Nedotsukov wrote: > >> Hey guys, >> >> After squashing yet another "too small thread stack size" bug in >> software developed on Linux. I decided to ask gurus for the comment. Why >> we still insist that 64K is good enough for 32bit archs? I do understand > > I suggested we double the stack size for 64-bit archs > (making it 128K). I could see going to 256K for 32-bit > and 512K for 64-bit. Why still keep it low? > We haven't worried too much about stack size since > everything has been running with libc_r for years without > too many problems and it's been using the same stack size. I don't think this 'without too many problems' give a good answer and reason, which this is rather blind to all of us. Maybe, it's me that don't understand the point or you still haven't answer to a real reason so far from what I have read the follows up to your lastest email (Fri, 19 Nov 2004 14:53:07 -0500 (EST)). My view is that if we higher the stack size and FreeBSD will have lesser problem by not have 'too small thread stack size', not have to hack in apps and etc. Unless anyone know what real reason (no theory, pls) why FreeBSD keeps the stack size smaller other than Bill Gates's 640 K quote excuse? Cheers, Mezz From owner-freebsd-threads@FreeBSD.ORG Fri Nov 19 22:01:11 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E00516A4CE; Fri, 19 Nov 2004 22:01:11 +0000 (GMT) Received: from rooster.cisco.com (hen.cisco.com [64.102.19.198]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C52E43D2D; Fri, 19 Nov 2004 22:01:10 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from [171.69.89.46] (dhcp-171-69-89-46.cisco.com [171.69.89.46]) by rooster.cisco.com (8.11.7+Sun/8.8.8) with ESMTP id iAJM19e03334; Fri, 19 Nov 2004 17:01:09 -0500 (EST) Message-ID: <419E6D2D.5040402@FreeBSD.org> Date: Fri, 19 Nov 2004 17:01:17 -0500 From: Joe Marcus Clarke Organization: FreeBSD, Inc. User-Agent: Mozilla Thunderbird 0.9 (Macintosh/20041103) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: X-Enigmail-Version: 0.86.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: Alexander Nedotsukov cc: freebsd-threads@FreeBSD.org Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2004 22:01:11 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel Eischen wrote: | On Fri, 19 Nov 2004, Joe Marcus Clarke wrote: | | |>Daniel Eischen wrote: |>| The thing I worry about is these piggy applications being the |>| driving force behind our stack size. If they really are designed |>| to need a huge stack size, they should be the ones that change |>| to support it, not the other way around. Do they know their own |>| stack space requirements or do they just ignore it because it |>| isn't a problem so far (on Linux)? |> |>The bottom line is that they don't know their stack requirements, but |>the OS is accommodating, so they never have to really find out until we |>submit a bug to them. However, some applications (e.g. gstreamer, |>libgnomecups, etc.) cannot be fixed without massive architectural |>changes. Just to be clear, these applications _are_ overrunning the |>default stack. | | | FYI, I don't suggest they change their stack usage, just that | they create threads with thread attributes specifying a larger | stack size. If they recognize they have large stack space | requirements, it should be easily solved without an architectural | overhaul. The problem is they don't realize they have such requirements until the FreeBSD camp complains about weird crashes. The architectural change to which I referred deals with GThreadPools. There is no way to set a per-thread stack size with GThreadPools. Therefore, I had to patch the source (glib20) such that all threads are allocated with a default stack size of 1 MB unless a function is called that explicitly sets a different size. ~ I assume your patches do exactly this; are the GNOME | developers reluctant to incorporate your patches? Yes. I have brought these patches up to them, but they prefer not to touch this code since FreeBSD seems to be the only platform affected. | | I'm not going to argue very strongly against changing the | default stack size. I just think the onus should be on | the larger applications to recognize that they need larger | stacks and to explicitly set it. There may also be other | applications that create a lot of threads which may need | to lower the stack size if the default were changed. That may be true, but I don't know of any such applications. This is something we can do in HEAD, and let it sit for a while to see if there are any hidden problems. If there are, we can always revert this change. However, I would imagine that those applications would have already had problems on Solaris or Linux if that were the case. Joe | - -- Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBnm0sb2iPiv4Uz4cRAk3gAKCMFLftovlRk3f/f4XDdONlRJTg9QCglEGP PhH7tz1ZPXT8rRKbOdgKC00= =Inkz -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Sat Nov 20 10:10:54 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 88EEE16A4CE for ; Sat, 20 Nov 2004 10:10:54 +0000 (GMT) Received: from psych.st-andrews.ac.uk (psych.st-and.ac.uk [138.251.11.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2C3FA43D1F for ; Sat, 20 Nov 2004 10:10:53 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by psych.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id KAA18545 for ; Sat, 20 Nov 2004 10:10:51 GMT From: "Niall Douglas" To: freebsd-threads@freebsd.org Date: Sat, 20 Nov 2004 10:10:25 -0000 MIME-Version: 1.0 Message-ID: <419F1811.27008.22997A86@localhost> Priority: normal In-reply-to: <20041119211734.LESF29395.lakermmtao08.cox.net@smtp.east.cox.net> X-PM-Encryptor: IDWPGP-PM32, 4 X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT Subject: Re: Question about our default pthread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Nov 2004 10:10:54 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I assume that FreeBSD merely reserves the virtual address space for thread stacks, committing memory as needed? Otherwise 1Mb per thread seems awfully expensive. Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBQZ8YEcEcvDLFGKbPEQIalQCgpoRGlSxFTd+L8L3EC1kiTDL//qUAn3lc ONKq6ZcwtygIhluHb4Lq1rbr =ZmB2 -----END PGP SIGNATURE----- From owner-freebsd-threads@FreeBSD.ORG Sat Nov 20 16:37:33 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D082916A4CE for ; Sat, 20 Nov 2004 16:37:33 +0000 (GMT) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD82843D1F for ; Sat, 20 Nov 2004 16:37:33 +0000 (GMT) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) iAKGbXxw072496 for ; Sat, 20 Nov 2004 08:37:33 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost)iAKGbXxK072495 for freebsd-threads@freebsd.org; Sat, 20 Nov 2004 08:37:33 -0800 (PST) (envelope-from sgk) Date: Sat, 20 Nov 2004 08:37:33 -0800 From: Steve Kargl To: freebsd-threads@freebsd.org Message-ID: <20041120163733.GA72479@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: firefox stuck in kserel X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Nov 2004 16:37:33 -0000 Fired up firefox and went to espn.com. Firefox becomes unresponse. gdb -p shows (gdb) where #0 0x48a7774b in pthread_testcancel () from /usr/lib/libpthread.so.1 #1 0x48a6f97c in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #2 0x48a73c74 in pthread_setconcurrency () from /usr/lib/libpthread.so.1 #3 0x48a78371 in pthread_exit () from /usr/lib/libpthread.so.1 #4 0x48a63013 in pthread_create () from /usr/lib/libpthread.so.1 #5 0x48b2a06f in _ctx_start () from /lib/libc.so.6 (gdb) list while top(1) shows 44682 kargl 20 0 56716K 49236K kserel 0:25 0.00% 0.00% firefox-bin -- Steve