From owner-freebsd-threads@FreeBSD.ORG Mon Feb 17 11:06:57 2014 Return-Path: Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F085FE0A for ; Mon, 17 Feb 2014 11:06:57 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C27FF11D8 for ; Mon, 17 Feb 2014 11:06:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1HB6vpD033245 for ; Mon, 17 Feb 2014 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1HB6vAo033243 for freebsd-threads@FreeBSD.org; Mon, 17 Feb 2014 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Feb 2014 11:06:57 GMT Message-Id: <201402171106.s1HB6vAo033243@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 11:06:58 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/186309 threads [patch] pthread_kill() broken code - signal to current o threa/184073 threads wrong signal delivery to multithreaded processes in Pe o threa/180652 threads [patch] compat32 problem in clock_getcpuclockid2 o threa/180496 threads clock_gettime() does not return CPU-time for zombie pr o threa/168417 threads pthread_getcpuclockid() does not work to specification o threa/163512 threads libc defaults to single threaded o threa/160708 threads possible security problem with RLIMIT_VMEM o threa/150959 threads [libc] Stub pthread_once in libc should call _libc_onc o threa/148515 threads Memory / syslog strangeness in FreeBSD 8.x ( possible o threa/141721 threads rtprio(1): (id|rt)prio priority resets when new thread o threa/135673 threads databases/mysql50-server - MySQL query lock-ups on 7.2 o threa/128922 threads threads hang with xorg running o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/30464 threads [patch] pthread mutex attributes -- pshared 21 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Feb 17 15:30:53 2014 Return-Path: Delivered-To: freebsd-threads@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC744C95; Mon, 17 Feb 2014 15:30:53 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 911A51DD4; Mon, 17 Feb 2014 15:30:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1HFUrDw019468; Mon, 17 Feb 2014 15:30:53 GMT (envelope-from eadler@freefall.freebsd.org) Received: (from eadler@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1HFUqD2019467; Mon, 17 Feb 2014 15:30:52 GMT (envelope-from eadler) Date: Mon, 17 Feb 2014 15:30:52 GMT Message-Id: <201402171530.s1HFUqD2019467@freefall.freebsd.org> To: jlw@xinuos.com, eadler@FreeBSD.org, freebsd-threads@FreeBSD.org From: eadler@FreeBSD.org Subject: Re: threads/186309: [patch] pthread_kill() broken code - signal to current thread sent twice X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 15:30:53 -0000 Synopsis: [patch] pthread_kill() broken code - signal to current thread sent twice State-Changed-From-To: open->closed State-Changed-By: eadler State-Changed-When: Mon Feb 17 15:30:52 UTC 2014 State-Changed-Why: Committed. Thanks! http://www.freebsd.org/cgi/query-pr.cgi?pr=186309 From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 14:47:42 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72126E56; Wed, 19 Feb 2014 14:47:42 +0000 (UTC) Received: from mail.myota.org (mail.myota.org [85.10.206.105]) by mx1.freebsd.org (Postfix) with ESMTP id D88621D31; Wed, 19 Feb 2014 14:47:41 +0000 (UTC) Received: from mobile.client (177.189.166.190.f.sta.codetel.net.do [190.166.189.177] (may be forged)) (authenticated bits=128) by mail.myota.org (8.14.7/8.14.7) with ESMTP id s1JElZoj064223; Wed, 19 Feb 2014 15:47:37 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Received: from submit.client ([127.0.0.1]) by schlappy.local (8.14.7/8.14.7) with ESMTP id s1JElS4B003075; Wed, 19 Feb 2014 15:47:29 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Received: (from mu@schlappy.local) by schlappy.local (8.14.7/8.14.7/Submit) id s1JElS5L003074; Wed, 19 Feb 2014 15:47:28 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Date: Wed, 19 Feb 2014 15:47:28 +0100 From: Andre Albsmeier To: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) Message-ID: <20140219144728.GA3036@schlappy> References: <20140218180646.GA67861@schlappy> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140218180646.GA67861@schlappy> X-Echelon: 757, CIO, secure, Tower, SWAT X-Advice: Drop that crappy M$-Outlook, I'm tired of your viruses! User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Not delayed on 85.10.206.105, ACL: AUTH(59), Origin: DO, OS: FreeBSD 9.x X-Virus-Scanned: clamav-milter 0.98.1 at colo X-Virus-Status: Clean X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 14:47:42 -0000 [Commenting my own mail from below and CC'ing freebsd-threads.] I have tested my code below on a Linux box (3.2.0) and here the behaviour is different and seems correct: While watching with top(1), VIRT climbs up to a few GB and collapses then to a few MB (somehow reminding me of some kind of garbage collection). Important thing is that RES always stays below 1MB. On FreeBSD SIZE and RES are about 100MB apart but both are increasing util 2GB and pth1 dies with Cannot map anonymous memory Out of memory So the question is: Is my programme buggy and Linux works around this bug or is there some kind of memory leak in the pthreads code in FreeBSD? -Andre ----- Forwarded message ----- Well, as these are my first steps regarding thread programming, it's probably my fault... Why does this programme slowly grow and grow until it hits resource limits? ----- snip pth1.c ----- #include #include #include void* mythread( void* arg ) { return NULL; } int main( int argc, const char* const argv[] ) { pthread_t pthr; int i; while( 1 ) { for( i=1000; i; i-- ) if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 ) fprintf( stderr, "pthread_create\n" ); else pthread_detach( pthr ); putchar( '.' ); fflush( stdout ); usleep( 25000 ); } } ----- snap ----- Just to be sure I have also created the non-detaching version which behaves in the same way: ----- snip pth2.c ----- #include #include #include #define M 1000 pthread_t pthr[M]; void* mythread( void* arg ) { return NULL; } int main( int argc, const char* const argv[] ) { int i; while( 1 ) { for( i=M; i; i-- ) if( pthread_create( &pthr[i], NULL, mythread, NULL ) != 0 ) fprintf( stderr, "pthread_create\n" ); for( i=M; i; i-- ) if( pthread_join( pthr[i], NULL ) != 0 ) fprintf( stderr, "pthread_join\n" ); putchar( '.' ); fflush( stdout ); usleep( 25000 ); } } ----- snap ----- Compile them using -pthread and watch their ps output in another window (FreeBSD-9.2 but that shouldn't matter). So what am I doing wrong here? Thanks, -Andre From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 15:08:34 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B958D6B; Wed, 19 Feb 2014 15:08:34 +0000 (UTC) Received: from alogt.com (alogt.com [69.36.191.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 31DAA108C; Wed, 19 Feb 2014 15:08:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=r2BurpOF7RHq+4+fPAPU4MSaKWX+ZIxYpP0sHV0sbW8=; b=vjeSb0vc/viWwRfyfO0J9+rV4y5V/gtNz9nLtinU8ddCL8efsPQnwvpXqRuW8yLBruXYovQh+PHqQ3/GB551Ckdqclkdpp1msteVDkauMiVznjL68QwaXUqeY/l98m26srxGwG8Y2giKzdXgrSYuiZCrdvwOYV2MoBDIoV4BdBI=; Received: from [39.212.216.37] (port=15745 helo=X220.alogt.com) by sl-508-2.slc.westdc.net with esmtpsa (SSLv3:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WG8lC-002Ag6-N2; Wed, 19 Feb 2014 08:08:32 -0700 Date: Wed, 19 Feb 2014 23:08:24 +0800 From: Erich Dollansky To: Andre Albsmeier Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) Message-ID: <20140219230824.0f2ba24b@X220.alogt.com> In-Reply-To: <20140219144728.GA3036@schlappy> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erichsfreebsdlist@alogt.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 15:08:34 -0000 Hi, as I understand your program, it creates 1000 threads, waits a but and then starts again creating 1000 threads until something kill it.=20 =46rom my point of view, your program depends very much on the default settings of the environment. If the environment allows the immediate execution of the new thread, you will never get many threads. Otherwise, the number of threads hanging around could add up. It also depends on the number of CPUs/cores your system has. But your are right, it should not crash on a modern machine but it still could use some amount of memory. Erich On Wed, 19 Feb 2014 15:47:28 +0100 Andre Albsmeier wrote: > [Commenting my own mail from below and CC'ing freebsd-threads.] >=20 > I have tested my code below on a Linux box (3.2.0) and here the > behaviour is different and seems correct: While watching with > top(1), VIRT climbs up to a few GB and collapses then to a few > MB (somehow reminding me of some kind of garbage collection). > Important thing is that RES always stays below 1MB. >=20 > On FreeBSD SIZE and RES are about 100MB apart but both are > increasing util 2GB and pth1 dies with=20 >=20 > Cannot map anonymous memory > Out of memory >=20 > So the question is: Is my programme buggy and Linux works around > this bug or is there some kind of memory leak in the pthreads > code in FreeBSD? >=20 > -Andre >=20 > ----- Forwarded message ----- >=20 > Well, as these are my first steps regarding thread programming, > it's probably my fault... >=20 > Why does this programme slowly grow and grow until it hits > resource limits? >=20 > ----- snip pth1.c ----- >=20 > #include > #include > #include >=20 > void* mythread( void* arg ) > { > return NULL; > } >=20 > int main( int argc, const char* const argv[] ) > { > pthread_t pthr; > int i; >=20 > while( 1 ) { >=20 > for( i=3D1000; i; i-- ) > if( pthread_create( &pthr, NULL, mythread, NULL ) !=3D 0 ) > fprintf( stderr, "pthread_create\n" ); > else > pthread_detach( pthr ); >=20 > putchar( '.' ); > fflush( stdout ); > usleep( 25000 ); > } > } >=20 > ----- snap ----- >=20 > Just to be sure I have also created the non-detaching version > which behaves in the same way: >=20 > ----- snip pth2.c ----- >=20 > #include > #include > #include >=20 > #define M 1000 >=20 > pthread_t pthr[M]; >=20 > void* mythread( void* arg ) > { > return NULL; > } >=20 > int main( int argc, const char* const argv[] ) > { > int i; >=20 > while( 1 ) { >=20 > for( i=3DM; i; i-- ) > if( pthread_create( &pthr[i], NULL, mythread, NULL ) !=3D 0 ) > fprintf( stderr, "pthread_create\n" ); >=20 > for( i=3DM; i; i-- ) > if( pthread_join( pthr[i], NULL ) !=3D 0 ) > fprintf( stderr, "pthread_join\n" ); >=20 > putchar( '.' ); > fflush( stdout ); > usleep( 25000 ); > } > } >=20 > ----- snap ----- >=20 > Compile them using -pthread and watch their ps output in another > window (FreeBSD-9.2 but that shouldn't matter). >=20 > So what am I doing wrong here? >=20 > Thanks, >=20 > -Andre > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to > "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 15:29:45 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65FF4668; Wed, 19 Feb 2014 15:29:45 +0000 (UTC) Received: from mail.myota.org (mail.myota.org [85.10.206.105]) by mx1.freebsd.org (Postfix) with ESMTP id E1CEE127E; Wed, 19 Feb 2014 15:29:43 +0000 (UTC) Received: from mobile.client (177.189.166.190.f.sta.codetel.net.do [190.166.189.177] (may be forged)) (authenticated bits=128) by mail.myota.org (8.14.7/8.14.7) with ESMTP id s1JFTcoh064412; Wed, 19 Feb 2014 16:29:40 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Received: from submit.client ([127.0.0.1]) by schlappy.local (8.14.7/8.14.7) with ESMTP id s1JFTWmn003381; Wed, 19 Feb 2014 16:29:32 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Received: (from mu@schlappy.local) by schlappy.local (8.14.7/8.14.7/Submit) id s1JFTWq9003380; Wed, 19 Feb 2014 16:29:32 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Date: Wed, 19 Feb 2014 16:29:32 +0100 From: Andre Albsmeier To: Erich Dollansky Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) Message-ID: <20140219152932.GA3324@schlappy> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140219230824.0f2ba24b@X220.alogt.com> X-Echelon: Colonel, Surveillance, secret, Satellite, Uzi X-Advice: Drop that crappy M$-Outlook, I'm tired of your viruses! User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Not delayed on 85.10.206.105, ACL: AUTH(59), Origin: DO, OS: FreeBSD 9.x X-Virus-Scanned: clamav-milter 0.98.1 at colo X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 15:29:45 -0000 On Wed, 19-Feb-2014 at 23:08:24 +0800, Erich Dollansky wrote: > Hi, > > as I understand your program, it creates 1000 threads, waits a but and > then starts again creating 1000 threads until something kill it. Right. 1000 threads are started, do nothing and thus terminate immediately. The value of 25 ms for usleep is just a good one for my machine to demonstrate the problem quickly while being sure that all 1000 threads have gone (verified with ps -H). You can chose a bigger value here and it will just take more time... > > From my point of view, your program depends very much on the default > settings of the environment. If the environment allows the immediate > execution of the new thread, you will never get many threads. > Otherwise, the number of threads hanging around could add up. It also > depends on the number of CPUs/cores your system has. But I never have more than 1000 threads. This is why I made pth2 where I explicitly wait for all 1000 threads to terminate before firing up the next bunch. So for me it seems as if every (already died away) thread leaves some memory behind which does not get free'ed or whatever. But it was my understanding that pthread_detach() or pthread_join() would clean all this up after a thread has finished. > > But your are right, it should not crash on a modern machine but it > still could use some amount of memory. But should it still grow with every bunch of 1000 threads being started while the previous 1000 are all gone already? I'd agree if the behaviour was like on the Linux machine where it seems that some kind of garbage collection kicks in for VIRT after a while but RES never goes above 1MB there. On FreeBSD it even makes the box swap... -Andre > > Erich > > On Wed, > 19 Feb 2014 15:47:28 +0100 Andre Albsmeier > wrote: > > > [Commenting my own mail from below and CC'ing freebsd-threads.] > > > > I have tested my code below on a Linux box (3.2.0) and here the > > behaviour is different and seems correct: While watching with > > top(1), VIRT climbs up to a few GB and collapses then to a few > > MB (somehow reminding me of some kind of garbage collection). > > Important thing is that RES always stays below 1MB. > > > > On FreeBSD SIZE and RES are about 100MB apart but both are > > increasing util 2GB and pth1 dies with > > > > Cannot map anonymous memory > > Out of memory > > > > So the question is: Is my programme buggy and Linux works around > > this bug or is there some kind of memory leak in the pthreads > > code in FreeBSD? > > > > -Andre > > > > ----- Forwarded message ----- > > > > Well, as these are my first steps regarding thread programming, > > it's probably my fault... > > > > Why does this programme slowly grow and grow until it hits > > resource limits? > > > > ----- snip pth1.c ----- > > > > #include > > #include > > #include > > > > void* mythread( void* arg ) > > { > > return NULL; > > } > > > > int main( int argc, const char* const argv[] ) > > { > > pthread_t pthr; > > int i; > > > > while( 1 ) { > > > > for( i=1000; i; i-- ) > > if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 ) > > fprintf( stderr, "pthread_create\n" ); > > else > > pthread_detach( pthr ); > > > > putchar( '.' ); > > fflush( stdout ); > > usleep( 25000 ); > > } > > } > > > > ----- snap ----- > > > > Just to be sure I have also created the non-detaching version > > which behaves in the same way: > > > > ----- snip pth2.c ----- > > > > #include > > #include > > #include > > > > #define M 1000 > > > > pthread_t pthr[M]; > > > > void* mythread( void* arg ) > > { > > return NULL; > > } > > > > int main( int argc, const char* const argv[] ) > > { > > int i; > > > > while( 1 ) { > > > > for( i=M; i; i-- ) > > if( pthread_create( &pthr[i], NULL, mythread, NULL ) != 0 ) > > fprintf( stderr, "pthread_create\n" ); > > > > for( i=M; i; i-- ) > > if( pthread_join( pthr[i], NULL ) != 0 ) > > fprintf( stderr, "pthread_join\n" ); > > > > putchar( '.' ); > > fflush( stdout ); > > usleep( 25000 ); > > } > > } > > > > ----- snap ----- > > > > Compile them using -pthread and watch their ps output in another > > window (FreeBSD-9.2 but that shouldn't matter). > > > > So what am I doing wrong here? > > > > Thanks, > > > > -Andre > > _______________________________________________ > > freebsd-threads@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > > To unsubscribe, send any mail to > > "freebsd-threads-unsubscribe@freebsd.org" > > -- echo "cdec cdec efg~ efg~ L8gagfL4ec L8gagfL4ec cc~ cc~" >/dev/speaker From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 15:30:57 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 128E477D; Wed, 19 Feb 2014 15:30:57 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id E609412E4; Wed, 19 Feb 2014 15:30:56 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id C584B1A3C38; Wed, 19 Feb 2014 07:30:53 -0800 (PST) Message-ID: <5304CE2C.1060305@mu.org> Date: Wed, 19 Feb 2014 07:30:52 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Erich Dollansky , Andre Albsmeier Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com> In-Reply-To: <20140219230824.0f2ba24b@X220.alogt.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 15:30:57 -0000 On 2/19/14 7:08 AM, Erich Dollansky wrote: > Hi, > > as I understand your program, it creates 1000 threads, waits a but and > then starts again creating 1000 threads until something kill it. > > From my point of view, your program depends very much on the default > settings of the environment. If the environment allows the immediate > execution of the new thread, you will never get many threads. > Otherwise, the number of threads hanging around could add up. It also > depends on the number of CPUs/cores your system has. > > But your are right, it should not crash on a modern machine but it > still could use some amount of memory. OK. Maybe use a global locked with a mutex or atomic that gets increased each time the main thread spawns a thread and then decreased by that thread right before the child thread exits. Then the main loop can print the value of that. If it gets huge then your main loop is making threads faster than the system can remove them. If not , then it's a leak. > > Erich > > On Wed, > 19 Feb 2014 15:47:28 +0100 Andre Albsmeier > wrote: > >> [Commenting my own mail from below and CC'ing freebsd-threads.] >> >> I have tested my code below on a Linux box (3.2.0) and here the >> behaviour is different and seems correct: While watching with >> top(1), VIRT climbs up to a few GB and collapses then to a few >> MB (somehow reminding me of some kind of garbage collection). >> Important thing is that RES always stays below 1MB. >> >> On FreeBSD SIZE and RES are about 100MB apart but both are >> increasing util 2GB and pth1 dies with >> >> Cannot map anonymous memory >> Out of memory >> >> So the question is: Is my programme buggy and Linux works around >> this bug or is there some kind of memory leak in the pthreads >> code in FreeBSD? >> >> -Andre >> >> ----- Forwarded message ----- >> >> Well, as these are my first steps regarding thread programming, >> it's probably my fault... >> >> Why does this programme slowly grow and grow until it hits >> resource limits? >> >> ----- snip pth1.c ----- >> >> #include >> #include >> #include >> >> void* mythread( void* arg ) >> { >> return NULL; >> } >> >> int main( int argc, const char* const argv[] ) >> { >> pthread_t pthr; >> int i; >> >> while( 1 ) { >> >> for( i=1000; i; i-- ) >> if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 ) >> fprintf( stderr, "pthread_create\n" ); >> else >> pthread_detach( pthr ); >> >> putchar( '.' ); >> fflush( stdout ); >> usleep( 25000 ); >> } >> } >> >> ----- snap ----- >> >> Just to be sure I have also created the non-detaching version >> which behaves in the same way: >> >> ----- snip pth2.c ----- >> >> #include >> #include >> #include >> >> #define M 1000 >> >> pthread_t pthr[M]; >> >> void* mythread( void* arg ) >> { >> return NULL; >> } >> >> int main( int argc, const char* const argv[] ) >> { >> int i; >> >> while( 1 ) { >> >> for( i=M; i; i-- ) >> if( pthread_create( &pthr[i], NULL, mythread, NULL ) != 0 ) >> fprintf( stderr, "pthread_create\n" ); >> >> for( i=M; i; i-- ) >> if( pthread_join( pthr[i], NULL ) != 0 ) >> fprintf( stderr, "pthread_join\n" ); >> >> putchar( '.' ); >> fflush( stdout ); >> usleep( 25000 ); >> } >> } >> >> ----- snap ----- >> >> Compile them using -pthread and watch their ps output in another >> window (FreeBSD-9.2 but that shouldn't matter). >> >> So what am I doing wrong here? >> >> Thanks, >> >> -Andre >> _______________________________________________ >> freebsd-threads@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-threads >> To unsubscribe, send any mail to >> "freebsd-threads-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > -- Alfred Perlstein From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 15:53:05 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C820144; Wed, 19 Feb 2014 15:53:05 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B49151569; Wed, 19 Feb 2014 15:53:04 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id x13so758901qcv.6 for ; Wed, 19 Feb 2014 07:53:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=LojqmoGwSaJXOshXdbS7hU+6Q+ODcxN2ZcE1RtagsTM=; b=UA/d/woNjPZunaVim3CtOU9KF3hxcZxNdT+5WnPPbpbKicAtI09MjH0sFDsLzM9J6k LR2UUVhgHBte7RydRAaQ/k5oPLjZkuLru9tOIsGLeVk30nVext6nnyLEapaxk4stCFwk CczCcAgkPcPkT6veP6sVbap1qK2Fihk5OJvUwgvUqyIlAMcQA94JZ1ch0uFm2FAFgYzH B+VDw3ZZTxpQnzciQJpSobiCMDuauGytFF+9DeK6UBbZB3JvMFxcxA7cDWJaze/5RD/B fpymv7mO5NEhF4/bNd3+yqDStkhWXLIo/so98eGAiAQKgdWy4QMoPXynxNIGs0dMZJZ9 C+Pw== MIME-Version: 1.0 X-Received: by 10.140.23.52 with SMTP id 49mr47830537qgo.17.1392825183925; Wed, 19 Feb 2014 07:53:03 -0800 (PST) Sender: carpeddiem@gmail.com Received: by 10.140.31.68 with HTTP; Wed, 19 Feb 2014 07:53:03 -0800 (PST) In-Reply-To: <20140219230824.0f2ba24b@X220.alogt.com> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com> Date: Wed, 19 Feb 2014 10:53:03 -0500 X-Google-Sender-Auth: 1fP6CbASp3MqYFUsCHtCwlsb_4M Message-ID: Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) From: Ed Maste To: Erich Dollansky Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 15:53:05 -0000 On 19 February 2014 10:08, Erich Dollansky wrote: > Hi, > > as I understand your program, it creates 1000 threads, waits a but and > then starts again creating 1000 threads until something kill it. > > From my point of view, your program depends very much on the default > settings of the environment. If the environment allows the immediate > execution of the new thread, you will never get many threads. > Otherwise, the number of threads hanging around could add up. It also > depends on the number of CPUs/cores your system has. > But your are right, it should not crash on a modern machine but it > still could use some amount of memory. Andre's second example program demonstrates the problem as well. It calls pthread_join() for each of the created threads, so should not be able to leave them around. From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 16:24:59 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF198E9E; Wed, 19 Feb 2014 16:24:59 +0000 (UTC) Received: from mail.myota.org (mail.myota.org [85.10.206.105]) by mx1.freebsd.org (Postfix) with ESMTP id 508241870; Wed, 19 Feb 2014 16:24:58 +0000 (UTC) Received: from mobile.client (177.189.166.190.f.sta.codetel.net.do [190.166.189.177] (may be forged)) (authenticated bits=128) by mail.myota.org (8.14.7/8.14.7) with ESMTP id s1JGOmb4065093; Wed, 19 Feb 2014 17:24:52 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Received: from submit.client ([127.0.0.1]) by schlappy.local (8.14.7/8.14.7) with ESMTP id s1JGOfDw003648; Wed, 19 Feb 2014 17:24:42 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Received: (from mu@schlappy.local) by schlappy.local (8.14.7/8.14.7/Submit) id s1JGOfDj003647; Wed, 19 Feb 2014 17:24:41 +0100 (CET) (envelope-from mail@ma17.ata.myota.org) Date: Wed, 19 Feb 2014 17:24:41 +0100 From: Andre Albsmeier To: Alfred Perlstein Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) Message-ID: <20140219162441.GA3564@schlappy> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com> <5304CE2C.1060305@mu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5304CE2C.1060305@mu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Not delayed on 85.10.206.105, ACL: AUTH(59), Origin: DO, OS: FreeBSD 9.x X-Virus-Scanned: clamav-milter 0.98.1 at colo X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 16:24:59 -0000 On Wed, 19-Feb-2014 at 07:30:52 -0800, Alfred Perlstein wrote: > On 2/19/14 7:08 AM, Erich Dollansky wrote: > > Hi, > > > > as I understand your program, it creates 1000 threads, waits a but and > > then starts again creating 1000 threads until something kill it. > > > > From my point of view, your program depends very much on the default > > settings of the environment. If the environment allows the immediate > > execution of the new thread, you will never get many threads. > > Otherwise, the number of threads hanging around could add up. It also > > depends on the number of CPUs/cores your system has. > > > > But your are right, it should not crash on a modern machine but it > > still could use some amount of memory. > OK. Maybe use a global locked with a mutex or atomic that gets > increased each time the main thread spawns a thread and then decreased > by that thread right before the child thread exits. > Then the main loop can print the value of that. > > If it gets huge then your main loop is making threads faster than the > system can remove them. If not , then it's a leak. While my second programme (pth2) explicitly waits for all threads to terminate, I tried what you suggested and here is the result (although these are my first steps w.r.t. threads I think I did it correctly ;-)): ----- snip ----- #include #include #include #define THREADS 1000 int c = 0; pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER; void* mythread( void* arg ) { pthread_mutex_lock( &mtx ); c--; pthread_mutex_unlock( &mtx ); return NULL; } int main( int argc, const char* const argv[] ) { pthread_t pthr; int i; while( 1 ) { pthread_mutex_lock( &mtx ); c += THREADS; pthread_mutex_unlock( &mtx ); for( i=THREADS; i; i-- ) if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 ) fprintf( stderr, "pthread_create\n" ); else pthread_detach( pthr ); pthread_mutex_lock( &mtx ); fprintf( stderr, "%d:", c ); pthread_mutex_unlock( &mtx ); usleep( 25000 ); pthread_mutex_lock( &mtx ); fprintf( stderr, "%d ", c ); pthread_mutex_unlock( &mtx ); } } ----- snap ----- As we can see, I print c before the usleep() and after, both values separated by a colon. One of the results is here (after running it through fmt(1)): 1:0 0:0 1:0 0:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 0:0 1:0 1:0 1:0 1:0 1:0 322:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 126:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 16:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 131:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 66:0 1:0 98:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 132:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 18:0 1:0 1:0 1:0 1:0 1:0 1:0 123:0 1:0 1:0 1:0 1:0 1:0 1:0 24:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 146:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 121:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 3:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 125:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 52:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 7:0 1:0 1:0 1:0 2:0 1:0 573:0 1:0 1:0 1:0 1:0 1:0 1:0 465:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 19:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:0 1:^C We can see that in most case one thread is still active before the usleep() but after that all are gone. In rare cases there are hundreds active but they will be gone for sure after the usleep()... But apart from that we might stick to pth2.c as this definitely waits for all threads to terminate. I'd also be interested to know how pth2 behaves on FreeBSD boxes other than 9.2... -Andre > > > > > > Erich > > > > On Wed, > > 19 Feb 2014 15:47:28 +0100 Andre Albsmeier > > wrote: > > > >> [Commenting my own mail from below and CC'ing freebsd-threads.] > >> > >> I have tested my code below on a Linux box (3.2.0) and here the > >> behaviour is different and seems correct: While watching with > >> top(1), VIRT climbs up to a few GB and collapses then to a few > >> MB (somehow reminding me of some kind of garbage collection). > >> Important thing is that RES always stays below 1MB. > >> > >> On FreeBSD SIZE and RES are about 100MB apart but both are > >> increasing util 2GB and pth1 dies with > >> > >> Cannot map anonymous memory > >> Out of memory > >> > >> So the question is: Is my programme buggy and Linux works around > >> this bug or is there some kind of memory leak in the pthreads > >> code in FreeBSD? > >> > >> -Andre > >> > >> ----- Forwarded message ----- > >> > >> Well, as these are my first steps regarding thread programming, > >> it's probably my fault... > >> > >> Why does this programme slowly grow and grow until it hits > >> resource limits? > >> > >> ----- snip pth1.c ----- > >> > >> #include > >> #include > >> #include > >> > >> void* mythread( void* arg ) > >> { > >> return NULL; > >> } > >> > >> int main( int argc, const char* const argv[] ) > >> { > >> pthread_t pthr; > >> int i; > >> > >> while( 1 ) { > >> > >> for( i=1000; i; i-- ) > >> if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 ) > >> fprintf( stderr, "pthread_create\n" ); > >> else > >> pthread_detach( pthr ); > >> > >> putchar( '.' ); > >> fflush( stdout ); > >> usleep( 25000 ); > >> } > >> } > >> > >> ----- snap ----- > >> > >> Just to be sure I have also created the non-detaching version > >> which behaves in the same way: > >> > >> ----- snip pth2.c ----- > >> > >> #include > >> #include > >> #include > >> > >> #define M 1000 > >> > >> pthread_t pthr[M]; > >> > >> void* mythread( void* arg ) > >> { > >> return NULL; > >> } > >> > >> int main( int argc, const char* const argv[] ) > >> { > >> int i; > >> > >> while( 1 ) { > >> > >> for( i=M; i; i-- ) > >> if( pthread_create( &pthr[i], NULL, mythread, NULL ) != 0 ) > >> fprintf( stderr, "pthread_create\n" ); > >> > >> for( i=M; i; i-- ) > >> if( pthread_join( pthr[i], NULL ) != 0 ) > >> fprintf( stderr, "pthread_join\n" ); > >> > >> putchar( '.' ); > >> fflush( stdout ); > >> usleep( 25000 ); > >> } > >> } > >> > >> ----- snap ----- > >> > >> Compile them using -pthread and watch their ps output in another > >> window (FreeBSD-9.2 but that shouldn't matter). > >> > >> So what am I doing wrong here? > >> > >> Thanks, > >> > >> -Andre > >> _______________________________________________ > >> freebsd-threads@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-threads > >> To unsubscribe, send any mail to > >> "freebsd-threads-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-threads@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > > > > > -- > Alfred Perlstein > -- Stuxnet? Find ich gut. Manche lernen nur auf die harte Tour... From owner-freebsd-threads@FreeBSD.ORG Wed Feb 19 16:52:47 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87A0BE0C; Wed, 19 Feb 2014 16:52:47 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 329B51B68; Wed, 19 Feb 2014 16:52:47 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so1151077qgd.1 for ; Wed, 19 Feb 2014 08:52:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9ZiijEZEsJSOC1EaFiT39Lg3SZbZTDYrMjgc5HyA5gs=; b=drAMunIHj68LradsofqLTvaqXHiOpBu4J3Xr2+pW712EqlSi8lfPDdKUa0HnrdVA8b tly0ATMU90Mun9+4KJvv68CMVQ/vk4f/haWKKu0Cn54J4YfRhSVXnIeJi+la25qw8Nol uXLVlQSSVxxH3YoD9fwekLKA/0VgbBLeBn2WsMIG0HvXR+lVeb1dTm4cArRah5igVNya x8K8mgFhAGDH4guncS1g8NYx7NDOP3czPywggGkHIz3sQI8szF5FVzZ5mhnx9LHaHRXl dab2lLalc4pQLKOCH2/jHWWDO5PRp9wncILWcxOFlAQNpNrZUTiouky431ZtIhvQIzKn 5y2Q== MIME-Version: 1.0 X-Received: by 10.224.114.78 with SMTP id d14mr2781415qaq.19.1392828766359; Wed, 19 Feb 2014 08:52:46 -0800 (PST) Sender: carpeddiem@gmail.com Received: by 10.140.31.68 with HTTP; Wed, 19 Feb 2014 08:52:46 -0800 (PST) In-Reply-To: <20140219162441.GA3564@schlappy> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com> <5304CE2C.1060305@mu.org> <20140219162441.GA3564@schlappy> Date: Wed, 19 Feb 2014 11:52:46 -0500 X-Google-Sender-Auth: sZr8r4zDLYiKWc8YN1wY2augiY0 Message-ID: Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) From: Ed Maste To: Andre Albsmeier Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 16:52:47 -0000 On 19 February 2014 11:24, Andre Albsmeier wrote: > I'd also be interested to know how pth2 behaves on FreeBSD boxes > other than 9.2... It is reproducible on FreeBSD-CURRENT from about 2 weeks ago; it looks like we probably do have an issue in libthr. From owner-freebsd-threads@FreeBSD.ORG Thu Feb 20 03:39:20 2014 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A715059E; Thu, 20 Feb 2014 03:39:20 +0000 (UTC) Received: from alogt.com (alogt.com [69.36.191.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7BD9F1850; Thu, 20 Feb 2014 03:39:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=yre7YymE+a8m2Vg/eil5/04v5d3NrHpt+xAzof75zV4=; b=m7k/cvOPFziIj1Lq9k6QCG88JmE3SoZZa0j3bp0ekiSkvjkzPWPsbnNdvL6ObZXEoCCu05h7iR95cC0PGNUbYIUqKQME1VGwlDON7w9f/W45vCLQW48jFruqbBix4yqjLPJCnsspyhJdCOSdBvORPnA3RGvhqKk1XBG+pGEXq3E=; Received: from [39.199.31.111] (port=10635 helo=X220.alogt.com) by sl-508-2.slc.westdc.net with esmtpsa (SSLv3:DHE-RSA-AES128-SHA:128) (Exim 4.82) (envelope-from ) id 1WGKTd-003x4N-Ri; Wed, 19 Feb 2014 20:39:11 -0700 Date: Thu, 20 Feb 2014 11:39:02 +0800 From: Erich Dollansky To: Andre Albsmeier Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) Message-ID: <20140220113902.63e97b00@X220.alogt.com> In-Reply-To: <20140219162441.GA3564@schlappy> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com> <5304CE2C.1060305@mu.org> <20140219162441.GA3564@schlappy> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erichsfreebsdlist@alogt.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Feb 2014 03:39:20 -0000 Hi, On Wed, 19 Feb 2014 17:24:41 +0100 Andre Albsmeier wrote: I am now running this program: > #include > #include > #include > > #define THREADS 1000 > > int c = 0; > pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER; > > void* mythread( void* arg ) > { > pthread_mutex_lock( &mtx ); > c--; > pthread_mutex_unlock( &mtx ); > > return NULL; > } > > int main( int argc, const char* const argv[] ) > { > pthread_t pthr; > int i; > > while( 1 ) { > > pthread_mutex_lock( &mtx ); > c += THREADS; > pthread_mutex_unlock( &mtx ); > > for( i=THREADS; i; i-- ) > if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 ) > fprintf( stderr, "pthread_create\n" ); > else > pthread_detach( pthr ); > > pthread_mutex_lock( &mtx ); > fprintf( stderr, "%d:", c ); > pthread_mutex_unlock( &mtx ); > > usleep( 25000 ); > > pthread_mutex_lock( &mtx ); > fprintf( stderr, "%d ", c ); > pthread_mutex_unlock( &mtx ); > } > } It runs on an i7 with two cores plus two hyper threads. The machine has 8GB of RAM and 16GB of Swap. The resident part of the program stays below 8GB (what wonder) but the rest is swapped out. I run this program on 10.0 from some 2 weeks ago. As already said, it looks like there stays some memory left after a thread terminates. To say something positive, the program really cleaned the memory. Erich