Date: Wed, 19 Feb 2014 07:30:52 -0800 From: Alfred Perlstein <bright@mu.org> To: Erich Dollansky <erichsfreebsdlist@alogt.com>, Andre Albsmeier <mail@ma17.ata.myota.org> Cc: freebsd-hackers@freebsd.org, freebsd-threads@freebsd.org Subject: Re: pthread programming eats up resources (My or FreeBSD's fault?) Message-ID: <5304CE2C.1060305@mu.org> In-Reply-To: <20140219230824.0f2ba24b@X220.alogt.com> References: <20140218180646.GA67861@schlappy> <20140219144728.GA3036@schlappy> <20140219230824.0f2ba24b@X220.alogt.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2/19/14 7:08 AM, Erich Dollansky wrote:
> Hi,
>
> as I understand your program, it creates 1000 threads, waits a but and
> then starts again creating 1000 threads until something kill it.
>
> From my point of view, your program depends very much on the default
> settings of the environment. If the environment allows the immediate
> execution of the new thread, you will never get many threads.
> Otherwise, the number of threads hanging around could add up. It also
> depends on the number of CPUs/cores your system has.
>
> But your are right, it should not crash on a modern machine but it
> still could use some amount of memory.
OK. Maybe use a global locked with a mutex or atomic that gets
increased each time the main thread spawns a thread and then decreased
by that thread right before the child thread exits.
Then the main loop can print the value of that.
If it gets huge then your main loop is making threads faster than the
system can remove them. If not , then it's a leak.
>
> Erich
>
> On Wed,
> 19 Feb 2014 15:47:28 +0100 Andre Albsmeier <mail@ma17.ata.myota.org>
> wrote:
>
>> [Commenting my own mail from below and CC'ing freebsd-threads.]
>>
>> I have tested my code below on a Linux box (3.2.0) and here the
>> behaviour is different and seems correct: While watching with
>> top(1), VIRT climbs up to a few GB and collapses then to a few
>> MB (somehow reminding me of some kind of garbage collection).
>> Important thing is that RES always stays below 1MB.
>>
>> On FreeBSD SIZE and RES are about 100MB apart but both are
>> increasing util 2GB and pth1 dies with
>>
>> Cannot map anonymous memory
>> Out of memory
>>
>> So the question is: Is my programme buggy and Linux works around
>> this bug or is there some kind of memory leak in the pthreads
>> code in FreeBSD?
>>
>> -Andre
>>
>> ----- Forwarded message -----
>>
>> Well, as these are my first steps regarding thread programming,
>> it's probably my fault...
>>
>> Why does this programme slowly grow and grow until it hits
>> resource limits?
>>
>> ----- snip pth1.c -----
>>
>> #include <stdio.h>
>> #include <unistd.h>
>> #include <pthread.h>
>>
>> void* mythread( void* arg )
>> {
>> return NULL;
>> }
>>
>> int main( int argc, const char* const argv[] )
>> {
>> pthread_t pthr;
>> int i;
>>
>> while( 1 ) {
>>
>> for( i=1000; i; i-- )
>> if( pthread_create( &pthr, NULL, mythread, NULL ) != 0 )
>> fprintf( stderr, "pthread_create\n" );
>> else
>> pthread_detach( pthr );
>>
>> putchar( '.' );
>> fflush( stdout );
>> usleep( 25000 );
>> }
>> }
>>
>> ----- snap -----
>>
>> Just to be sure I have also created the non-detaching version
>> which behaves in the same way:
>>
>> ----- snip pth2.c -----
>>
>> #include <stdio.h>
>> #include <unistd.h>
>> #include <pthread.h>
>>
>> #define M 1000
>>
>> pthread_t pthr[M];
>>
>> void* mythread( void* arg )
>> {
>> return NULL;
>> }
>>
>> int main( int argc, const char* const argv[] )
>> {
>> int i;
>>
>> while( 1 ) {
>>
>> for( i=M; i; i-- )
>> if( pthread_create( &pthr[i], NULL, mythread, NULL ) != 0 )
>> fprintf( stderr, "pthread_create\n" );
>>
>> for( i=M; i; i-- )
>> if( pthread_join( pthr[i], NULL ) != 0 )
>> fprintf( stderr, "pthread_join\n" );
>>
>> putchar( '.' );
>> fflush( stdout );
>> usleep( 25000 );
>> }
>> }
>>
>> ----- snap -----
>>
>> Compile them using -pthread and watch their ps output in another
>> window (FreeBSD-9.2 but that shouldn't matter).
>>
>> So what am I doing wrong here?
>>
>> Thanks,
>>
>> -Andre
>> _______________________________________________
>> freebsd-threads@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-threads
>> To unsubscribe, send any mail to
>> "freebsd-threads-unsubscribe@freebsd.org"
> _______________________________________________
> freebsd-threads@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-threads
> To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org"
>
--
Alfred Perlstein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5304CE2C.1060305>