Date: Wed, 30 Aug 2006 18:56:02 +0300 From: Andriy Gapon <avg@icyb.net.ua> To: Daniel Eischen <eischen@vigrid.com> Cc: ace-users@cse.wustl.edu, Sergey Matveychuk <sem@freebsd.org>, freebsd-threads@freebsd.org Subject: Re: ace/freebsd: THR_NEW_LWP problem with libpthread/pthread_setconcurrency Message-ID: <44F5B512.5060708@icyb.net.ua> In-Reply-To: <Pine.GSO.4.64.0608301112310.5392@sea.ntplx.net> References: <44F59F3C.3040303@icyb.net.ua> <Pine.GSO.4.64.0608301112310.5392@sea.ntplx.net>
next in thread | previous in thread | raw e-mail | index | archive | help
on 30/08/2006 18:19 Daniel Eischen said the following: > On Wed, 30 Aug 2006, Andriy Gapon wrote: >> DESCRIPTION: >> ACE_Thread_Manager::spawn(..., THR_NEW_LWP) fails with EAGAIN. It seems >> that this happens because when creating such threads ACE calls >> pthread_setconcurrency with incrementally increasing concurrency levels. >> pthread_setconcurrency of libpthread in turn seems to call kse_create() >> which fails with EPROCLIM. As I understand this happens because kernel >> does not allow to have more KSE-s per process as there are processors. > > pthread_setconcurrency() should return EAGAIN, not EPROCLIM. kse_create() > may set errno to EPROCLIM, but I don't see how the code in libpthread > can return anything other than EAGAIN if kse_create() fails. That's exactly what I actually was trying to say, sorry for unclear wording. > POSIX states that pthread_setconcurrency() can return EAGAIN, so I > think ACE is at fault if it is not handling that properly. I > would also think that THR_NEW_LWP should not increment the > concurrency, but create a new system scope thread since that > seems more in the spirit of "spawning a new thread with a new > LWP". Yes, indeed, it seems like a good approach, at least for FreeBSD+libpthread. On the other hand, ACE also defines the following thread flags: THR_SCOPE_SYSTEM, THR_SCOPE_PROCESS. The whole THR_NEW_LWP seems to be inspired by Solaris threads (of old?): http://docs.sun.com/app/docs/doc/805-5080/6j4q7emhv?a=view THR_NEW_LWP--Increases the concurrency level for unbound threads by one. The effect is similar to incrementing concurrency by one with thr_setconcurrency(3T), although THR_NEW_LWP does not affect the level set through the thr_setconcurrency() function. Typically, THR_NEW_LWP adds a new LWP to the pool of LWPs running unbound threads. >> SAMPLE FIX/WORKAROUND: >> Straightforward workaround is to explicitly pass thread flags everywhere >> and not use THR_NEW_LWP. Maybe this could be patched in ACE sources as a >> part of FreeBSD port patch step. >> IMHO, better fixes would be: >> 1. make ACE_Thread_Manager::spawn() more robust with respect to >> pthread_setconcurrency() failing with EAGAIN. >> 2. make FreeBSD libpthread dumb-happy in pthread_setconcurrency(), i.e >> pretend to always succeed. AFAIK, POSIX leaves it up to implementations >> to interpret concurrency levels, so making any level be equivalent to >> default level should be OK. > > Yes to 1, No to 2, and the third option should be to treat THR_NEW_LWP > as creating a new system scope thread. Yes, I also feel that ACE should not try to mimic Solaris threads THR_NEW_LWP and instead treat it as an alias for THR_SCOPE_SYSTEM (or no-flag). At least on FreeBSD. Whether this is to be done through changes in ACE itself or through a patch in the FreeBSD port, I am not sure. -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44F5B512.5060708>