Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 23 Nov 2006 18:48:17 +0200 (EET)
From:      Andriy Gapon <avg@icyb.net.ua>
To:        FreeBSD-gnats-submit@FreeBSD.org
Cc:        sem@FreeBSD.org
Subject:   ports/105794: devel/ace: can not create more than ncpu threads with default thread flags
Message-ID:  <200611231648.kANGmHNl027945@oddity.topspin.kiev.ua>
Resent-Message-ID: <200611231650.kANGoIuL013644@freefall.freebsd.org>

next in thread | raw e-mail | index | archive | help

>Number:         105794
>Category:       ports
>Synopsis:       devel/ace: can not create more than ncpu threads with default thread flags
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-ports-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Nov 23 16:50:16 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator:     Andriy Gapon
>Release:        FreeBSD 6.1-RELEASE-p2 i386
>Organization:
>Environment:
System:
FreeBSD 6.1-RELEASE-p2 i386
ace-5.5.1
libpthread is used as threads lib

>Description:
Please see the attached program.
It fails with the following output:
spawning thread #0
spawned thread #0 (thr_id=8053400)
spawning thread #1
spawn failure: Resource temporarily unavailable
Exit 71

ktrace shows the following information:
 27360 thread   GIO   fd 2 wrote 19 bytes
       "spawning thread #1
       "
 27360 thread   RET   write 19/0x13
 27360 thread   CALL  kse_create(0x804bf0c,0)
 27360 thread   RET   kse_create -1 errno 67 Too many processes

It seems that the problem occurs because flag THR_NEW_LWP, included into default set of ACE thread flags,
is interpreted by ACE library as an instruction to call pthread_setconcurrency with concurrency level
set to current number of threads created through ACE. FreeBSD libpthread refuses to set
concurrency higher than a number of CPUs available. ACE library treats pthread_setconcurrency() failure
as a fatal failure for thread creation.

Any solution for this problem should probably be submitted to ACE maintainers for
inclusion into ACE platform-specific source base.

>How-To-Repeat:
1. Change the attached program so that thr_num variable is greater than the number of CPUs on your test machine.
2. Compile the attached program:
$ c++ thread.cpp -o thread -I /usr/local/include -L /usr/local/lib -lACE -pthread
3. Make sure that libpthread is used as your thread support library:
$ ldd thread
thread:
        libACE.so.5 => /usr/local/lib/libACE.so.5 (0x48077000)
        libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x48196000)
        libm.so.4 => /lib/libm.so.4 (0x4826c000)
        libpthread.so.2 => /usr/lib/libpthread.so.2 (0x48283000)
        libc.so.6 => /lib/libc.so.6 (0x482aa000)

4. run the program

>Fix:

Workaround: use libthr as a threading library

Solution 1: change a meaning of THR_NEW_LWP on FreeBSD as term "LWP" is not directly
applicable to FreeBSD threading model. Reasonable approach is to alias THR_NEW_LWP
to THR_SCOPE_SYSTEM.

Solutuion 2: do not treat failure of pthread_setconcurrency() as a fatal error
for thread creation.

No patches, sorry.

--- thread.cpp begins here ---

#include <sysexits.h>

#include <ace/Log_Msg.h>
#include <ace/Thread.h>
#include <ace/Thread_Manager.h>

ACE_THR_FUNC_RETURN thr_func(void*)
{
	ACE_DEBUG((LM_DEBUG, ACE_TEXT("in thr_func\n")));
	ACE_OS::sleep(1);
	ACE_DEBUG((LM_DEBUG, ACE_TEXT("out thr_func\n")));
	return 0;
}

int ACE_TMAIN(int /*argc*/, ACE_TCHAR* /*argv*/[])
{
	ACE_TRACE(ACE_TEXT("main\n"));

	const int thr_num = 2;
	ACE_thread_t t_id[thr_num];

	for (int i = 0; i < thr_num; ++i) {
		ACE_DEBUG((LM_DEBUG, ACE_TEXT("spawning thread #%d\n"), i));
		if (ACE_Thread::spawn(thr_func, 0, THR_NEW_LWP | THR_JOINABLE | THR_INHERIT_SCHED, &t_id[i]) == -1) {
			ACE_ERROR_RETURN((LM_ERROR, ACE_TEXT("spawn failure: %m\n")), EX_OSERR);
		}
		ACE_DEBUG((LM_DEBUG, ACE_TEXT("spawned thread #%d (thr_id=%X)\n"), i, reinterpret_cast<unsigned long>(t_id[i])));
	}

	for (int i = 0; i < thr_num; ++i) {
		ACE_DEBUG((LM_DEBUG, ACE_TEXT("joining thread #%d\n"), i));
		ACE_Thread::join(t_id[i], 0, 0);
		ACE_DEBUG((LM_DEBUG, ACE_TEXT("joined thread #%d\n"), i));
	}

	return 0;
}
--- thread.cpp ends here ---


>Release-Note:
>Audit-Trail:
>Unformatted:



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200611231648.kANGmHNl027945>