Date: Thu, 16 Aug 2018 16:28:33 +0000 From: Double Tong <doubleble@outlook.com> To: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Create a thread with a separate file descriptor table (set RFFDG flag) Message-ID: <BN6PR2201MB1523127E9938E9CC1FADB944A23E0@BN6PR2201MB1523.namprd22.prod.outlook.com>
next in thread | raw e-mail | index | archive | help
Hello, I want to create a thread with a separate file descriptor table to have bet= ter performance with kevent(2). On Linux, I was using unshare(2) syscall to= achieve this, which as far as I know there is no equivalent or similar sys= call in FreeBSD. I have posted on freebsd forums (https://forums.freebsd.org/threads/create-= a-thread-with-a-separate-file-descriptor-table-set-rffdg-flag.67143/), and = now I understood the following: rfork_thread(3) is deprecated in favor of = pthread_create(3). rfork_thread(3) is written in assembly language to perfo= rm stack swapping, which means if rfork_thread(3) no longer exists in the b= uild, it can damage our program's portability if it relies on rfork_thread. With the above consideration, I still have the following questions: 1. Is there an elegant way to create a thread with a separate file descr= iptor table? 2. If you are thinking about using rfork_thread(3) to do this, I am work= ing on this direction. I am using waitpid to join these "threads", and the = thread exits in the middle of execution with status 0x8b collected by waitp= id. I guess this status means invalid page access. I wrote a tiny program (= attached below) to reflect the code I am using in my program, I appreciate = if you would like to take a look at it to see if there is anything I was no= t doing correctly. 3. As I was reading the code of pthread_create, it allocates a pthread s= truct on the top of thread, and then calls clone, which freebsd implemented= its version of clone that actually calls rfork (I did not find the source = of freebsd's clone, can someone provides a link?). So I believe theoretical= ly there should be a way to achieve this in the user space. And if I am not= using pthread related APIs, then missing pthread struct should be fine as = well? 4. On Linux, after calling unshare(CLONE_FLIES), I got performance incre= ase around 10% with 1000 concurrent TCP connections. I am instructed to imp= lement this by my supervisor, and I do not have much details about why the = performance would increase. Would this also works for freebsd as well (keve= nt calls)? Thank you for any help, comments in advance! my_rfork_test.cc: -------------------------- // This program runs well, except status code is non zero. In my bigger pro= gram, it terminates in the middle of the routine with status 0x8b #include <iostream> #include <unistd.h> #include <sys/wait.h> #include <sys/mman.h> #include <vector> const size_t VEC_SIZE =3D 1000; static int thread_routine(void* arg) { std::cout<<"init thread "<<arg<<std::endl; // the problem seems related to some memory allocation std::vector<int>** vectors =3D new std::vector<int>*[VEC_SIZE]; for(int i =3D 0; i < VEC_SIZE; i++) { vectors[i] =3D new std::vector<int>(10000); std::cout<<"vec "<<i<<" initialized"<<std::endl; } } int main() { const int STACK_SIZE =3D 8000000; //void* stackaddr =3D malloc(STACK_SIZE); // should also work void* stackaddr =3D mmap(NULL, STACK_SIZE, PROT_READ|PROT_WRITE, MAP_PRIV= ATE|MAP_ANONYMOUS|MAP_STACK, -1, 0); std::cout<<"stackaddr: 0x"<<std::hex<<stackaddr<<std::endl; void* stacktop =3D (char*)stackaddr + STACK_SIZE; // assuming stack goin= g downwards pid_t child =3D 0; child =3D rfork_thread(RFFDG|RFPROC|RFMEM|RFSIGSHARE,stacktop,&thread_rou= tine, reinterpret_cast<void*>(2)); int status =3D 0; waitpid(child, &status, 0x0); std::cout<<"return status 0x"<<std::hex<<status<<std::endl; // should ret= urn 0? but usually not } -------------------------- Best regards, Shuangyi Tong
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BN6PR2201MB1523127E9938E9CC1FADB944A23E0>