Date: Thu, 16 Aug 2018 10:38:51 -0600 From: Alan Somers <asomers@freebsd.org> To: Double Tong <doubleble@outlook.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: Create a thread with a separate file descriptor table (set RFFDG flag) Message-ID: <CAOtMX2j8p_ZMd=4nfSe2jY-ZuBnqVzw_yv=Hs6uRCJC4zX6mug@mail.gmail.com> In-Reply-To: <BN6PR2201MB1523127E9938E9CC1FADB944A23E0@BN6PR2201MB1523.namprd22.prod.outlook.com> References: <BN6PR2201MB1523127E9938E9CC1FADB944A23E0@BN6PR2201MB1523.namprd22.prod.outlook.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Aug 16, 2018 at 10:28 AM, Double Tong <doubleble@outlook.com> wrote: > Hello, > > I want to create a thread with a separate file descriptor table to have > better performance with kevent(2). On Linux, I was using unshare(2) syscall > to achieve this, which as far as I know there is no equivalent or similar > syscall in FreeBSD. > > I have posted on freebsd forums (https://forums.freebsd.org/ > threads/create-a-thread-with-a-separate-file-descriptor- > table-set-rffdg-flag.67143/), and now I understood the following: > rfork_thread(3) is deprecated in favor of pthread_create(3). > rfork_thread(3) is written in assembly language to perform stack swapping, > which means if rfork_thread(3) no longer exists in the build, it can damage > our program's portability if it relies on rfork_thread. > > With the above consideration, I still have the following questions: > > 1. Is there an elegant way to create a thread with a separate file > descriptor table? > Sort of. The usual way to do this is to create a separate process, not a separate thread. If you also need a shared address space, then you can probably satisfy that by creating a shared memory object, and using that for whichever data structures actually need to be shared. See shm_open(2) for details. > 2. If you are thinking about using rfork_thread(3) to do this, I am > working on this direction. I am using waitpid to join these "threads", and > the thread exits in the middle of execution with status 0x8b collected by > waitpid. I guess this status means invalid page access. I wrote a tiny > program (attached below) to reflect the code I am using in my program, I > appreciate if you would like to take a look at it to see if there is > anything I was not doing correctly. > 3. As I was reading the code of pthread_create, it allocates a pthread > struct on the top of thread, and then calls clone, which freebsd > implemented its version of clone that actually calls rfork (I did not find > the source of freebsd's clone, can someone provides a link?). So I believe > theoretically there should be a way to achieve this in the user space. And > if I am not using pthread related APIs, then missing pthread struct should > be fine as well? > 4. On Linux, after calling unshare(CLONE_FLIES), I got performance > increase around 10% with 1000 concurrent TCP connections. I am instructed > to implement this by my supervisor, and I do not have much details about > why the performance would increase. Would this also works for freebsd as > well (kevent calls)? Does your Linux program use select(2), epoll(2), or something else? select(2) has well-known performance problems with large file-descriptor tables, but epoll(2) and kevent(2) do not. If your program is using select(2), then you should convert it to use epoll(2)/kevent(2) and ditch unshare(2). > > Thank you for any help, comments in advance! > > -Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2j8p_ZMd=4nfSe2jY-ZuBnqVzw_yv=Hs6uRCJC4zX6mug>