Date: Mon, 24 Apr 2000 16:44:05 -0500 From: "Richard Seaman, Jr." <dick@seaman.org> To: Jason Evans <jasone@canonware.com> Cc: A G F Keahan <ak@freenet.co.uk>, freebsd-hackers@FreeBSD.ORG Subject: Re: Multithreaded server performance Message-ID: <20000424164405.G370@seaman.org> In-Reply-To: <20000423212115.E31925@sturm.canonware.com>; from jasone@canonware.com on Sun, Apr 23, 2000 at 09:21:15PM -0700 References: <3903AEA6.FA7CBBAB@freenet.co.uk> <20000423212115.E31925@sturm.canonware.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Apr 23, 2000 at 09:21:15PM -0700, Jason Evans wrote: > On Mon, Apr 24, 2000 at 05:17:10AM +0300, A G F Keahan wrote: > > I am currently porting a multithreaded TCP server from NT (yech!) to > > UNIX using pthreads. The server has a fairly straightforward design -- > > it opens a thread for each connection, and each thread spends most of > > its life blocked in a call to read() from a socket. As soon as it > > receives enough of a request, it does quite a bit of processing and > > sends something back to the client. > > This design isn't ideal on any OS, but the fact that you do significant > processing every time a request arrives on a socket probably hides most of > the inefficiency due to thread switching and lack of cache locality due to > many thread stacks. > > > How would FreeBSD 4.0 perform in such a scenario? We are talking > > hundreds, maybe thousands of threads, a lot of them doing blocking reads > > and writes. Is the standard pthreads library adequate for the task, or > > would "Linuxthreads" be a better choice? What is the main difference > > between the standard FreeBSD pthreads and "Linuxthreads" -- it seems > > both are implemented using a version of clone(). > > FreeBSD's threads should perform adequately given the design of your program > and the hardware you listed. Actually trying it on the various operating > systems would be a good exercise, but I have found that FreeBSD's threads > perform at least as well as Solaris's for such an application. Interesting. I would have thought Solaris would be much superior. > LinuxThreads will definitely bog down with so many threads because the > kernel scheduler has to deal with so many clone()d processes. Somewhat. But its nothing compared to the slowdown in the libc_r scheduler, in my experience. Each context switch (or each time the libc_r scheduler examines its state to consider a context switch), it executes a poll call to examine the state of *each* file descriptor that is considered "blocked" (of course its not really blocked in the kernel -- libc_r converts all i/o calls to non-blocking and polls their status to determined when i/o can be done). If you have hundreds/thousands of threads and each has an open file descriptor that has to be polled, linuxthreads is much faster, at least for the tests I've run. However, linuxthreads will consume a lot more kernel resources, so this only holds if you can tolerate the load on your resources. That said, if you want more than 1022 threads using linuxthreads (either in the FreeBSD port or in linux emulation, or in native linux) you will have to tweak the linuxthreads pthread library. It has a default limit of 1024 threads including the "main" thread and the thread manager thread. You have to raise the limit, and fiddle with the default thread stack spacing, if I recall. (I think the default stack spacing is 2MB. This doesn't work too well when you have "thousands" of threads :)) > FreeBSD's libc_r does not use clone() or anything similar. Instead, it is > a userland call conversion library that multiplexes threads in a single > process. This style of threads library should perform well for the type of > application you are dealing with. Perhaps. The proposed "new" threads model that I understand you are working on should be *much* superior in this kind of application to either libc_r or linuxthreads, I would think. -- Richard Seaman, Jr. email: dick@seaman.org 5182 N. Maple Lane phone: 262-367-5450 Chenequa WI 53058 fax: 262-367-5852 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20000424164405.G370>