Date: Tue, 02 Nov 1999 18:50:07 -0500 From: "Daniel M. Eischen" <eischen@vigrid.com> To: Nate Williams <nate@mt.sri.com> Cc: Julian Elischer <julian@whistle.com>, freebsd-arch@freebsd.org Subject: Re: Threads models and FreeBSD. (Next Step) Message-ID: <381F78AF.D5073BFB@vigrid.com> References: <Pine.BSF.4.10.9911020810090.2283-100000@current1.whistle.com> <19991102173736.9E34E1FCD@io.yi.org> <199911022319.QAA26200@mt.sri.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Nate Williams wrote: > > [ > > Jumping in headlong into this thing, after reading the 'Scheduler > Activations paper. I may be a bit confused, so feel free to correct me, > but I have a potential issue with the implementation described in the > paper. > > ] > > First, some background. One of the big reasons that FreeBSD is chosen > over other platforms is it's ability to handle heavy loads well. It's > both robust *AND* makes very good use of available hardware. > > The types of loads that FreeBSD excels usually involve lots of I/O (WWW > servers, disk farms for video, network servers, etc...). Being able to > effeciently pump out *LOTS* of bandwidth in an extremely effecient > manner is one of FreeBSD's strengths. We get close-to-hardware speeds > for networking performance, file system performance, and the like, and > that's one of our big selling points. > > IMO, any threading solution we implement must maintain this advantage. > > However, I'm not sure using 'scheduler activation' would allow us to do > that well. In particular, the threading model proposed assumes that > Activation Upcalls to userland will be minimized. > > However, in the multi-threaded applications I'm aware, the design tends > to have *many* threads, where each thread is responsible for one/more > I/O channels (files, sockets, clients, etc...) > > One of the biggest advantages of kernel threads (vs. userland threads) > is that regardless of the type of I/O, a kernel thread can 'block' > waiting for data and/or waiting for data to be written while allowing > other 'threads' of execution to continue. Therefore, you'd want to have > lots of 'kernel threads' active at any one time if you had lots of > channels for I/O to occur. (Note, select/poll don't always work, and > are not a good 'map' into threading models, IMO. Feel free to point out > why I'm out to lunch, as I'd love to have a discussion on this.). > > So, assuming we're not using select/poll, there are potentially a *LOT* > of kernel threads blocked in the kernel. In the SA model, once a > 'thread' was blocked in the kernel, an SA (which is in effect the kernel > thread) would be passed back via an upcall to the userland scheduler to > allow it to continue working on other threads, so the process is allowed > to continue using the rest of its quantum. > > Given that there are N userland threads 'blocked' in the kernel, and M > active 'SA' (kernel threads), everytime one of the userland threads > 'unblocks' due to I/O coming in or completing, there is potentially a > *LOT* of upcall traffic, especially in applications that manage lots of > I/O channels (WWW servers, ftp servers, etc..). > > It would seem to me that the SA model is a poor match for this kind of > application, which tends to be the kind of application that FreeBSD is > best suited for. > > What am I missing? Is it just a fact that there is no good threading > solution for this kind of application? Solaris seems to do a pretty > good job of optimizing this case, although it may be that we have > nothing better to compare it to simply because no one has implemented a > robust enough implementation on any other OS/hardware platform? It could also take many kernel wakeups for a heavy I/O bound thread to leave the kernel. I thought of this ;-). This is how binding a thread to a LWP can be useful. For a thread bound to a LWP, you only notify the user level threads library if it blocks because it's time quantum expired (so the threads library can see if it is in a critical region). If the thread blocks due to a tsleep or something like that, you can assume it's not holding any critical resources that the user threads library implementation needs. In this case, you don't notify the threads library. So by extending SA to allow binding threads to LWPs, you can achieve just about the same thing as async call gates. But it's the application that gets to decide exactly how to use these features, and not left to the implementation in the kernel. Dan Eischen eischen@vigrid.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?381F78AF.D5073BFB>