Date: Tue, 03 Jul 2001 03:06:08 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Idea Receiver <receiver@blueskybbs.yi.org> Cc: "E.B. Dreger" <eddy+public+spam@noc.everquick.net>, Chris Costello <chris@calldei.com>, freebsd-smp@FreeBSD.ORG, freebsd-hackers@FreeBSD.ORG Subject: Re: libc_r locking... why? Message-ID: <3B419910.BF346FB4@mindspring.com> References: <Pine.BSF.4.21.0106301129090.8701-100000@RedDust.BlueSky.net.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Idea Receiver wrote: > On Fri, 29 Jun 2001, Terry Lambert wrote: > > If you "need" kernel threads, look at the Linux kernel > > threads in the ports collection (it's a kernel module > > that builds and installs as a package). You probably > > don't, since performance of kernel threads is really only > > about a 20% increment, if you implement them the SVR4 or > > Solaris (pre-2.7) or Linux way. It's probably better to > > implement with FreeBSD threads as they currently exist, > > and get massive SMP scalability when KSE's are brought > > into the source tree. > > > > just a quick question... > I konw KSE will brought in after SMPng. > but it will be really helpful to konw when it will first appear > in the source tree? They went over the design at Usenix in Boston last week; there was a big FreeBSD BOF. I believe the design is now frozen. > or what other OS can help with SMP vs pthread problem? Solaris 8 & 9 have pretty good code in this area, but are limited on scaling due to a lot of bus contention; most SVR4 derivatives claim that 4 CPUs in one box are the point of diminishing returns. Big iron from Sun is actually semi-NUMA architecture, i.e. they have loosely coupled clusters of hardware, with only a small number of shared memory multiprocessors in the same contention domain, thus avoiding the scaling issue. A couple of good starting points for looking at NUMA are: http://citeseer.nj.nec.com/12857.html http://www.ibm.com/servers/eserver/xseries/numa/index.html The last one there is the former Sequent, which was bought out by IBM. Historically, Sequent has been at the forefront of SMP systems; they were able to scale to 32 processors; they had a special bus, and ran without Intel APICs (you can only have 32 APICs, since that's all the ID space can handle, and at least one of them has to be an I/O APIC, which means using Intel's approach, you are maxed out at 31 processors and one I/O APIC; usually 30/2, actually. Sequent had a BSD-based OS called Dynix, which had a lot of smart things in it, including per processor resource pools, which is what enabled it to scale so large: it removed everything it could from the inter-CPU contention domain. FreeBSD is trying to take much of that approach. Unfortunately, they went to System V (SVR3), which then introduced a big giant lock on SMP-unsafe subsystems; in particular, only one processor was allowed into the VFS at a time, which sucked -- if you started two "ls -R" processes on two processors, then one would complete, and then the other -- but the second one wouldn't start until the lock was let go, so they were effectively being serialized, while one CPU was idle. It really ruined the usefulness of the machine. Other OSs have their own problems: VxWorks, the NetApp OS, and NetWare are all pretty allergic to SMP, since they use voluntary cooperative multitasking, where you either have to call an explicit yield, or you have to run to completion; this is very hard to SMP-ize, since you end up having to add in all of the locking that you left out to get the light weight multitasking, and they generally do not implement a seperate protection domain at all, which makes it hard to have more than one processor running at once in any case; in FreeBSD, you can have multiple CPUs in user space, but only one CPU in the kernel at a time. The -current branch tries to change this, but it's really rough going. Frankly, I predict that a fork is likely; I expect that many people will not move off the 4.x branch to 5.0, when it becomes available. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B419910.BF346FB4>