Date: Thu, 15 Feb 2001 10:06:51 -0700 (MST) From: Nate Williams <nate@yogotech.com> To: Seth Leigh <seth@pengar.com> Cc: nate@yogotech.com (Nate Williams), freebsd-smp@FreeBSD.ORG Subject: Re: possible problem with SMP? Message-ID: <14988.3243.652929.139953@nomad.yogotech.com> In-Reply-To: <5.0.2.1.0.20010215020051.01b9fc70@hobbiton.shire.net> References: <5.0.2.1.0.20010214234050.026959f8@hobbiton.shire.net> <5.0.2.1.0.20010214222040.00aac5c0@hobbiton.shire.net> <Pine.GSO.3.96.1010214142231.6504A-100000@p1> <20010214191110.D78224@mollari.cthul.hu> <5.0.2.1.0.20010215020051.01b9fc70@hobbiton.shire.net>
next in thread | previous in thread | raw e-mail | index | archive | help
> There are problems of cpu starvation with a Many to Many model in that > usually (unless you manually step up the concurrency) the Many to Many ends > up being Many to Number of CPUs. If Number of CPUs is equal to 1, then > usually you will only get one LWP beyond the ones that are sleeping. Using > the Solaris libpthread, a user-level thread will not give up the LWP it's > using until it voluntarily makes a call into libthread which gives > libthread an opportunity to deschedule that thread and schedule a different > one. This is the same 'model' that Green Threads use, which isn't very effecient. It is possible to have user-space threads be pre-emtive (setjmp/longjmp), so don't blame the implementation issues solely on user threads. > To illustrate this, I talked to a customer the other day whose > application had around 200 threads. He had around 40 LWPs created, but 39 > of them were blocked in the kernel, and 1 appeared to be running when > looking at the threads in dbx. The application appeared to be hung, > though. Finally he figured out that the one thread that was actually > running on the one free LWP was in fact running a hugely inefficient code > chunk that was simply taking forever to complete, and since it wasn't > calling into libthread, it was hogging that cpu and the other 160 or so > threads in the program simply were not getting *any* time whatsoever. This is a thread scheduler problem. > The solution to this was either to attempt to set the concurrency manually > to something like "a few" LWPs higher than the number he expected to block, > or throw in some sched_yield()s into all of his threads to make sure others > got some time, or else simply make them bound. I advised him to just make > them bound, run some tests to see how this affected his performance, and > then decide from there. In all likelihood the majority of his threads can > and do block at some point, and I see very little reason if that is the > case why he shouldn't just make his threads bound. Context switching is now *MUCH* higher. In properly written threaded programs (which are arguable hard to design, but not necessarily any more so than properly written high-performance multi-process software), context switching can be a big deal, especially in the case of *LOTS* of threads. At some point, the more threads you use, the higher the context switching becomes (this is fairly obvious). > I am starting to believe, based on the number of customers I have talked to > writing threaded code, that most real-world applications that use threads > use threads that sometimes block in the kernel. No argument there, see previous email re: I/O. > That being the case, I > honestly see no real reason not to make them bound. Only those threads that block need to be bound. And, in a good design, you can design the system so these threads have their own kernel context. However, with the current FreeBSD kernel-thread design, this can be some more 'automatically', since the kernel can make this decision for you. This is the essence of kernel/scheduler activations. > Fact is, once they > block, they cause a new LWP to be created anyways No, they don't. The number of LWP's is fixed at compile time. > and then basically if > all your threads block at least once in a relatively short amount of time > you end up with most of your threads having their "own" LWP anyhow, plus > you have the additional overhead of the more complicated thread > library. In short, for many if not most real-world applications, > user-level threads don't really buy you anything. Again, this is pure conjecture, and not based on any real-world benchmarks or experience. I can point to *lots* of both that show the opposite. Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?14988.3243.652929.139953>