Date: Sun, 26 Aug 2001 00:15:53 -0700 From: Julian Elischer <julian@elischer.org> Cc: "bsd-api-discuss@wasabisystems.com" <bsd-api-discuss@wasabisystems.com> Subject: Re: changes to BSD APIs for THREADS support Message-ID: <3B88A229.C07CCBB4@elischer.org> References: <Pine.WNT.4.33.0108260058190.1340-100000@todd>
next in thread | previous in thread | raw e-mail | index | archive | help
Todd Vierling wrote: > > On Sat, 25 Aug 2001, Julian Elischer wrote: > [Note: I'm not a Solaris weenie; in fact, I really don't like Solaris much > at all. However, I have cited it several times below, mostly because it's > already traveled down all these roads successfully where we are treading > anew.] > > : how does PS extract state from the kernel when a process is simultaniously in > : several states? > > : Which wmesg is shown if there are 8 sleeping threads? > > I'd presume that the only useful way to deal with this is to provide > per-thread info in a ps display. Solaris may be one possible prior art to > look at (particularly with a multithreaded process under their compatibility > /usr/ucb/ps). the question as far as interfaces is to decide whether in a copy of ps being run with the 'threaded' option not set (in solaris the "-L" option) whether the ps goes the same as it does now and gets just process info, and the kernel agregates that for it, or it always looks for thread info and agregates it itself and then only displays the totals. for procfs there is no choice.. the kernel does it, but procfs must always show all the information, so how would we show this information in the procfs hierarchy? /proc/curproc/threads/12332/status /proc/curproc/lwps/1234/status? > > : What does "%cpu" actually indicate? > > There's two components to this. > > * Whether 100% = 1 CPU or 100% = all CPUs; > > * What to show for a process without expanding the individual threads in a > display. > > Personally I prefer that 100% = all CPUs, but that's my take. Other prior > art may be in Solaris /usr/bin/ps and /usr/ucb/ps for this. I looked at the man pages but haven't got the answer there. I'd like to see one in action.. > > : Inside the kernel, how are the proiority adjustments calculated when > : a process might be running and sleeping at the same time? > > Process priorities are a really implementation-specific concept, as a > "complete" threading subsystem will have more complex scheduling than simple > priorities (for RT ability) and multi-level scheduling (for limiting time > allocation to all threads within a process and/or for honoring CPU affinity > requests). So there's no real way to provide a standard for this. We have a cunning plan... well we're covering it.. see the KSE doc at http://www.freebsd.org/~jasone for some starter information. > > What's displayed to the user via ps or top isn't all that important > otherwise; it might as well be a formula averaging the threads' priorities > or somesuch, appropriate for the particular threading implementation. > > : How does ^T decide what thread/proc to report? > > Again, we might see if Solaris provides decent prior art with its LWPs. It's not that important but it's one of the points I encountered while threading the FreeBSD kernel that really I had to 'kludge' because I couldn't figure out what the right answer is.... > > : All this is separate to what syscalls we will be wanting to add to > : actually give the multithreading libraries access to the new kernel > : support. > : > : On that front FreeBSD is looking at at least 4 new syscalls. > : I'm going to describe them by function rather than name as the names are > : not decided: > : > : 1/ Add a new kernel schedulable entity (KSE). > > I'm using the term "kernel thread" below, to define it as a thread of > program flow scheduled by the kernel. It's less confusing to me, but only > because of prior experience with kernel-scheduled threading. 8-) Ahhhh but you see in this world a KSE and a thread are two different things... A KSe is similar I thinkl to a schedular activation in the landmark SA paper that is being implemented in NetBSD. A thread is the short term line of execution that enters the kernel, and might block.. it's saved to one side, and hte KSE continues on by upcalling to the Userland scheduler, and asking for more work. > > : 2/ A yield() call which tells the kernel that the user thread scheduler has no > : present use for the current processor and it may be used by another process. > > By this I presume that you're talking about some sort of many-to-one > userland thread scheduler? well, yes, I agree that the win isn't neccesarily that great, but we're going to try it out.. > > You might want to know that, after some personal conversation with Solaris > developers at USENIX, they've decided that a two-level (userspace plus > kernelspace) thread system is actually more overhead than it is worth and > will be phasing it out in future releases. It turns out that reusing the > existing context switch code available in the kernel is simpler and faster > than jumping around with the userland scheduler's setjmp/longjmp-alike. I have a cunning plan sire.. (again) We have a scheme that simplifies this tremendously. if it works it'll make things so simple that it won'e be much overhead at all. > > I believe that the forthcoming NetBSD implementation also will leave > scheduling entirely to the kernel, but the person in charge of writing it > would know better than I.... No, actually as an SA (Scheduler activations) implememtnation it has a userland scheduler in control. (If not then ist isnt an SA implementation) > > : 3/ A call that can be used by the user thread scheduler to 'unyield' > : a previously yielded processor. i.e. it could do with it, now, if it's > : available. > > Unyield a processor, or a kernel thread? What it sounds like you're > proposing here is to combine processor affinity and kernel thread > pause/resume actions into one group. The Scheduler activation is released from the processor. One KSE can run many threads.. It's a kind of CPU reservation on behalf of the user thread scheduler. if the UTS doesn't need it it needs to release the reservation. > > Processor affinity should really be separate from the operation of kernel > threads. To define the differences, a kernel thread should be able to run > on any available processor by default, but can be set to prefer a specific > CPU or CPU group. CPU-specific operation is available as a subset of the > available operations in this arrangement. threads can run on any KSe they are allowed to. and KSEs MIGHT be linked to certain processors. > > : 4/ A call by which a KSE suicides.. It will never return > : and the #1 frame (upcall frame) associated with this KSE will never be used > : again and can be freed. > > AKA a kernel thread exit point; normal. yep > > : 5/ It is possible that we may need a call by which the user thread scheduler can > : 'cancel' the syscall that a particular kernel thread is currently operating > : on. (particularly if it is waiting). > > Solaris also has a way to do this, but I don't remember what it is off the > top of my head. It is kind of the bottom end of what happens now when a signal is delivered to a process that is in the kernel. Except now it is a thread. All the same logic should remain pretty much untouched. > > : In addition there may be the need for one or two synchronisation primatives > : though this is not certain. > > NetBSD will require kernel-level synchronization primitives as a fallback, > because not all processors NetBSD supports have atomic instructions. I don't think FreeBSD has the 4004 as a target architecture. (Isn't there a MIPS like that?) > > There's one more thing having serious interactions with threads that should > be addressed: > > Multithreaded process signal delivery. There's an intricate way that this > is handled in Solaris, as "async signals". More prior art that's probably > worth imitating for code compatibility's sake. I covered this in another email > > -- > -- Todd Vierling <tv@wasabisystems.com> * Wasabi NetBSD: Run with it. > -- NetBSD 1.5.2 available on CD-ROM soon! -- http://www.wasabisystems.com/ -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B88A229.C07CCBB4>