Date: Tue, 21 Nov 2000 15:55:16 -0800 From: Julian Elischer <julian@elischer.org> To: arch@FreeBSD.ORG Cc: smp@FreeBSD.ORG Subject: Re: Threads (KSE etc) comments Message-ID: <3A1B0B64.6D694248@elischer.org> References: <Pine.SUN.3.91.1001121160717.7102A-100000@pcnet1.pcnet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Note to all on FreeBSD-SMP.. this is to ARCH, and CC'd to SMP
I'll trim SMP next time around so if you are reading this in SMP and are
not on 'arch'......
Daniel Eischen wrote:
>
> On Tue, 21 Nov 2000, Terry Lambert wrote:
> > > yes, but that gives the ability to use M times as much CPU as a
> > > nonthreaded process.
> >
> > If you won't give it to me, I'll just take it, instead, either
> > by using rfork() and a shared memory segment for my global data,
> > which gets me the equivalenet of a threaded environment, for all
> > practical purposes, or I'll just fork() and run multiple instances
> > of myself. Either way, you don't get to tell me not to compete
> > as multiple processes, and I can throw a KSEG based policy out the
> > window, without needing your permission to do it. Worse, there
> > is additional context switch overhead introduced by this (the
> > same reason Linux kernel threads are a bad idea), and everyone
> > gets to pay the penalty for my application.
>
> I have to agree with Terry. You can't dictate what application
> writers will do. If they want more CPU and they can't get it
> with the threading model we provide, then they will get it
> another way. Limiting PTHREAD_SCOPE_PROCESS threads to 1 KSEG
> with only 1 quantum doesn't stop someone from [r]fork()'ing
> to get more CPU.
At this stage I'm having the following thoughts..
1/ I think we've almost talked ourselves out of the need for a
KSEG.
2/ I don't want to actually say that until I've thought about
it a bit.. there were some REALLY GOOD REASONS for it before,
I just can't think what they were. I'll probably remember
at 3AM this morning while I'm mostly asleep.
My remaining use for KSEGs is for housekeeping:
3/ My KSEG thought was:
The user can take as many quanta as he wants by having many
KSEGs. If he uses default actions and simply fires off a lot
of threads, he gets N quanta, per 'round-robin cycle,
where N is the number of CPUs, because by default the threads
package only fires up a max of N KSEs to run the threads
on in each KSEG.
The KSEG is an 'official' way for him to be more greedy.
but he has to ask for it..
One KSEG gives a single timeslot per round-robin, on each CPU.
but KSEs in the same KSEG don't compete with each other in
the KSEG.
4/ I am not sure that #3 is completely useful these days either,
I'm thinking about it.
5/ it's possible that if KSEs in the same KSEG can never be
on the same CPU then they can never 'pre-empt' each other.
This means that locking between the UTS instances in the same KSE
can predict that locks on structures held private within the KSE
can never 'block', but will always 'spin-out' in a short time
(assuming the UTS is written correctly). This means that
(for example) threads assigned to the same KSEG can be migrated
around easily within the KSEG with very simple locking. If
there are two KSE's in teh same KSEG on teh same CPU, there
is always the danger that one may pre-empt the other while
it holds a lock on a KSEG internal structure, and then we
might deadlock. (this thought is also rather vague)
Basically, KSE's are to provide concurrency across processors,
where KSEGs are to provide competition in the scheduler.
I see the two as being orthogonal.
KSEGs may go away, or may be simply a 'virtual element' that
never actually exists. but I still see the use of confining
'normal threads' to some bounded box. The fact that we give
you the ability to make more boxes at you leasure shouldn't
detract from the usefulness of the box.
Scheduling within the box (and migration therin) needs to be
fast and may happen very frequently. Moving between boxes is
probably less frequent.
For that reason I like having the boxes well defined in a way
that ensures that they are well behaved. making each box
contain only one KSE from each CPU allows as much concurrency
as you can get, yet makes sure that we don't have to
deal with KSEs tripping over each other. I think it will
make the use of simple (tiny) spinlocks the most that will be
needed for such things as process migration within the box.
If you start having KSEs (there is one UTS 'incarnation'
per KSE) tripping over each other's CPUs it starts to get
more complicated.
Having said that, I'm still thinking about whether KSEGs
can go away.
> Let's admit that this is what some applications will want to
> do and allow them to do it within our threading model. No,
> we won't do it by default, but we can provide simple hooks to
> allow an application to request more "CPU reservations"
> (a KSEG under the current definition).
exactly We allow them to be greedy. but we make them do
slightly more work for it. At least they KNOW they are being
greedy.
>
> Note that I am only talking about scope process threads.
> Scope system threads are bound to their own KSEG/KSE pair.
KSEG/KSEs
If A SCOPE_SYSTEM thread creates a new thread, that
thread competes with it on however many KSEs it has,
within it's KSEG.
>
> --
> Dan Eischen
Was my explanation of how KSEs communicate with the UTS
clear enough? It's hard to decide that for your own writing....
--
__--_|\ Julian Elischer
/ \ julian@elischer.org
( OZ ) World tour 2000
---> X_.---._/ presently in: Budapest
v
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A1B0B64.6D694248>