Date: Tue, 28 Nov 2000 12:46:37 -0800 From: Julian Elischer <julian@elischer.org> To: "Brian F. Feldman" <green@FreeBSD.org> Cc: arch@FreeBSD.org, jasone@FreeBSD.org Subject: Re: Threads .. chopping up 'struct proc' Message-ID: <3A2419AD.43A14605@elischer.org> References: <200011262239.eAQMd0576413@green.dyndns.org>
next in thread | previous in thread | raw e-mail | index | archive | help
"Brian F. Feldman" wrote:
>
> Julian Elischer <julian@elischer.org> wrote:
> > I'v been looking a the proc srtucture..
> >
> > The aim is to eventually move some of the fields into a
> > struct KSE (struct schedbox?)
> > struct KSEC (struct threadcontext?)
> > struct KSEG (struct schedgroup?)
>
> Sounds about right, as far as I've been following the discussion (I read all
> of -arch, but don't follow -smp at all since I just don't have SMP ;)
>
> My question thus far is, okay, given a proc has one of each; will a set of
> threads, in any form, ALWAYS have a proc backing it up? It would make sense
> as such, and in that case I'd think that you would reduce a lot of the
> complexity in the switchover.
>
> > Initially we would simply include one of each of these in the struct proc,
> > but link them together as if they were correctly connected up.
> > we would use macros such as:
> > #define p_estcpu p_kse.kse_estcpu
> > to keep present code working....
> > eventually functions that get changed to receive a kse directly
> > would just use kse->kse_estcpu and if they need proc they
> > can use kse->kse_proc. But until then, we'd start by simply
> > separating the fields and using macros. Then we can convert
> > calls at our leasure.
>
> What would be the difference between doing it "right" for struct proc in the
> first place rather than dummying them up? I wouldn't want an artificial
> discrepancy here, if possible. Perhaps you could explain a bit more of the
> vision you have here? I haven't been able to pick that bit up from your
> posts as of yet. A KSE of just one thread would seem to logically be
> handled the exact same as a process.
>
> > However when going through the fields in struct proc,
> > some difficulties become obvious. Here's my initial
> > division of the fields. I've added a comment at the
> > beginning of each line that indicates where I think
> > it should go, however I'm not convinced about some of them:
> >
> > P = stays in struct proc
> > E = goes to 'KSE' struct (schedulable entity)
> > G = goes to 'group' struct
> > C = goes to 'sleepable Context' struct.
>
> Does each KSE get a sleepable context? I don't know if I really see where
> it fits in; sounds like it would have a 1:1 mapping with KSEs.
>
Ok I'm going to only answer this question here as I'm off to school inthe
morning and it's 12:30
AM now.. but you have a misconception so I'll try clear that up quickly..
A KSE doesn't have a stack. It doesn't have any state WRT system call execution.
When a system call happens, controll passes from userland, to a waiting KSE that
is presently assigned to teh processor you are on, and your process. The KSE
grabs a spare "KSEC - KSE CONTEXT) (maybe it already has is sitting ready) and
uses it. The KSEC supplies a stack and storage for anything that describes the
state of the processor at any moment during the syscall.
When the system call blocks, the KSEC is left on the sleep queue, and the KSE
grabs another one, and performs an upcall to the Userland Thread scheduler,
which schedules another thread. When THAT thread does a system call, the system
call is executed, storing a set of frames and state onto the stack in the NEW
KSEC. If, in turn, that blocks, it too is thrown onto the sleep queue.
Everything needed to complete the system calls is in the KSECs, which is
hibernating on the Sleep queues. When the system call is reawakenned, the
kernel, waits for a scheduling event in which a KSE from that process (possibly
the same one) is being scheduled. It then reassociates the first KSEC (with it's
stack and stored processor context) with that KSE and then completes the system
call (including any copyout()s or copyin()s). However, instead of crossing back
to user space when it gets back up to the boundary, it puts the syscall's return
information in the mailbox that the Thread system configured (I skipped that
bit) for that thread (don't worry it's trivial), and checks if there are any
more awakened syscalls to complete. It keeps doing this until there are no more
awakening KSECs, at which time it does an upcall to the process. This results in
the Userland Thread Scheduler (UTS) picking up all the completed threads,
deciding which is the highest priority, and running it, as if it were just
returning from the kernel.
I forgot to mention that the mailboxes for the completed threads are linked
together by the kernel before doing the upcall, and the resulting list is passed
as a single pointer to UTS.
Note: the thread that was running when the KSE was pre-empted is also in the
list of threads that is returned to the UTS when the upcall happens, so the UTS
may decide to let it continue running.
It didn't voluntarily do a syscall, but it did cross to the kernel when the
timer interrupt occured, so it can be faked up to look the same. If it was in a
critical region, then of course it should have marked that fact, so it would be
scheduled first. A process may have a KSE for each physical processor. When it
creates a new KSE (upto the maximum of N) it sets up a KSE mailbox. When it
shedules a thread, it places a pointer to the Thread mailbix in the KSE mailbox.
The KSE always knows where it's mailbox is so it can always find the thread
mailbox of the thread that just made the systemcall. When the syscall blocks,
that thread mailbox address is stored int the KSEC, and it is zero's out from
the KSE's mailbox.
When an upcall happens, the KSE adds the linked list of all completed syscall's
mailboxes in that same KSE mailbox, as well. The UTS just takes that list, and
adds the threads mentionned onto it's lists of runnable threads, and then makes
a schedulaing decision and runs the highest priority thread. It sets the mailbox
address of that thread into the KSE's mailbox, and jumps into the thread..
etc.etc.
I haven't mentionned KSEGs here but if you are limited to N KSEs, you want a
container into which you want to put extra competeing KSEs (for example a super
High prority thread).
usually you just have one KSEG, but you may start another, in which they are
treated by teh system much like two separate processes. each with it's own KSEs.
more later.
Julian
--
__--_|\ Julian Elischer
/ \ julian@elischer.org
( OZ ) World tour 2000
---> X_.---._/ presently in: Budapest
v
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A2419AD.43A14605>