Date: Fri, 26 Nov 1999 14:50:31 -0800 (PST) From: Julian Elischer <julian@whistle.com> To: arch@freebsd.org Subject: Re: Threads stuff (fwd) Message-ID: <Pine.BSF.4.10.9911261450200.544-100000@current1.whistle.com>
next in thread | raw e-mail | index | archive | help
---------- Forwarded message ----------
Date: Fri, 26 Nov 1999 13:42:34 -0800 (PST)
From: Julian Elischer <julian@whistle.com>
To: Daniel M. Eischen <eischen@vigrid.com>
Subject: Re: Threads stuff
On Fri, 26 Nov 1999, Daniel M. Eischen wrote:
> Julian Elischer wrote:
> > Ok I fetched it and have been looking at it.
> >
> > comments:
> >
> > circles P1 and P2 are?
>
> Just the UTS' view of its allocated subprocesses. The UTS asks the kernel
> (or rforks) for additional subprocesses. Either P1 or P2 is the main
> process, and the other one is the rfork'd process.
>
> > I assume the two boxes are the subprocesses that the program has forked to
> > give itself parallelism.
>
> Yes.
>
> >
> > there are two aync nootify contexts because there is one per subprocess
> > right?
>
> Yes.
>
> >
> > kse_current..
> > ok
> >
> > kse_notify..
> > this is effectively the saved context that will be used to upcall?
>
> Yes. There is only one for now, there may be a need for more than one
> though.
>
> >
> > kse_unblockedq
> > ok
> >
> > kse_blockedq..
> > hmmm maybe.. the KSE's that are blocked are onthe sleep queues but I guess
> > you need a way of being able to find them, so, ok..
>
> Right. The UTS might also want to cancel them.
>
> > I presume that t4, t6, and t9 are blocked in userspace..
>
I screwed up reading it..
ignore that one..
> No, those are blocked in kernel space. Their respective KSEs are
> shown hung off of proc P1 and P2. Oops, the kse_blockedq and
> kse_unblockedq tags on proc P1 are reversed. P1->kse_blockedq
> should read kse_unblockedq, and P1->kse_unblockedq should read
> kse_blockedq.
ok but then in your diagram you should show some threads blocked in
userland as well.
>
> > can we presume that threads blocked in user space and threads blocked in the
> > kernel are indetical? (At this moment I don't think we can, though it is a
> > design goal')
>
> Threads blocked in user space (let's say waiting on a mutex), don't
> have a KSE.
exactly.
But I was thinking of what context the UTS had access to for the thread.
For example, could the UTS time-out an IO and return a 'failed' result
and restart a thread that had made an IO request, even though the IO
hadn't returned? If all the context that was needed to restart a thread
blocked in the kernel was available to teh UTS then that saved thread info
must be pretty much the same as what a thread blocked in the userland must
be.
> There are only KSEs for threads blocked in the kernel,
> the UTS event notifications, and the currently running threads.
agreed
> When
> a thread blocked in user space unblocks, the UTS can simply do a
> _longjmp to resume it (using the same KSE as the thread being swapped
> out). Perhaps we should show examples of threads blocked in user
> space in the diagram also.
yes that was my comment above.
>
> I don't think we want to require a kernel call to resume a thread
> that is blocked in user space, right? And I really don't think we
> want KSEs for every thread in an application, right?
no, though you are suggesting it in other email because you say you need
to block/unblock signals, and your {s,g}et_context calls would require it.
>
> So other than the above, threads blocked in user space and threads
> blocked in kernel space are exactly the same. There will just be
> a flag and perhaps KSE ID in the user thread struct to indicate
> whether it can be resumed with a _longjmp or a thread_resume(2).
After a UTS has been notified that a thread has blocked in the kernel,
it should end up being identical to one that is blocked in user space,
except it is waiting on an event that the kernel must supply
(i.e. an IO completion block or something.)
Until it is notified of the blocking it is 'running'
>
> >
> > kde_freeq..
> > I don't think this is needed. we can have a systemwide cache of free KSEs
> > without much problem, and even a per-processor cache maybe..
>
> Good point. I'll remove them. We might want the proc to know the
> maximum number of KSEs it's suppose to have, though. Consider
> thread groups and subprocesses at other-than-default-priority.
> Once a processes blocked KSE limit is reached, the process can
> be put to sleep until at least one of the KSEs wakeup.
sure
>
> > > We need some more diagrams, but I wanted to make sure we're in
> > > general agreement before I make any more.
> >
> > basically I agree.
> > I had an interesting thought yesterday..
> >
> > If every thread stores all it's context on the end of it's stack, then
> > we only have to store the stack right?
>
> OK, makes sense.
>
> > so at teh moment using the current system call-gate, we store all teh
> > context in the kernel stack,
>
> Hmm, I was under the impression it was on the USER stack. Isn't
> that why we have to copyin/copyout to access the trapframe from
> the kernel?
I need my x86 book, but it's at work..
/*
* Call gate entry for FreeBSD ELF Linux/NetBSD syscall (int 0x80)
*/
SUPERALIGN_TEXT
IDTVEC(int0x80_syscall)
subl $8,%esp /* skip over tf_trapno and tf_err
*/
pushal <----
pushl %ds <----
pushl %es <----
pushl %fs <----
movl $KDSEL,%eax /* switch to kernel segments */
movl %ax,%ds
movl %ax,%es
MOVL_KPSEL_EAX
movl %ax,%fs
movl $2,TF_ERR(%esp) /* sizeof "int 0x80" */
FAKE_MCOUNT(13*4(%esp))
MPLOCKED incl _cnt+V_SYSCALL
ALTSYSCALL_LOCK
call _syscall
and in ipl.s in doreti()
.globl doreti_popl_fs
doreti_popl_fs:
popl %fs
.globl doreti_popl_es
doreti_popl_es:
popl %es
.globl doreti_popl_ds
doreti_popl_ds:
popl %ds
popal
addl $8,%esp
.globl doreti_iret
doreti_iret:
iret
we've already switched to the kernel stack.
and we save all the regs (pushal) abd then the segment registers.
the stack segment has already been changed I think.
Of course if I need my x86 book it might not work for alpha..
but if the syscall itself did: (this is my version of the 'syscall()' syscall.)
ENTRY(syscall)
pop %ecx /* rta */
pop %eax /* syscall number */
push %ecx
pushal <-----------
pushl %ds <-----------
pushl %es <-----------
pushl %fs <-----------
KERNCALL
call _UTS_syscall_end <----------- may decide to schedule something else.
popal <-----------
popl %fs <-----------
popl %es <-----------
popl %ds <-----------
push %ecx /* need to push a word to keep stack frame intact
upon return; the word must be the return address. */
jb 1f
ret
1:
PIC_PROLOGUE
jmp PIC_PLT(HIDENAME(cerror)) <-- not needed.. already in return status b
then the same stuff would not be needed in the first code segment
and all that context would be available to the UTS immediatly without
needing it to cross the kernel boundary at all.
The only problem I see is we would have to be very careful about getting a pagefault
while saving it.. pagefaults on stack segments are allowed to block?
>
> > but if we were to save it onto the USER stack before it did the
> > KERNCALL, and didn't do it onto the kernel stack
> > then the information we needed to move a blocked syscall thread onto the
> > blocked thread list (as you show) would already all be in the right place.
> > returned values would be placed into a separate async completion block
> > that is allocated at the beginning of every thread's stack rather than
> > straight into the context. Basically this all happens anyhow, but
> > at the present it's all done on the kernel side of the line.
> > I suggest instead moving it to the user side, and making each syscall
> > load it's return values from the status block after it returns from the
> > kernel. ERRNO lives in the block permanently and need not be moved out.
>
> I think I follow you, but diagrams would help :-) Would there be some
> sort of unique ID returned from a blocked syscall, so that the UTS could
> later resume/cancel the thread?
the blockage notification is from the syscall you think you are doing.
>
> > This is the kind of thing that mean when I say that using a different
> > protocol might make sense.
>
> I'm open to the idea, but it just seems like there might be a real
> easy and fast way to switch to a predefined context/trapframe that
> would take us directly to the UTS. The only thing the UTS needs
> to know is what process blocked and a unique ID for resuming/suspending
> the blocked system call. Based on which process blocked, the UTS
> will know what thread blocked, and can tag it with the unique ID
> and place it in the blocked queue. For threads that are preempted,
> you can additionally send the program counter so the UTS can decide
> if it was in a critical region.
>
> > what are you using to make the drawings?
> > If it's tgif or xfig, then maybe we can get at the sources so we can
> > submit suggestions for changes in picture format :-)
>
> I'm using xfig. I wish there was something better, though. Let me
> make your changes, and I'll send you the .fig file.
>
> Dan Eischen
> eischen@vigrid.com
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.10.9911261450200.544-100000>