Date: Tue, 28 Nov 2000 14:20:31 -0800 From: Julian Elischer <julian@elischer.org> To: Daniel Eischen <eischen@vigrid.com> Cc: arch@FreeBSD.ORG Subject: Re: Thread-specific data and KSEs Message-ID: <3A242FAF.313295F0@elischer.org> References: <Pine.SUN.3.91.1001128124003.15989A-100000@pcnet1.pcnet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Daniel Eischen wrote:
>
> On Mon, 27 Nov 2000, Julian Elischer wrote:
> >
> > One thing I just realised:
> >
> > If we are using defered FP state saving and restoring in the kernel, then we
> > will have troubles with that when switching threads in userland, since the
> > handler for that is in the kernel. Of course we could set the place for it in
> > the KSE mailbox and let the kernel save the information when it needs it.
>
> Our current threads library knows when to save and restore FP state;
> it currently only happens when a signal is received (for i386, I think
> alpha FP state is always saved both in jmp_buf and ucontext_t)
That's comforting.
I was looking at the ia64 specs..
that thing presents some interesting challenges in regards to
the 'intelligent' stack it has. It will be very hard to play
games with it's stack when it's cached inside the chip. I
presume they have a scheme to allow such things as threads,
but it looks a mess from here.
>
> I think we want to avoid saving and restoring FP state unless it's
> necessary. That's probably only when a fault occurs or when the
> KSE is preempted. I like the idea of having the kernel save the
> FP state in the thread state storage area (ucontext_t?) in the
> KSE mailbox thingy.
The question is, what happens to the FPU context when you swap threads?
should each thread have it's own FPU context?
If there is one for the KSE, might that be not enough?
especially if the KSE was pre-empted. If a thread is migrated to
another KSE, having last been pre-empted, it becomes important
that the FPU state go with it because it may have been part way
through some calculations when that KSE was stopped. And what if
the new KSE already has one that wsa stopped in thasame way?
it looks to me like you need to have one per thread.
It's not much different fromt eh point of view of the kernel.
WHen you create a KSE you give it's mailbox.
when you schedule a thread onto the KSE you set a pointer in that
mailbox to the thread's context and state storage area.
The kernel can easily follow that link when it pre-empts the KSE
to store the General regs, the FPU regs etc.
Theoretically it might only store the regs there in a syscall
if it looks like the syscall will block. but the aim would be
to make allthreads look the same when stopped so that the UTS
can restart any one it chooses.
>
> Also, are we going to allow the kernel to follow links out of
> the mailbox, or are we going to limit UTS<->kernel communication
> to just this one page? I think it might be preferable to only
> communicate via the mailbox and never have the kernel attempt
> to read/write to other areas of KSE/thread storage.
The kernel already has to follow links etc for (for example)
the readv() syscall. it's not that big a step. If you allocate
all the thread-context blocks together, the pages they are in will
be pretty hot. There are great advantages to having the KSEs
being able to follow links. For example it means that the kernel
can ALWAYS deliver a linked list of ALL completed and 'ready-to-run'
threads. It can set them up so that each one will look exactly
as if it has just returned from the syscall. If you only deal
with one structure, you have to consider what happens when you
cannot fit all returning threads into the single structure.
As the kernel takes control (as the syscall or trap is entered)
it notes where the context block is and when and if it decide
it needs to save context, it knows where to put it. The UTS is
given everything on a plate, and it's almost easier to do it
this way for the kernel too. It can store this address with
the KSEC and use it without any fear of ever having a clash with
some other returning syscall (for example). I can imagine where
a syscall starts on one KSE and is completed on another.
it makes sence for the context to travel with the thread/KSEC
rather than the KSE, which may suddenly have 4 syscalls
all coming back within the same upcall. (where do you save
all that data?)
> For instance,
> we could place the pointer to the thread state storage area
> in the mailbox. But that would require a copyin, and then a
> copyout to another page that might be paged out.
Since the thread storage is part of the thread control block
that the UTS has just used to schedule the thread, it's
unlikely to be paged out. Even less so if it shares a page
with other thread control blocks
And you could always protect it with madvise().
In any case what you suggest above is EXACTLY what jason and I
were planning on doing. The kernel will define a structure
in /sys/i386/include/kse.h (or somewhere) called something like
struct user_process_context
which you would include in your thread control block.
it would include a link to other such blocks (so we can
return a linked list of completed or pre-empted threads (KSECs)
and a status word that says whether it is a completed syscall, or a
preempted thread, or whatever, and a cookie that the kernel
doesn't touch so you can extend it with whatever else you need.
> The drawback
> of only using the mailbox is that it requires an additional copy
> by the UTS every time an upcall is made (to copy the thread state
> from the mailbox to the storage area in the thread).
You forget that a single upcall may want to return 37 completed
syscalls or pre-empted threads.
Using my scheme.. there is an upcall, and bingo the UTS has ALL
the completed items at once.
It sorts them onto the runnable queues, selects it's favourite
and puts that address into it's mailbox, loads the context,
and it's off an running the next thread.
>
> --
> Dan Eischen
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-arch" in the body of the message
--
__--_|\ Julian Elischer
/ \ julian@elischer.org
( OZ ) World tour 2000
---> X_.---._/ presently in: Budapest
v
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A242FAF.313295F0>