Date: Tue, 24 Oct 2000 13:59:53 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: eischen@vigrid.com (Daniel Eischen) Cc: tlambert@primenet.com (Terry Lambert), jasone@canonware.com (Jason Evans), seth@pengar.com (Seth Leigh), smp@FreeBSD.ORG Subject: Re: SA project (was Re: SMP project status) Message-ID: <200010241359.GAA27114@usr05.primenet.com> In-Reply-To: <Pine.SUN.3.91.1001024080432.1904A-100000@pcnet1.pcnet.com> from "Daniel Eischen" at Oct 24, 2000 08:24:19 AM
next in thread | previous in thread | raw e-mail | index | archive | help
> > Perhaps the biggest pain is the default signal behaviour, which
> > no longer results in system call restart. This means that the
> > signal code must self-mask anything that it wants to pretend is
> > a system call, for every wrapping library routine. POSIX really
> > screwed us over on that account, kind of like what they did to
> > locks when you close one of several file handles to the same file.
>
> Hmm. If the method used by the kernel to notify the threads library
> that a thread has blocked in the kernel can also tell the threads
> library that it is/isn't blocked in a restartable system call,
> then the threads library can DTRT WRT interrupting the thread.
> Basically, all it would need is the system call number. It could
> probably get it from the context of the interrupted thread (which
> is passed out to the threads library under SA).
This may be one approach; it assumes a scheduler context in user
space which is updated from kernel space, and which contains the
call number. It's doable, but potentially gross.
> There are only a few system calls that require restarting (assuming
> SA_RESTART is set), so I'm not quite sure what you mean by self-masking
> library routines. Perhaps library routines that use read(), for
> instance, in their implementation?
OK, if I have a call that can be interrupted, e.g.:
int
foo( struct fooargs *args)
{
}
And I want to wrap it in user space so that I can do a call
conversion reschedule based on it blocking, and me getting an
activation, then I have:
int
foo( struct fooargs *args) /* wrapper*/
{
...
rv = realfoo( args);
... /* Danger, Will Robinson!*/
}
..so I need the code that occurs before and after the call to the
realfoo() call to appear as atomic as the realfoo() call, in the
event that I either need to restart it, or, worst case, I need the
library wrapper foo() to return EINTR as if it were realfoo() in
an unwrapped library, not running threaded.
This means that I need to mask off any additional signal processing
between the time that realfoo() returns EINTR, and processing to
the end of the function is completed to the point that foo() can
return -- in other words, foo() must look like a system call to
realfoo() to the caller, even if we end up with a blocking call in
the kernel itself, coupled with an activation. It wouldn't do to,
for example, disable SIGINT or SIGHUP, merely because I had some
background processing going on in another thread.
Under activations, this applies to everything the uses non-blocking
fd's in the current situation, coupled with a scheduler entry in
user space to convert a call that would have blocked into a call
that returns EWOULDBLK. Under activations, this becomes once again
a blocking call, with an activation occurring at the point that
the current kernel would normally put the process to sleep (the one
exception to this would be an involuntary context switch as a result
of the quantum being used up).
Unfortunately, it gets more complicated, as Julian and Archie
could also tell you. Since an involuntary context switch can
occurr during the processing of an activation, in other words, in
the user space scheduler, instead of the kernel, activations that
are pending delivery to user space have to be stalled until the
user space scheduler hits a "safe" place, where it can be concurrent
again, so it can safely process the pending activations. You need
one of these per CPU on which you might be scheduled to run, or more,
if you are capable of being in the scheduler more than once on a
single CPU (this is necessary for some priority handling).
As I said before, this is really more complicated than it would
have to be, if all non-blocking scheduling events occurred in
user space. You could do this, if you had either a non-blocking
system call interface, or a generic AST mechanism, like VMS and NT
have, since either approach will result in either all the work being
done in user space, or all the work being done in the kernel. The
DEC MTS services use AST-based call conversion to schedule several
concurrent operations simultaneously; I used that to add the missing
interlocks into Mentat Streams on VMS back in 92 or so, since the
MTS as it shipped from DEC didn't have timers or interlocks for
doing inter-thread semaphoring.
OBTW: the signal processing following an interrupted system call
is not the only situation where blocking additional interruptions
until processing has completed will be an issue. On any architecture
with register windows, such as SPARC, you will need an explicit
kernel assist to flush the register windows, so it will have to be
stubbed in, even it it's a NOOP on x86 architectures (at least until
the new Intel stuff becomes common, I think).
Personally, I'd probably approach the problem as if I were going
to allow multiple concurrent outstanding system calls from a
single process on a uniprocessor system (which means that you can
have more than one entry in the scheduler for a given process),
and then sort out the migration and other issues, as well as the
SMP issues, later.
Terry Lambert
terry@lambert.org
---
Any opinions in this posting are my own and not those of my present
or previous employers.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200010241359.GAA27114>