Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 10 Jan 2002 16:27:13 -0700
From:      Nate Williams <nate@yogotech.com>
To:        Terry Lambert <tlambert2@mindspring.com>
Cc:        Nate Williams <nate@yogotech.com>, Daniel Eischen <eischen@pcnet1.pcnet.com>, Dan Eischen <eischen@vigrid.com>, Peter Wemm <peter@wemm.org>, Archie Cobbs <archie@dellroad.org>, Alfred Perlstein <bright@mu.org>, arch@FreeBSD.ORG
Subject:   Re: Request for review: getcontext, setcontext, etc
Message-ID:  <15422.9041.555403.902698@caddis.yogotech.com>
In-Reply-To: <3C3E2017.D8F55078@mindspring.com>
References:  <Pine.SUN.3.91.1020110161827.1485C-100000@pcnet1.pcnet.com> <3C3E168B.B768CDC8@mindspring.com> <15422.7120.512072.974803@caddis.yogotech.com> <3C3E2017.D8F55078@mindspring.com>

next in thread | previous in thread | raw e-mail | index | archive | help
> > > 1)    It is the first occurance of floating point in the
> > >       context; this is complicated.
> > 
> > It's actually not complicated.  It can be treated as a completely
> > separate context (in a separate thread, or whatever).
> 
> I'm trying like a bandit to avoid the "speerate thread"
> treatment of this issue, to avoid the need to call into
> the kernel to maintain the correct context on a per thread
> basis.  8-(.

'Separate userland thread context' is the entire crux of the entire
thread.  If we don't consider it, then the whole point is moot, since
the kernel already handles separate FPU process contexts already.

We don't (necessarily) want to make a syscall to do context switching
either, and I don't believe it's going to be necessary at all.

> > > 2)    It is an occurance subsequent to the first in the
> > >       context, in which case, when the context was brought
> > >       back, the state was brought back, as well, so it's
> > >       already dealt with, since the previous context
> > >       could have been lazy-save at that point.
> > 
> > Daniel's example stated it wasn't in the same context, hence
> > the need to restore the old (prior) context that was saved.
> 
> I gotta say: this works now, when we are talking about
> processes, and it only gets as complicated as everyone is
> trying to make it when people try to use multiple FP
> contexts in a single multithreaded process.

Actually, it works in multi-threaded processes as well, but there may be
subtle issues that are missing.  (See my Java example posted which works
'correctly' in all versions of FreeBSD since 2.2.6.)

> > > My answer to #1 is to treat the FP context as being "per
> > > process", rather than "per thread", and then do explicit
> > > (non-lazy) saves _only_ when migrating between CPUs.
> > 
> > But, Terry.  The context is per-thread.  You can't treat as a
> > per-process context, becuase there *are* multiple contexts
> > (threads) per process, and each of these context has (the potential)
> > to have separate FPU context.
> > 
> > This is just the way it is, like it or not.
> > 
> > Here's a *very* simple Java example.
> 
> [ ... Java example elided ... ]
> 
> > This java program runs under FreeBSD as a single process, but has two
> > different thread contexts that do FPU operations.
> 
> It's tempting to force this example to use explicit context
> saves, based on the JVM's knowledge of the use of the "double"
> class...

FWIW, all context switches in the JVM do a complete save/restore of the
FPU context.  This may not be efficient, but it's safe.  (At least, I
think it's safe, the jury (Bruce) appears to be out on that one still.)

> Let's take another example: let's do exactly the same thing
> in a linear Java program, and then run two instances of the
> program at the same time.
> 
> This will undeniably work right now in FreeBSD, since the
> lazy binding of the contexts is OK.

Except that it's *much* less effecient, even ignoring for the fact the
JVM startup is doubled.

The entire purpose of threads is to be 'lightweight' processes.  Threads
are by definition lighter-weight than processes, so you can have many
more of them running on a system than you could have proceses.

At my previous job, we had a Java server (I/O bound) that had 10K
threads running in a single JVM instance, serving 3K clients.  It used
up 25MB of memory, and was incredibly effecient.  This would have been
nearly *impossible* to implement and field with the same hardware
resources using non-threaded processes.

Plus, the other huge advantage was this software was deployed on 3
different OS's.   FreeBSD, Solaris, and WinXX (95/98/NT).

> This lets us recast the question in terms of "what happens
> that lets this work on process context switches with lazy
> FPU binding, and why do we think this same approach can not
> be used with use space threads?".

The kernel has knowledge about the internal state on whether or not the
process used the FPU.  (However, do we know if it was used during the
particular timeslice, or that it was used at least once, and hence the
FPU state is saved/restored for every context switch from that point
on.)

If the latter, then the user-space solution won't work, simply because
the kernel only knows the 'process' has used the FPU, and
(intentionally) is not aware of which user-context thread has used the
FPU.

If the kernel knows on a per time-slice basis whether or not the FPU has
been used, then either the kernel must export this information to
userland, or the userland thread scheduler must make a system
call at every context switch to determine if the FPU state should be
saved.

In the latter, is the system call overhead to determine the FPU state
*greater* than the cost of simply saving the FPU state at every thread
context.

(I have no idea the answer to this, but for now my gut feeling is that
the cost of saving the FPU everytime is close to the same as making the
system call to determine *IF* the context should be saved.)

> > > It's tempting to say that the FPU enable/disable (and therefore
> > > the save/restore) should be a system call, and handled on a per
> > > thread basis by the user space scheduler.  If the FPU is used,
> > > then you pay an extra system call per context switch penalty
> > > in the user space scheduler for the use of it.
> > 
> > This conflicts with what you are saying above, because each 'thread' is
> > a different context in the same process.
> 
> It's arguable whether it's a different FPU context, and even
> if it has to be... why the same approach to lazy binding will
> not work for it.

See above.  If the FPU has been used, the lazy binding won't work.

> Dan (and Julian) have already argued that the kernel needs to
> be able to find out, after an involuntary switch, which user
> space thread has the context (in particular, to know if they
> are in the user space scheduler at the time), right?

That's question is beyond the scope of this discussion.



Nate

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15422.9041.555403.902698>