Date: Sat, 22 Jun 1996 01:40:11 -0700 (PDT) From: "James E. [Jed] Donnelley" <jed@webstart.com> To: Thomas Pfenning <thomaspf@microsoft.com> Cc: Ron Minnich <rminnich@sarnoff.com>, davidg@Root.COM, smp@freebsd.org, mail@ppgsoft.com Subject: RE: SMP version? Message-ID: <199606220840.BAA00118@aimnet.com>
next in thread | raw e-mail | index | archive | help
At 12:33 AM 6/22/96 -0700, you wrote: >Without getting in what Ron has to say but how could you get the >impression this work is on message passing? He is working on shared >memory for years now which is pretty clearly documented on his WEB site. >What do you mean you want to run the UNICOS model but not virtual shared >memory. SCI implements virtual shared memory when I understood it >correctly. > >Don't worry, I was just getting a bit confused by your mail. No problem. I will try to explain how I see it. Looking again at Ron's Web site, I can see the references to "Distributed Shared memory" and "virtual shared memory". I am not sure just what these terms mean, but I have a guess. I don't see anything on the page that refers to memory sharing hardware. My guess is that what he is working on is essentially what SGI refers to in their shared memory model - namely essentially paging through the network. You set up a shared memory map between processors (this is a part I am interested in) so that if a process on one machine references some memory, a copy of that memory is moved to that machine. If the reference is a write then all other copies are invalidated. If it is "another" read then copies can reside read-only on multiple machines. I am just hacking this wording out, but the model is pretty tried and true. It can work reasonably well if there is no significant distributed "simultaneous" reading and writing of the same memory. What we are working on is more like "real" SMP shared memory. Namely with latencies in at least the 1-10 microsecond range and hardware support for "remote" reads and writes through SCI - passing individual cache lines and blocking instruction execution while waiting for "remote" memory. Perhaps this is what Ron is doing and I just didn't catch it at first glance. If so, it was a simple (perhaps inept) misunderstanding. I don't see how he can be doing remote reads and writes of cache lines over 100BaseT. It would seem that at least some hardware to "redirect" memory references is needed. I didn't see any reference to that. Reducing the software overhead of remote memory access is one of the key areas that we want to explore. Of course even 1-10 microseconds is pretty nonuniform as memory access goes these days, but I believe it is worth exploring. My comment about message passing and MPI was inappropriate. It seems clear that he is not going down that path. If he is doing remote memory "paging" with software and getting latencies in the 1-10 microsecond range then I will be interested to read further to see how he does it. In that case perhaps "real" shared memory is not needed... - "virtual" shared memory may be enough for most applications. I guess some applications may push memory uniformity down to the 10-100 nanosecond range, but I would guess that such applications are somewhat unusual. If he is doing remote memory accesses and running a "standard" (i.e. dumb - like Unicos) SMP operating system on his machines then we will be happy to run with his software (should it be an available "open" Unix version such as FreeBSD and/or Linux). We are getting quick turn around in this dialog, but perhaps the clarity is suffering a bit - my apologies. I'll download the postscript from some of Ron's papers and try to clarify my understand of what he has been doing. It will probably be next week before I can get to it. I am working only a very small portion of my time on this effort (for some other folks) and trying to bore directly to my needed answer to see if further effort is justified. Thanks for the quick responce. I hope it helps clarify what I am looking for. Sorry for not being clearer. --Jed http://www.webstart.com/cc/jed-signature.html
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199606220840.BAA00118>