Date: Thu, 02 Aug 2001 04:05:20 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Rik van Riel <riel@conectiva.com.br> Cc: craig <craiglei@pasia.com.cn>, freebsd-hackers@FreeBSD.ORG Subject: Re: How to visit physical memory above 4G? Message-ID: <3B6933F0.FC88449A@mindspring.com> References: <Pine.LNX.4.33L.0108020728100.5582-100000@duckman.distro.conectiva>
next in thread | previous in thread | raw e-mail | index | archive | help
Rik van Riel wrote: > > > Only if you want to use it all within one process. > > > > No. It still bites you if you want to do IPC, etc., since you > > can not guarantee the structures used for this are all within > > the non-segmented region of memory. > > Wrong. Your process can have pages from all over the > 64 GB mapped into its page tables. Try doing this with code pages in one segment, the stack in another, and the data being referenced in a third. It will not work. > Each process has 3 GB of virtual memory mapped to any > of the pages of the 64 GB of physical space. Like I said before, this is not useful. The only marginal use for such a thing is for implementing an L3 cache in software, or for implementing multiple virtual machines on one box. > > > Only the FreeBSD memory management subsystem doesn't > > > support it (yet?). > > > > It's not a question of "supporting it", it's a question of > > whether or not it's a useful idea at all. > > > I have yet to see one person using it for anything. So far, > > it is nothing more than marketing fodder: I haven't seen one > > motherboard capable of more than 4G worth of SIMMs. > > I've seen a bunch of the machines. They're rather > popular with the database folks. Name an OS that supports this; more than likely, you will have to appeal to a purpose built embedded system. > > > Problem is that you'll have to fit all of FreeBSD's VM > > > data structures in the 2GB of KVA. This just isn't going > > > to fit with the size the data structures have today ... > > > > So it's not unreasonable to think of putting 8G or 16G in > > a box, and being able to map it all. > > You can never map it all, since your virtual address space > is limited to 4GB... Overlays. A technology from the dawn of time, I know, but so are segment registers. > Basically the database folks are really keen on keeping > their 3GB user addressable memory, so the kernel will > remain limited to 1GB of KVA. They shouldn't care, since they are getting more memory. > On the really large machines, this can lead to the > situation where even the page tables hardly fit into > KVA. 4MB pages seem like the only solution ... This is why we use 64 bit processors for really large machines, and we say that 36 bit address spaces are really pretty useless, and will be obsolete by the time the code is complete, because of ia64 being a cheaper and faster soloution. 8-p. 36 bits only gives you 2^4 * 4G, or 64G, anyway, and it is hardly worth the segment thrashing and instruction and data cache shootdowns to be able to handle it. You are better off throwing 16 machines at the problem: your major cost item is going to be memory, anyway, and getting 64G in one box is going to cost you significantly more than just putting together multiple boxes. If the locality is such that 2G per process is OK, you might as well be on seperate boxes with non-segmenet swapped memory. > (well, there's also the mess of shared page tables, > but nobody is keen on the locking issues those imply) It's much, much worse than that. Like I said before, you could do it pretty trivially, but the expense of doing it will be so high, relatively, that you might as well buy an Alpha or PA-RISC box and call it a day, if you need it now, or just wait for IA64, rather than throwing developement effort into something that will end up in a scrap heap before it gets reasonable performance, or perhaps before it's even deployed. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B6933F0.FC88449A>