Date: Fri, 03 Aug 2001 22:24:00 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Rik van Riel <riel@conectiva.com.br> Cc: Julian Elischer <julian@elischer.org>, craig <craiglei@pasia.com.cn>, freebsd-hackers@FreeBSD.ORG Subject: Re: How to visit physical memory above 4G? Message-ID: <3B6B86F0.EAAA016@mindspring.com> References: <Pine.LNX.4.33L.0108031243370.11893-100000@imladris.rielhome.conectiva>
next in thread | previous in thread | raw e-mail | index | archive | help
Rik van Riel wrote: > > Original poster said he was working on it for Linux, which > > means it's not done, which means "not Linux". > > It's been running for a while now, integrated > in the 2.4 kernel. > > The way Linux manages to avoid the horrors you > describe is by simply not letting the kernel > use more than its 1GB (or 2GB) of KVA. No special > tricks to make the kernel use more memory, let it > go on a diet instead. This is a trivial implementation. I'm not very impressed. In particular, it is the kernel, not the user space, which needs access to more memory. User space is swappable (by the kernel, of course), while only a small fraction of the kernel memory is itself swappable. If the memory were accessible to the kernel, the kernel could swap programs to it, and not have to worry about going to disk. Further, making the RAM only available to user space is trivial; given access to the hardware, I think I could do this with less than a week of elapsed time; it is, IMO, about a days worth of real work. Personally, I'm not interested in a huge user space, so much as I am in supporting more TCP connections, or in supporting full TCP windows sizes for all connections. I've run FreeBSD at 1.6M simultaneous connections, and we are shipping product that can do 1M connections, and do caching and load balancing on top of that. Fully populating both the transmit and receive windows for 1M connections is 32G of RAM, right there... and it better be kernel RAM, or you're screwed. > > FWIW, on the "Linux has it, so FreeBSD should have it" > > arguments: > > > > "If all your friends jumped off a cliff..." > > Fully agreed, but as long as you don't do any > weird tricks trying to let the kernel itself use > more memory, it's not at all like jumping from a > cliff ... I still don't see it being useful, unless it applies to the kernel memory, and then you are basically stuck doing the virtual memory thing in software, like on the PPC or Alpha or MIPS chips, so that you can manage the full address space using descriptor structures for the memory you are accessing, using a fully-faulted address space. For the user space processes, it's not pretty, moving system call parameters from user space to kernel space, if you push the user space up to 4G. I admit that it _may_ be useful for large user spaces, even in the absence of massive amounts of RAM, but then I'm only using two descriptors -- one for the kernel, and one for the user space -- and I need to use a gate to talk between the two, or my kernel won't be there to take the call, so my system calls get more expensive by about a factor of 3. Doing the "graceful" expansion based on when the user space grows above a certain size is extremely problematic. IMO, it is not worth doing this: it complicates and makes more expensive the common case, and there is hardware that you can get that doesn't have this problem. I still haven't seen a useful user space application for this which would not be better handled by having the program in question mmap/munmap regions on its own, since it's guaranteed that the program would do a better job of picking what should or shouldn't be directly accessible without a fault than the kernel ever could... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B6B86F0.EAAA016>