Date: Thu, 16 Nov 2000 16:00:34 -0500 From: Shannon <shannon@widomaker.com> To: Mitch Collinsworth <mitch@ccmr.cornell.edu> Cc: freebsd-questions@freebsd.org, Mitch Collinsworth <mitch@mercury.ccmr.cornell.edu> Subject: Re: Linux malloc better on FreeBSD than FreeBSD malloc? Message-ID: <20001116160034.A28365@widomaker.com> In-Reply-To: <Pine.LNX.4.10.10011111746430.16421-100000@dragon.ccmr.cornell.edu>; from mitch@ccmr.cornell.edu on Sat, Nov 11, 2000 at 05:59:03PM -0500 References: <200011112225.eABMPcW332585@saturn.cs.uml.edu> <Pine.LNX.4.10.10011111746430.16421-100000@dragon.ccmr.cornell.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Nov 11, 2000 at 05:59:03PM -0500, Mitch Collinsworth wrote: > > The malloc limit difference between FreeBSD and Linux will not be > > significant. You can tweak both systems for a few percent more > > memory, but you won't ever reach 4 GB on either. > > This is what I'm trying to find out. But your say-so is insufficient. > Could you cite a reference for this assertion? The references is basic OS design, at least for an OS that uses a system call interface like (most) UNIX. The OS always has to have some space in a process so you can never get all of it for your program. Just think about how UNIX works a moment... when a process makes a system call you are calling a routine in the kernel. Unless you are in ring 0 (or the equivalent on your system/OS combination), you cannot just jump anywhere in memory. A process can only be in it's own address space. Therefore, the OS needs to be mapped into the process address space. This is why you can't give all 4GB of addresses to a Linux or BSD process. You can verify this yourself at any number of OS web sites, or by looking into the kernel code and documentation. I think you have to move to a pure message-passing model before you can get the OS out of your processes address space. There may be (probably are) other techniques used, but I am fairly certain that all of the current major CPUs security features enforce this idea. The Linux 2.4 kernels can use a 36-bit addressing modes but I think this is basically just a selector/offset scheme that gives you a lot of large processes, not 36-bit pointers in a process. Heh heh... Intel didn't even want anyone to know thier CPUs could do this at all. > > 3. split up your program > > 4. have your program map/unmap as it operates > > This implies knowledge of the programs that you simply don't have. > Some of our programs can be improved along these lines. Others can not. Indeed. Plus, splitting programs up can often have huge costs in time and money, not to mention hair and fingernails. > The latter group will benefit most from the largest arrays possible. How large? Does another 512MB really mean that much to you? You cannot get that on currentIntel systems so again, the Alpha might be your best bet here. Are you running FORTRAN programs? I would mention 64-bit SPARC, but they are underpowered for the price and BSD and Linux support is still unstable and immature. If you really need lot's of memory, either you split things up or you get a bigger architecture. Alternatively, you could map data structures into very large files, moving sections into memory for processing. A lot of times you can do this and get more performance than letting the OS swap. In your case, it seems you don't have a choice. If I'm not mistaken, I have seen libraries to do this kind of thing that are reasonably free sources. I've only seen you say the word array so far, so right now I'm assuming you are doing some numeric processing or something similar. Somewhere around here I have some books on stuff like this, but everything is in boxes or I'd give you a reference or some URLs. > We've been able to run up to about 3.5 GB total in a single process on > Linux, though still limited to a single malloc being no larger than 2 > GB. This has been demonstrated on both 2.2.x and 2.4.x. Just curious: are you sure about this? Linux allows overcommit and just because you can malloc() 3.5GB doesn't mean you really have it. If you don't touch it, it isn't mapped. I'm asking because I believe the default Linux kernel setup is 1G OS and 3GB process, so I'm curious about you getting to 3.5. > Again, it does have a 2 GB per malloc limitation, which I'm looking to > find a way around. This is an example of tools not keeping up with the OS. NetBSD for 64-bit SPARC is like this. The kernel is new and few of the tools and libraries have been moved up yet. > It's possible the alpha is the only solution for this but I want to be > absolutely sure. Regarding performance, again I need a reference. Your > say-so alone is insufficient. This is a tough call. Linux 2.4 did make a large leap forward, making me question my own belief that FreeBSD is faster. The problem is that it is difficult to really test this. Here's my take on things though: * FreeBSD filesystem is slower, but perhaps more reliable * you can help equalize the fs performance with the BSD soft updates patches * Linux is a lazy swapper... so it's fast at first, but takes a huge hit when you run out of memory and it has to start swapping aggressively * FreeBSD runs almost like it is on a meter... unloaded it seems slower than Linux but as the load goes up it tends to maintain that speed. You can think of it as "streaming" while Linux is "bursty". Linux can be tuned of course, but I'm not sure it ever handles a load as well as FreeBSD. If you are compute-bound, you might not care of course. If you are I/O bound, you owe it to yourself to do some extensive tests. At any rate, beware people on either side telling you that X whips Y because that will often depend on your point of view and the exact applications you are running. Linux feels a lot faster as a workstation, but I'm currently of the opinion that FreeBSD is the superior load handler. -- shannon@widomaker.com _________________________________________________ ______________________/ armchairrocketscientistgraffitiexenstentialist "And in billows of might swell the Saxons before her,-- Unite, oh unite! Or the billows burst o'er her!" -- Downfall of the Gael To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20001116160034.A28365>