Date: Sat, 11 Nov 2000 17:59:03 -0500 (EST) From: Mitch Collinsworth <mitch@ccmr.cornell.edu> To: freebsd-questions@freebsd.org Cc: Mitch Collinsworth <mitch@mercury.ccmr.cornell.edu> Subject: Re: Linux malloc better on FreeBSD than FreeBSD malloc? Message-ID: <Pine.LNX.4.10.10011111746430.16421-100000@dragon.ccmr.cornell.edu> In-Reply-To: <200011112225.eABMPcW332585@saturn.cs.uml.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Someone wrote: > > recently changed jobs, into a dept that's heavily Linux-based. I > > of course have different preferences, but need to demonstrate > > technical superiority if any changes are going to happen. An > > opportunity just arose to make some comparisons between Linux and > > FreeBSD in maximum process size. We have some new systems with > > 4 GB RAM each that will be made available for running large batch > > jobs. > > You are already at FreeBSD's absolute physical memory limit. > Linux can go up to 64 GB on a PC. It would be a grave mistake > to switch to an OS that can't grow with your needs. > > You take the blame too. > > > We have a very simple test program that malloc's successively larger > > and larger blocks of memory until it fails, freeing the current blocks > > after each successful trial. The strange thing is that the test > > program runs much farther on FreeBSD using the Linux binary in > > compatibility mode than it does using the binary compiled on FreeBSD! > > Included below are the output from each binary and the source. Can > > anyone explain these results? > > Linux libc is good for something! > > > A second question, which is the primary issue we were trying to > > discover is if it is possible for a single process to malloc more than > > 2 GB of memory. FreeBSD supposedly supports up to 4 GB (on Intel > > hardware), but does anyone know if there is still a 2 GB per process > > limit? And if a single process can be larger than 2 GB, is there a > > 2 GB limit on any single malloc with that process? > > The malloc limit difference between FreeBSD and Linux will not be > significant. You can tweak both systems for a few percent more > memory, but you won't ever reach 4 GB on either. This is what I'm trying to find out. But your say-so is insufficient. Could you cite a reference for this assertion? > Don't be short sighted. Your choices are: > > 1. get stuck fighting a losing battle to reach 3 or 4 GB > 2. get an Alpha, and forget about these old limits We are working on getting some Alphas. In the mean time, we have just been given 3 systems with quad 700 MHz processors and 4 GB RAM each. We may as well make the best use of them possible. > 3. split up your program > 4. have your program map/unmap as it operates This implies knowledge of the programs that you simply don't have. Some of our programs can be improved along these lines. Others can not. The latter group will benefit most from the largest arrays possible. > > For comparison purposes, the Linux answers to these questions are that > > with a special "bigmem" patch installed, a process can be larger than > > 2 GB, but any single malloc is still limited to 2 GB. > > This must be wrong. The bigmem patch I've heard of will decrease > available space in order to support more physical memory on > obsolete Linux systems. Also, there isn't any 2 GB limit. You > should be able to allocate almost 3 GB on a Linux box. I may have the patch name wrong. I'm not really up on Linux yet. The answers are correct though and have been proven with the same test program below. We've been able to run up to about 3.5 GB total in a single process on Linux, though still limited to a single malloc being no larger than 2 GB. This has been demonstrated on both 2.2.x and 2.4.x. > You should be running Linux 2.4.xx on these systems. It has > no problem with all your memory, and it whips FreeBSD for > performance too. Again, it does have a 2 GB per malloc limitation, which I'm looking to find a way around. It's possible the alpha is the only solution for this but I want to be absolutely sure. Regarding performance, again I need a reference. Your say-so alone is insufficient. -Mitch To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.LNX.4.10.10011111746430.16421-100000>