From owner-freebsd-hackers Tue Jan 6 13:18:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.7/8.8.7) id NAA22261 for hackers-outgoing; Tue, 6 Jan 1998 13:18:34 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (8.8.7/8.8.7) with ESMTP id NAA21728 for ; Tue, 6 Jan 1998 13:13:08 -0800 (PST) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.8.8/8.8.8) id OAA01103; Tue, 6 Jan 1998 14:13:00 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpd001039; Tue Jan 6 14:12:50 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id OAA10782; Tue, 6 Jan 1998 14:12:46 -0700 (MST) From: Terry Lambert Message-Id: <199801062112.OAA10782@usr08.primenet.com> Subject: Re: Weird malloc problem. To: steve@visint.co.uk (Stephen Roome) Date: Tue, 6 Jan 1998 21:12:46 +0000 (GMT) Cc: joelh@gnu.org, freebsd-hackers@FreeBSD.ORG In-Reply-To: from "Stephen Roome" at Jan 6, 98 02:24:45 pm X-Mailer: ELM [version 2.4 PL23] Content-Type: text Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk > > IIRC (and I'm no expert), it is possible to sbrk your processes' > > entire addressable memory space (all 2^32 bits), and never use it. > > Not very logical though that I can allocate more memory than I have. > To me at least. Actually it's very reasonable. There are many reasons why you might want a discontinuous virtual address space, where not all pages are backed by real pages. For one, you might want to implement statistical data protection (a one out of 2^20th chance of someone guessing your page), etc.. This would be highly useful to avoid protection domain crossing in an OS simulator, etc.. There are other reasons. Maybe you want a "perfect" hash into memory, but will store some finite (small) number of hashed objects. Maybe you are going to rfork, and you want to be able to marshal data between "kernel threads" using an IPC mechanism instead of a heavyweight mechanism for reinstancing them (like Microsoft screwed up and required by putting instanced OLE/ActiveX interfaces in thread local storage)... and to do that, you need to ensure that the address space will not be private -- and you do that by preallocating it before the first rfork(). There's lots of other examples that I won't bore you with; suffice to say, it *can* be a useful thing to do. > Actually, as someone just pointed out, it's fine to set the limits to > anything, but malloc should never think it suceeded in allocating virtual > memory which clearly just doesn't exist! This is a standard feature of memory overcommit. Other standard features are: (1) The process that can't get the new page dies, even though it may be the longest running process on your system. (2) When you are using an executable file as a swap store, and the image is on a remote FS, if the server goes down, your machine hangs in paging in via the vnode pager until the server comes back up. (3) When you are doing #2, the VEXEC bit can't be set on the NFS server, and therefore the image you are using as swap store can be overwritten by another NFS client, or by the server process, and your application can crash catastrophically (including doing such things as deleting all your files, if the new image's code is juuuuuuust right). (4) You can't do a system suspend/resume, unless you have a seperate disk area for backing your primary RAM and restart information, in addition to the SWAP space. Etc.. Overall, with the exception of "committed" applications and remote FS vagries (which I think should be fixed, since they are fixable), memory overcommit is worth the trouble it causes. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.