From owner-freebsd-current@FreeBSD.ORG Fri Jan 4 11:18:07 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 245B416A469 for ; Fri, 4 Jan 2008 11:18:07 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by mx1.freebsd.org (Postfix) with ESMTP id 8F5D913C448 for ; Fri, 4 Jan 2008 11:18:06 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: by ug-out-1314.google.com with SMTP id y2so3601808uge.37 for ; Fri, 04 Jan 2008 03:18:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=hZfaHCNwrnh7haN2GtWepr9fSx+5siwIR0QBCmYuHTE=; b=qijGKmlMMuy6qHCR0raPj80VtDcVLSvggq/5JCWQH66RGmKTBrusqNn7Zu63RHUzO9l8D85VZP4Zm+oJjHIBSe8f5Meh3pML1np2EFSNdu3Vq90L1Z6xvZGM0Fs40bjKIB0BTqStWNlO7mciiZo+ZT+wbRETJoWRoxBrwxsqLKM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=qGgqa8y3/6AySyXYAYV+9myTenJk84Vs8oX4RDr7muQwE4UqIIpLPuqSVb4biUgPM52jEqaQh0XmRtDnnVsFuaqM6SguuQ8NaW3iycUr8CHmizUhD+Mphyjgqg0TsPoZ841XM5k28HvigPkI2INJii9eFJXwTzjdkRyhGIpeagw= Received: by 10.66.219.11 with SMTP id r11mr16713450ugg.31.1199445485104; Fri, 04 Jan 2008 03:18:05 -0800 (PST) Received: by 10.66.248.11 with HTTP; Fri, 4 Jan 2008 03:18:05 -0800 (PST) Message-ID: Date: Fri, 4 Jan 2008 11:18:05 +0000 From: "Igor Mozolevsky" Sender: mozolevsky@gmail.com To: "=?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?=" In-Reply-To: <86bq81c12d.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <477C82F0.5060809@freebsd.org> <863ateemw2.fsf@ds4.des.no> <20080104002002.L30578@fledge.watson.org> <86bq81c12d.fsf@ds4.des.no> X-Google-Sender-Auth: 0adb89dd6b28915f Cc: freebsd-current@freebsd.org, Robert Watson , Jason Evans Subject: Re: sbrk(2) broken X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2008 11:18:07 -0000 On 04/01/2008, Dag-Erling Sm=F8rgrav wrote: > For the same reason as it has for the last 20 years or so: memory > overcommit, which means that malloc() allocates address space, not > memory. Actual memory is allocated on-demand when the address space is > used (read from or written to). If there is no RAM left and none can be > freed by swapping out, the process gets killed. The process that gets > killed is not necessarily the memory hog, it is merely the process that > is unlucky enough to touch a new page at the wrong moment, i.e. when all > RAM and swap is exhausted *or* everything in RAM is wired down and > unswappable. Broadcasting SIGDANGER would be a much better option; followed by SIGTERM to the memory hogger (to allow for graceful termination) and only then SIGKILL. I can imagine a few (legitimate) scenarios when a user process would want to hog as much RAM as possible... > Of course, if you're afraid of memory overcommit and you know in advance > how much memory you need, you can simply allocate a sufficient amount of > address space at startup and touch it all. This way, you will either be > killed right away, or be guaranteed to have sufficient memory for the > rest of your (process) lifetime. Alternatively, do what Varnish does: > create a large file, mmap it, and allocate everything you need from that > area, so you have your own private swap space. Just make sure to > actually allocate the disk space you need (by filling the file with > zeroes, or at the minimum writing a zero to the file every sb.st_blksize > bytes, preferably sequentially to avoid excessive fragmentation) Surely you can just fseek() on the file at the correct lenght? > The ability to specify a backing file to use instead of anonymous > mappings would be a cool addition to jemalloc. That would be really cool and even better if it allocated it in a contiguous chunk. Igor :-)