From owner-freebsd-current@FreeBSD.ORG Thu May 15 00:32:38 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E30EA37B401 for ; Thu, 15 May 2003 00:32:38 -0700 (PDT) Received: from mail04.svc.cra.dublin.eircom.net (mail04.svc.cra.dublin.eircom.net [159.134.118.20]) by mx1.FreeBSD.org (Postfix) with SMTP id 879E243FA3 for ; Thu, 15 May 2003 00:32:37 -0700 (PDT) (envelope-from pmedwards@eircom.net) Received: (qmail 80291 messnum 1208317 invoked from network[159.134.237.78/webmail01.eircom.net]); 15 May 2003 07:32:35 -0000 Received: from webmail01.eircom.net (HELO webmail.eircom.net) (159.134.237.78) by mail04.svc.cra.dublin.eircom.net (qp 80291) with SMTP; 15 May 2003 07:32:35 -0000 From: "Peter Edwards" To: current@freebsd.org Date: Thu, 15 May 2003 08:32:35 +0100 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit X-Originating-IP: 155.208.225.212 X-Mailer: Eircom Net CRC Webmail (http://www.eircom.net/) Organization: Eircom Net (http://www.eircom.net/) Message-Id: <20030515073237.879E243FA3@mx1.FreeBSD.org> Subject: Re: vm_page_max_wired and gpg... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2003 07:32:39 -0000 (This bounced first time around due to DNS problems) Hi, >For some time now, gpg has been having issues on my -CURRENT system >with being unable to mlock() the ~8k buffer it uses to hold the >decrypted secret key. A ktrace shows that it's being returned an >EAGAIN from mlock(), which my peeking around has shown could be: > >1) I hit the RLIMIT_MLOCK limit on memory locking, or >2) I hit the system-wide "wired pages" limit. I ran into this when trying to get "memcheck" working, so I suppose I should share my results. The kernel has a "max wired pages" limit, that's set when the swapper starts up to be one third of physical memory. You can see this in src/sys/vm_pageout.c, on about line 1414: > if (vm_page_max_wired == 0) > vm_page_max_wired = cnt.v_free_count / 3; This is pretty much a third of what you see at boot time (and in /var/log/messages or dmesg) for "avail memory = " Now, unfortunately, pretty much the only thing that tries to obey this limit is "mlock". So, just creating dirty buffers can be enough to stop any user process locking any memory at all. On my 128M laptop, There's about 14M wired at boot time. However, creating dirty filesystem buffers will rapidly raise this. For example, "dd if=/dev/random of=/tmp/dirtyMeSomeBuffers" will rapidly bring me up to about 34 megs. There's obviously some more checks and balances to ensure the number of buffers doesn't consume all available memory, but there's other places in the kernel that pages get wired down, and it's pretty easy to hit that one-third limit. I really don't know the VM well enough to say how to fix it, but it seems that mlock() is at the mercy of the rest of the kernel, and the rest of the kernel doesn't really care. Maybe either a better heuristic to decide wheather to allow mlock() to complete, or even allow mlock() to launder itself a few new pages when things are tight might be an idea. (Or, maybe the I/O system shouldn't be robbing a share of "wired pages", and have its own category for that data.) For your purpose, making vm_page_max_wired a sysctl would probably fix the problem in the short term. Anyone got any ideas for the long term? :-) -- Peter Edwards