From owner-freebsd-current Wed Oct 23 2:36:23 2002 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52B8E37B401; Wed, 23 Oct 2002 02:36:22 -0700 (PDT) Received: from axe-inc.co.jp (axegw.axe-inc.co.jp [61.199.217.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 90E0E43E6E; Wed, 23 Oct 2002 02:36:20 -0700 (PDT) (envelope-from tanimura@axe-inc.co.jp) Received: from t-axegw.t.axe-inc.co.jp ([218.230.241.250]) by axe-inc.co.jp (8.9.3+3.2W/3.7W) with ESMTP id SAA13089; Wed, 23 Oct 2002 18:36:14 +0900 (JST) Received: from shojaku.t.axe-inc.co.jp ([192.168.6.103]) by t-axegw.t.axe-inc.co.jp (8.12.6/3.7W-Axe-Gwhost-Tokyo) with ESMTP id g9N9Zg14072575 ; Wed, 23 Oct 2002 18:35:42 +0900 (JST) Received: from shojaku.t.axe-inc.co.jp.t.axe-inc.co.jp (localhost [127.0.0.1]) by shojaku.t.axe-inc.co.jp (8.12.6/3.7W-Axe-Tokyo-NoARR) with ESMTP id g9N9ZgoK086922 ; Wed, 23 Oct 2002 18:35:42 +0900 (JST) Message-Id: <200210230935.g9N9ZgoK086922@shojaku.t.axe-inc.co.jp> Date: Wed, 23 Oct 2002 18:35:42 +0900 From: Seigo Tanimura To: Bruce Evans Cc: Seigo Tanimura , current@FreeBSD.ORG, Subject: Re: Dynamic growth of the buffer and buffer page reclaim In-Reply-To: <20021023143249.D20521-100000@gamplex.bde.org> References: <200210220949.g9M9nroK026750@shojaku.t.axe-inc.co.jp> <20021023143249.D20521-100000@gamplex.bde.org> User-Agent: Wanderlust/2.8.1 (Something) SEMI/1.14.4 (Hosorogi) FLIM/1.14.4 (=?ISO-8859-1?Q?Kashiharajing=FE-mae?=) APEL/10.3 MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) Organization: AXE, Inc. MIME-Version: 1.0 (generated by SEMI 1.14.4 - "Hosorogi") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 23 Oct 2002 16:44:06 +1000 (EST), Bruce Evans said: bde> I should be the last to defend the current design and implementation of bde> the buffer cache, since I think it gets almost everything wrong (the bde> implementation is OK, but has vast complications to work around design bde> errors), but I think buffer_map is one of the things that it gets right bde> (if we're going to have buffers at all). (snip) bde> I use the following changes in -current to enlarge the buffer cache and bde> avoid fragmentation. These only work because I don't have much physical bde> memory (512MB max). Even i386's have enough vm for the pure form of bde> buffer_map to work: bde> - enlarge BKVASIZE to MAXBSIZE so that fragmentation can not (should not?) bde> occur. bde> - enlarge nbuf by a factor of (my_BKVASIZE / current_BKVASIZE) to work bde> around bugs. The point of BKVASIZE got lost somewhere. bde> - enlarge nbuf and associated variables by another factor of 2 or 4 to bde> get a larger buffer cache. bde> This is marginal for 512MB physical, and probably wouldn't work if I had bde> a lot of mbufs. nbuf is about 4000 and buffer_map takes about 256MB. bde> 256MB is a lot, but nbuf = 4000 isn't a lot. I used buffer caches bde> with 2000 * 1K buffers under Minix and Linux before FreeBSD, and ISTR bde> having an nbuf of 5000 or so in FreeBSD-1.1. At least 2880 buffers are bde> needed to properly cache a tiny 1.44MB floppy with an msdosfs file bde> system with a block size of 512, and that was an important test case. bde> End of FreeBSD-[2-5] history. Incidentally, Solaris 7 on sun4u reserves a space of 256MB in the KVM according to Solaris Internals. On i386 (x86), the size is only 4MB. Not sure whether they use those spaces in a pure form, or they cluster some consecutive pages (which leads to fragmentation), though... NetBSD UBC also makes a map dedicated to buffers in kernel_map. Maybe there is a point to have a map dedicated to the buffer space for a better stability, and the size of the buffer map could be much smaller than now. During my testing, I found that only up to 6-7MB of the buffers out of 40-50MB were wired down (ie busy, locked for background write or dirty) at most. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message