From owner-freebsd-stable@FreeBSD.ORG Sat Feb 13 19:39:14 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52A701065693 for ; Sat, 13 Feb 2010 19:39:14 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta13.emeryville.ca.mail.comcast.net (qmta13.emeryville.ca.mail.comcast.net [76.96.27.243]) by mx1.freebsd.org (Postfix) with ESMTP id 37D568FC2A for ; Sat, 13 Feb 2010 19:39:13 +0000 (UTC) Received: from omta15.emeryville.ca.mail.comcast.net ([76.96.30.71]) by qmta13.emeryville.ca.mail.comcast.net with comcast id hVp81d0081Y3wxoADXfErW; Sat, 13 Feb 2010 19:39:14 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta15.emeryville.ca.mail.comcast.net with comcast id hXfD1d00D3S48mS8bXfERP; Sat, 13 Feb 2010 19:39:14 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id AB4081E301B; Sat, 13 Feb 2010 11:39:12 -0800 (PST) Date: Sat, 13 Feb 2010 11:39:12 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Message-ID: <20100213193912.GA4894@icarus.home.lan> References: <4B759E70.4030809@comcast.net> <4B75ADC7.6000308@comcast.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: ZFS ARC being limited below what is defined in /boot/loader.conf X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Feb 2010 19:39:14 -0000 On Fri, Feb 12, 2010 at 01:10:51PM -0800, Artem Belevich wrote: > vm.kmem_size_max/vm.kmem_size_min define the range vm.kmem_size can be set to. > vm_kmem_size specifies the actual kmem size. > > ARC size in turn limited by vm.kmem_size. > > If you want to bump ARC size, you do need to bump vm.kmem_size. > > --Artem > > On Fri, Feb 12, 2010 at 11:36 AM, Steve Polyack wrote: > > On 02/12/10 13:47, Artem Belevich wrote: > >> > >> On Fri, Feb 12, 2010 at 10:31 AM, Steve Polyack > >>  wrote: > >> > >> > >>> > >>> Has anyone had an issue with the ZFS ARC max being limited below what has > >>> been defined in /boot/loader.conf?  I just upgraded the RAM in a > >>> ZFS-equipped system and attempted to devote 4GB to the ARC cache by > >>> placing > >>> the following in loader.conf: > >>>  vfs.zfs.arc_max="4096M" > >>> > >>> However, after rebooting, querying the sysctl gives me this: > >>> $ sysctl vfs.zfs.arc_max > >>> vfs.zfs.arc_max: 1726489600 > >>> > >>> or about 1.7GB, an odd number that I can't find any references to.  For > >>> reference, I'm running 8-STABLE (as of Jan 19th) on an amd64 system with > >>> 8GB > >>> of RAM.  The system was previously very stable with 4GB of RAM and a > >>> 512MB > >>> arc_max.  I have not modified vm.kmem_size_max (defaults to ~330GB on > >>> amd64) > >>> or any other ZFS tunables.  I'd also like to avoid syncing up to the > >>> current > >>> 8-STABLE if at all possible. > >>> > >>> Thanks, > >>> Steve Polyack > >>> > >>> _______________________________________________ > >>> freebsd-stable@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > >>> > >>> > >> > >> Check your vm.kmem_size. Default setting is way too low. Set it to at > >> least double of desired arc size. > >> > >> --Artem > > > > I mentioned it briefly, but vm.kmem_size_max was left at the default for > > amd64.  At 330GB it is way above and beyond what will ever be allocated to > > ARC: > > $ sysctl vm.kmem_size_max > > vm.kmem_size_max: 329853485875 I'm concerned about adjusting vm.kmem_size, since vm.kmem_size_max is supposed to be auto-adjusting as of this point in time. How does adjusting vm.kmem_size affect things like kern.maxdsiz, kern.dfldsiz, and kern.maxssiz? These tunings are required for things like userland apps which require a large amount of memory (read: mysqld). The amount of VM-related tunables requiring is getting out of hand. We need documentation of some sort, outlining how all these things fit together as to avoid potential kernel panics or memory exhaustion issues; for example, people with 8GB of RAM installed who utilise both ZFS on the server in addition to a memory-hungry mysqld. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |